Email Security Transparency Dashboard Added to Office 365 Defender
Microsoft has announced a new e-mail security dashboard in Microsoft Defender for Office 365, offering customers visibility into threat detection metrics and benchmarking data.
"At Microsoft, we believe that transparency is foundational to trust," wrote Microsoft's Ramya Chitrakar and Scott Woodgate. "As both an e-mail platform and a security provider, we want to work together with our ecosystem and do more to empower customers to understand e-mail security effectiveness."
The dashboard is accessible through the Microsoft 365 Defender portal. It provides tenants with real-time data on e-mail threat volumes, filtering outcomes, and false positive/negative rates based on their own message traffic.
According to Microsoft, the dashboard includes the following capabilities:
- Visual summaries of e-mail classifications;
- Detailed statistics on detection actions, such as blocking or allowing messages;
- Reporting on false positives and false negatives; and
- A comparison of tenant-specific detection rates against Microsoft-wide averages.
The dashboard pulls data from Exchange Online Protection (EOP) and Microsoft Defender for Office 365 signals, including telemetry from Secure by Default settings and user-submitted reports. Customers can use the portal to drill into specific detection events and investigate potential gaps in filtering coverage.
New E-mail Security Benchmarks Released
Along with the new dashboard, Microsoft introduced two types of benchmark reports (found in the dashboard) as part of its new Email Security Transparency initiative in Microsoft Defender for Office 365. These reports are intended to help organizations evaluate how well Defender is performing within their environment and how their results compare with broader Microsoft-wide metrics.
Microsoft-wide benchmark: The Microsoft-wide benchmark provides aggregate performance data collected across Defender for Office 365 tenants. According to Microsoft, the benchmark includes:
- Miss rate for malicious e-mails: 0.003%
- Incorrectly blocked clean e-mail rate: 0.001%
These metrics are updated quarterly and are intended to provide a point of comparison for individual customers evaluating the effectiveness of their own configurations.
Customer-specific benchmarks: The Email Security Transparency Dashboard also shows each organization's own filtering performance using Defender for Office 365 data. Metrics visible in the dashboard include:
- Proportion of messages that are spam, malicious, or clean;
- Filtering results from Secure by Default;
- Detection corrections based on user and analyst submissions; and
- Miss rates and false positive rates based on tenant-specific traffic.
Customers can compare their internal metrics side by side with Microsoft’s aggregate data. The dashboard uses data from both Exchange Online Protection and Defender for Office 365, and includes information enriched by manual and automated submissions.
The new dashboard is available in public preview to customers licensed for Microsoft Defender for Office 365 Plan 2. Microsoft stated that future updates to the benchmarks will be published in the Microsoft 365 Defender documentation and the Microsoft Security Blog.
For more information, read the Microsoft blog post.