AegisAI Perspectives · 135
Measure catch rate as threats caught divided by every threat that arrived, misses included, and log the report or search that holds each count monthly.
See it on your own Google Workspace or Microsoft 365 tenant. No MX change.
Or get the Q3 2026 State of the AI Attack report (free, instant download)
Not ready for a demo? Check your domain (free, about 10 seconds).
Email security efficacy is measured with catch rate: the threats caught before or after delivery, divided by every threat that arrived in the same 30 days, misses included3. Miss rate and false positive rate are the two other fractions you need.
AegisAI investigates every email for what it is trying to do, and removes the ones that fail from every mailbox they reached19. If you run IT for a company of 20 to 500 people on Microsoft 365 or Google Workspace, with a small IT or security team and no SOC, this page gives you the formulas and the logs to run them on your own mail. The product side is on the inbound email security page. The program scorecard, with time to detect and time to respond, is in email security metrics: MTTD and MTTR. A vendor test on a labelled attack set is in how to evaluate email security detection accuracy. This page is the monthly count on live mail.
Key takeaways
- Catch rate is threats stopped before delivery plus threats removed after delivery, divided by those two plus the threats missed.
- Microsoft only counts a missed threat as Uncaught after you submit it, so the miss count is a floor2.
- Admins can submit a missed message only while it is 30 days old or less and still in the mailbox8.
- NIST defines a false positive as benign content a security tool classifies as malicious17.
- The Google Workspace Spam filter report shows only the last 30 days, so export it every month13.
| Question | Answer | Source |
|---|---|---|
| How to measure email security efficacy / catch rate | (Stopped before delivery + removed after delivery) divided by (stopped + removed + missed), times 100, over one 30 day window | 1 |
| What is the miss rate? | Threats missed divided by the same total. It equals 100 minus the catch rate | 18 |
| What is the false positive rate? | AegisAI's working formula: legitimate messages flagged divided by all legitimate messages in the window | 7 |
| Where are the counts in Microsoft 365? | The Phish / Malware Efficacy card, the User reported messages report, the Mailflow status report | |
| Where are the counts in Google Workspace? | The Spam filter report, the User report, Email Log Search | 1415 |
Write these three fractions at the top of your monthly sheet before you pull a single count.
Short on time? Watch the explainer, or book a demo.
Catch rate = (threats stopped before delivery + threats removed after delivery) divided by (stopped + removed + missed), times 100. Microsoft's Plan 2 hunting query uses this form over a 30 day window.
Each rate needs a population and a clock. The population is the mail the rate counts. The clock is the window, and whether a removal after delivery still counts as a catch.
The denominator of catch rate is the hard part. It holds threats the filter never saw as threats. Microsoft leaves undetected false negatives out of the efficacy figures until you submit them. A miss that nobody reports never reaches your denominator, so the catch rate reads higher than it is.
Count misses from admin submissions and user reports, then add the incidents your own tickets record.
No. Detection rate is measured on a labelled test set you build in advance. Catch rate is measured on live mail, where nobody labels the threats for you. On live mail the missed count comes only from what people find and submit.
A detection test is a one-off decision tool, and how to evaluate email security detection accuracy sets one up. Use catch rate for the month-on-month trend of the tools you already run.
Read them side by side, never alone: email security efficacy is both numbers read together. A filter can raise its catch rate by flagging more legitimate mail, and the catch rate alone will not show it. Microsoft also removes the false positives it detected from its efficacy calculation, so the efficacy card does not carry that cost.
Report catch rate and false positive rate on the same line every month, with both denominators shown.
No source opened for this page publishes a neutral benchmark for live mail. AegisAI reports that 3 in 10 confirmed phishing emails cleared native Gmail and Microsoft filters in Q3 202624. That is a miss share on phishing, not a target catch rate.
Use your own first full month as the baseline, then compare each month against it.
Pick the window: one calendar month, the same every time. Count a threat removed after delivery inside the window as caught, as Microsoft's Post-delivery category does for zero-hour auto purge.
Export the threats stopped and removed. In Microsoft 365, read Pre-delivery and Post-delivery from the Phish / Malware Efficacy card. In Google Workspace, filter the Spam filter report by Time classified, then by the Phishing or Malware classification.
Submit every confirmed miss as an admin submission while it is 30 days old or less. It then counts as Uncaught.
Pull user reports. In Microsoft 365, the User reported messages report lists mail users reported with the built-in Report button. In Google Workspace, the User report shows Phishing labels among the user labels. Label each one threat or clean.
Count legitimate mail flagged. Use quarantine releases, which Microsoft calls likely false positives11, plus Not spam labels in Google. Confirm each one by hand.
Count all legitimate mail for the denominator: good email in the Mailflow status report, or Clean in the Spam filter report.
Save the exports on the last day of the month. Google keeps Email log search for 30 days and Gmail log events for 6 months16.
Run steps 1 to 3 in the first week, before the submission window closes on the earliest misses.
| Count | Microsoft 365 | Google Workspace | Window | Source |
|---|---|---|---|---|
| Stopped before delivery | Efficacy card, Pre-delivery | Spam filter report, Pre-delivery | 30 days | |
| Removed after delivery | Efficacy card, Post-delivery (zero-hour auto purge) | Spam filter report, Post-delivery | 30 days | |
| Missed | Uncaught, only after an admin submission | User report, Phishing label | 30 days to submit; 31 days shown | |
| Delivery result per message | EmailEvents: DeliveryAction, OriginalThreatTypes, LatestDeliveryLocation | Email Log Search, up to 1,000 messages | Past 30 days in Google | 5 |
| Legitimate mail flagged | Quarantine Statistics, share released | User report, Not spam label | Previous 30 days | |
| All legitimate mail | Mailflow status report, good email | Spam filter report, Clean | 90 days in the Mailflow view | |
| One message, start to end | Message trace, up to 90 days | Email Log Search | 2 day default in message trace | 10 |
The Google security dashboard needs the Security center administrator privilege, and its predefined reports lag by about 4 hours12. Grant that role to whoever runs the count before month end.
Illustrative example, figures are fictional: Larch Valley Dental Group (fictional) has 140 people on Microsoft 365 and one IT admin. In October the Efficacy card shows 1,860 threats stopped before delivery and 42 removed after delivery. She submits 18 confirmed misses from user reports and tickets. Catch rate is 1,902 divided by 1,920, or 99.1 percent, and miss rate is 18 divided by 1,920, or 0.9 percent. Six more phish turn up after their 30 day submission window, so she adds them to her own sheet: with them, catch rate is 1,902 divided by 1,926, or 98.8 percent. For false positives she confirms 52 quarantine releases and 12 Not junk reports, 64 in all, against 96,400 good messages delivered. False positive rate is 64 divided by 96,464, or about 0.07 percent, 1 in every 1,507 legitimate messages.
Run this same arithmetic on your own October exports this week.
Track three numbers, each from an export you keep.
To compare a second layer on the same month of mail, set up that comparison.
Divide threats stopped before delivery plus threats removed after delivery by those two plus the threats missed, over one 30 day window. Count misses from submissions and user reports. For a second count on the same mail, AegisAI runs in monitoring mode first.
Only once you submit them. Undetected false negatives are not in the figures until a submission adds them to Uncaught. Treat the Uncaught count as a floor.
Within 30 days: admins can submit messages as old as 30 days if they are still in the mailbox. Submit in the first week of each month.
The Spam filter report shows the last 30 days. Email log search keeps 30 days and Gmail log events keep 6 months. Export monthly.
No. Microsoft calls the released share likely false positives. Confirm each release by hand before it enters your false positive count.
