AegisAI Perspectives · 097
Pick an AI red teaming tool that tests your people with approved, per-person lures and logs every run; tools that probe AI models test something else.
See it on your own Google Workspace or Microsoft 365 tenant. No MX change.
Or get the Q3 2026 State of the AI Attack report (free, instant download)
Look for a tool that tests your people with per-person AI-written lures10 that your team approves11, and returns a full log of each run as evidence13. Tools that probe AI models test software instead of employees1. AegisAI's red team agent researches each person and writes a lure just for them. In 2025, IC3 received more than 22,000 complaints reporting AI-related information7.
Choose the type that attacks employees, because the other type attacks AI models. NIST SP 800-53 defines a red team exercise as a simulated adversarial attempt, reflecting real-world conditions, to compromise an organization's missions or business processes22.
Model-testing tools aim that exercise at software: one cloud AI platform documents its red teaming agent for finding safety risks in generative AI systems during design and development. That agent supports only text-based scenarios2 and scores each attack and response pair3.
People-testing tools aim it at staff, with an AI agent that researches each person and writes a lure for them, the way attackers now work. The NCSC describes spear phishing the same way: the attacker uses information about your employees or company16.
Write "test employees with AI-written lures" as the first line of your requirements, so model-probing tools drop off the list early.
Short on time? Watch the explainer, or book a demo.
IC3's 2025 report counts more than 22,000 complaints reporting AI-related information, with adjusted losses above $893 million8. Businesses reported losses over $30 million to business email compromise scams involving AI in the same year9.
Put the IC3 figures in the business case for a people-testing tool, labelled as national complaint data.
Set the scope in writing: only the domain, teams, roles or named people you approve14.
Have your security team scope and approve every run before it starts.
Check the plan with your HR department, since the NCSC warns that phishing simulations can create legal risk18.
Let the agent research each person in scope and write one lure per person.
Confirm that passwords and secrets are never collected12.
Send the lures only to the domain, teams, roles or named people in the approved scope.
Separate human clicks from scanner clicks before you count a single result15.
Export the evidence: each lure with its sources, plus every event in the run.
File the signed scope and the evidence export in one record for each run.
A spear phishing simulation writes a different lure for each person and logs what happened afterwards. A tool that sends one template to a whole department skips the step the NCSC names, the attacker's use of information about your employees.
Check three capabilities in a demo run. The agent researches each person. The run stays inside the approved domain, teams, roles or named people. The results arrive as a log of every lure, source and event. For the approval and legal steps when the targets are executives, follow how to run an AI spear phishing test on executives safely.
Ask each vendor for one sample lure written for a named role in your company, with the sources the agent used.
Count them separately, so the click rate describes people. Ask for the logged event behind each counted click, since every event in a run comes back as evidence.
NIST warns that click rates analyzed on their own can create a false sense of security21. Its Phish Scale rates five elements of a lure on a 5-point scale and ranks the exercise by difficulty20. The NCSC adds a second count: how many people reported the message, next to how many clicked19.
Ask every tool on your shortlist to show one run where scanner clicks were removed from the total.
Score each candidate on the six criteria below; the last row is where model-probing tools fit and a people-testing tool has no edge.
| Criterion | Question to ask | People-testing tool | Model-probing tool | AegisAI |
|---|---|---|---|---|
| Consent and scope | Who approves each run, and can targets be limited? | Required | Not applicable | Security team approves every run |
| Per-person lures | Does an agent research each target first? | Required | Not applicable | Researches each person |
| What is stored | Are passwords or secrets ever kept? | Must be none | Not applicable | Never collected |
| Evidence returned | Is every lure returned with its sources and events? | Required | Attack Success Rate per attack and response pair | Logged and handed back |
| Human and scanner clicks | Are scanner clicks removed from results? | Required | Not applicable | Separated in simulation training |
| Testing your own AI apps | Can it probe an LLM app for safety risks? | Not its job | Built for it during design and development | No edge: its agent researches and writes lures for people |
Drop any people-testing tool that cannot show a scope approval record and a scanner-free click count from a past run.
Start with one team under one approver, with a written scope. A run on one team shows whether the evidence export and the click counts hold up before the tool reaches the whole company.
Illustrative example, figures are fictional: A 400-person firm tests its 40-person finance team. The security lead approves a scope of the 40 finance staff only, with no executives. The agent writes 40 lures, one per person. The raw log shows 15 clicks. After scanner clicks are removed, 6 are human, and 3 of those 6 people also reported the message. The firm records a human click rate of 15 percent, where the raw log would have shown 37.5 percent, and files the evidence export with the approval.
Write the pass mark before the pilot: a signed scope, one lure per person, and a human click count you can trace to each event.
An AI agent researches each person and writes a lure just for them, the way attackers now work10. Passwords and secrets are never collected. Your security team scopes and approves every run, and each run reaches only the domain, teams, roles or named people you approve. Every lure, every source and every event is logged and handed back to you as evidence. AegisAI replays phishing it caught in your tenant to the people it targeted, and separates human clicks from scanner clicks.
Track three numbers from the first runs:
Review the three numbers at day 30 and widen the scope only when every run has its approval on file.
Look for a tool that tests people with per-person AI-written lures. Your security team should approve each run's scope first. It should never collect passwords, and it should return every lure with its sources and events as evidence.
They research each target and write a lure built for that person, then log what happened. A good one also separates human clicks from scanner clicks, so the result reflects people.
No. Model-probing agents look for safety risks in generative AI systems during design and development, and one documented agent supports only text-based scenarios. A phishing test aims NIST's simulated adversarial attempt at employees.
No. The NCSC says no training package, including phishing simulations, can teach users to spot every phishing attempt. Count reports as well as clicks, and check each plan with HR first.
Read it next to the lure's difficulty and the report count. NIST warns that click rates analyzed on their own can create a false sense of security, and its Phish Scale ranks each exercise by difficulty.
IC3 received more than 22,000 complaints reporting AI-related information in 2025, with adjusted losses above $893 million. Business email compromise scams involving AI cost businesses over $30 million.
