AegisAI Perspectives · 046
The team deploying Copilot or Gemini in the inbox owns the tool scopes per assistant type, the approval line, content isolation, logs and red team tests.
To protect an AI email assistant from prompt injection, assume some injected emails will get through8 and limit what the assistant can reach and do, as OWASP LLM01:2025 sets out in 7 prevention strategies15. AegisAI inspects incoming mail for concealed payloads written to manipulate an inbox assistant, and acts on a message that carries one at delivery. Its post on hidden instructions in Gemini and Copilot explains that attack. Least privilege and human approval, owned by the team that deploys the assistant, limit what an injection can do; content isolation lowers the odds it works; logging and red teaming show when it did.
Build the controls around the assistant, because the model cannot keep instructions and email text apart. The NCSC states that current LLMs do not enforce a security boundary between instructions and data inside a prompt19. Microsoft's guidance is to design on the expectation that some attacks will succeed, since no single solution is sufficient8.
Microsoft Defender for Office 365 Plan 2 detects prompt injection content in inbound email before it reaches a user or an AI assistant1. Detections are filed under the High confidence phishing verdict with the detection technology value Prompt injection protection. Verify: search for that value in your email reports.
Microsoft says runtime safeguards in Copilot or another assistant remain important for attempts that depend on its instructions, permissions, grounded data or tools. In Google Workspace, Gemini applies six layered defenses12.
Give the agent only the minimal number of short-lived privileges its task needs. Scope each tool as read-only or write and to specific resources, and fail closed for any tool the policy does not know17.
The NCSC rule is that an LLM processing information from a party has that party's privileges, so an assistant reading external email should not have privileged tools25.
Wrap each external message body in clear delimiters or metadata tags that label it as provided content11. Apply the same wrapper to quoted replies and attachment text. Spotlighting adds data marking and metaprompting to isolate external content7. Both lower the odds the model obeys the email; neither creates the missing boundary.
Microsoft calls verifying risky actions with the user the last line of defense6. Google's user confirmation framework asks for explicit confirmation before operations such as deleting calendar events. Bind each approval to the exact action18, and run the check in code outside the model.
Log prompts and model outputs, and send them to Microsoft Sentinel or another log analytics tool to find repeated injection attempts10. MITRE ATLAS lists AML.M0024 AI Telemetry Logging as a mitigation for prompt injection.
NIST AI 600-1 action MS-2.7-007 calls for AI red-teaming to assess resilience against attacks such as prompt injection21. Keep every attack that worked as a regression test.
Each assistant type gets the narrowest mailbox scope and tool set its job needs, and every step up in power adds an approval. Use the table as the starting scope list for a deployment in Microsoft 365 or Google Workspace.
| Assistant type | Mailbox scope | Tools it gets | Waits for a person |
|---|---|---|---|
| Summarize and triage | Read the user's own mailbox | None that write, send or share | Nothing, since it cannot act |
| Draft replies | Read the mailbox, write to Drafts | Create draft | The user sends every draft |
| Sends mail | Read, write drafts, send as the user | Send, forward | Any send to an external address, any forward |
| Changes rules or files | Read, write, mailbox rules, file links | Rule edit, delete, share | Every rule change, delete or share |
| Acts on the calendar | Read and write the user's calendar | Create, move, delete events | Every delete, any invite to an external address |
A summarizer gets no send tool because the email it reads comes from an outside party. An assistant that sends on a person's behalf is a tool the email's author can try to steer, so each send to an external address carries an approval.
Illustrative example, figures are fictional. A 400 person firm turns on an assistant that summarizes mail and drafts replies. On one Tuesday, 3 messages arrive with hidden text telling the assistant to forward the last 20 invoices to an outside address. Inbound screening quarantines 2 of them. The third reaches a mailbox, and the assistant proposes a forward. The scope list gives a drafting assistant no forward tool, so the call fails closed and is logged with the source message id. The alert on refused tool calls fires within the hour. The security lead pulls the message and adds its text to the regression set.
No. The NCSC says the best that can be hoped for is reducing the likelihood or impact of attacks20, because the model draws no boundary between instructions and data.
Screening detects injected emails before the assistant reads them. Scopes and approvals cap what one successful injection can do. Logs and tests show when a layer slipped.
Write the failure case down for each assistant. Name the worst action it can take with its current scopes. If that action is not acceptable without a person, remove the tool or add the approval.
The minimum its task needs. Grant it per tool and only for a short time.
Give it read access to the user's mailbox and nothing else. Mark each tool read-only or write against named resources, and refuse any tool the policy does not list. Review the permissions the assistant's app holds in Microsoft 365 or Google Workspace, and remove any write or send scope the job does not use.
An assistant that reads mail from external senders runs at those senders' privilege level, so privileged tools stay out of its reach. Where a workflow needs both, split it in two. One assistant reads and summarizes external mail. A second acts only on instructions a signed-in user types.
Any action that moves data outside the organisation or cannot be undone waits for a person. Microsoft names verifying risky actions with the user as the last line of defense. OWASP LLM01 lists human approval for high risk actions among its 7 strategies.
In an inbox, that means any send to an external address, forward, delete, shared file or link, mailbox rule change or calendar delete. Gemini already asks for explicit confirmation before operations such as deleting calendar events.
Bind the approval to the exact action, so approving a reply to one colleague does not also approve an attachment the model added. Enforce it in code outside the model, because the model itself enforces no boundary between instructions and data. Show the person the recipients and arguments the tool will receive as raw values.
Run red-team tests against every channel the assistant reads, and keep the attacks as a regression set. NIST AI 600-1 action MS-2.7-007 calls for AI red-teaming against attacks such as prompt injection. OWASP's seventh strategy is adversarial testing and attack simulation.
Cover the message body, hidden HTML text, quoted replies in long threads and attachment text. NIST describes indirect injection as prompts placed in data likely to be retrieved22, and every one of those parts is retrieved.
Write each test as an email plus the action it tries to trigger. Record whether screening, the scope list or the approval step stopped it. A test passes only when the attempted action never runs. Re-run the set after every model, prompt or tool change.
They write instructions into an email and wait for the assistant to read it. NIST defines indirect prompt injection as adversaries exploiting LLM-integrated applications by injecting prompts into data likely to be retrieved. An inbox is that data, and the same inbox also receives AI-written phishing lures.
OWASP's Scenario #5 records one case in an LLM-powered email assistant, CVE-2024-518416.
The AegisAI post linked in the opening describes how AegisAI inspects incoming mail for concealed payloads aimed at inbox assistants.
Track three numbers from logs the assistant and the mail platform already keep.
Yes. Defender for Office 365 Plan 2 detects prompt injection content in inbound email before it reaches a user or an AI assistant, and files it under High confidence phishing. Microsoft says runtime safeguards in the assistant remain important next to it.
It is the first layer. AegisAI keeps hidden regions of the email body in scope, so an instruction written for Gemini or Copilot is judged before either assistant sees it. Scopes and approvals then limit what any message that still gets through can do.
No. Google lists six layered defenses for Gemini, from content classifiers to a user confirmation framework. The controls a Workspace team owns are the assistant's scopes, approvals, logs and tests.
Assume some injected emails will reach the assistant, then cap the damage with least privilege and approval before risky actions, and keep logs plus red team tests.
No. Delimiters and metadata tags make the model less likely to treat email text as instructions. They do not create a security boundary, which current models lack. Use them together with scopes and approvals.
For every action, log the source message id and the tool called. Add its arguments, the outcome, plus the approver. OWASP says to log every agent decision and tool call along with its outcome. Microsoft points to Microsoft Sentinel or another log analytics tool to find repeated injection attempts.
