Letting an AI Read Email Without Letting It Read Everyone'seric

We were asked to give two colleagues an AI assistant that could work with their email — read the mailbox, search it, pull an attachment, draft a reply for them to review and send. Standard stuff. Microsoft Graph exposes all of it, and wiring an AI agent to Graph through a small server takes an afternoon.

The hard part wasn't the plumbing. It was one sentence in the requirements: each person's assistant must be able to touch only that person's mailbox. Not "should." Must. And "must" is a security claim — the kind you have to be able to prove, not merely intend.

This is the story of how we made that boundary real, and the three places it tried to fool us into thinking it was already there when it wasn't.

Application permissions are tenant-wide by default

To act on a mailbox without a signed-in user, a Graph app uses application permissions and the client-credentials flow: the app authenticates as itself and calls the API. The permission we needed was Mail.ReadWrite — it covers reading too, so there's no separate read permission to add. Crucially, we did not grant Mail.Send. The app can create drafts but physically cannot transmit them; every reply lands in a Drafts folder for a human to send.

Here's the trap. Mail.ReadWrite as an application permission is tenant-wide. Once an admin consents, the app can read and write every mailbox in the organisation — not the one you had in mind, all of them.

We confirmed this the blunt way, before trusting anything. We took the app's own credentials, asked Microsoft for a token, and called the API against a different colleague's mailbox. It returned their inbox, cheerfully, with a message count. The app we had registered "for one person" could read the whole company.

So the app permission is not the boundary. It's the blast radius. The boundary has to come from somewhere else.

Put the wall where a bug can't reach it

There are two ways to constrain which mailbox an app can touch.

The tempting one is to do it in your own code. The server knows which user's token is calling, so it just... refuses any request for a different mailbox. One shared app, a lookup table, a check before every call.

Don't. That wall lives entirely in your code, which means one missed check, one refactor, or one clever prompt-injection that talks your agent into passing a different address, and it's gone. When the requirement is "must," you don't want the enforcement sitting in the same place as the bugs.

The better option is a control the provider enforces: an Application Access Policy in Exchange Online. You create one small mail-enabled group per app, containing exactly the mailbox (or mailboxes) that app is allowed to touch, then bind a RestrictAccess policy from the app to that group. After that, Microsoft itself refuses any Graph call from that app to any mailbox outside the group — a 403, at the source, regardless of what your code does or doesn't check.

So we registered one app per person — not one shared app with code checks — and pinned each to a single mailbox with its own access policy. Now the wall between the two users is enforced by Microsoft. A bug in our server cannot leak one person's mail to the other, because the credential itself cannot see it.

The general principle: when you can push a boundary down to the layer that owns the data, do it. Your code should rely on the wall, not be the wall.

The setup, in PowerShell

Application Access Policies live in Exchange Online, so the whole thing is a short, one-time PowerShell run — one mail-enabled group and one RestrictAccess policy per app. (Placeholders below: contoso.com, two people alice and bob, and the Application/client ID of each person's app registration.)

Copy # Run once, interactively, signed in as an Exchange/Global admin on a managed device.
Connect-ExchangeOnline -UserPrincipalName admin@contoso.com

# One app registration per person; paste each app's Application (client) ID.
$aliceApp = '<alice-app-client-id>'
$bobApp   = '<bob-app-client-id>'

# One mail-enabled security group per app, holding only that person's mailbox(es).
New-DistributionGroup -Name 'sg-mail-alice' -Type Security `
  -PrimarySmtpAddress '[email protected]' -Members '[email protected]'
New-DistributionGroup -Name 'sg-mail-bob' -Type Security `
  -PrimarySmtpAddress '[email protected]' -Members '[email protected]'
# (A group can hold more than one mailbox — add every address that app may touch.)

# Pin each app to its group. After this, Graph refuses any mailbox outside it.
New-ApplicationAccessPolicy -AppId $aliceApp -PolicyScopeGroupId '[email protected]' `
  -AccessRight RestrictAccess -Description 'mail agent: alice -> own mailbox only'
New-ApplicationAccessPolicy -AppId $bobApp   -PolicyScopeGroupId '[email protected]' `
  -AccessRight RestrictAccess -Description 'mail agent: bob -> own mailbox only'

# Verify the policy DEFINITION: Granted for own mailbox, Denied for the other.
Test-ApplicationAccessPolicy -Identity alice@contoso.com -AppId $aliceApp   # Granted
Test-ApplicationAccessPolicy -Identity bob@contoso.com   -AppId $aliceApp   # Denied
Test-ApplicationAccessPolicy -Identity bob@contoso.com   -AppId $bobApp     # Granted
Test-ApplicationAccessPolicy -Identity alice@contoso.com -AppId $bobApp     # Denied

Those last four lines are Microsoft telling you the policy is correct. Which brings us to the first trap.

"Configured" is not "enforced"

We created the policies and ran Microsoft's own checker, Test-ApplicationAccessPolicy, for every combination — each app against its own mailbox, each app against the other's. All four came back exactly right: Granted where it should be, Denied where it shouldn't. Green across the board.

Then we ran the real test — the app's credentials against the live API, the way the agent would actually use it — and every mailbox still returned 200. Wide open. The policy was defined, verified by Microsoft's own tool, and completely un-enforced at runtime.

Test-ApplicationAccessPolicy tells you what the policy says. It does not tell you the enforcement layer has caught up. Those are different systems, and on the day we set this up they were about four hours apart.

The lesson generalises well beyond email: verify a security boundary the way it is actually exercised, not with the vendor's "is it configured?" checker. For an API boundary, that means real calls, with the real credentials, returning the real deny.

Propagation is slow and uneven

The four-hour wait had a sting in the tail.

We polled the live API in the background, watching for the cross-mailbox calls to start returning 403. Eventually they did — and we nearly declared victory on the spot, because we were only watching the deny cases.

Good thing we looked at the whole picture. Enforcement hadn't landed atomically. The blanket "deny everything" propagated first; the per-mailbox allow list lagged behind it. For a window, each app was denied its own mailbox — the two users would have been locked out of their own email, while strangers' mail was (correctly) refused. The deny rule was live; the "except your own" hadn't caught up.

If we had gated go-live on "cross-mailbox returns 403," we'd have shipped a broken permission set. The right gate was the entire matrix correct at once — every allow returning 200 and every deny returning 403 — not a convenient subset. When a distributed system rolls out a rule, don't assume the parts you didn't check moved in lockstep with the parts you did.

A detour: you can't always automate the admin step

One more trap, because it cost real time. Creating those access policies needs Exchange Online admin PowerShell, and we wanted to run it unattended from a server. We couldn't, for reasons worth knowing up front:

• The old, universally pre-consented "Exchange Online Remote PowerShell" app that used to make headless sign-in easy has been disabled by Microsoft.
• The device-code sign-in flow — the usual fallback for a machine with no browser — cannot show a consent screen, so it dead-ends on any app that isn't already consented.
• And even past that, the tenant's Conditional Access policy refused to issue a token to an unmanaged, unregistered device. (The error, if you ever meet it, is AADSTS530035.)

The lesson here isn't a workaround; it's an expectation. Admin actions in a well-secured Microsoft 365 tenant are designed to resist unattended automation. Plan to run them interactively, once, from a managed device — and build everything else around that single human step instead of fighting to remove it.

What to check if you're doing this

1. Prove the blast radius before you trust anything. Take the app's own credentials and call a mailbox it shouldn't reach. If it answers, your boundary isn't there yet.
2. Enforce isolation at the provider, not in your code. One app per scope, each pinned by an Application Access Policy. Make a bug in your code harmless by construction.
3. Never grant more than the job needs. Drafting replies needs Mail.ReadWrite, not Mail.Send. If the app can't send, no bug or injection can make it send.
4. Verify with real calls, not the "is it configured" checker. Configured and enforced are different systems, sometimes hours apart.
5. Gate go-live on the whole matrix. Every allow and every deny correct, at the same moment — not the deny cases alone.

The reassuring part: once it settles, it is genuinely solid. The isolation isn't a promise in our code or a line in a runbook — it's Microsoft returning 403 to a credential that was never allowed to see the data in the first place. That's the kind of wall you can point at and call "must."