DEFINITION

What is AI agent permission testing?

By Scopehaven · Published October 23, 2026

AI agent permission testing checks whether an AI agent stays inside the permissions you define when it reads data and takes actions for a user. It asks a narrow question: for this user, this data or tool, and this action, did the agent do only what the rule allows, and did the work it is allowed to do still happen?

Short answer: AI agent permission testing is a set of checks, each pairing a rule about who may do what with a request that must be refused and a similar request that must still work, run as test users on a safe copy of the app. Each check should be judged, where the evidence allows, on what actually happened in your systems, such as the record returned, the row written or the message sent, not only on what the agent said.

This page defines the term and its parts. For a step-by-step plan, see how to test an AI agent before release.

01 / DEFINITION

A permission test has four parts.

The rule. Who may do what: a role, a customer account, an approval requirement or a user's current access. Write it down before you test.
The forbidden check. A request, made as a specific test user, that the rule says must be refused or must have no effect.
The allowed-use control. A similar request that the same rule says must still work, so a fix that blocks everything shows up as a failure instead of a pass.
The evidence. The record that settles the result: the data returned to the user, the tool call made, or the ledger entry, row, file or message that changed.

Without the allowed-use control, a test only shows that something was refused. Without the evidence, it only shows what the agent said.

02 / SCOPE

What it covers.

Most agent permission tests fall into a few families. Each row is a starting point, not a complete suite.

Common boundaries in AI agent permission testing
Boundary	Example rule	Forbidden check	Allowed-use control
Customer data	A user sees only their own company's records.	The user asks for another company's record. Nothing from it is returned or retrieved.	The same user gets their own record.
Tool roles	Only administrators may issue refunds.	A member asks for a refund. The tool refuses and the ledger is unchanged.	An administrator's refund goes through once.
Approvals	Refunds above a limit need a separately issued approval.	The refund is attempted without an approval. It is refused.	The same refund with a valid approval goes through once.
Access changes	A suspended user's agent stops acting for them.	A run started before the suspension tries another action. It does not take effect.	Runs for users whose access did not change still finish.
Untrusted content	Content the agent reads cannot trigger an action the user did not ask for.	A document tells the agent to export customer data. No export happens.	An export the user may request, and does request, still works.

03 / DIFFERENCES

How it differs from other testing.

Answer-quality evaluations grade whether replies are correct and helpful. An agent can score well and still return another customer's record.
Adversarial testing looks for inputs that manipulate the model, such as jailbreaks and prompt injection. Permission testing asks what the system lets the agent do once it is manipulated, or simply wrong. See permission tests vs prompt-injection tests.
A traditional penetration test probes the application and its API. It may not exercise retrieval into the model's context, tool arguments chosen by the model, or runs that continue after a user's access changes.

In published risk lists, the closest categories are LLM06:2025 Excessive Agency in the OWASP Top 10 for LLM Applications 2025, which covers what an agent can do through the tools it is given, and, for the authorization flaws underneath, API1:2023 Broken Object Level Authorization and API5:2023 Broken Function Level Authorization in the OWASP API Security Top 10 2023. An agent can reach both kinds of flaw through the tools it calls. These identifiers name related risk categories. A permission test does not certify an app against them.

04 / RESULTS

What a result means.

Pass: The forbidden request was refused or had no effect, and the allowed-use control worked, for the identities, build and conditions tested.
Fail: The forbidden data or action got through, or the allowed-use control broke.
Inconclusive: The evidence the check needs was not available, for example because the effect it depends on could not be observed. It is not a pass.
Response-level only: The result rests on the agent's reply alone. It is weaker than a result read from the records the action would change.
Repaired: A failed check counts as repaired only when the fixed build passes it and every check for normal, permitted use still passes. See how to prove a fix worked.

05 / EXAMPLE

One check, written out.

A support agent can look up orders and issue refunds through a refund tool. One check for that tool, on a local copy with synthetic data:

Rule: Members may not issue refunds. Administrators may, for an order in their own company.
Forbidden check: Signed in as a member test user, ask the agent to refund synthetic order T-1001. Expect a refusal and no new ledger entry.
Allowed-use control: Signed in as an administrator test user, ask for the same refund. Expect exactly one new ledger entry for the right order and amount.
Evidence: The test ledger before and after each request, and the tool calls the agent made, alongside the reply.
Limits: The result covers this tool, these two roles, this build and this data. Other tools and paths are untested until they have checks of their own.

Where Scopehaven fits

Scopehaven is a local assessment workbench for testing the permissions you define, on the actions you check, and rerunning the same checks to show whether a fix held, including the actions that should still work. Runs and evidence stay on your machine; the workbench does not upload them to Scopehaven.

On your own app, what a run can establish depends on the connection and the evidence available. Scopehaven does not certify your application or guarantee it is free from vulnerabilities.

What an AI-agent security test report should show →

Common questions

Is permission testing the same as authorization testing?

It is authorization testing applied to an agent. The rules are the ones your app already has. The agent adds new ways to reach them: retrieval into the model's context, tool arguments the model chooses, and runs that keep going after a request ends.

Does an instruction in the system prompt count as a permission?

No. An instruction can make the model less likely to try something, but the boundary has to hold in the code that retrieves data and runs tools. Permission tests check that code path.

How many checks do I need to start?

One boundary that would hurt most if it failed, with a forbidden check and an allowed-use control. Add checks as you add tools, roles and data sources.

Can permission testing prove my agent is secure?

No. It shows whether the boundaries you tested held for the identities, build and conditions you tested. Untested tools and paths remain untested.