Scopehaven
Request a first test
What is AI agent permission testing?
The four parts of a permission test, what it covers, how it differs from evals and red teaming, and what a result means.
Read the definitionGUIDES / AI AGENT SECURITY
Guides to testing what your AI agent can see and do
Practical guides for engineering teams shipping AI agents that read customer data or take real actions. Each one covers one question: the checks to run, the normal use that must keep working, and what the evidence can and cannot show.
01 / BEFORE YOU SHIP
Plan the tests and the release gate.
How do I test an AI agent before release?
Three kinds of tests, what a pass on each one shows, and a pre-release plan for agents that take real actions.
Read the pre-release guideIs my AI agent safe?
Learn how to scope permission checks, preserve allowed use, and identify what the evidence can establish before launch.
Read the permission-testing guideCan I test without uploading our data?
Run a local copy with synthetic data, catch email, webhooks and payments locally, and decide how to handle the model call.
Test permissions locallyWhat should I retest on every release?
Model, prompt, tool and permission changes: rerun the same checks, compare like with like, and block on regressions.
Use the release checklist02 / PERMISSION BOUNDARIES
Test one boundary at a time.
Can one customer see another's data?
Set up two test tenants and check retrieval, tool calls, caches and concurrent requests, not only the assistant's reply.
Test tenant isolationCan your agent act without approval?
Test refunds, exports and account changes: role checks inside each tool, approvals bound to one exact action, and the effects that actually happened.
Test tool permissions and approvalsWhat happens after you remove a user's access?
Test in-flight runs, paused actions, caches and tokens, not only the user's next request.
Test access removalCan a document make your agent act?
Plant instructions in synthetic content, then check the tools and outbound channels, not the reply.
Test prompt injection against your toolsPermission tests or prompt-injection tests?
What each one checks, where they overlap when injected text reaches a tool, and which kinds of tools cover each.
Compare the two03 / EVIDENCE
Check what actually happened.
Did your agent do what it said?
Compare each claim with the record: false successes, silent actions, wrong targets and duplicates.
Verify agent actionsDid the fix work?
Rerun the same checks, keep the requests that must still work, and know when a failure counts as repaired.
Prove a fix workedWhat should a test report show?
Use a synthetic demo to understand permission rules, observations, evidence limits, and repair comparisons.
Learn how to read the evidence