WORKS WITH

Works with your existing tools, and when you don't need us.

By Scopehaven · Draft · Vendor docs read October 29, 2026

Most teams shipping an agent already run an evaluation or red-team tool, and they should keep it. Scopehaven is meant to sit beside those tools before a release. Below is what each one's own documentation says it checks, where we fit, and when you can skip us.

WHAT EACH ONE DOCUMENTS

In their own docs, as of October 29, 2026.

This describes documentation, not a test of these tools. Each is good at what it is built for, and docs change: follow the links for the current page.

What each tool's documentation says it evaluates (read 2026-10-29)
Tool	What its docs say it evaluates	Source
Google Gen AI evaluation for agents (Gemini Enterprise Agent Platform, preview)	The agent's final response and its trajectory, the sequence of tool calls, against a reference, with computed metrics and optional model-based ones. A separate environment simulation feature can intercept chosen tool calls and return mocked data or simulated errors.	Evaluate agents; Agent evaluation; ADK environment simulation
Microsoft Foundry agent evaluators	Agent messages: the query and response, or the message list, with tool definitions for the tool evaluators. Most use a model as judge and return pass or fail or a score.	Agent evaluators
AWS Bedrock AgentCore Evaluations	Agent traces, at the session, trace or tool-call level. Built-in evaluators use a model as judge; a custom evaluator is a Lambda function you write, which receives the session's spans and may call external APIs.	Evaluations; Code-based evaluators
Promptfoo	Model and agent output, with deterministic and model-graded assertions, tool-use trajectory assertions and trace assertions. Custom JavaScript, Python or Ruby assertions are ordinary code, so a team can write its own check of app state.	Assertions and metrics; JavaScript assertions
OpenAI Evals	Being retired: existing evals become read-only on October 31, 2026, and the dashboard and API are scheduled to shut down on November 30, 2026. OpenAI's notice points to a Promptfoo migration guide.	Deprecations
Runtime guards and agent identity tools	Whether a request should be allowed before the agent's call runs. They control the request; a test of what the request changed is a different job.	Category, no single source
Your own integration tests	Whatever your team writes: often the database state after the paths you designed.	Your repository

WHERE SCOPEHAVEN FITS

Beside them, on the records your agent changes.

Before a release, Scopehaven runs your agent on a local copy of your app and decides each check from the app's own records. What it adds beside an evaluation tool:

Records your API doesn't show. Database rows, ledger tables and other tenants' records, read directly on the local copy, not through the app's own endpoints.
Attribution. Each write is tied to the acting user and tenant, so a write that landed on the wrong customer shows.
Signed evidence. A report can ship with a signed bundle and a small script that re-checks the verdicts. See the evaluation protocol.
Inconclusive is never a pass. Where no observer can see an effect, the check stays inconclusive.
A retest after each fix, with allowed-use controls, so a fix that blocks legitimate work shows up.

WHAT OUR COMPARISONS SHOWED

On records an API exposes, a capable team can match us.

In our rounds so far, graders given the same records caught what our checks caught, and on defects visible in the transcript they caught more. Scopehaven's difference is that it gathers those records from your app, signs them, and retests the fix.

In the most recent round, sealed and timestamped before it ran, on 24 defects we planted in two public agent templates, an open-source evaluation tool using its built-in features and Scopehaven each flagged the same 23. Both missed the same one: a write to records that no test account could see through the app's API.

So, honestly: a capable engineer can match this on anything your app exposes over its API. What you would own is the per-action checks, kept current every release, the attribution, the ledger and other-tenant reads your API doesn't show, and the signed evidence.

WHEN YOU DON'T NEED US

Skip Scopehaven if any of these is true.

Your agent doesn't move money or change accounts. Answer quality and safety evals cover a read-only assistant well.
Every record your agent touches is visible through your own API, and your team has the time to write and keep a state check for each action, every release.
You need production monitoring. Scopehaven runs before a release, on a local copy; it does not watch live traffic.
You need a penetration test, red-teaming breadth across prompts, or a certification. Those are other jobs, and other tools do them.

HONEST LIMITS

What is true today.

Scopehaven is a private alpha, tested on our own apps and open-source templates with synthetic data, not yet on a customer's app. It isn't a certification or a general penetration test. Test accounts and made-up data only: never production, never patient data.
We have no customers yet, and nobody outside the project has installed it.
Our comparisons above ran on public templates and apps we built, with defects we planted. They are descriptive, not a ranking of tools.

KEEP YOUR TOOLS

Bring one action your agent takes on money or accounts.

We'll tell you on the call whether your existing tools already cover it, and send a written page either way.

Book 20 minutes