SAMPLE REPORT / SYNTHETIC

Sample report: a synthetic test app, before and after a fix.

By Scopehaven · Draft · Runs of October 29, 2026

This is the shape of the report a team gets after a Scopehaven assessment, filled in from runs on our owned support-agent test app. The three pages below are the report; the assessor-pack sample after them is the version written for an auditor or a security reviewer.

PAGE 1 OF 3 / SCOPE AND RESULTS

Support agent on a test app: refunds, records and approvals.

Scope
App	Our deliberately vulnerable test support app, with two companies (Aster and Cobalt) as tenants, and its repaired build.
Checks	Suite 2.3.0: 44 checks, of which 26 are boundary checks (something must not happen) and 18 are allowed-use controls (something permitted must still work).
Where it ran	A local copy on our own machine, over loopback, with synthetic users and data.
Agent replies	The test app's fixed replies. No AI model was called, so this says nothing about how a model would behave.
Runs	One run per build on October 29, 2026, 18:30 UTC, each on fresh data.

Results at a glance
Build	Boundary checks	Allowed-use controls	Inconclusive
Vulnerable, as built	26 of 26 failed	18 of 18 passed	0
Repaired	26 of 26 passed	18 of 18 passed	0

The allowed-use controls are there so a fix that blocks legitimate users shows up. They are not a security score.

Inconclusive is never a pass. This sample has no inconclusive rows; a report on your app lists each one as a gap.

PAGE 2 OF 3 / FINDINGS AND RETEST

Two findings, in full, and what the retest showed.

Finding: a member of one company received another company's record.

On our deliberately vulnerable test app, the policy says a member of one company, Aster, must never receive another company's record, Cobalt. Scopehaven doesn't grade the chatbot's reply; it checks what the server did, and the test app's backend record confirms the Cobalt record reached the Aster user.

Smallest fix: enforce tenant access before retrieval. A refusal after retrieval cannot undo data already supplied to the model.

Finding: the reply said a blocked refund was processed.

On our deliberately vulnerable test support app, a synthetic member asked for a refund. The app correctly blocked it, recording a denial and no refund, because only admins may issue refunds. But the test app's reply said "Your refund of 20 CAD has been processed." A check that grades only the reply would call that a success; Scopehaven flags it as a false success.

The other 24 boundary checks that failed on the vulnerable build include cached access after a role change, self-approval and approvals reused for a changed recipient, amount or currency, forwarding to an outside destination, an access change in the middle of a task, refunds and discounts promised outside policy, internal notes that record a blocked action as done, and actions landing on the wrong object.

Retest: vulnerable build compared with the repaired build
Fixed	Still failing	New failures	Allowed use preserved
26	0	0	18 of 18

The two runs are comparable: the same suite, policy and test data, by hash. The repaired build is a demonstration build we wrote, not an automatic repair of a customer's code.

PAGE 3 OF 3 / EVIDENCE AND LIMITS

How to check this report, and where it stops.

Both run files were signed together in one envelope with a key made for this sample (key id 0e245be0…8a43; its private half was never written to disk). Signed files, by SHA-256: vulnerable build e2218657…cbfe, repaired build e3d8a393…3d33. The redacted sample bundle lists the full digests.

Our re-checker, run on the signed files, re-derived all 88 verdicts from the kept observations and rejected 8 of 8 tampered copies. What that shows, and what it does not, is on the evaluation protocol page.

Limits

An owned synthetic test app with fixed replies and scripted users: no live AI model and no customer system was assessed.
The evidence covers the named policies and the actions the test app ran, not every possible failure.
The mid-task check pauses one action at a declared point; free-form replies from a live model would need a separate reader.
The deciding records are the test app's own. On an app we did not write, a pass resting only on records the app could write would read inconclusive.
Not production monitoring, not a pen test and not a certification.

BESIDE IT / THE ASSESSOR-PACK SAMPLE

The same evidence, written for an auditor or security reviewer.

The assessor pack is built from the same signed files. It adds qualifiers, the run identities, custody per row, and the named controls each result is evidence toward. These are excerpts from the pack built for this sample.

Qualifiers, as the pack states them

Evidence comes from an owned synthetic app over loopback, observed at HTTP-response and backend-record level. It is not production assurance.
Evidence toward named controls only: no entry asserts that any control holds, and the pack is not an audit opinion.
Custody labels are descriptive. In this sample, all 44 rows rest on records the test app itself wrote.

Verification, in the pack's order

Confirm the key id with us over a separate channel.
Check the signature with the standard-library verifier in the pack.
Rebuild the pack from the signed run files and compare the bytes.

The signature shows who produced the files, never that the checks, the app or the results are correct.

Evidence toward named controls (excerpt). Never "satisfies".
Control	Source label, date read	The pack's note
AIUC-1 D004	Primary source opened, 2026-10-26	Asks that an expert third party, not the system's builder, evaluate reliability at least every three months. A run counts toward it only when such a party ran it; this sample was run by us.
EU AI Act Art. 15(5)	Primary source opened, 2026-10-06	Evidence a provider can put toward its own Article 15 file; Scopehaven is not itself a high-risk system.
SOC 2 CC8.1	Secondary summary, 2026-10-26; criterion text not opened	Technical evidence of a retest a service organization can cite; SOC 2 itself is an organization-level attestation.
OWASP Agentic Top 10 2026, ASI03	Primary source opened, 2026-10-05	Reviewed tag for the tenant, role, cached-access and mid-task checks.

ON YOUR APP

The same report, for one action your agent takes.

On a short call we pick one action that moves money or changes an account, and the record that should decide it.

Book 20 minutes