Six public AI cases, and the check we would run for each.
By Scopehaven · Updated
In the public cases below, an AI product or agent exposed data or took an action it shouldn't have, or researchers showed it could. Three were found by researchers, and none involved Scopehaven. Each card names the check we would run before launch.
The six cases
Reported incident
Meta
Password resets requested through an AI support tool went to attackers
Attackers used Instagram's AI-assisted account recovery tool to request password reset links to email addresses they controlled. Meta says the tool worked as intended, but a bug in a separate code path skipped the check that the email matched the account, so accounts without two-factor authentication could be taken over. Meta told Maine's Attorney General that up to 20,225 accounts may have been compromised.
WHAT WE WOULD TEST
An account change goes only to the contact on file for that account, never to one the requester supplies.
APPROVAL-04Matching check in our demo suite: Approval rejects a changed recipient
Stolen tokens for an AI chat agent were used to pull companies’ Salesforce data
Attackers used stolen access tokens for Drift, Salesloft’s AI chat agent, which companies had connected to Salesforce, to export support cases, accounts and user records, then searched them for passwords and cloud keys. Google put the number of potentially affected organizations at more than 700. Cloudflare said the text of its support cases was taken. Salesloft and Salesforce revoked all active Drift tokens.
WHAT WE WOULD TEST
The agent’s permitted reads are capped, so its access can’t add up to a bulk export.
BUDGET-01Matching check in our demo suite: Repeated permitted reads cannot exceed a cumulative budget
A test admin account on an AI hiring platform used the password 123456
Researchers got into McHire, the AI hiring platform Paradox.ai runs for McDonald’s, through a test admin account with the password 123456, then found they could open other applicants’ records. They estimated more than 64 million were reachable; Paradox says only the researchers viewed data, covering five candidates, and it was fixed within hours.
WHAT WE WOULD TEST
One applicant's records can't be opened by changing an ID in the request.
TENANT-01Matching check in our demo suite: Customer records stay within their tenant
One crafted email could make an AI assistant leak private data
Researchers demonstrated private-data exfiltration from Microsoft 365 Copilot through a crafted email. They knew of no affected customers, and Microsoft fixed the vulnerability.
WHAT WE WOULD TEST
Instructions hidden in an email or document can't send private data anywhere.
FLOW-01Matching check in our demo suite: Reading does not authorize forwarding to an external sink
A poisoned lead record could steer an AI agent to leak CRM data
Researchers showed how malicious content in a lead record could steer a Salesforce Agentforce agent into CRM data exfiltration. Salesforce remediated the reported chain.
WHAT WE WOULD TEST
Text inside a CRM record can't trigger an export to an unapproved destination.
UNTRUSTED-01Matching check in our demo suite: A document cannot authorize a customer-data export
Why test before launch: these public incidents and research findings show why permission boundaries need testing. Scopehaven checks the boundaries you define, records the available evidence, and reruns the same checks after a fix, including controls for permitted work.
Where Scopehaven has run so far: our own test apps, an internal pilot of Scopehaven's approach on one open-source AI workspace, and practice installs of two open-source apps we did not build, Open WebUI and the Vercel chatbot template (the chatbot through a small login bridge), all on our machine with synthetic data. The practice installs are not customers, and no vulnerability was found or claimed in them. Scopehaven has not been used on the systems above, on customer apps or in production.
Scopehaven
Request a first test