Published:
Human reviewers approved 1 in 3 malicious AI agent commands as routine across 40,000 simulated oversight sessions, according to research published August 5, 2026 by security researcher Alex Wauters at Scale X. The study analysed 409,000 individual approve/deny decisions and found a mean threat-detection accuracy of 66.3%, meaning the average human-in-the-loop reviewer missed 33.7% of threats presented to them.
What the Scale X Study Found
The study used a browser-based simulation at llmgame.scalex.dev where participants act as the human overseer for an AI coding agent, approving or denying terminal commands under time pressure. Approximately 34% of commands shown were threats — higher than typical production rates, making the simulation harder than most real deployments. Across 409,000 decisions, Scale X recorded the following outcomes:
- 32.9% of sessions ended with a negative score, meaning penalties for missed threats outweighed correct approvals.
- 35.2% of participants caught every threat correctly, but only 20.8% did so while also blocking fewer than 1 in 5 safe commands — the dual-accuracy benchmark that matters for real operations.
- 7% of participants approved every single prompt presented, including all threats.
For a practical picture of how AI agents work in practice in business settings, including the command types they issue, Scale X’s category breakdown is the most empirically grounded data published to date on human oversight performance.
Which Threats Humans Miss Most
Miss rates varied sharply by threat category. Obvious destructive commands such as rm -rf / carried an 11.7% miss rate — reviewers recognised and blocked the bulk of them. Persistent mutation commands such as crontab injection, which add recurring tasks that survive session end, were missed 23.8% of the time. Exfiltration and code-execution commands such as curl calls to unknown APIs were missed 33.4% of the time. Scope-violation commands — for example, cat ~/.aws/credentials — were missed 35.0% of the time.
The single most-missed command in the study was npm run analyze, approved 64.7% of the time despite containing a data-exfiltration script visible in the agent’s history log directly above the permission prompt. According to Alex Wauters: “Two thirds of players approved it anyway, indicating the history log just above the permission prompt may not be read closely.”
Wauters noted the pattern across categories: “The blatantly destructive commands are caught most reliably. The commands that actually exfiltrate your credentials are missed three times as often.”
What This Means for Businesses Deploying AI Agents
The practical implication for enterprise teams: human-in-the-loop approval, the most common AI agent safety control, carries a measurable and category-specific failure rate. The threats that most reliably evade human review — credential-scope violations and ambiguous run scripts — are also the ones that cause the most operational damage. A 35% miss rate on cat ~/.aws/credentials means that roughly 1 in 3 attempts to harvest cloud credentials would clear an attentive human reviewer.
A separate 2026 Gravitee report found that only 21% of executives have complete visibility into agent permissions, tool usage, or data access patterns — suggesting the oversight gap extends well beyond individual reviewers to organisational governance.
The data validates the case for technical safeguards that operate at the infrastructure layer rather than relying on human attention. Approaches that address this include limiting what AI agents can do with scoped execution budgets and preventing agents from accessing credentials at the identity layer — both of which remove the need for a human to catch the threat in the first place.
Our Take: “Human-in-the-loop” has become a compliance checkbox, not a safety mechanism. A 64.7% miss rate on
npm runscripts — where the exfiltration evidence sits directly above the prompt — shows that ambiguous commands at speed defeat even attentive reviewers. Businesses deploying coding agents need technical controls: permission scopes, credential isolation, inference-level interception. Human approval alone is not an adequate backstop at any meaningful scale.
For Context: The AI Agent Oversight Debate in 2026
The Scale X findings contribute to a growing body of evidence that AI agent deployment has outpaced governance tooling. Vint Cerf’s June 2026 calls for formal AI agent protocols addressed the same structural gap from a standards perspective. On the vendor side, infrastructure-level controls are multiplying: Cloudflare’s scoped agent wallets address execution-budget limits; 1Password’s agent integration addresses preventing agents from accessing credentials by design. The Scale X study now provides empirical grounding for why these controls matter — not as theoretical safeguards, but as corrections for a measured 33.7% human failure rate.
