I click approve on agent commands all day. Turns out I’m bad at it. In a browser game where you play human-in-the-loop, 40,000 runs and 409,000 decisions produced a mean accuracy of 66.3% — one in three threats waved through. The failure isn’t where you’d guess. rm -rf / gets caught; only 11.7% missed. It’s the scope violations at 35.0% and exfiltration at 33.4% — the quiet ones that move your credentials somewhere else.
The part that got me: npm run analyze was approved 64.7% of the time, the single most-missed command, even though the history log right above the prompt showed the script piping bundle stats into a curl POST. Pooled with npm run setup and npm run deploy, those three were missed 52.5% of the time versus 28.4% for other exfiltration attacks. A familiar script name roughly doubles the payload’s success rate. As dns_snek put it on HN, npm run build is just “run an arbitrary shell command written in package.json” — and the agent can edit that file without asking anyone.
So I’m treating my own approvals as decoration. Sandbox the agent, keep secrets out of .zshrc and source them from a separate file instead, and stop pretending the prompt is a control.
The story — Alex Wauters published statistics from a browser game where players approve or deny commands for a simulated AI coding agent under time pressure. Across 40,000+ runs, 32.9% of sessions ended with a negative score, 7% approved every single prompt, and benign commands like rm -rf dist/ were blocked 45% of the time — noise that the author argues breeds the fatigue Anthropic has already flagged in Claude Code. (Source)