The standard answer to "how do we stop an AI agent doing something dangerous?" is: keep a human in the loop. The agent proposes a command — delete this, run that, read these credentials — and a person approves or denies it. A new set of numbers, going around this week, suggests that safeguard is leakier than most people assume. The humans waved through roughly one in three malicious commands.

Before anyone panics, the honest framing matters, because it isn't the framing the headline gives you.

It's a game, not a study — and the author says so

The data comes from a browser-based game built by Alex Wauters, a software engineer who writes about building and scaling software on his personal blog, Scale X. Players act as the human reviewer, approving or denying an AI agent's requests against the clock. He is upfront about what it is: "it's just a game and not an academic study." Across more than 40,000 playthroughs and 409,000 individual decisions, players caught threats with a mean accuracy of 66.3% — missing about a third — and a third of sessions ended in the red.

The caveats are real and the author lists them. The game runs under artificial time pressure. Around 34% of the commands were malicious — wildly higher than any real workflow, where dangerous requests are rare needles in a haystack of safe ones. Players couldn't always see what a command would actually touch. So the precise percentage is an artefact of the game, not a measurement of the world, and nobody should quote "humans miss 33% of attacks" as a field statistic. It isn't one.

Why the result is still worth your attention

Strip away the exact number and two findings underneath it are harder to wave off.

The first is which threats got missed. Obviously destructive commands — the rm -rf sort — were waved through only 11.7% of the time. People catch the scary-looking ones. But scope violations, like an agent quietly asking to read your cloud credentials or Kubernetes config, slipped past 35% of the time, and exfiltration or code-execution commands 33.4%. The single most-approved malicious command was npm run analyze — nodded through nearly 65% of the time, because it looks utterly routine, even though it can run whatever a project's configuration tells it to. The danger isn't the command that looks dangerous. It's the one that looks like Tuesday.

The second is permission fatigue, and it's the part that generalises. Miss rates climbed the longer a session went on — the more approvals someone waved through, the less attention each one got. That isn't a quirk of the game; Anthropic has made the same observation about real supervision, that "the more approvals a user sees, the less attention they pay to each." A human asked to approve the fortieth command of the day is not the same reviewer who approved the first.

The takeaway

It is one engineer's self-built game rather than a controlled study, so read the exact figures as illustration, not measurement. But the underlying point doesn't depend on the number, and it lands on something the whole industry currently leans on. "Keep a human in the loop" is treated as the backstop that makes autonomous agents safe to deploy. What this suggests — even as a game — is that the backstop erodes exactly where it's needed most: on the un-scary commands, and after the human has been clicking approve all day. A safeguard that works best against threats you'd have caught anyway, and worst against the ones you wouldn't, is worth being honest about before you build your agent strategy around it.