Approval prompts are not a control when people approve 93-97% of them: what Anthropic's data means for policy
Per-action human approval is not an effective security control for coding agents at normal volume. Anthropic reports that Claude Code users approve 93% of permission prompts (Anthropic Engineering, March 2026) and, five months later, 97% (Anthropic, August 2026). In a study of 1,053 paid testers, human review caught 143 planted dangerous commands, or 13.6%, while Anthropic's classifier blocked 937, or 89%. Keep human sign-off for a short, named list of irreversible actions. Put everything else under deterministic rules and a decision log.
The consensus, stated fairly
Most acceptable-use policies and control matrices written for AI coding agents in 2025 and 2026 contain some version of one sentence: a human approves agent actions before they run. The sentence exists for good reasons, and its authors were not careless.
Human oversight is what auditors know how to test. It maps onto the four-eyes principle that already governs production change, payments and access grants, and it gives an organisation a named person to point to when something goes wrong. It also matched how the tools behaved. Claude Code, Cursor and Copilot all shipped with a permission prompt in front of shell commands and file writes, so "a human approves each action" described the product as it arrived.
There is a second, quieter reason. A human approval step moves liability onto a decision that a person made, which is comfortable for everyone except the person. That is a principal-agent problem dressed as a control. The policy author gets a clean sentence. The developer gets three hundred prompts a day.
The cracked assumption: the human is reading
The control only works if the person reading the prompt evaluates it. Anthropic's August 2026 post gives the clearest public measurement of whether they do.
The company swapped one permission prompt in a test session for a clearly dangerous command, across 1,053 paid testers who knew they were being evaluated but not what was being measured. Approving the prompt never ran anything harmful. Human review caught 143 of the 1,053, which is 13.6%. The classifier running in auto mode blocked 937, which is 89%. Head to head, auto mode blocked 800 commands that humans approved, and humans blocked 6 that auto mode allowed (Anthropic, 7 August 2026).
The same post reports a decay curve. Humans blocked about 17% early in a session and about 5% after 50 or more prior prompts, while the classifier's block rate stayed flat. Earlier secondary coverage attributed this curve to Anthropic without a primary link; the primary now exists, in that post.
Behaviour outside the study points the same way. As of June 2026, according to the same source, 49.5% of active CLI users had written a Bash allow-rule, 5% allowed any shell command outright, 62% had used bypass mode or clicked "don't ask again" on Bash, and 25% of interactive sessions started in bypass mode. Developers are not failing the control. They are routing around it, rationally, because it costs them attention on every call.
Reconciling 93% and 97%
The two approval figures come from the same vendor and measure similar things, so a reader will reasonably ask which one is right. Both can be.
Figure Source and date What the post says What it does not say
93% Anthropic Engineering, 25 Mar 2026 "users approve 93% of population, period,
permission prompts" prompt mix
97% Anthropic blog, 7 Aug 2026 users approve 97% of population, period,
permission prompts prompt mixNeither post publishes a denominator, a measurement window or a definition of which prompts count. The two figures were published roughly five months apart, during a period when auto mode went from launch to default for Pro, Max and Team plans on 14 August 2026. The most likely causes of the gap are a different user population, a different period and a different mix of prompts. One plausible mechanism, which is an inference rather than a reported finding: users who adopted allow-rules removed the prompts they would have rejected least often, which leaves a residual stream of prompts that are even easier to approve. The safe editorial position is to cite both with dates and treat the range, not either point, as the finding.
One detail in the August post deserves more attention than the headline number. Users reject 39% of plans presented for approval but only 3% of individual permission requests. People do review when the unit of review is an intent rather than a command string. That is the mechanism the policy should be built on.
Where the classifier is not the answer either
Replacing a tired human with a classifier swaps one error profile for another. Anthropic's March 2026 evaluation reports a 0.4% false-positive rate on 10,000 real internal tool calls, but a 17% false-negative rate on 52 real "overeager" actions and 5.7% on 1,000 synthetic exfiltration attempts (Anthropic Engineering). The August post reports a 7% miss rate on adversarial red-team sets and warns that this "should not be taken as the miss rate on real traffic." The engineering post says auto mode "is not a drop-in replacement for careful human review on high-stakes infrastructure."
So the vendor's own position is layered. A probabilistic reviewer handles volume; humans handle a small set of high-stakes decisions; and neither is a substitute for a rule that never fires probabilistically, such as refusing a package that is already known to be malicious.
The action-tier matrix
The alternative mechanism fits in one sentence: decide in advance which actions are blocked by rule, which are logged and allowed, which go to automated review, and which deserve a human, and make the human tier small enough that a person will read it.
Tier What belongs here Enforced by Evidence for auditors
------------------- ----------------------------------------- ------------------------- --------------------------
0 Block by rule Known-bad packages, skills, MCP servers; Deterministic hook or Rule text, policy version,
writes to credential stores; disabling managed deny rule; block events in the log
logging; force-push to protected branches never asks
1 Log and proceed Reads, local edits, tests, builds inside Allow rules plus a Decision log sampled
a sandboxed workspace decision log monthly
2 Automated review Unknown packages, new network Classifier or auto mode, Classifier denials, review
destinations, shell outside the workspace logged of a sample of allows
3 Human sign-off Production infrastructure, IAM changes, Named owner approves the Approver, timestamp, the
data deletion, releases, payments plan, not each command diff or plan they approvedThree conditions make Tier 3 a real control rather than a ritual. It must be rare, measured in a handful of decisions per person per day rather than hundreds, because the August data shows catch rates falling with volume. It must present an intent, a plan or a diff, because that is where the 39% plan-rejection rate shows people still engage. And it must leave evidence: who approved what, when, against which version of the policy.
Tier 0 matters more than its size suggests. A deterministic block on a known-bad artifact does not depend on anyone's attention at 6 p.m., and it is the only tier whose failure rate an auditor can test by replaying the rule.
Rewriting the policy sentence
Most organisations do not need a new policy. They need one sentence replaced.
Before: "All actions taken by AI coding agents are reviewed and approved
by a human before execution."
After: "AI coding agent actions are governed by an action-tier policy
(version N). Tier 0 actions are blocked by deterministic rules.
Tier 1 and 2 actions are logged; Tier 2 actions pass automated
review. Tier 3 actions, listed in Appendix A, require approval
of the plan by a named owner. All decisions are logged with the
policy version that produced them."The second sentence is longer and less comforting. It is also true, which the first one is not for most teams running agents today.
Where this argument would be wrong
Every figure here comes from one vendor studying its own product. The testers knew they were being evaluated, approving the planted prompt never ran anything dangerous, and Anthropic has not published the populations behind the 93% and 97% figures. An independent replication on a different agent could produce different numbers.
The argument would weaken if an independent study showed that humans catch a large share of dangerous actions when prompts are rare and presented with context. That would not rescue per-action approval at today's volumes. It would strengthen the case for Tier 3 as designed above. The argument would also need revisiting if a vendor shipped approval interfaces that measurably hold catch rates flat across a long session.
None of this says humans are useless reviewers. It says they are expensive ones, and expensive controls should be spent where the consequence is irreversible.
The Monday-morning decision
Pull your agent policy and find the sentence that names human approval as the control. List the actions your agents actually take in a week, sort them into the four tiers, and count how many land in Tier 3. If that count is more than a person could read with care, the tiering is wrong, not the person.
If you want a starting list for Tier 0, the threat.cafe board tracks publicly documented malicious skills, MCP servers and packages, each credited to its finder.