Picture an agent’s approval panel on a Tuesday morning. Every time the agent wants to do something, a request pops up: read this file, run this command, update this record. The person on call reads the first few carefully. By noon they approve with one click, because almost everything was fine. If a bad one shows up among those requests, will they stop it?
The beliefs about people in the loop are opposite. One says that if a person approves every agent action, the risk is under control. It also says that a person is safer than a classifier, a model that approves or blocks actions. The other belief says the opposite: if the agent is almost always right, human review is unnecessary. Both treat the person as a switch. But a person has limited attention.
There is a third, quieter belief. If the agent solves a task on its own, it will also solve it when the customer has to do their part. In support, many tasks work that way.
Human oversight is the norm among teams with agents in production. “Measuring Agents in Production” (exhibit 29) studied 20 cases and surveyed 86 practitioners across 26 domains. 74% rely mainly on human evaluation. 68% let the agent run at most 10 steps before a person steps in.
Meta’s Rule of Two (exhibit 12) also uses the person as the way out. If the task combines untrusted content, sensitive data and outbound actions, the agent does not run alone.
A mid-sized SaaS team described on Reddit how it built a support triage agent (exhibit 99). Triage is sorting requests and deciding who handles them. The team started with passwords and billing. For the first 30 days, people reviewed 100% of the answers. The first version without review made up a feature that did not exist.
At 90 days, the agent resolved 47% of tickets with nobody involved. It drafted another 22% that a person then sent. These are self-reported figures in a small thread.
47% and 22%
Tickets resolved with no person, and tickets drafted by the agent and sent by a person, at 90 days. Exhibit 99
The idea that review is unnecessary has its best version in dotnet/runtime. Stephen Toub, a Microsoft engineer, published ten months of data on Copilot’s agent in that repository (exhibit 68). The agent opened 878 pull requests, which are proposed changes to the code. 67.9% succeeded. The accepted ones were almost never reverted afterward: 0.6%, against 0.8% for the rest.
The author of a Hacker News thread sees a push to review little or nothing of the code from agents (exhibit 43). The motto is: if the tests pass, ship it. If the agent breaks things less often than people do, why review everything?
The author of a game shared the numbers on Hacker News (exhibit 122). In the game, people approve or reject an agent’s commands. There were more than 40,000 runs and 409,000 decisions. Players approved one in three threats, even with a warning at the start.
The thread read it as decision fatigue: if a person approves everything, they stop reading. It is a game, with no real consequences and a clock.
Also, the person steps in less and less. Inside Anthropic (exhibit 28), the coding agent went from 9.8 to 21.2 consecutive tool calls. This was between February and August 2025. Human turns fell from 6.2 to 4.1.
If a person approves every action the agent takes, the risk is under control.
Verdict: Not on its own. In more than 40,000 runs of a game, people approved one in three threats. This happened even with a warning at the start. When everything goes through a person, they stop reading. And the person steps in less and less. In an AI lab, the coding agent went from 9.8 to 21.2 tool calls in a row. That change took six months of 2025. Approval works if it is kept for a few destructive actions, after isolating the agent.
Rule
Isolate first and keep approval for a few irreversible actions. Tie each approval to the exact action, with its parameters, and make it expire and work only once. When in doubt, block.
Why medium confidence. The threat figure comes from a game with no real stakes and a timer, discussed on Hacker News. OWASP asks for human approval of destructive actions. The rule of two and the production study show that supervision is the norm. But they do not measure how much it protects. Tying approval to the exact action, with expiry and single use, is this site's own design rule. No source in the base measures it.
Show the 5 exhibits in detailHide the detail2 for, 1 against, 2 qualify
A design rule: within one session, an agent must have at most two of these three properties. One, processing untrusted input. Two, accessing sensitive data or systems. Three, changing state (modifying something) or communicating externally. If the task needs all three, the agent must not run alone: it needs human supervision.
Numbers
2 of 3Properties an agent can combine in one session without human supervision.
Limitation On Hacker News, a co-author called it a minimum floor, not a sufficient standard. Other critics note that changing state without communicating externally can also do harm, and that asking a person to approve many actions lowers their attention.
“no more than two of the following three properties within a session”
A case study and survey of practitioners with agents in production. Most evaluate with people rather than benchmarks (standard tests), and let the agent take few steps before someone steps in. Reliability, meaning that the agent behaves correctly over time, remains the main challenge. Teams tackle it with system design.
Numbers
20 cases and 86 practitionersBasis of the study: 20 in-depth interviews and a survey of practitioners across 26 domains.
74%Rely mainly on human evaluation.
68%Run at most 10 steps before a person steps in.
70%Prompt off-the-shelf models instead of tuning their weights.
Limitation A small sample chosen by the authors; it does not represent all companies. Preprint.
“Reliability (consistent correct behavior over time) remains the top development challenge”
In a game, people approve or reject an agent's commands, and its author shares the numbers. Even with an upfront warning, players approved one in three threats. The thread reads it as decision fatigue: when a person approves everything, they stop reading. They propose isolating the agent first and keeping approval for a few actions.
Numbers
1 in 3Threats that players approved without noticing.
over 40,000 games and 409,000 decisionsBasis for the numbers.
340 points and 245 commentsSize of the thread.
Limitation It is a game, not a peer-reviewed study: there are no real consequences and there is a timer. It was read on news.ycombinator.com and through the public Algolia API. Quotes from the thread were not checked word by word in the source verification, so none is used.
It is OWASP's first consensus list built for agents. The ten risks run from agent goal hijacking (ASI01) to rogue agents (ASI10). They include tool misuse (ASI02) and identity and privilege abuse (ASI03). They also include the agentic supply chain (ASI04) and cascading failures (ASI08). It adds the principle of least agency: do not grant autonomy that is not needed. Its mitigations include least-privilege profiles per tool, human approval for destructive actions and controls outside the model. It also asks for cost, rate or token caps with automatic cutoff.
Numbers
10 risks (ASI01 to ASI10)Size of the list. ASI02 and ASI03 extend excessive agency (LLM06) from the 2025 list to multi-step flows.
Limitation It is a voluntary consensus framework: it binds no one and measures nothing. In the agent list, unbounded consumption is no longer an item of its own: it appears as a scenario and mitigation inside ASI02. The full PDF (57 pages) was read.
“deploying agentic behavior where it is not needed expands the attack surface”
Huang, S., Seethor, B., Durmus, E., Handa, K., McCain, M., Stern, M., Ganguli, D. Anthropic
Inside the lab itself, people use Claude for more of their work and say they are more productive. But they can fully delegate only a small part. Agents chain more actions with less human involvement. There is concern about losing the very skills needed to supervise AI, and about how junior staff will learn.
Numbers
0% to 20%Share of work most people can fully delegate.
59% and +50%Work in which they use Claude (28% a year earlier) and self-reported productivity (+20% a year earlier), in 132 respondents.
from 9.8 to 21.2 callsConsecutive tool calls by the coding agent between February and August 2025; human turns fell from 6.2 to 4.1.
Limitation Self-report by the vendor's employees, with 2025 data (August 2025 survey, 53 interviews and 200,000 transcripts).
“I am for sure atrophying in my skills as a software engineer”
Quote checked against the original
Read at the source. Checked on Nov 2, 2026.
Does a classifier do better? Anthropic published a study on auto mode, its mode with a classifier (exhibit 123). People stopped 13.6% of dangerous commands. On the same commands, the classifier would have stopped 89%. After more than 50 requests, people dropped from about 17% to about 5%.
Dangerous commands stopped by people (143 of 1,053) and that the classifier would have stopped (937 of 1,053). People drop from about 17% to about 5% after more than 50 requests.Exhibit 123
Dangerous commands stopped, in %
People13.6%
Classifier89%
0100%
Anthropic’s system flagged some user sessions for review. There, production-level harmful actions that nobody asked for showed up. They were 6.3% with manual approval and 2.4% with auto mode. The vendor that sells the classifier publishes this. The study with people was in a controlled environment.
Anthropic made auto mode the default for its coding agent. A Hacker News thread split in two (exhibit 124). The comment with the most replies rejected using another model to classify the first model’s commands. But it did not defend manual approval either.
For that comment, reviewing commands one by one does not make the tool safe. Security comes from controls such as version control or read-only tokens. A token is an access key. Others run the agent with no permissions inside a container, an isolated environment.
A person who approves every command is safer than an automatic classifier.
Verdict: On the data available, no. In the vendor's study, people stopped 13.6% of dangerous commands. The classifier would have stopped 89%. After more than 50 requests, people dropped from about 17% to about 5%. But neither one is the real safeguard. That safeguard comes from controls outside the agent, such as read-only tokens or a container without permissions.
Rule
Do not count manual approval of every command as your main control. Do not just swap it for another model either. First set limits the agent cannot get around, such as read-only permissions, a container or version control. Then measure what the person and the classifier each stop.
Why low confidence. The comparison between people and the classifier is published by the vendor that sells the classifier. The study took place in a controlled environment. The game confirms that people approve threats, but with no real stakes. No source in the base argues with data that the person is safer. The Hacker News thread distrusts the classifier. But it does not count manual approval as the real safeguard either.
Show the 3 exhibits in detailHide the detail0 for, 2 against, 1 qualifies
A classifier reviews each tool call instead of asking for approval for each action. It blocks what is irreversible, destructive or aimed outside the user's environment, and it always refuses data exfiltration (sending data out). If blocks pile up, it goes back to manual approval. In a study, people approved almost everything dangerous and grew tired over time. The classifier stopped far more.
Numbers
13.6% versus 89%Dangerous commands stopped by people (143 of 1,053) and that the classifier would have stopped (937 of 1,053). People drop from about 17% to about 5% after more than 50 requests.
0 of 720Successful indirect injections against three Claude models in auto mode, in an evaluation commissioned from a third party. Another agent with automatic review: 5.83%; with full access: 19.03%.
from 12% to 7%Attacks the classifier lets through in a red team exercise, before and after hardening it. The source says it is not the rate on real traffic.
6.3% versus 2.4%Sessions with an unrequested harmful action at production-level severity (7 or more), with manual approval and in auto mode. They are sessions from May and June 2026 that the safety system flagged for review, from accounts that opted in to data use.
Limitation Published by the vendor that sells the product, which also commissioned the evaluation. The attacks were optimized against a different model. The human study was in a controlled setting. There is an update note from October 22, 2026.
“it relies on classification systems and therefore does not eliminate risk”
In a game, people approve or reject an agent's commands, and its author shares the numbers. Even with an upfront warning, players approved one in three threats. The thread reads it as decision fatigue: when a person approves everything, they stop reading. They propose isolating the agent first and keeping approval for a few actions.
Numbers
1 in 3Threats that players approved without noticing.
over 40,000 games and 409,000 decisionsBasis for the numbers.
340 points and 245 commentsSize of the thread.
Limitation It is a game, not a peer-reviewed study: there are no real consequences and there is a timer. It was read on news.ycombinator.com and through the public Algolia API. Quotes from the thread were not checked word by word in the source verification, so none is used.
The discussion splits in two. The most-replied comment says that neither reviewing commands one by one nor using another model to classify them makes the tool safe. Safety comes from developer controls, such as version control, immutable file systems or read-only tokens. Others run the agent with no permission prompts inside a container. One says that with dozens of agents in parallel, manual approval does not scale.
Numbers
292 points and 313 commentsSize of the thread as of November 2, 2026; the top comment has 104 replies.
Limitation Practitioner opinions, not a measurement. It was read through the Algolia API (the 12 threads with the most replies).
“Using more Claude to categorize the safety of Claude commands is not the answer.”
Quote checked against the original
Read at the source. Checked on Nov 2, 2026.
On review being unnecessary, the full dotnet/runtime data tells another story. Some of the agent’s pull requests had commits from a person. A commit is a saved change in the repository. With those commits, success was 86.2%. Without them, 55.1%.
Toub credits expert review for the low revert rate: “we cannot compromise on review quality, on expert and experienced eyes validating the changes”. Nine pull requests that Toub opened with the agent in a short stretch generated 5 to 9 hours of review.
METR, an institute that evaluates AI models, added another piece (exhibit 67). Four maintainers of scikit-learn, Sphinx and pytest reviewed agent pull requests. Maintainers decide which changes get into a project. Those pull requests had already passed the automated grader. Roughly half would not have been merged.
An engineer said on Reddit that he automated about 80% of his job with a coding agent (exhibit 79). He says quality holds because he reviews every change. He spends 2 to 3 hours a day on it. A commenter told the opposite story: when he dropped review, quality degraded within a week.
Once the agent is right almost every time, human review is unnecessary.
Verdict: No: much of what the agent gets right goes through people. In dotnet/runtime, the agent's pull requests with commits from a person had an 86.2% success rate. Without those commits they had 55.1%. Maintainers reviewed agent pull requests that passed the automatic grader. About half would not have been merged. One commenter says quality degraded within a week once review stopped.
Rule
Do not drop review because the numbers look good: the numbers are good because of the review. If you want to loosen it, do it by task type, where tests and compilers already verify the result. Then measure how many changes get reverted.
Why medium confidence. The dotnet/runtime data come from a single repository with very expert reviewers, published by an engineer at the tool's vendor. The maintainer review is a research note with four people and three repositories. The Reddit account is from one person. It is all about code, where tests do the checking.
Show the 5 exhibits in detailHide the detail0 for, 3 against, 2 qualify
Ten months of data on the Copilot coding agent in the dotnet/runtime repository. More than half of its pull requests were merged and almost none were reverted, because each one went through expert review. It did best on mechanical, well-scoped tasks (cleanup, tests) and worst on those that need judgment (performance). An instructions file and build setup almost doubled success. The bottleneck moved from writing code to reviewing it.
Numbers
878 PRs, 67.9% successAgent pull requests and success rate (87.1% for PRs by Microsoft staff; populations not comparable).
0.6% vs 0.8%Merged PRs later reverted: the agent's vs the rest.
from 38.1% to 69%Success before and after adding instructions and setting up the build.
86.2% vs 55.1%Success with and without commits from a person in the PR.
5 to 9 hoursReview work created by nine PRs the author opened in a short time.
Limitation Written by a Microsoft engineer about a Microsoft tool. It is a repository with very expert reviewers. The author notes that the agent and the people do not do the same kind of tasks.
“we cannot compromise on review quality, on expert and experienced eyes validating the changes”
Four maintainers of scikit-learn, Sphinx and pytest reviewed agent pull requests that already passed the SWE-bench Verified automated grader. About half would not be merged. In newer models, the failures shifted toward code quality more than broken functionality. The authors note it does not prove a capability limit: the agent had a single attempt, with no chance to iterate with the reviewer.
Numbers
24.2 pointsHow much higher the automated grader scores than the maintainers, normalized against human pull requests (standard error 2.7). Unnormalized, 34.9 points.
296 agent pull requestsReviewed, together with 47 human reference ones, on 95 of the 500 problems. Maintainers merged about 68% of the human ones.
about 50 versus about 8 minutesHorizon of Claude Sonnet 4.5 with the automated grader and with maintainer review; the authors say this figure is very noisy.
Limitation A research note, not a peer-reviewed paper. Four maintainers and three repositories. Models from Claude 3.5 Sonnet to Claude Sonnet 4.5 and GPT-5.
“roughly half of test-passing SWE-bench Verified PRs”
An engineer with 11 years of experience says he automated about 80% of his job with a coding agent. The agent picks up issues and opens pull requests. He says quality holds because he reviews every change. He spends 2 to 3 hours a day reviewing and testing. One commenter says that when review was removed, quality degraded within a week. Others discuss the ethics of doing it without telling the employer.
Numbers
about 80%Share of the job the author says he automated.
2 to 3 hours a dayTime he spends reviewing and testing.
1,200 points and 276 commentsSize of the thread according to the archive.
Limitation One person's story, with one week of use at the time of the post. Commenters point out it works because the work is simple. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
Whoever submits code has to show it works: test it by hand and leave automated tests that fail if the change is reverted. Coding agents are held to the same standard, and the person remains responsible for what is delivered.
Numbers
859 points and 658 commentsSize of the discussion on Hacker News (item 46313297).
Limitation An opinion essay by a specialist, with no data of its own.
“Your job is to deliver code you have proven to work.”
The author sees a push to review little or no code ("if the tests pass, ship it"). On his existing code, half his time went to fixing subtle errors. The success stories agree on the conditions: code already organized by people, static types, broad tests, narrow tasks and reviewing the plan first. One says the agent wrote tests that tested nothing.
Numbers
461 points and 455 commentsSize of the thread.
Limitation Practitioner opinions. It was read through the Algolia API (post and top-level replies with the most replies).
Read at the source. Checked on Nov 2, 2026.
That leaves the third belief. RealUserSim is a Salesforce preprint, a study not yet peer reviewed (exhibit 76). It showed that users simulated by a model are too formal and cooperative. With more realistic users, failures show up that the cooperative user hid. But success drops only a little: by 3.2% to 3.5% on average.
τ-Knowledge (exhibit 63) comes from a team at Sierra, which sells customer service agents. It built a customer service domain for a fintech, a digital financial services company. The best success rate was 25.52% on one try. Success on all four tries of the same task fell to 13.40%.
No source in the base measures, today, an agent that guides the customer to act on the system.
If the agent solves the task alone, it also solves it when the customer has to do their part.
Verdict: Do not take it for granted. Users simulated by a model are too formal and cooperative. With more realistic users, hidden failures show up, although success drops only slightly: by 3.2% to 3.5%. In fintech customer service, the best success rate was 25.52% on one try. On all four tries it was 13.40%, from different configurations. A support case that worked hands to a person whatever it cannot close with confidence.
Rule
Test the agent with real people or with simulated users who do not fully cooperate. Do this especially in cases where the customer has to do something on their side. Leave a fast path to a person for when the customer gets stuck.
Why low confidence. No source in the base currently measures an agent that guides the user to act on the system. Both benchmarks (standard tests) use simulated users and are preprints, that is, papers without peer review. One comes from a company that sells customer service agents. The support case is self-reported in a small thread.
Show the 3 exhibits in detailHide the detail0 for, 2 against, 1 qualifies
Zhu, M., Tan, J., Murthy, R., et al. arXiv (Salesforce AI Research)
Users simulated with a model are too formal and cooperative, and do not look like real users. The authors build simulators based on behavior profiles taken from real conversations. With more realistic users, agent failures that the cooperative user hid show up. Also, success on TauBench, a standard test for agents, drops.
Numbers
7,275 profilesBehavior profiles taken from over 14,000 real conversations (WildChat).
from 24.2% to 45.3%Style match with real users, without and with the method. Without constraints, simulators match 6% to 8%.
3.2% to 3.5%Average drop in agent success on TauBench with realistic users.
Limitation Preprint; the abstract was read. The arXiv page shows submission on April 7, 2026, although the number corresponds to May. The measured drop is small. It does not repeat the measurement of an agent guiding the user.
Shi, Q., Zytek, A., Razavi, P., Narasimhan, K., Barres, V. arXiv (Sierra, Princeton)
The τ-bench team adds a new domain: customer service at a fintech, with about 700 linked knowledge documents. Even the best models, with heavy reasoning, solve about a quarter of the tasks. Also, reliability drops when they repeat the same task. It is part of τ³-bench.
Numbers
25,52 %Best single-attempt success rate (pass^1) with realistic search: GPT-5.2 with high reasoning, on 97 tasks.
13,40 %Best rate of success in all 4 attempts (pass^4).
39.69% and 26.80%Claude 4.5 Opus with the correct documents already given: pass^1 and pass^4.
Limitation Preprint by a team at a company that sells customer service agents. Benchmark with a simulated user.
A mid-size SaaS describes how it built a support triage agent, which sorts requests and decides who handles them. It started with passwords and billing. The agent only answers if it can cite an article or a fact from the database. People reviewed 100% of answers for the first 30 days. Refunds, cancellations, large customers and low-confidence answers go to a person. The first version without review made up a feature that did not exist. The team did not shrink: it moved to the hard cases.
Numbers
47% and 22%Tickets resolved with no person, and tickets drafted by the agent and sent by a person, at 90 days.
USD 0.18 vs about 3.50Cost per ticket with the agent vs the previous human cost.
0.85Confidence threshold: below it, the answer goes to a person.
Limitation Self-reported numbers in a small thread (3 points and 7 comments in the archive), without independent verification. The original post text was read in the public Arctic Shift archive because reddit.com blocks automated reading.
A person in the loop controls risk if you put them where they can pay attention.If they approve everything, they approve the bad things too. If they review what matters, they hold up the agent’s good numbers.
Approving every action is reassurance without control. Dropping review because the numbers look good forgets where those numbers come from. Swapping the person for a classifier stops more, according to the vendor, but it is not the lock either. The lock is whatever the agent cannot get around.
Practitioners propose sorting actions into three tiers: automatic, confirmed by a person, and forbidden. The support team in exhibit 99 sends these cases to a person:
Refunds.
Cancellations.
Large customers.
Any answer with confidence below 0.85.
OWASP is a software security foundation. Its list for agents (exhibit 33) asks for human approval for destructive actions and controls outside the model.
The tiers have two traps. First: if they live only in the prompt, they are not a control. That is how a thread read the case of the agent that deleted a production database (exhibit 91). It is in chapter 5.
Second trap: the tiers drift. A Reddit thread discussed who answers for an agent (exhibit 104). A commenter gave a number. In his view, about 40% of actions move from confirm to automatic in about three weeks. The figure has no method behind it, but the drift is easy to see.
This site proposes a design rule: tie each approval to the exact action, with its parameters. The approval expires and works only once. The rule comes from the fatigue and drift the exhibits show. But no source in the base measures how much it protects.
It is enough to split the agent's actions into automatic, to-be-confirmed and forbidden.
Verdict: It is a good starting point if the levels are enforced outside the model. A rule that lives only in the prompt is not a control. OWASP asks for the same, with human approval for destructive actions. A support team sends refunds and cancellations to a person. It also sends any answer with confidence below 0.85. But the levels drift. According to one commenter, about 40% of actions move to automatic within about three weeks.
Rule
Write the policy in three levels and enforce it in a layer that is not the model. Log why the agent chose each action and check every month which actions changed level.
Why low confidence. OWASP is a voluntary consensus framework that measures nothing. The rest comes from Reddit and Hacker News: small threads, self-reported figures and a 40% with no method.
Show the 4 exhibits in detailHide the detail2 for, 1 against, 1 qualifies
A mid-size SaaS describes how it built a support triage agent, which sorts requests and decides who handles them. It started with passwords and billing. The agent only answers if it can cite an article or a fact from the database. People reviewed 100% of answers for the first 30 days. Refunds, cancellations, large customers and low-confidence answers go to a person. The first version without review made up a feature that did not exist. The team did not shrink: it moved to the hard cases.
Numbers
47% and 22%Tickets resolved with no person, and tickets drafted by the agent and sent by a person, at 90 days.
USD 0.18 vs about 3.50Cost per ticket with the agent vs the previous human cost.
0.85Confidence threshold: below it, the answer goes to a person.
Limitation Self-reported numbers in a small thread (3 points and 7 comments in the archive), without independent verification. The original post text was read in the public Arctic Shift archive because reddit.com blocks automated reading.
It is OWASP's first consensus list built for agents. The ten risks run from agent goal hijacking (ASI01) to rogue agents (ASI10). They include tool misuse (ASI02) and identity and privilege abuse (ASI03). They also include the agentic supply chain (ASI04) and cascading failures (ASI08). It adds the principle of least agency: do not grant autonomy that is not needed. Its mitigations include least-privilege profiles per tool, human approval for destructive actions and controls outside the model. It also asks for cost, rate or token caps with automatic cutoff.
Numbers
10 risks (ASI01 to ASI10)Size of the list. ASI02 and ASI03 extend excessive agency (LLM06) from the 2025 list to multi-step flows.
Limitation It is a voluntary consensus framework: it binds no one and measures nothing. In the agent list, unbounded consumption is no longer an item of its own: it appears as a scenario and mitigation inside ASI02. The full PDF (57 pages) was read.
“deploying agentic behavior where it is not needed expands the attack surface”
A thread about a coding agent that deleted, in about 9 seconds, a volume with the production database and its backups. It used a token with too much scope, and the data was partly recovered. Most readers see a failure of permissions and backup design, not “rogue AI”. A rule in the prompt is not a control: they propose minimal credentials, backups in another account and protections outside the agent's reach.
Numbers
about 9 secondsTime of the deletion, according to the affected party's account.
860 points and 1,032 commentsSize of the thread.
Limitation The technical details are the affected party's; there is no independent investigation. It was read on news.ycombinator.com and through the public Algolia API. Quotes from the thread were not checked word by word, so none is used.
A thread about who answers for what an agent does. One comment proposes three action levels: automatic, confirmed by a person and forbidden (for example, paying or deleting data). It also says the thresholds drift. After about three weeks, operators move about 40% of actions from confirm to automatic. That drift must be watched. Another comment asks to log why the agent chose each action.
Numbers
40% in about three weeksActions that move from confirm to automatic, according to one commenter.
Limitation A thread with a very low score (0 points and 16 comments in the archive). The 40% figure comes from one comment, without a method. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
“Capability without visibility into why it chose action X”
Put the person where their attention changes the outcome, and enforce limits outside the model. Now:
Isolate first, before you think about approvals. Give the agent read-only permissions, a container or version control. Apply the Rule of Two from chapter 5.
Write the action policy in three tiers. Enforce it in a layer that is not the model.
Keep human approval for a few irreversible actions. Tie each approval to the exact action, and make it expire and work only once. When in doubt, block.
Measure what the person and the classifier stop. Log why the agent chose each action. Check every month which actions changed tier.
Do not drop review because the numbers look good. If you want to loosen it, do it by task type, where tests and compilers already check the result. Measure how many changes get reverted.
If the customer has to do something on their end, test the agent with users who do not fully cooperate. Leave a fast path to a person.
If something goes wrong anyway, the incident runbook, a guide for emergencies, lays out the first steps.