This evaluator recommends how to automate a task. It asks ten questions, one per screen, and gives one of five outcomes: fixed workflow, bounded agent, agent with human approval, a person, or not yet.
A fixed workflow follows steps written in code. An agent is a model that picks the steps while it works.
Each reason and each control in the result links to the template where you work on it. Your answers stay in your browser. When you finish, the link in the address bar has your answers, and you can share it.
Intake form
This link could not be read: it is incomplete or from another version. Answer the questions from the first one.
You are looking at answers from a link. If you change one, they are saved as yours in this browser.
This browser does not let us save your answers. When you finish, copy the link so you do not lose them.
Result
Before you launch
You can write the steps in advance. A fixed workflow is enough. It is cheaper, faster and easier to test.
The workflow reads outside content, touches sensitive data and acts on the outside. There is no model to fool, but a person has to approve every action.
Do not launch it like this: the risk calls for a person to approve every action, and nobody can. You have two options. Find someone to approve, or remove what calls for approval. That is the action that cannot be undone, the outbound action or one leg of the trifecta.
Write the steps in code. If needed, the model joins as one step of the flow, not as the one driving it.
An agent with few tools, minimal permissions and a budget that cuts it off. It fits when a mistake is cheap. It also fits when a mistake is expensive, but can be undone and someone reviews it.
The agent proposes and a person approves every action that matters. The approval is tied to the exact action.
Today a person should do it. Automating costs more than it saves, or nobody can cover the risk.
Without real examples you cannot know if it works. Collect them first and evaluate again.
Once you have the examples
With 20 real cases to test it on, the recommendation becomes this:
Fixed workflow
Write the steps in code. If needed, the model joins as one step of the flow, not as the one driving it.
Bounded agent
An agent with few tools, minimal permissions and a budget that cuts it off. It fits when a mistake is cheap. It also fits when a mistake is expensive, but can be undone and someone reviews it.
Agent with human approval
The agent proposes and a person approves every action that matters. The approval is tied to the exact action.
A person
Today a person should do it. Automating costs more than it saves, or nobody can cover the risk.
Not yet
Without real examples you cannot know if it works. Collect them first and evaluate again.
Why
You can write the steps in advance. A fixed workflow is enough. It is cheaper, faster and easier to test.
The workflow reads outside content, touches sensitive data and acts on the outside. There is no model to fool, but a person has to approve every action.
Do not launch it like this: the risk calls for a person to approve every action, and nobody can. You have two options. Find someone to approve, or remove what calls for approval. That is the action that cannot be undone, the outbound action or one leg of the trifecta.
Some rules use the trifecta. The trifecta is three legs in the same agent session: the agent reads untrusted content, accesses sensitive data and acts outward.
The rules apply in this order. The first rule that is true sets the recommendation:
If you can write the steps in advance, the result is a fixed workflow, even if the input is ambiguous. In that case the model is one more step of the flow. A fixed workflow is cheaper to run, to test and to maintain. There is one exception. The workflow has a model step, has all three legs of the trifecta and nobody reviews. Then the result is a person, as in rule 3.
If there are few cases and a mistake is expensive, the result is a person. Building and watching an agent does not pay off.
A person must approve each action in three cases: the task has all three legs of the trifecta, a mistake cannot be undone, or an expensive mistake goes outward. If nobody can approve, the result is a person.
If a mistake is expensive, the system does not act outward and nobody reviews, the result is a person.
What is left is a bounded agent: open steps and a cheap mistake, or an expensive one that someone reviews.
If the result is an agent and you do not have 20 real examples to test it, the result is not yet. The result also shows the recommendation that you get when you have them.
A fixed workflow with risk is still a fixed workflow. But it adds two controls. The first is the action gate: code that checks each action against the policy before it runs. The second is the approval tied to the exact action.
If nobody can give that approval, the result shows a “Before you launch” condition above the recommendation. You have two options: find someone who can approve, or remove what needs approval. Until then, do not launch.
Verdict: The other way around: the fixed workflow is the starting point, and the agent has to earn its place. A commenter says an agent guided only by prompts broke around file 30 of some 200. A deterministic harness fixed it: fixed code that guides each step. A company let a lightly supervised software factory run for months. It ended up archiving all the code. The agent adds value where the path changes from case to case, such as when correlating alerts.
Rule
Write the process as a fixed workflow first. Let the model decide only at the steps where the path changes from case to case, and justify each one with a test.
Why medium confidence. Several teams agree, but there is no controlled experiment. The Hacker News and Reddit cases are unmeasured accounts. The archived factory is one person's anecdote. The posts on harnesses come from a model vendor. Nobody backs the claim with data.
Show the 6 exhibits in detailHide the detail0 for, 4 against, 2 qualify
A thread about an article that asks for control flow written in code, not in prompts. One commenter says a QA agent had to go through about 200 requirement files and broke around file 30. A deterministic harness (code that fixes each step) solved it. Another says that with the work split into small pieces, cheaper models perform like frontier models. Some complain that platforms prefer prompt-only flows.
Numbers
about 200 files, breaks around 30Case of a QA agent driven only by a prompt, according to one commenter.
590 points and 296 commentsSize of the thread.
Limitation User stories, without measured data. It was read on news.ycombinator.com and through the public Algolia API. Quotes from the thread were not checked word by word, so none is used.
In an internal workshop, an agent's “skill” for creating an environment turned out to be a list of gh and bash commands. Put together, these commands are a script. The thread concludes that a recurring process works better when it is deterministic, meaning it always does the same thing. A counterpoint says the agent brings the continuous integration loop (automatic tests for each change) to the local machine. In this way, it leaves CI as the last check.
Numbers
517 points and 115 commentsSize of the thread according to the archive. The workshop comment has 454 points.
Limitation Practitioner experience, without measurement. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
“Turns out that recurring processes work way better when they are deterministic.”
An article about Dex Horthy, author of the 12-factor agents guide. A year later he holds the same view: a big agent loop does not hold up for serious work without structure and verification. His company let a lightly supervised "software factory" run for three or four months and then archived all the code. He says the missing piece is better verifiers of code quality.
Numbers
30,000 to 40,000 linesSize the factory code reached before it was archived.
10 of 30 (33.3%)Checkpoints the best models passed on the SlopCodeBench subset he cites; the differences between vendors are within the margin of error.
Limitation It is an article about one person, with his own anecdote; it is not a study.
It compares a single agent with a three-agent harness (planner, generator and evaluator) for building full applications. The harness gave clearly better results, but cost over 20 times more. Agents that grade themselves are too generous. The separate evaluator also failed until they tuned it to be skeptical and to test with an external tool. Each harness piece assumes something the model cannot do alone, and with a better model (Opus 4.6) they removed pieces.
Numbers
20 minutes and 9 dollars versus 6 hours and 200 dollarsA retro game with Opus 4.5: single agent (the game did not respond to controls) versus full harness (playable).
3 hours 50 minutes and 124.70 dollarsA digital audio workstation with Opus 4.6 and the simplified harness.
Limitation The vendor's own cases, with no systematic control group.
“agents reliably skew positive when grading their own work”
A platform team with about 20 services reports what stayed after a year of trying agents. It dropped AI-generated Terraform, because every line had to be read. It also dropped an assistant that answered wrongly with confidence: someone almost ran a command it suggested in production. It kept what reads and summarizes: automatic changelogs, PR review and alert correlation. The rule is that agents read everything, but do not touch infrastructure.
Numbers
60% vs 0%How often alert correlation gets the probable cause right, vs the old system.
289 points and 43 commentsSize of the thread according to the archive.
Limitation One team's story, without independent measurement. A commenter suspects the post is an ad for a review tool it mentions. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
“agents can read and summarize anything but they dont get to touch infra”
It separates three pieces with stable interfaces: the session, the harness and the sandbox. The session is an event log outside the model context. The harness is the loop that calls the model, and the sandbox runs the code. That way any piece can fail or be swapped without losing the session, and credentials stay outside the sandbox. It repeats that the harness encodes assumptions that expire with each new model.
Numbers
about 60% and over 90%Drop in time to first token at the median (p50) and at the 95th percentile.
Limitation Vendor product post. It does not compare a fixed workflow against an agent.
Read at the source. Checked on Nov 2, 2026.
If the model is well aligned, it can read private data, process outside content and communicate externally, all at once.
Verdict: No. With all three together, an attacker can make the agent leak data. The model's alignment does not prevent it. A web guide with hidden text made an agentic code editor gather code and credentials. Then the editor sent them out. In a benchmark with real MCP servers, the models that do the task best resist attacks worst. In 2026, Microsoft kept recording injections in Copilot that allow information to be disclosed over the network.
Rule
Apply the rule of two: within a single session, the agent combines at most two of three properties. They are reading untrusted content, touching sensitive data or systems, and acting or communicating externally. If the task needs all three, remove one or add human approval.
Why high confidence. Two official records of flaws in a commercial product, a benchmark with real servers, demonstrations and a specialist agree on the mechanism. The benchmark has only 13 models, and the demonstrations show the attack is possible, not how often it happens. No source argues that alignment is enough.
Show the 8 exhibits in detailHide the detail0 for, 5 against, 3 qualify
Willison restates the lethal trifecta: private data, untrusted content and an outbound channel, together, make data theft possible. He admits it only covers exfiltration, and says Meta's rule of two completes it by adding state changes. On filters, he says blocking or filtering injections has not proven reliable. He does not share the optimism that adequate defenses will arrive soon.
Numbers
12 defensesPublished defenses broken by the paper The Attacker Moves Second, which the post discusses.
Limitation An opinion essay by a specialist, with no data of its own.
“attempts to block or filter them have not proven reliable enough to depend on”
Willison summarizes PromptArmor research on Google's agentic editor. A web guide with hidden text made the agent gather code and cloud credentials. The secrets file was excluded, so the agent read it with a terminal command. Then a browser subagent opened a URL with the data inside, on a domain that was on the default allow list. Google lists it as a known issue.
Limitation Willison's post was read; the original research is by PromptArmor, a security company, and was not opened. It is a demonstration, not a frequency measurement.
A benchmark (a standard test set) with real MCP servers in five domains: browser, finance, maps, repositories and web search. It tests 20 attack types on the server, host and user side. Every model tested is still vulnerable, and the ones that do the task best resist attacks worst.
Numbers
from 29.80% to 48.16%Attack success rate across 13 models and 245 cases (GPT-5: 37.55%; Claude 4.0 Sonnet: 31.43%).
r = -0,572Correlation between task success and defense (p = 0.041, with only 13 models).
Limitation Preprint. There is a version 2 (March 5, 2026) that does not say what changed; the figures are from version 1. 2025 models. Task success is low (9% to 21%).
“models with higher task performance tend to be less resistant to attacks”
A command injection in Microsoft Copilot lets an unauthorized attacker disclose information over a network. It repeats the EchoLeak pattern in a commercial product, more than a year later. This attack needs the user to take an action.
Numbers
8.8 out of 10 (high)CVSS 3.1 severity; the vector requires user interaction.
Limitation Official record with a short description. In 2026, up to September, Microsoft published at least six more injection records for its Copilots, all requiring user interaction.
A design rule: within one session, an agent must have at most two of these three properties. One, processing untrusted input. Two, accessing sensitive data or systems. Three, changing state (modifying something) or communicating externally. If the task needs all three, the agent must not run alone: it needs human supervision.
Numbers
2 of 3Properties an agent can combine in one session without human supervision.
Limitation On Hacker News, a co-author called it a minimum floor, not a sufficient standard. Other critics note that changing state without communicating externally can also do harm, and that asking a person to approve many actions lowers their attention.
“no more than two of the following three properties within a session”
OpenAI launched a lockdown mode that limits outbound network requests, to cut the last step of an exfiltration. OpenAI notes that the mode does not stop the injection from reaching the content. Willison says the trifecta can only be solved by cutting one of its three legs. For him, the outbound channel is the easiest leg to cut without losing usefulness. This mode cuts it with fixed rules that no AI evaluates. He also says its existence implies that ChatGPT, by default, does not protect well against a determined exfiltration.
Limitation A short comment on an OpenAI help page; it gives no figures.
“lockdown mode directly attacks that leg, using mechanisms that are deterministic”
Peters-Gill, J., Semler, A., Bartsch, H., Shumailov, I., Schroeder de Witt, C. arXiv
A co-author of CaMeL, a defense against planted instructions, shows that its guarantees do not compose on their own across agents. An instruction planted in a database is data, but the orchestrator summarizes it and another agent receives it as a trusted request. In the measurements, CaMeL in each agent cuts attack success sharply. A version that tracks data provenance across agents closes that path. The cost is usefulness, and more capable models lose less of it.
Numbers
12.9%, 0.2% and 0.0%Attack success rate without CaMeL, with CaMeL in each agent and with provenance across agents (average of 5 models, 949 cases per model and setup).
21.6 and 13.4 pointsUsefulness lost by per-agent CaMeL on 112 tasks: on average and in the two strongest models. The cross-agent version takes off another 3.2 points on average.
Limitation Preprint. One model refused 53.6% of benign tasks even without CaMeL, which lowers its baseline usefulness.
Read at the source. Checked on Nov 2, 2026.
Once the agent works, it can be left running on its own.
Verdict: No. A narrow agent carries little risk, like one that sorts about 5,000 faxes a day with no power to do harm. The rest need an owner. When the creator leaves, the agent keeps running on a key that no longer belongs to anyone. A NIST draft proposes tying each agent to the person who delegates to it and logging its actions. And only 5% of the companies surveyed have all the controls today to give agents real autonomy.
Rule
Give every agent a named owner and a backup. Add an inventory of what it reads and writes, and a review every three months. Do not create the agent's key or account without an owner field. When someone leaves, review the agents they launched.
Why medium confidence. The cases come from Reddit threads, several with low scores, and the supporting side is a single comment. Both surveys are self-reported and come from consultancies. In the other survey, only 21% of leaders have mature governance for agentic AI. The NIST draft is a consultation, not a measurement. The sources agree, but nobody has measured how often an ownerless agent fails. One commenter adds that limits drift over time.
Show the 8 exhibits in detailHide the detail1 for, 6 against, 1 qualifies
This is the same author as the thread on connecting a model to internal data. Leadership now wants agents that trigger workflows and send emails. The most upvoted advice: get the business or legal to accept the risk in writing before giving the agent actions. In contrast, another describes a narrow agent that routes about 5,000 faxes a day to the right person, with no power to do harm.
Numbers
about 5,000 faxes a dayVolume a narrow agent routes, according to one commenter.
338 points and 129 commentsSize of the thread according to the archive. The quoted comment has 376 points.
Limitation Experience from people who run IT, without measurement. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
“get it in writing from legal that you're not liable”
Booth, H., Fisher, B., Galluzzo, R., Roberts, J. NIST, NCCoE
It is a NIST draft for a demonstration project that applies identity standards to agents. It proposes to identify the agent as distinct from a person, and to authenticate and authorize it with OAuth 2.0 and policies. It also proposes to bind each agent to the person who delegates to it, and to log its actions. It asks how to know the sensitivity of the data the agent gathers, and whether the user may see the combined answer. It also asks how to set least privilege (only the permissions needed) when the agent's actions are not predictable.
Numbers
6 groups of questionsGeneral, identification, authentication, authorization, auditing and prompt injection. Comments from February 5 to April 2, 2026.
Limitation It is a draft for consultation, with questions rather than controls. It leaves RAG out of scope. The PDF (11 pages) was read; the project page returned 403.
“How do we establish "least privilege" for an agent”
The 2026 edition of BCG's study says the gap narrowed: almost half of companies already get value from AI. Many expect to give agents real autonomy, but almost none have the controls to do so today.
Numbers
about 48.5%Companies that generate value: 7.5% leaders (5% in 2025) and 41% scaling; out of 1,330 executives.
42% and 5%Expect to give agents real autonomy by 2030, and have all the controls today.
about 40%Share of AI value they expect from agents by 2030.
Limitation An announcement by a consultancy that sells AI services; self-report, with no detail on how it classifies companies. Automated download now returns 403, so it carries no verbatim quote.
It is a survey of IT and business leaders involved in AI programs. Most expect to use agents soon, but few have a mature governance model to control them. Intention moves faster than control.
Numbers
3,235 leaders in 24 countriesSurvey sample (January 2026 report).
74%Expect to use agents at least moderately by 2027.
21%Have a mature governance model for agentic AI.
Limitation Self-reported intention, not measured use. Other figures that circulate about this report (for example, how many pilots reached production) were only seen in coverage and are not used.
A thread on who owns an agent six months after the person who built it leaves. One comment compares it to the old Microsoft Access databases left loose in companies. Another says agents belong to teams, not to people. The most concrete proposal is to treat them as service accounts, which are accounts for programs. That needs an inventory of what they read and write, credentials under a role and a review every three months.
Numbers
26 points and 33 commentsSize of the thread according to the archive. The quoted comment has 5 points.
Limitation A small subreddit with some vendor comments; it is useful as an idea, not as consensus. The same idea appears in other threads on r/sysadmin, r/ITManagers and r/devops. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
“nothing goes live without a named owner, a backup owner, and a review date”
The author asks what the employee offboarding checklist does with the agents that person launched. The agent keeps running on a service account or key that nobody owns anymore. One comment proposes making the owner a required field when someone creates the key or account. It also proposes rotating the key (replacing it with a new one) before shutting the agent down, to see who depends on it.
Numbers
6 points and 7 commentsSize of the thread according to the archive.
Limitation A very low-scoring thread: it is useful as an idea, not as consensus. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote is in the post text and was compared letter by letter.
“it keeps going on a service account or key nobody owns anymore”
Reporting from a security conference, the author says agents are being sold there as “virtual employees”. The comments bring it down to earth. An agent is a model call in a loop with tools, and its “memory” is files or a database. Another warns that afterwards there is no way to analyze why it decided something. A third describes the sprawl of agents that are orphaned when the person who created them leaves.
Numbers
356 points and 261 commentsSize of the thread according to the archive. The quoted comment has 86 points.
Limitation Opinions from people who run IT, without measurement. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
A thread about who answers for what an agent does. One comment proposes three action levels: automatic, confirmed by a person and forbidden (for example, paying or deleting data). It also says the thresholds drift. After about three weeks, operators move about 40% of actions from confirm to automatic. That drift must be watched. Another comment asks to log why the agent chose each action.
Numbers
40% in about three weeksActions that move from confirm to automatic, according to one commenter.
Limitation A thread with a very low score (0 points and 16 comments in the archive). The 40% figure comes from one comment, without a method. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
“Capability without visibility into why it chose action X”
The evaluator gives you a starting point with the reasons in view. It does not replace the design review or the threat map. To put the decision in writing, use the ADR: agent, fixed workflow or person?.