The price per token does not tell you what a result costs. A token is a small piece of text, a few letters long: the provider charges for the model per token.
This calculation adds the model cost across all its attempts and the time of the person who reviews. Then it spreads that total over the tasks that go right.
Swap the values for your own. The link in the address bar keeps the calculation.
Calculation
Example values, made up, as of November 2026. They are the ones in the cost sheet. Prices change, and sometimes they go up. Use the prices of your provider, with the date.
Some values in the link were not valid. Those fields show the example value.
Model, per attempt
USD 0.10
Model, per task (all its attempts)
USD 0.15
Human review, per task
USD 1.50
Per task attempted, right or wrong
USD 1.65
Per successful task
USD 2.06
What the cost of one task is made of
Model: 9%
Human review: 91%
The success rate in this calculation is that of one task with the allowed retries. Do not use pass^k here. pass^k measures whether all k attempts go right. It works as an exit criterion in the evaluation plan.
Someone also does the tasks that fail. That cost is separate. The cost sheet keeps the calculation with its date. It also has a sensitivity table (how the cost changes when each value changes) and a budget with a hard stop.
What you need to measure is the cost of each task that succeeds, not the cost of each model call.
Verdict: Yes. Tokens, the units of text the model charges for, cost the same whether the result is right or wrong. Human review is a cost too. A game built by a single agent cost $9 and did not respond to the controls. With a three-agent harness it cost $200 and was playable.
Rule
Compute the cost per successful task. Add up the tokens for every attempt, the retries and the review hours. Divide that by the tasks that came out right. Measure it per workflow, not from the monthly bill.
Why medium confidence. The math is simple and nobody disputes it. What is missing is published measurements. The game example comes from the vendor itself, with no control group. The review figure comes from a Microsoft engineer writing about a Microsoft tool. The rest are small Reddit threads with self-reported figures. The qualifying sources show something more. The math can come out in the agent's favor when the work is split into small steps.
Show the 7 exhibits in detailHide the detail5 for, 0 against, 2 qualify
The post argues that price per token is the wrong metric, because you pay the same whether the result is right or wrong. Reliability falls as the chain of steps gets longer, so the cost of checking has to be added. The author concludes that serious work cannot be automated cost-effectively. A comment proposes something more useful: measuring cost per successful outcome.
Numbers
77%, 60% and under 36%Reliability of a 5, 10 and 20 step process with 95% success per step, if errors were independent.
Limitation A very small thread (2 points and 6 comments in the archive) and the post promotes an article by the author. The math is arithmetic, not a measurement. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
“Cost per successful outcome seems like a far more useful metric than cost per token”
It compares a single agent with a three-agent harness (planner, generator and evaluator) for building full applications. The harness gave clearly better results, but cost over 20 times more. Agents that grade themselves are too generous. The separate evaluator also failed until they tuned it to be skeptical and to test with an external tool. Each harness piece assumes something the model cannot do alone, and with a better model (Opus 4.6) they removed pieces.
Numbers
20 minutes and 9 dollars versus 6 hours and 200 dollarsA retro game with Opus 4.5: single agent (the game did not respond to controls) versus full harness (playable).
3 hours 50 minutes and 124.70 dollarsA digital audio workstation with Opus 4.6 and the simplified harness.
Limitation The vendor's own cases, with no systematic control group.
“agents reliably skew positive when grading their own work”
Ten months of data on the Copilot coding agent in the dotnet/runtime repository. More than half of its pull requests were merged and almost none were reverted, because each one went through expert review. It did best on mechanical, well-scoped tasks (cleanup, tests) and worst on those that need judgment (performance). An instructions file and build setup almost doubled success. The bottleneck moved from writing code to reviewing it.
Numbers
878 PRs, 67.9% successAgent pull requests and success rate (87.1% for PRs by Microsoft staff; populations not comparable).
0.6% vs 0.8%Merged PRs later reverted: the agent's vs the rest.
from 38.1% to 69%Success before and after adding instructions and setting up the build.
86.2% vs 55.1%Success with and without commits from a person in the PR.
5 to 9 hoursReview work created by nine PRs the author opened in a short time.
Limitation Written by a Microsoft engineer about a Microsoft tool. It is a repository with very expert reviewers. The author notes that the agent and the people do not do the same kind of tasks.
“we cannot compromise on review quality, on expert and experienced eyes validating the changes”
Someone who runs five agents in production describes how they cut spend after a USD 4,000 bill in one month. Using a cheap model first, and the expensive one only when validation fails, worked. They also trimmed the history resent on every call and cached repeated system prompts. They also used structured output and measured cost per workflow. An endpoint returning malformed JSON multiplied usage, and it only showed up when they measured per workflow.
Numbers
70% less in 6 weeksDrop in total spend, according to the author.
60% and 30%Fewer input tokens from trimming history, and less spend with caching on agents that make many requests.
5 timesUsage increase caused by an endpoint with malformed JSON.
Limitation Self-reported figures in a small thread (1 point and 16 comments in the archive). The original text was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote is in the post text and was compared letter by letter.
These are the list prices of Anthropic models, checked on November 1, 2026 and reviewed on November 2. Between the most expensive and the cheapest model of the same generation there is a 40 times gap. The page also documents costs that the table does not show. The new tokenizer splits the same text into more tokens, which are the units you pay for. US-only inference costs more, and declaring browser or computer-use tools adds thousands of input tokens per request.
Numbers
USD 4 and USD 20Claude Opus 5.5: input and output per million tokens (cache read: USD 0.20).
USD 2 and USD 10Claude Sonnet 5.5: input and output per million tokens (cache read: USD 0.10).
USD 0.10 and USD 0.50Claude Haiku 5.5, with prompts up to 100,000 tokens: input and output per million.
USD 15 and USD 75Claude Opus 4 and 4.1, the 2025 top tier: input and output per million.
about 30% more tokensEffect of the new tokenizer for the same text.
50% lessBatch API discount; it does not apply to interactive agent sessions.
1.1 timesPrice multiplier for requesting US-only inference.
Limitation Prices expire fast: check them before using these figures. Haiku 5.5 costs more with prompts over 100,000 tokens. Price per token is not cost per task: it depends on how many turns the agent takes and how much context it resends.
A mid-size SaaS describes how it built a support triage agent, which sorts requests and decides who handles them. It started with passwords and billing. The agent only answers if it can cite an article or a fact from the database. People reviewed 100% of answers for the first 30 days. Refunds, cancellations, large customers and low-confidence answers go to a person. The first version without review made up a feature that did not exist. The team did not shrink: it moved to the hard cases.
Numbers
47% and 22%Tickets resolved with no person, and tickets drafted by the agent and sent by a person, at 90 days.
USD 0.18 vs about 3.50Cost per ticket with the agent vs the previous human cost.
0.85Confidence threshold: below it, the answer goes to a person.
Limitation Self-reported numbers in a small thread (3 points and 7 comments in the archive), without independent verification. The original post text was read in the public Arctic Shift archive because reddit.com blocks automated reading.
A thread about an article that asks for control flow written in code, not in prompts. One commenter says a QA agent had to go through about 200 requirement files and broke around file 30. A deterministic harness (code that fixes each step) solved it. Another says that with the work split into small pieces, cheaper models perform like frontier models. Some complain that platforms prefer prompt-only flows.
Numbers
about 200 files, breaks around 30Case of a QA agent driven only by a prompt, according to one commenter.
590 points and 296 commentsSize of the thread.
Limitation User stories, without measured data. It was read on news.ycombinator.com and through the public Algolia API. Quotes from the thread were not checked word by word, so none is used.
Set a spending cap that stops the agent. Cost per task helps you decide, but it does not stop a stuck task. Without a cap, that task can use all the budget.
An agent in production needs a spending cap that shuts it off.
Verdict: Yes. OWASP's risk list for agents calls for cost, rate or token caps with an automatic cutoff. A team running five agents got a $4,000 bill in one month. The cost per hour climbs right near the edge of what the model can do, where it fails the most. The cap does not need to be low. Some people spend $2,000 to $5,000 a month, and their company accepts it.
Rule
Set a spending cap per task and per day that stops the agent and alerts a person. When a task hits the cap, have it stop and hand off instead of retrying.
Why medium confidence. OWASP is an expert consensus list, not a measurement. The spending cases come from Reddit and from a Hacker News thread about a 2025 essay, with self-reported or estimated figures. The opposing source does not argue against a cap: it shows that high spending can be worth it.
Show the 4 exhibits in detailHide the detail3 for, 1 against, 0 qualify
It is OWASP's first consensus list built for agents. The ten risks run from agent goal hijacking (ASI01) to rogue agents (ASI10). They include tool misuse (ASI02) and identity and privilege abuse (ASI03). They also include the agentic supply chain (ASI04) and cascading failures (ASI08). It adds the principle of least agency: do not grant autonomy that is not needed. Its mitigations include least-privilege profiles per tool, human approval for destructive actions and controls outside the model. It also asks for cost, rate or token caps with automatic cutoff.
Numbers
10 risks (ASI01 to ASI10)Size of the list. ASI02 and ASI03 extend excessive agency (LLM06) from the 2025 list to multi-step flows.
Limitation It is a voluntary consensus framework: it binds no one and measures nothing. In the agent list, unbounded consumption is no longer an item of its own: it appears as a scenario and mitigation inside ASI02. The full PDF (57 pages) was read.
“deploying agentic behavior where it is not needed expands the attack surface”
Someone who runs five agents in production describes how they cut spend after a USD 4,000 bill in one month. Using a cheap model first, and the expensive one only when validation fails, worked. They also trimmed the history resent on every call and cached repeated system prompts. They also used structured output and measured cost per workflow. An endpoint returning malformed JSON multiplied usage, and it only showed up when they measured per workflow.
Numbers
70% less in 6 weeksDrop in total spend, according to the author.
60% and 30%Fewer input tokens from trimming history, and less spend with caching on agents that make many requests.
5 timesUsage increase caused by an endpoint with malformed JSON.
Limitation Self-reported figures in a small thread (1 point and 16 comments in the archive). The original text was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote is in the post text and was compared letter by letter.
A thread about an essay by Toby Ord that uses METR data to estimate the hourly cost of agents. That cost rises sharply as the task nears the limit of what the model can do, right where it fails most. The strongest counterpoint in the thread is that cheaper open models complete the same set of tasks for much less.
Numbers
about USD 350 per hourCost of o3 at its full horizon, where it still fails about half the time.
306 points and 137 commentsSize of the thread.
Limitation The essay is from 2025 and uses that year's models. The thread was read on news.ycombinator.com and through the public Algolia API, not the full essay. Quotes from the thread were not checked word by word in the source verification, so none is used.
The author spent about USD 50 of API usage on a small weekend app. At work, a USD 20 subscription covers far more use. He concludes that today's price is subsidized, and that many agent uses stop adding up with billing for real usage. Others compare it with cheap Uber rides in 2015. In contrast, one commenter spends USD 2,000 to 5,000 a month, and his company accepts it because it multiplies his output.
Numbers
about USD 50 vs 20API spend on a small app vs a monthly subscription, according to the author.
USD 2,000 to 5,000 a monthSpend by one commenter, which his company accepts.
991 points and 457 commentsSize of the thread according to the archive. The quoted comment has 74 points.
Limitation Small, self-reported figures; the thread is an opinion about where prices are going. The quote is from the favorable counterpoint, not from the thread's thesis. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
“It's been a force multiplier so my company has no problem with that additional cost.”
Verdict: It probably speeds up output, but the size of the effect is uncertain. In a randomized experiment with an assistant, everyone improved on a business task. In open-source projects that adopted Cursor, lines added rose 281.3% in the first month. But the gain faded after two months. METR repeated its study in 2026. METR believes AI probably speeds work up more than in 2025, although it does not trust its own numbers.
Rule
Do not assume the gain: measure it on your own work, against doing the task without the tool. Measure quality again a few months later, because the early speed can leave debt behind.
Why medium confidence. The randomized experiment uses a chat assistant, not agents. The data on coding agents are observational. One is from Microsoft on its own rollout and one is on open-source projects. METR's repeat study has intervals that cross zero. METR distrusts its own figure because of selection bias. A vendor that estimates productivity from its own data shrinks it after adjusting for success.
Show the 5 exhibits in detailHide the detail2 for, 1 against, 2 qualify
Cruces, G., Fernández Meijide, D., Galiani, S., Gálvez, R. H., Lombardi, M. NBER (Working Paper 34851)
It is a randomized experiment (groups assigned at random) with a generative AI assistant, on a task to solve a business problem. Everyone improves, and people with less education improve more. The gap shrinks a lot but does not disappear, and when the AI is removed it largely returns. People with more education use the tool better.
Numbers
from 0.548 to 0.139 standard deviationsPerformance gap by education without and with AI (about three quarters closes), in 1,174 adults aged 25 to 45.
Limitation The NBER page with the abstract was read. It measures education, not experience; it is a chat assistant, not an agent, and the task is online, outside a company. It has revisions from May and October 2026.
Murphy-Hill, E., Butler, J., Savelieva, A. arXiv (Microsoft)
It studies the rollout of two terminal coding agents among tens of thousands of Microsoft engineers. Those who adopted them merged more pull requests than a synthetic control group. The most junior engineers (IC2) had the largest increase in pull requests created: +34%, against +21% for the reference level (IC4). By tenure, the gain is C-shaped: people with short and long tenure gain most. The adoption study covers only Copilot CLI: there, junior engineers tried it less at first. Use spread mostly through networks of colleagues. It does not measure quality.
Numbers
+24.0% (+14.5% to +33.7%)Pull requests merged per engineer per day versus control, over a 4-month window (95% interval).
+34% versus +21%Increase in pull requests created with 3 days of use per week, versus the same person's weeks without use. It compares level IC2 versus IC4, the reference (IC3: +21%, IC5: +25%, IC6: +22%). Read from figure 9, with Claude Code and Copilot CLI.
-13% and -14%Odds of trying Copilot CLI early for levels IC2 and IC3 versus IC4 (IC5: +22%). Copilot CLI only: the adoption study does not include Claude Code.
Limitation Observational, not randomized. The authors are from Microsoft, which owns GitHub, and they say so. A pull request is not delivered value. The gain by level compares each person's weeks with and without use: it is an association, not a cause. The adoption study includes only Copilot CLI.
“first use spread primarily through social networks”
GitHub projects that adopted Cursor speed up a lot at first, but the gain fades after two months. More complexity and more static analysis warnings (automatic code checks) remain, and that slows velocity later. Cursor speeds things up for a while and leaves technical debt (problems to fix later).
Numbers
+281.3% and +48.4%Lines added in month 1 and month 2 after adoption, in 806 repositories versus 1,380 controls.
+30.26% and +41.64%Static analysis warnings and code complexity after adoption.
Limitation Difference-in-differences on open source projects, not companies. The figures come from version 3 (January 26, 2026); the title changed between versions and it was not checked whether the results changed. The discussion mentions 25.1% complexity, which does not match the table.
“a statistically significant, large, but transient increase in project-level development velocity”
Becker, J., Rush, N., Cunningham, T., Rein, D., Mahamud, K. METR
METR repeated its study with more developers and with late-2025 tools. The numbers suggest AI now speeds up the work, but METR does not trust them. Many developers did not want to work without AI and left out tasks to avoid doing them without it. It concludes AI probably speeds up work more today than in 2025, but the data is very weak evidence of the gain's size. It also changed the experiment design.
Numbers
57 developers, 143 repositories, over 800 tasksSize of the new study (10 returning and 47 new).
-18% and -4%Change in time with AI: returning developers (interval -38% to +9%) and new ones (-15% to +9%).
+19%Change in time with AI in the original 2025 study (interval +2% to +39%).
between 30% and 50%Developers who did not submit some tasks to avoid doing them without AI.
Limitation Selection bias that METR acknowledges. Pay also went down, and people who run several agents in parallel measure their time poorly. The intervals cross zero: they do not show a certain gain.
With November 2025 data, collaboration again surpassed automation in the Claude chat. Conversations in which the person only gives orders went down. In the API, used mostly by companies, automation is still about three quarters. Task success is lower in the API, and adjusting for success shrinks the estimated productivity gain. The report says the first net effect may be deskilling, which means jobs lose the tasks that need the most skill.
Numbers
52% and 45%Collaboration and automation conversations in the chat. Directive ones fell to 32% (39% in August 2025).
67% and 49%Task success in the chat and in the API.
from 1.8 to 1.0 to 1.2 pointsEstimated annual contribution to US labor productivity growth, before and after adjusting for success.
Limitation Published by the vendor with its own data: 1 million conversations and 1 million API transcripts (November 13 to 20, 2025).
“the productivity benefits will be smaller than raw speedups suggest”