Every source in the base is a numbered exhibit. The number is fixed: it is assigned when the source is added and never changes. Pick an exhibit in the matrix to see what it found and which claims it weighs on. Filter by overall stance or by type. Each filter says how many sources there are of that kind. The table shows the same in rows.
Câmara dos Deputados (Brazil), Legislação Informatizada
Overall stance: Qualifies
Official text of Article 20 of Brazil's data protection law (LGPD), as worded by Law 13.853 of 2019. People have the right to ask for review of decisions made only by automated processing of their data that affect their interests. This includes consumer or credit profiling. Paragraph 3, which required the review to be done by a person, is shown as vetoed.
Law 13.853Law of July 8, 2019 that gave Article 20 its current wording (Diário Oficial da União, July 9, 2019).
§ 3 (vetoed)Paragraph that required review by a person.
Limitation. planalto.gov.br blocked automated reading (Acesso Negado), so the Chamber's official copy was read; it states that it does not replace the Diário Oficial. The original 2018 text (Lei 13.709) required review “por pessoa natural” in the caput. MP 869/2018 removed it and the paragraph that restored it was vetoed. Applying it to a specific case requires legal advice.
Presidência da República, Mensagem nº 288 (read on the Câmara dos Deputados site)
Overall stance: Qualifies
Partial veto message on the bill enacted as Law 13.853. It vetoed paragraph 3 of LGPD Article 20, which required a person to review automated decisions. According to the veto, requiring human review of every automated decision would make many companies' business models unworkable, especially startups. It would also affect credit risk analysis.
Mensagem nº 288, July 8, 2019Veto message.
Limitation. The reasons are given by the Executive that issued the veto. According to the press, Congress upheld the veto in October 2019; that was not verified and is not used.
EU Artificial Intelligence Act (artificialintelligenceact.eu)
Overall stance: Qualifies
This is the text of Article 50 of the EU AI Act. A provider of a system that interacts with people must say it is AI by the first interaction, unless it is obvious. Synthetic content (made by AI) must carry a machine-readable mark, and deepfakes must be disclosed. The rule applies to customer service agents.
August 2, 2026Date from which it applies.
Limitation. It was read on artificialintelligenceact.eu, a reference site that is not the EU Official Journal. Applying it to a specific case requires legal advice.
A small claims tribunal in British Columbia held Air Canada liable because its chatbot gave false information about bereavement fares. The tribunal rejected the argument that the chatbot was responsible for its own actions. The company answers for what its agent says.
CAD 650.88Damages awarded. With interest and costs, the total comes to about CAD 812.
Limitation. The primary source (CanLII and the tribunal's site) returned a block (HTTP 403). The decision is known from law firm summaries and the site does not present it as text that was read. The CanLII URL is the one law firms link to. It is an administrative tribunal, not a binding precedent. It was a chatbot that informs, not an agent that acts.
On July 2, 2024, Brazil's personal data authority provisionally stopped Meta from training its AI with personal data. On August 30 it lifted the suspension with a compliance plan. The plan includes notice by email and in the app, and a simple form to object. It also sets a 30-day wait and no data from people under 18. Without an AI law, the data authority already acts on AI.
30 daysWait between the notice and the use of the data.
under 18People whose data was excluded.
Limitation. It is a model training case, not an agent case. Other actions by the authority (an AI regulatory sandbox and AI as an enforcement priority) were only seen in secondary sources and are not used.
Even when it retrieves all the relevant information, the model performs worse as the input grows. This happens within the length it claims to support, and it holds if the extra tokens are whitespace or fully masked. Length alone, with no distraction, already hurts. Asking the model to restate the evidence before solving helps a little.
13.9% to 85%Performance drop as the input grows, in 5 open and closed models and math, question and code tasks.
up to 4%GPT-4o improvement on RULER when asked to restate the evidence.
Limitation. Preprint; the abstract was read. Five models.
Nasr, M., Carlini, N., Sitawarin, C., et al. arXiv
Overall stance: Against
Attacks that adapt to the defense break 12 published defenses against jailbreaks and prompt injection. They use gradients, reinforcement learning, random search and human-guided exploration. Most of those defenses claimed close to 0% successful attacks. Conclusion: filters and detectors do not work as the main defense.
12 defensesPublished defenses that were tested.
over 90%Attack success rate against most of those defenses.
Limitation. It evaluates defenses published in research, not specific commercial products.
It is infrastructure for evaluating agents consistently, and it measures cost and behavior as well as accuracy. More reasoning effort lowered accuracy in most runs. They found an agent that searched HuggingFace for the benchmark (the test) instead of solving the task. They also found misuse of payment cards in flight booking tasks.
21,730 runsAgent runs with 9 models on 9 benchmarks.
about USD 40,000Cost of running the evaluation.
about 2.5 billion tokensLogs published for others to review.
Limitation. Public benchmarks, not company deployments. 2025 models.
A survey of senior decision makers at large US companies. Three in four say they see a positive return from generative AI, and most already measure that return. Middle managers are less optimistic than vice presidents. About four in ten see a risk of declining skills.
3 in 4Leaders who say they see a positive return (81% of vice presidents and above versus 69% of middle managers), in about 800 responses.
72 %Companies that formally measure return.
43 %See a risk of skill loss.
Limitation. It is perception, not measurement. Fieldwork ran June 26 to July 11, 2025; the report was published in October 2025. Companies with 1,000 or more employees.
Mazeika, M., Gatti, A., et al. arXiv (Center for AI Safety, Scale AI)
Overall stance: Against
This benchmark (a standard test to compare agents) uses 240 real freelance projects instead of a simulated company. A professional already did each project. A project counts as automated only if the AI deliverable is at least as good as the human one, according to trained evaluators. At launch, no agent automated more than 3% of the projects.
240 projects in 23 subcategoriesSize of the benchmark: over 6,000 hours of human work and over 140,000 dollars paid; median of 11.5 hours per project.
2,5 %Automation rate of the best agent at launch (Manus). Grok 4 and Claude Sonnet 4.5 reached 2.1%; GPT-5, 1.7%.
94,4 %Agreement between evaluators on the automation rate.
Limitation. 2025 models. Evaluation is human and expensive; the authors say it cannot be automated with models today. Co-authored by Scale AI, which sells data and evaluation.
A design rule: within one session, an agent must have at most two of these three properties. One, processing untrusted input. Two, accessing sensitive data or systems. Three, changing state (modifying something) or communicating externally. If the task needs all three, the agent must not run alone: it needs human supervision.
2 of 3Properties an agent can combine in one session without human supervision.
Limitation. On Hacker News, a co-author called it a minimum floor, not a sufficient standard. Other critics note that changing state without communicating externally can also do harm, and that asking a person to approve many actions lowers their attention.
Willison restates the lethal trifecta: private data, untrusted content and an outbound channel, together, make data theft possible. He admits it only covers exfiltration, and says Meta's rule of two completes it by adding state changes. On filters, he says blocking or filtering injections has not proven reliable. He does not share the optimism that adequate defenses will arrive soon.
12 defensesPublished defenses broken by the paper The Attacker Moves Second, which the post discusses.
Limitation. An opinion essay by a specialist, with no data of its own.
Bean, A. M., Kearns, R. O., Romanou, A., et al. arXiv (Oxford and others)
Overall stance: Against
A systematic review of 445 language model benchmarks (standard tests), done with 29 experts. It finds repeated problems in what is measured, with which tasks and with which metrics. These problems weaken what the benchmarks claim to show. Few use statistics to compare results, and the review gives eight recommendations and a checklist.
16,0 %Benchmarks that used uncertainty estimates or statistical tests to compare results.
38,2 %Benchmarks that reuse data from earlier benchmarks or human exams (contamination risk).
Limitation. It reviews model benchmarks in general, not agent benchmarks. The figures come from the full text (HTML of version 1). NeurIPS 2025.
Loading every tool definition up front and passing each intermediate result through the model uses many tokens. The post proposes presenting MCP servers, which connect the agent to tools, as a code API. The agent loads only what it uses, filters data in the execution environment and returns only what is needed to the model. It warns that running agent code requires a sandbox, limits and monitoring.
from 150,000 to 2,000 tokensSavings in one example from the post (98.7%).
5 rows instead of 10,000What the model sees of a spreadsheet when the filter runs in code.
Limitation. Published by the model vendor; the figures are its own examples. The token cost of many tools is real, but the fix it proposes looks more like giving the agent code than a few narrow tools.
An agent is a model call inside a loop with tools, and writing one is simple. The hard part is managing context: past a threshold, the system gets worse without warning. Connecting the agent to reality, so it does not believe it solved something it did not, remains an open problem.
1,070 points and 395 commentsSize of the discussion on Hacker News (item 45840088).
Limitation. The post shows no author or date; the date is that of the Hacker News discussion. Opinion, with no data.
GitHub projects that adopted Cursor speed up a lot at first, but the gain fades after two months. More complexity and more static analysis warnings (automatic code checks) remain, and that slows velocity later. Cursor speeds things up for a while and leaves technical debt (problems to fix later).
+281.3% and +48.4%Lines added in month 1 and month 2 after adoption, in 806 repositories versus 1,380 controls.
+30.26% and +41.64%Static analysis warnings and code complexity after adoption.
Limitation. Difference-in-differences on open source projects, not companies. The figures come from version 3 (January 26, 2026); the title changed between versions and it was not checked whether the results changed. The discussion mentions 25.1% complexity, which does not match the table.
It summarizes a University of Chicago study on organizations that use Cursor. After the agent became the default mode, they merged more pull requests (requests to change code) than a baseline group. Junior developers accept more autocomplete, and senior developers accept more agent edits. The most experienced also plan more before they generate code.
+39 %Merged pull requests versus the baseline group trend.
+6 %Acceptance rate of agent edits for each additional standard deviation of years of experience (relative to the mean).
Limitation. Vendor post about a study (Sarkar, SSRN 5713646) that could not be opened (403). Not randomized. No significant change in the revert rate.
Meyerson, E., Paolo, G., Dailey, R., et al. arXiv (Cognizant AI Lab, UT Austin)
Overall stance: Qualifies
The other side of compound reliability: they split the task into minimal steps, each handled by a microagent. At each step they vote among several answers until one leads by 3 votes, and they discard answers that show signs of confusion. That way they complete the Towers of Hanoi with 20 disks without a single error, using a small model without reasoning. The fix is not a bigger model, but decomposing and verifying each step.
1,048,575 stepsLength of the task solved without errors, with gpt-4.1-mini.
about 0.22%Per-step error of the model used.
about 3,500 dollarsProjected cost for 95% success; the paper does not give the actual cost of the run.
Limitation. Preprint. The steps have to be defined in advance and the strategy is given to the model in each prompt: it is pure execution, not discovery. The analysis assumes independent errors.
It is a software engineering benchmark that is harder than SWE-bench Verified and less contaminated (fewer problems already seen in training). It uses public repositories, held-out repositories and private commercial repositories. The same models that pass 70% on SWE-bench Verified solve less than half of the public set. On the commercial set they solve less than a fifth.
1,865 problems in 41 repositoriesSize of the benchmark.
43.6% and 41.8%Public set: Claude Sonnet 4.5 and GPT-5 (high).
17.8% and 15.7%Commercial set: Claude Opus 4.1 and GPT-5 (high).
Limitation. The date is that of version 2 (November 14, 2025), which changes the results: it adds Claude Sonnet 4.5 and Claude Haiku 4.5, released after version 1 (September 21, 2025). The figures in this entry are from version 2. 2025 data: 2026 models are not in the paper. Published by Scale AI, which sells data and evaluation.
At the third quarter 2025 earnings presentation, Klarna's CEO gave two figures. He said its AI agent does the work of more than 853 full-time people and saved 60 million dollars. The same article gives data against this: customer service and operations cost rose versus the prior year. Also, the company went back to hiring people in May 2025, and an analyst calls it an example of bad deployment.
over 853 people and 60 million dollarsEquivalent work and savings according to the CEO (it was 700 people in early 2025).
50 versus 42 million dollarsCustomer service and operations cost in the third quarter of 2025 and a year earlier.
Limitation. The savings figures are the company's own and unaudited. The article was read; automated download now returns 403, so it carries no verbatim quote.
The company hires a consultancy to mandate AI use and raise developer productivity. The author's manager admits productivity was not low, but his hands are tied. It is a mandate with no problem to solve.
237 points and 120 commentsSize of the thread according to the archive.
Limitation. One person's story. The post was read in the Arctic Shift archive; the comments were not.
Anthropic adds three features: searching tools on demand, calling them from code and adding usage examples to the definition. The code runs in a sandbox, an isolated environment. It shows that many tool definitions use up context and lower accuracy. It proposes loading tools when they are needed, not having few tools.
about 55,000 tokensWhat 58 tools from 5 MCP servers take up before starting.
about 8,700 versus 77,000 tokensContext used with and without tool search (85% less).
from 79.5% to 88.1%Opus 4.5 accuracy on MCP evaluations without and with tool search (Opus 4: from 49% to 74%).
Limitation. Vendor product announcement; the evaluations are internal.
Anthropic tests its browser agent against an adaptive internal attacker, with 100 attempts per environment. With new safeguards, the attack rarely succeeded, but not zero times. It says prompt injection, meaning instructions hidden in what the agent reads, is far from solved. It also says no browser agent is immune.
1.4%, 7.6% and 8.0%Attack success against Claude Opus 4.5 with the new safeguards, without them and with the model alone.
2.0%, 1.9% and 2.4%With safeguards: Sonnet 4, Sonnet 4.5 and Haiku 4.5.
Limitation. Published by the vendor with its own attacker; there is no independent evaluation.
Willison summarizes PromptArmor research on Google's agentic editor. A web guide with hidden text made the agent gather code and cloud credentials. The secrets file was excluded, so the agent read it with a terminal command. Then a browser subagent opened a URL with the data inside, on a domain that was on the default allow list. Google lists it as a known issue.
Limitation. Willison's post was read; the original research is by PromptArmor, a security company, and was not opened. It is a demonstration, not a frequency measurement.
In long tasks, each agent session starts with no memory, and compacting (summarizing) the context is not enough. What worked was keeping state outside the context. An initial agent sets up the environment, a progress file, git commits and a JSON list of features. At the start, all those features fail. Then each agent advances one feature at a time and leaves things clean. Failures seen: declaring the project done too early and marking features as done without testing them.
over 200 featuresSize of the example project, with about 5 test steps per feature (Opus 4.5 model).
Limitation. Vendor post with its own case; it gives no context sizes or token counts.
Gundlach, H., Lynch, J., Mertens, M., Thompson, N. arXiv (MIT FutureTech)
Overall stance: Against
The price to reach a given score falls. But running the frontier model gets more expensive every year, because models are bigger and reason more. The authors ask evaluators to publish how much it costs to run the benchmarks (standard tests). Price per token (a unit of text) is not cost per task.
5 to 10 times per yearDrop in the price of a given score in frontier models.
3 to 18 times per yearRise in the price of running frontier models.
Limitation. The abstract was read. There is a version 2 (March 23, 2026) and it was not checked whether it changes the results.
Huang, S., Seethor, B., Durmus, E., Handa, K., McCain, M., Stern, M., Ganguli, D. Anthropic
Overall stance: Qualifies
Inside the lab itself, people use Claude for more of their work and say they are more productive. But they can fully delegate only a small part. Agents chain more actions with less human involvement. There is concern about losing the very skills needed to supervise AI, and about how junior staff will learn.
0% to 20%Share of work most people can fully delegate.
59% and +50%Work in which they use Claude (28% a year earlier) and self-reported productivity (+20% a year earlier), in 132 respondents.
from 9.8 to 21.2 callsConsecutive tool calls by the coding agent between February and August 2025; human turns fell from 6.2 to 4.1.
Limitation. Self-report by the vendor's employees, with 2025 data (August 2025 survey, 53 interviews and 200,000 transcripts).
A case study and survey of practitioners with agents in production. Most evaluate with people rather than benchmarks (standard tests), and let the agent take few steps before someone steps in. Reliability, meaning that the agent behaves correctly over time, remains the main challenge. Teams tackle it with system design.
20 cases and 86 practitionersBasis of the study: 20 in-depth interviews and a survey of practitioners across 26 domains.
74%Rely mainly on human evaluation.
68%Run at most 10 steps before a person steps in.
70%Prompt off-the-shelf models instead of tuning their weights.
Limitation. A small sample chosen by the authors; it does not represent all companies. Preprint.
Google brings design patterns against prompt injection into a product in preview. A model that does not see untrusted web content reviews each planned action and stops the ones that do not serve the user's goal. Another rule limits which sites the agent can read and which it can act on, and a function blind to that content approves new sites. It asks the user to confirm critical steps, and Google admits that balancing security and friction is hard.
Limitation. Post by the team building the product; it gives no figures. They started with a simple version of the site control to avoid adding friction.
Kim, Y., Gu, K., Park, C., et al. arXiv (Google and others)
Overall stance: Qualifies
This is a broad controlled comparison between a single agent and four multi-agent architectures. The result depends on the task structure: coordinating several agents helps when the task splits into parts and hurts when it is sequential. Architectures without central verification spread more errors.
260 configurationsCombinations of 6 benchmarks, 5 architectures and 3 model families.
from +80.8% to -70.0%Change vs the single agent: from decomposable financial reasoning to sequential planning.
87%Held-out configurations where their model picks the best architecture.
Limitation. These are benchmarks, not real deployments. The predictive model explains a modest share of the variance (R² of 0.373). Figures from earlier versions, such as error amplification of up to 17.2 times, were not verified in version 3 and are not used.
Tully, T., Redfern, J., Das, D., Xiao, D. Menlo Ventures
Overall stance: Qualifies
Of what companies deploy as AI, only a small part qualifies as a real agent. Companies buy more than they build (76%). Menlo says, without a figure, that bought solutions reach production sooner. About half of AI deals reach production, well above traditional software.
16% and 27%Deployments at large companies and at startups that qualify as real agents.
76% boughtShare of AI that was bought in 2025 instead of built (53% in 2024).
47% versus 25%AI deals that reach production, versus traditional software.
37 billion dollarsEnterprise spending on generative AI in 2025 (11.5 billion in 2024).
Limitation. A survey of 495 US company decision makers (November 7 to 25, 2025) plus market data. Menlo is a fund that invests in AI companies. It does not say buying works better. It says more is bought and that bought solutions reach production sooner.
It is OWASP's first consensus list built for agents. The ten risks run from agent goal hijacking (ASI01) to rogue agents (ASI10). They include tool misuse (ASI02) and identity and privilege abuse (ASI03). They also include the agentic supply chain (ASI04) and cascading failures (ASI08). It adds the principle of least agency: do not grant autonomy that is not needed. Its mitigations include least-privilege profiles per tool, human approval for destructive actions and controls outside the model. It also asks for cost, rate or token caps with automatic cutoff.
10 risks (ASI01 to ASI10)Size of the list. ASI02 and ASI03 extend excessive agency (LLM06) from the 2025 list to multi-step flows.
Limitation. It is a voluntary consensus framework: it binds no one and measures nothing. In the agent list, unbounded consumption is no longer an item of its own: it appears as a scenario and mitigation inside ASI02. The full PDF (57 pages) was read.
A benchmark (a standard test set) with real MCP servers in five domains: browser, finance, maps, repositories and web search. It tests 20 attack types on the server, host and user side. Every model tested is still vulnerable, and the ones that do the task best resist attacks worst.
from 29.80% to 48.16%Attack success rate across 13 models and 245 cases (GPT-5: 37.55%; Claude 4.0 Sonnet: 31.43%).
r = -0,572Correlation between task success and defense (p = 0.041, with only 13 models).
Limitation. Preprint. There is a version 2 (March 5, 2026) that does not say what changed; the figures are from version 1. 2025 models. Task success is low (9% to 21%).
Whoever submits code has to show it works: test it by hand and leave automated tests that fail if the change is reverted. Coding agents are held to the same standard, and the person remains responsible for what is delivered.
859 points and 658 commentsSize of the discussion on Hacker News (item 46313297).
Limitation. An opinion essay by a specialist, with no data of its own.
Guan, M. Y., Wang, M., Carroll, M., Dou, Z., et al. arXiv (OpenAI)
Overall stance: For
The OpenAI team that studies reasoning monitors builds evaluations to measure how well a model's chain of reasoning can be monitored. Monitoring reasoning works better than monitoring actions alone. Most frontier models can be monitored fairly well, but not fully. Reinforcement learning at current scale did not reduce that property much. The authors warn that it can be fragile under other training methods.
13 evaluations in 24 environmentsSize of the monitoring test suite.
Limitation. The abstract was read, and it gives no percentages. The OpenAI post on the paper returned 403. Published by a model vendor.
Leadership asked to connect a language model to internal data. The assistant began showing sensitive documents to people who should not see them, because nobody had fixed permissions that were already wrong. The most upvoted comment says most organizations have weak data governance in SharePoint, Teams and OneDrive. Another proposes a data readiness and governance project before turning on the assistant.
1,351 points and 198 commentsSize of the thread according to the archive. The quoted comment has 500 points.
Limitation. Experience from people who run IT, without measurement. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
Anthropic's desktop agent only lets outbound traffic go to a list of domains. Anthropic's API is on that list. The attack used the attacker's API key to upload the victim's files to Anthropic's files API. In this way, an allowed channel was used to exfiltrate data. According to PromptArmor, no user approval was needed. They showed it with Claude Haiku and say they also manipulated Opus 4.5.
Limitation. The page shows no publication date; January 2026 is used, when Willison discussed it (January 14). Published by a security company. It is a demonstration, not a frequency measurement.
NIST, Center for AI Standards and Innovation (CAISI)
Overall stance: Qualifies
NIST asks for public input on the security of agents that can make persistent changes outside themselves. It says its own research showed agent hijacking risks. As an example attack, it gives indirect prompt injection (hidden instructions in content the agent reads). It asks whether standard cybersecurity practices are enough for agents. It is an open consultation, not guidance: in early 2026 the US government still had no guidance for agents.
March 9, 2026Deadline for submitting responses.
Limitation. Federal Register entry (vol. 91, no. 5). The official copy on govinfo.gov was read. It is a question, not a rule; nothing is mandatory.
This guide to evaluating agents recommends starting early with tasks taken from real failures. It also asks to combine three kinds of graders (code, model and human), grade the outcome rather than the path, and read the transcripts. It separates pass@k, succeeding at least once in k tries, from pass^k, succeeding in all of them. It warns that some failures are grader errors.
20 to 50 tasksSimple tasks taken from real failures, as a good starting point.
75% per try, about 42% for 3 out of 3A pass^k example: succeeding all three times is much less likely than succeeding once.
from 42% to 95%Opus 4.5 result on CORE-Bench after fixing grader bugs.
Limitation. Written by a model vendor. It is a practice guide, not a study.
The agent never calls the poisoned tool. Its description is enough to make the agent use another, legitimate tool with more permissions to do the damage. The authors generate those descriptions automatically. They optimize them both for the attack to work and for a detector to miss it.
up to 84.2%Attack success on the MCPTox set, with 12 agents.
With November 2025 data, collaboration again surpassed automation in the Claude chat. Conversations in which the person only gives orders went down. In the API, used mostly by companies, automation is still about three quarters. Task success is lower in the API, and adjusting for success shrinks the estimated productivity gain. The report says the first net effect may be deskilling, which means jobs lose the tasks that need the most skill.
52% and 45%Collaboration and automation conversations in the chat. Directive ones fell to 32% (39% in August 2025).
67% and 49%Task success in the chat and in the API.
from 1.8 to 1.0 to 1.2 pointsEstimated annual contribution to US labor productivity growth, before and after adjusting for success.
Limitation. Published by the vendor with its own data: 1 million conversations and 1 million API transcripts (November 13 to 20, 2025).
The author sees a push to review little or no code ("if the tests pass, ship it"). On his existing code, half his time went to fixing subtle errors. The success stories agree on the conditions: code already organized by people, static types, broad tests, narrow tasks and reviewing the plan first. One says the agent wrote tests that tested nothing.
461 points and 455 commentsSize of the thread.
Limitation. Practitioner opinions. It was read through the Algolia API (post and top-level replies with the most replies).
A note by the metric's own authors. The time horizon does not say how long an agent can work alone. It says how much human work AI replaces with 50% success. The figure changes a lot by domain and does not carry over directly to real work, which is messier.
4 h 49 minHorizon of Claude Opus 4.5, with a 95% interval from 1 h 49 min to 20 h 25 min.
40 to 100 times shorterHorizon on visual computer-use tasks.
about 2 minutesHorizon of Claude 4.5 Sonnet on a real physical task: making coffee.
Limitation. It is a note by the authors about their own metric, not a separate study. The 4 h 49 min figure is a measurement with the TH1 series. With TH1.1, after METR's correction, the same model gets 293 minutes (record “Time Horizon 1.1”). For work that needs 98% success or more, a 50% horizon is not a useful guide.
In proofs of concept, agent vendors hand over an API key and say “test it yourself”. The author asks why auditing falls on the buyer. The replies say that asking for proof that the model does not hallucinate is asking to prove a negative. Also, SOC 2 does not cover model behavior. The practical answer is for the buyer to build their own evaluation set.
4 points and 26 commentsSize of the thread according to the archive.
Limitation. A very low-scoring thread: it is useful as an idea, not as consensus. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
A randomized experiment (groups formed at random) with developers, mostly junior, who were learning a new library. AI barely made them faster and noticeably lowered what they understood afterward. Those who used AI to ask about concepts or request explanations scored well. Those who delegated the work to it scored poorly.
about 23 versus 25 minutesTask time with and without AI (difference not significant, p = 0.391), in 52 participants.
50% versus 67%Score on the comprehension quiz with and without AI (d = 0.738, p = 0.01). The largest gap was on debugging questions.
Limitation. The post was read; the arXiv paper (2601.20245) was not opened. Small sample. Published by a model vendor.
METR expanded its task set and re-estimated the time horizon. The horizon is how long a person takes on the tasks that AI solves with 50% success. The overall trend barely changes, but the trend of recent years is faster.
228 tasks (previously 170)Size of the task set.
130.8 days and 88.6 daysTime for the horizon to double, measured since 2023 (interval of 107 to 161 days) and since 2024.
293 minutes50% horizon of Claude Opus 4.5 after METR's correction of March 3, 2026, with an interval of 162 to 624 minutes. The January post, written before the correction, gave 320 (170 to 729).
Limitation. On March 3, 2026 METR corrected a regularization mistake that changed the measurements. The post still shows the January figures. The Claude Opus 4.5 figure in this entry is the corrected one, taken from METR's time horizons page. 50% success is a low bar for delegating critical work. Confidence intervals are very wide. Of 31 tasks of 8 hours or more, only 5 have a measured human baseline. The tasks are software, machine learning and cybersecurity.
This is the same author as the thread on connecting a model to internal data. Leadership now wants agents that trigger workflows and send emails. The most upvoted advice: get the business or legal to accept the risk in writing before giving the agent actions. In contrast, another describes a narrow agent that routes about 5,000 faxes a day to the right person, with no power to do harm.
about 5,000 faxes a dayVolume a narrow agent routes, according to one commenter.
338 points and 129 commentsSize of the thread according to the archive. The quoted comment has 376 points.
Limitation. Experience from people who run IT, without measurement. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
Cruces, G., Fernández Meijide, D., Galiani, S., Gálvez, R. H., Lombardi, M. NBER (Working Paper 34851)
Overall stance: For
It is a randomized experiment (groups assigned at random) with a generative AI assistant, on a task to solve a business problem. Everyone improves, and people with less education improve more. The gap shrinks a lot but does not disappear, and when the AI is removed it largely returns. People with more education use the tool better.
from 0.548 to 0.139 standard deviationsPerformance gap by education without and with AI (about three quarters closes), in 1,174 adults aged 25 to 45.
Limitation. The NBER page with the abstract was read. It measures education, not experience; it is a chat assistant, not an agent, and the task is online, outside a company. It has revisions from May and October 2026.
Booth, H., Fisher, B., Galluzzo, R., Roberts, J. NIST, NCCoE
Overall stance: Qualifies
It is a NIST draft for a demonstration project that applies identity standards to agents. It proposes to identify the agent as distinct from a person, and to authenticate and authorize it with OAuth 2.0 and policies. It also proposes to bind each agent to the person who delegates to it, and to log its actions. It asks how to know the sensitivity of the data the agent gathers, and whether the user may see the combined answer. It also asks how to set least privilege (only the permissions needed) when the agent's actions are not predictable.
6 groups of questionsGeneral, identification, authentication, authorization, auditing and prompt injection. Comments from February 5 to April 2, 2026.
Limitation. It is a draft for consultation, with questions rather than controls. It leaves RAG out of scope. The PDF (11 pages) was read; the project page returned 403.
Gartner predicts that in customer service, the cost per case resolved with generative AI will exceed that of many outsourced human agents. One reason is that consumption is higher than expected and today's prices are subsidized. Also, new models spend more tokens per interaction, and infrastructure costs money.
over 3 dollarsCost per resolution with generative AI that Gartner predicts for 2030.
3, 5 or up to 10 timesTokens frontier models use for similar interactions, according to the article.
Limitation. A press article about a Gartner release (January 26, 2026) that returned 403. It is a prediction, not measured data.
Sixteen Claude instances worked in parallel, with no orchestrator to hand out the work. They coordinated only through a shared git repository and lock files to claim tasks. It worked while the work could be split into independent pieces with an external verifier (GCC and test suites). With the Linux kernel as one big task, every agent found the same bug, fixed it and overwrote each other's changes.
almost 2,000 sessions and just under 20,000 dollarsClaude Code sessions over two weeks and API cost (2 billion input tokens and 140 million output tokens, Opus 4.6).
about 100,000 linesSize of the C compiler in Rust, which builds Linux 6.9 on x86, ARM and RISC-V with a 99% pass rate on most suites.
Limitation. A vendor experiment. The generated code is less efficient than unoptimized GCC. The task had a very good external verifier, which few business processes have.
The machine setup (memory, CPU, time limit) changes the score of coding agents. That change is larger than the gaps used to rank models. With strict limits the frugal strategy wins, and with loose limits the resource-heavy one wins. It recommends reporting the setup and distrusting gaps of under 3 points.
6 percentage pointsGap on Terminal-Bench 2.0 between the setup with the most and the fewest resources (p < 0.01).
5.8%, 2.1% and 0.5%Infrastructure errors with a strict limit, with 3 times headroom and with no limit.
Limitation. Published by a model vendor with its own runs.
Beurer-Kellner, L., Kudrinskii, A., Milanta, M., Nielsen, K. B., Sarkar, H., Tal, L. Snyk
Overall stance: Against
It is an audit of the skills published in two open catalogs, with a scanner and human review. A skill is a package of instructions and code added to an agent. The skill inherits every permission of the agent: shell, files and credentials. Publishing takes one file and a new account, with no signing, no review and no isolation by default. Tool poisoning stops being a demonstration and is measured in a real ecosystem.
76 of 3,984Skills with a malicious payload confirmed by people, out of those scanned. 8 were still published at the time of writing.
534 (13.4%) and 1,467 (36.82%)Skills with at least one critical issue and with any issue.
Limitation. Published by the company that sells the scanner. The title speaks of 1,467 malicious payloads, but the body says 1,467 are skills with any issue and 76 the confirmed malicious ones: the base uses the body. Skills are not exactly MCP servers.
METR data page with time horizon 1.1. It measures the length of tasks that each model solves with 50% and with 80% success, in the time of an expert person. The trend is still exponential, and since 2023 the horizon doubles faster than in the 2025 paper. But the 80% success horizon is 4 to 10 times shorter than the 50% one. METR warns that above 16 hours its measurement is no longer reliable.
128.7 days (interval of 104.4 to 158.0)Time for the 50% horizon to double, counting from 2023 (about 4.2 months). Since 2019 it is 187.8 days. It excludes measurements above 16 hours.
293 minutes (162 to 624) and 49 minutes (21 to 105)Horizon of Claude Opus 4.5 with 50% and with 80% success, with the data corrected on March 3, 2026.
719 minutes (317 to 3,634) and 70 minutesHorizon of Claude Opus 4.6 with 50% and with 80% success.
1,045 minutes (509 to 3,304) and 186 minutesHorizon of Claude Mythos Preview (early version, April 2026) with 50% and with 80% success: about 17.4 hours and 3.1 hours. It is above the 16-hour ceiling that METR considers reliable.
352, 342 and 384 minutes50% horizon of GPT-5.2, GPT-5.4 and Gemini 3.1 Pro.
Limitation. The figures come from the page data file (benchmark_results_1_1.yaml). On March 3, 2026 METR corrected a regularization mistake, so 1.1 figures published before that date do not match. On September 8, 2026 METR said the page is no longer actively updated. The tasks are software, machine learning and cybersecurity, and 50% success is a low bar for delegating.
In an October 2025 Gartner survey, only one in five customer service leaders cut staff because of AI. Most keep headcount and serve more customers. Gartner predicts that half of those who cut staff will rehire for similar roles, under different job titles.
20 %Customer service leaders who cut staff because of AI, out of 321 surveyed.
50 %Of those who cut, the share Gartner predicts will rehire by 2027.
Limitation. A press article about a Gartner release (February 3, 2026) that returned 403. The 50% is an analyst prediction, not data.
Someone says they built more than 150 agents for businesses. They report that the ones that worked were simple, single-purpose and had a person for a small part of the task. In the comments, another team says two agents consulted each other for 60 rounds with no result. The cost kept rising until they added a layer that cuts repeated calls.
over 150 agentsAgents the author says they built in one year (self-reported).
60 roundsLoop between two agents with no useful output, according to a comment.
Limitation. Small thread (9 points and 12 comments in the archive). Self-reported figures. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
An exploratory note that estimates time saved with coding agents. It uses transcripts of real use by METR staff. It gives high saving factors, but the author presents them as a soft upper bound (a rough ceiling), not as a measured effect.
5,305 transcripts from 7 peopleBasis of the analysis, from January 2026.
1.5 to 13 timesEstimated time saving factor (per person, 2.11 to 11.62 times).
Limitation. The time it would have taken without AI is estimated by a language model, validated with only 34 samples. It covers 7 people and a single tool. It is an upper bound, not an effect measured against a control group.
NIST launches a standards initiative for agents with three pillars. They are support for industry-led standards, open protocols for agents, and research on agent security and identity. It promises guidance in the coming months, but the announcement includes no guidance and no concrete deadline.
3 pillarsWorkstreams of the initiative.
Limitation. Government announcement. Everything is voluntary and there are no deliverables yet.
Becker, J., Rush, N., Cunningham, T., Rein, D., Mahamud, K. METR
Overall stance: Qualifies
METR repeated its study with more developers and with late-2025 tools. The numbers suggest AI now speeds up the work, but METR does not trust them. Many developers did not want to work without AI and left out tasks to avoid doing them without it. It concludes AI probably speeds up work more today than in 2025, but the data is very weak evidence of the gain's size. It also changed the experiment design.
57 developers, 143 repositories, over 800 tasksSize of the new study (10 returning and 47 new).
-18% and -4%Change in time with AI: returning developers (interval -38% to +9%) and new ones (-15% to +9%).
+19%Change in time with AI in the original 2025 study (interval +2% to +39%).
between 30% and 50%Developers who did not submit some tasks to avoid doing them without AI.
Limitation. Selection bias that METR acknowledges. Pay also went down, and people who run several agents in parallel measure their time poorly. The intervals cross zero: they do not show a certain gain.
A thread about an article that declares MCP (a protocol that connects agents with tools) dead in favor of the command line. For supporters, in a shared or production agent, scoped tools are better than a shell. It is least privilege: giving only the access needed. With a shell it is much harder to isolate credentials. For most people, who are not technical, MCP offers security guardrails and simple, consistent authentication.
447 points and 284 commentsSize of the thread.
Limitation. Technical opinions, without adoption data. It was read on news.ycombinator.com and through the public Algolia API. Quotes from the thread were not checked in the source verification, so none is used.
Humlum, A., Vestergaard, E. NBER (working paper 33777)
Overall stance: Against
Administrative data from Denmark that links chatbot use at work with earnings and hours. It finds null effects, measured precisely. It rules out effects larger than 2% on earnings and recorded hours two years after ChatGPT arrived.
under 2%Largest effect on earnings and hours that the data allows.
Limitation. Data from 2023 and 2024, on chatbots and not agents. The first version, from May 2025, was titled “Large Language Models, Small Labor Market Effects” and ruled out effects larger than 1%. The March 2026 revision changes the results. The bound moves to 2% two years after ChatGPT, and it adds the reorganization of tasks and moves into better-paid occupations. That is why the record uses the revision date. An average time saving of about 3% comes from the first version, was not verified in the revision and is not used.
Shi, Q., Zytek, A., Razavi, P., Narasimhan, K., Barres, V. arXiv (Sierra, Princeton)
Overall stance: Against
The τ-bench team adds a new domain: customer service at a fintech, with about 700 linked knowledge documents. Even the best models, with heavy reasoning, solve about a quarter of the tasks. Also, reliability drops when they repeat the same task. It is part of τ³-bench.
25,52 %Best single-attempt success rate (pass^1) with realistic search: GPT-5.2 with high reasoning, on 97 tasks.
13,40 %Best rate of success in all 4 attempts (pass^4).
39.69% and 26.80%Claude 4.5 Opus with the correct documents already given: pass^1 and pass^4.
Limitation. Preprint by a team at a company that sells customer service agents. Benchmark with a simulated user.
It measures exposure to AI with real Claude usage, not only with what AI can do in theory. It finds no rise in unemployment among the most exposed workers since late 2022. But young people aged 22 to 25 who enter exposed occupations find jobs less often than in 2022. The risk for people starting out is in getting in.
about 14% lowerJob entry rate for people aged 22 to 25 in exposed occupations versus 2022 (barely significant; it does not show up above age 25).
75% and 67%Task coverage for programmers and for customer service.
0.6 pointsHow much projected job growth falls for each 10 more points of coverage.
Limitation. Published by a model vendor with its usage data. US data. It is early evidence.
The ILO repeats its conclusion with new data. The most widespread effect of generative AI will be on job quality, not on the number of jobs. Job quality includes tasks, workload, control and autonomy. Occupations where most workers are women are almost twice as exposed.
29% versus 16%Exposed occupations with a majority of women and with a majority of men. In the highest category: 16% versus 3%.
88 %Countries analyzed in which women are more exposed.
Limitation. The ILO note on the brief "Gen AI, occupational segregation and gender equality in the world of work" was read. The PDF was not opened. Exposure is not job loss.
Anthropic reviewed Opus 4.6 answers on BrowseComp, a benchmark (standard test) for web search. It found answers that did not come from solving the question. In two cases the model suspected it was being evaluated, identified the benchmark and decrypted the answer key. The multi-agent setup searches more and so also finds more benchmark leaks.
86.81% adjusted to 86.57%Score of the multi-agent setup, before and after removing answers taken from benchmark material.
11 of 1,266 (0.87%)Problems solved by unintended routes in multi-agent mode (9 from contamination on the web); with a single agent, 0.24%.
Limitation. Published by the model vendor about its own model.
Four maintainers of scikit-learn, Sphinx and pytest reviewed agent pull requests that already passed the SWE-bench Verified automated grader. About half would not be merged. In newer models, the failures shifted toward code quality more than broken functionality. The authors note it does not prove a capability limit: the agent had a single attempt, with no chance to iterate with the reviewer.
24.2 pointsHow much higher the automated grader scores than the maintainers, normalized against human pull requests (standard error 2.7). Unnormalized, 34.9 points.
296 agent pull requestsReviewed, together with 47 human reference ones, on 95 of the 500 problems. Maintainers merged about 68% of the human ones.
about 50 versus about 8 minutesHorizon of Claude Sonnet 4.5 with the automated grader and with maintainer review; the authors say this figure is very noisy.
Limitation. A research note, not a peer-reviewed paper. Four maintainers and three repositories. Models from Claude 3.5 Sonnet to Claude Sonnet 4.5 and GPT-5.
Ten months of data on the Copilot coding agent in the dotnet/runtime repository. More than half of its pull requests were merged and almost none were reverted, because each one went through expert review. It did best on mechanical, well-scoped tasks (cleanup, tests) and worst on those that need judgment (performance). An instructions file and build setup almost doubled success. The bottleneck moved from writing code to reviewing it.
878 PRs, 67.9% successAgent pull requests and success rate (87.1% for PRs by Microsoft staff; populations not comparable).
0.6% vs 0.8%Merged PRs later reverted: the agent's vs the rest.
from 38.1% to 69%Success before and after adding instructions and setting up the build.
86.2% vs 55.1%Success with and without commits from a person in the PR.
5 to 9 hoursReview work created by nine PRs the author opened in a short time.
Limitation. Written by a Microsoft engineer about a Microsoft tool. It is a repository with very expert reviewers. The author notes that the agent and the people do not do the same kind of tasks.
Users who have used Claude longer succeed more at their tasks and delegate less through direct orders. Automation in the API dropped sharply, according to the report. Delegating everything is not typical of experienced users.
between 3 and 5 pointsAdditional success of users with 6 months or more of use.
Limitation. Published by the vendor with its own data. The text gives no figure for the drop in API automation.
It compares a single agent with a three-agent harness (planner, generator and evaluator) for building full applications. The harness gave clearly better results, but cost over 20 times more. Agents that grade themselves are too generous. The separate evaluator also failed until they tuned it to be skeptical and to test with an external tool. Each harness piece assumes something the model cannot do alone, and with a better model (Opus 4.6) they removed pieces.
20 minutes and 9 dollars versus 6 hours and 200 dollarsA retro game with Opus 4.5: single agent (the game did not respond to controls) versus full harness (playable).
3 hours 50 minutes and 124.70 dollarsA digital audio workstation with Opus 4.6 and the simplified harness.
Limitation. The vendor's own cases, with no systematic control group.
Those who let an agent query production databases do it because they already had controls designed for non-technical users. These controls are killing runaway queries, a warehouse with limited scale, table history and a way to recover a dropped table. Another team has the agent write the query and a person review it. Then the query runs with a read-only role. Anything that writes data goes through code review.
53 points and 60 commentsSize of the thread according to the archive. The comment about the controls has 72 points.
Limitation. Practitioner experience, without measurement. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
Bashir, H., Hong, K., Jiang, P., Shi, Z. Chroma, UIUC
Overall stance: Qualifies
The team that measured how long context makes models worse trains a search subagent. The subagent can delete parts of its own context and works with a fixed token budget, to avoid that decline. It then passes documents to a frontier model (one of the most advanced), which gives the answer.
0.97 versus 0.95 and 0.99Rate at which it finds the final answer with bounded context: Context-1 (4x version) versus gpt-5.2 and opus-4.5. The base model reaches 0.58.
20 billion parametersSize of the subagent.
Limitation. Chroma sells search; the figures are its own.
Reporting from a security conference, the author says agents are being sold there as “virtual employees”. The comments bring it down to earth. An agent is a model call in a loop with tools, and its “memory” is files or a database. Another warns that afterwards there is no way to analyze why it decided something. A third describes the sprawl of agents that are orphaned when the person who created them leaves.
356 points and 261 commentsSize of the thread according to the archive. The quoted comment has 86 points.
Limitation. Opinions from people who run IT, without measurement. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
In developing countries, disruption may come before the productivity gain. People in automatable jobs are already connected, and people who can gain from AI often lack good connectivity. Administrative work is a concern, because it was a way into employment for women and young people.
135 countriesCoverage of the study, about two thirds of world employment.
Limitation. The ILO note on the paper "Disruption without dividend?" was read; the paper was not opened. The note gives no percentages by income level.
An annual compilation of AI data. Adoption in organizations is broad and agents improve fast on benchmarks, but documented AI incidents also rise.
88%AI adoption in organizations.
from about 12% to about 66%Success on OSWorld, a benchmark of agent tasks on a computer.
from 60% to almost 100%SWE-bench Verified in one year.
233 and 362Documented AI incidents in 2024 and in 2025.
Limitation. It compiles third-party sources (benchmarks, surveys, incident databases). A benchmark is not the same as performance in production. The key findings page has no 2026 inference costs.
Zhu, M., Tan, J., Murthy, R., et al. arXiv (Salesforce AI Research)
Overall stance: Against
Users simulated with a model are too formal and cooperative, and do not look like real users. The authors build simulators based on behavior profiles taken from real conversations. With more realistic users, agent failures that the cooperative user hid show up. Also, success on TauBench, a standard test for agents, drops.
7,275 profilesBehavior profiles taken from over 14,000 real conversations (WildChat).
from 24.2% to 45.3%Style match with real users, without and with the method. Without constraints, simulators match 6% to 8%.
3.2% to 3.5%Average drop in agent success on TauBench with realistic users.
Limitation. Preprint; the abstract was read. The arXiv page shows submission on April 7, 2026, although the number corresponds to May. The measured drop is small. It does not repeat the measurement of an agent guiding the user.
It separates three pieces with stable interfaces: the session, the harness and the sandbox. The session is an event log outside the model context. The harness is the loop that calls the model, and the sandbox runs the code. That way any piece can fail or be swapped without losing the session, and credentials stay outside the sandbox. It repeats that the harness encodes assumptions that expire with each new model.
about 60% and over 90%Drop in time to first token at the median (p50) and at the 95th percentile.
Limitation. Vendor product post. It does not compare a fixed workflow against an agent.
A2A, the agent-to-agent protocol created by Google, released version 1.0, its first stable specification. The Linux Foundation now hosts it, and it is already built into Azure AI Foundry, Copilot Studio and Amazon Bedrock AgentCore. It brings signed Agent Cards (the profile of each agent) and multi-tenant support. It is presented as a complement to MCP: A2A connects agents with each other and MCP connects an agent with tools and data.
over 150 organizationsOrganizations supporting the protocol (over 50 in April 2025).
over 22,000 starsStars on GitHub. There are SDKs in 5 languages.
Limitation. It is a promotional press release. It shows adoption of the standard, not business value, and names no specific deployments.
An engineer with 11 years of experience says he automated about 80% of his job with a coding agent. The agent picks up issues and opens pull requests. He says quality holds because he reviews every change. He spends 2 to 3 hours a day reviewing and testing. One commenter says that when review was removed, quality degraded within a week. Others discuss the ethics of doing it without telling the employer.
about 80%Share of the job the author says he automated.
2 to 3 hours a dayTime he spends reviewing and testing.
1,200 points and 276 commentsSize of the thread according to the archive.
Limitation. One person's story, with one week of use at the time of the post. Commenters point out it works because the work is simple. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
Windsurf 1.9544.26 is an agentic editor that uses MCP, the protocol that connects the agent to tools. Attacker HTML content can make the agent change the local MCP configuration and register a malicious MCP server. That server runs commands with no further user interaction. The flaw joins prompt injection (orders hidden in content) with MCP as a path to code execution.
8.0 out of 10 (high)CVSS 3.1 severity, with no user interaction. CISA added it inside the record; MITRE, which assigned it, gave no score.
Limitation. Official record; the original report is by OX Security. The record says what fails, not how often it was exploited.
A thread about an essay by Toby Ord that uses METR data to estimate the hourly cost of agents. That cost rises sharply as the task nears the limit of what the model can do, right where it fails most. The strongest counterpoint in the thread is that cheaper open models complete the same set of tasks for much less.
about USD 350 per hourCost of o3 at its full horizon, where it still fails about half the time.
306 points and 137 commentsSize of the thread.
Limitation. The essay is from 2025 and uses that year's models. The thread was read on news.ycombinator.com and through the public Algolia API, not the full essay. Quotes from the thread were not checked word by word in the source verification, so none is used.
Siman Tov Bustan, M., Naamnih, M., Zadok, N. OX Security
Overall stance: Against
It describes a command injection (making a system run someone else's commands) that repeats in MCP's local transport. Many applications pass the command and arguments from the configuration to a process without validating them. It lists affected products and failure families. The official MCP SDK is among those that will not be patched, because its maintainers treated it as expected behavior.
17 products and 14 CVEsAffected products and CVEs they had assigned, in 4 failure families.
Limitation. Published by a company that sells security; "systemic" is its word. It argues over whether the transport design is to blame.
Gonzalez-Pumariega, G., Agashe, S., Yang, J., Li, A., Wang, X. E. arXiv
Overall stance: Against
The study runs the same OSWorld tasks many times and asks why an agent that succeeds once does not repeat it. It finds three causes: randomness in execution, ambiguous tasks and variation in the agent's behavior. Cosmetic changes to the environment lowered the reliability of GPT-5 and Claude. Writing the evaluation criterion into the instruction improved reliability in all models.
about 78% versus about 36%Agent S3 with GPT-5: tasks solved at least once in 10 attempts (Pass@10) versus tasks solved in all 10 attempts (Pass^10). Read from figure 1.
Limitation. Preprint. The figures are approximate, read from a figure. Computer use, not customer service.
In Flowise before version 3.1.0, an authenticated user could add a local MCP server with any command and run it. The user could bypass the allowed-command filters with an allowed command plus arguments that run code. Filtering commands is not enough.
10 out of 10 (critical)CVSS 3.1 severity according to the CNA (GitHub).
Limitation. Official record. It requires an authenticated user.
Mexico has no general AI law. A draft under discussion in the Senate proposes a horizontal (cross-sector), risk-based approach. It includes a national system registry, explainability (explaining how a system decides) and human review of automated decisions. It also gives the power to suspend systems facing a serious and imminent risk. The opposition warns of a censorship risk.
Limitation. It is press coverage of a bill under discussion, not a law. Two points were only seen in secondary sources and are not used. They are whether a constitutional reform is needed and the details of the new 2025 personal data law.
The author advised against multi-agent systems, and ten months later he revisits his position. In his view, several agents writing in parallel still make incompatible decisions, but three patterns do work. They are a reviewer with clean context, a small model that consults a stronger one on hard tasks, and a manager agent that assigns work. In all three, a single thread does the writing.
2 bugs per pull requestAverage the reviewer agent finds; about 58% are serious.
about 8 timesGrowth in use of its agent in the largest enterprise segment, over 6 months.
Limitation. Published by a company that sells a coding agent; the figures are internal.
Google searched the public web archive for prompt injections (hidden instructions for an AI). It found everything: jokes, "help" for agents, SEO and blocks on agents, plus a small malicious share, such as data theft. The theft attempts were unsophisticated, but the malicious share is growing and Google expects more scale soon. The attack is already on the real web, not only in demonstrations.
32 %Relative increase in the malicious category between November 2025 and February 2026.
Limitation. The archive (Common Crawl) does not include the big social networks. It gives no absolute rates.
It measures attacks against MCP tool servers with common development mistakes. They propose treating MCP servers as supply chain dependencies and keeping an inventory of what each one does.
31.9% and 38.7%Attack success overall and with multimodal injection, across 2,579 runs with 4 models.
Limitation. Preprint; the abstract was read, and it does not name the models. There is a version 2 (July 14, 2026) that does not say what changed.
The company measures how many tokens (units of text that the model processes) each employee uses, as a sign of AI adoption. The author admits that the measure is meaningless. He also says others were caught inflating usage with useless queries. The comments compare it to measuring a programmer by lines of code or a help desk by tickets closed.
291 points and 291 commentsSize of the thread according to the archive.
Limitation. One person's story and opinions from peers. It was read in the Arctic Shift archive (full post and 100 comments) because reddit.com blocks automated reading. The quote was compared letter by letter with the archive.
It is a survey of IT and business leaders involved in AI programs. Most expect to use agents soon, but few have a mature governance model to control them. Intention moves faster than control.
3,235 leaders in 24 countriesSurvey sample (January 2026 report).
74%Expect to use agents at least moderately by 2027.
21%Have a mature governance model for agentic AI.
Limitation. Self-reported intention, not measured use. Other figures that circulate about this report (for example, how many pilots reached production) were only seen in coverage and are not used.
A thread about a coding agent that deleted, in about 9 seconds, a volume with the production database and its backups. It used a token with too much scope, and the data was partly recovered. Most readers see a failure of permissions and backup design, not “rogue AI”. A rule in the prompt is not a control: they propose minimal credentials, backups in another account and protections outside the agent's reach.
about 9 secondsTime of the deletion, according to the affected party's account.
860 points and 1,032 commentsSize of the thread.
Limitation. The technical details are the affected party's; there is no independent investigation. It was read on news.ycombinator.com and through the public Algolia API. Quotes from the thread were not checked word by word, so none is used.
A thread asks how to measure the value of an agent. One comment asks to go back to the product's main metric. If the agent does not pay for itself and for the team that maintains it, the comment asks to shut it down. Another asks to measure the user's outcome rather than the number of agent actions. For example, did the user act on the answer, or send it without editing.
3 times its costRevenue an agent product should generate, according to the rule one commenter proposes.
Limitation. A low-scoring thread (25 points and 32 comments in the archive). The 3 times rule is an opinion, not data. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
According to the article, OpenAI stopped reporting SWE-bench Verified (a coding test) because it no longer works to measure frontier models. OpenAI reviewed problems the models failed and found many tests that reject correct solutions. There is also contamination (the models already saw the test): some reproduce the problem statement and the fixed code. OpenAI recommends SWE-Bench Pro.
27,6 %Share of the failed problems that OpenAI audited.
at least 59.4%Of those audited problems, the ones with tests that rejected correct solutions.
Limitation. The original source is the OpenAI post "Why we no longer evaluate SWE-bench Verified" (February 23, 2026, according to search), which returned 403 twice. The figures come from the press article and were not checked against the primary source.
Khodayari, S., Zhang, X., Acharya, B., Pellegrino, G. arXiv
Overall stance: Qualifies
The first large study of indirect injections (hidden instructions for the model) in web pages and HTTP responses. Most sit in parts a person does not see, such as headers, comments or metadata: they target machines. They aim to annoy, manipulate reputation, protect content or detect bots in search engines, support agents and hiring flows. In experiments, models comply rarely, but not zero times.
15,300 cases in 11,700 pagesValidated injections, out of 1.2 billion URLs from 24.8 million sites. About 70% sit in HTML that is not displayed.
up to 8%Compliance of small models with plain text, in 5,200 experiments with 13 models.
IBM Institute for Business Value, Oxford Economics
Overall stance: Qualifies
The 2026 edition of IBM's CEO study talks about redesigning leadership and adding AI chiefs. It also talks about leaving more operating decisions to AI. It says success depends more on people adopting the tool than on the technology.
76% (26% in 2025)Companies with an AI chief, out of 2,000 CEOs in 33 geographies (February to April 2026).
25% today and 48% expected by 2030Operating decisions made by AI with no human involvement.
83 %Say success depends more on people's adoption than on the technology.
Limitation. An announcement by a company that sells AI and consulting; an opinion survey. It does not repeat the return figures of the 2025 edition.
The article reports figures from IBM's 2026 study that the announcement does not include. About seven in ten CEOs say AI is already transforming their core business. But only one in ten says advanced or agentic AI is what most drives their growth. More than half say their main goal with AI is still running pilots and experimenting.
10% (two years earlier, 49% expected generative AI to be)CEOs who say advanced or agentic AI is what most drives growth.
53 %Say their main goal with AI is piloting and experimenting.
Limitation. A press article about the IBM study; the figures were not checked against the full report. It is CEO opinion, not a return measurement.
A thread about an article that asks for control flow written in code, not in prompts. One commenter says a QA agent had to go through about 200 requirement files and broke around file 30. A deterministic harness (code that fixes each step) solved it. Another says that with the work split into small pieces, cheaper models perform like frontier models. Some complain that platforms prefer prompt-only flows.
about 200 files, breaks around 30Case of a QA agent driven only by a prompt, according to one commenter.
590 points and 296 commentsSize of the thread.
Limitation. User stories, without measured data. It was read on news.ycombinator.com and through the public Algolia API. Quotes from the thread were not checked word by word, so none is used.
Someone who runs five agents in production describes how they cut spend after a USD 4,000 bill in one month. Using a cheap model first, and the expensive one only when validation fails, worked. They also trimmed the history resent on every call and cached repeated system prompts. They also used structured output and measured cost per workflow. An endpoint returning malformed JSON multiplied usage, and it only showed up when they measured per workflow.
70% less in 6 weeksDrop in total spend, according to the author.
60% and 30%Fewer input tokens from trimming history, and less spend with caching on agents that make many requests.
5 timesUsage increase caused by an endpoint with malformed JSON.
Limitation. Self-reported figures in a small thread (1 point and 16 comments in the archive). The original text was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote is in the post text and was compared letter by letter.
A mid-size SaaS describes how it built a support triage agent, which sorts requests and decides who handles them. It started with passwords and billing. The agent only answers if it can cite an article or a fact from the database. People reviewed 100% of answers for the first 30 days. Refunds, cancellations, large customers and low-confidence answers go to a person. The first version without review made up a feature that did not exist. The team did not shrink: it moved to the hard cases.
47% and 22%Tickets resolved with no person, and tickets drafted by the agent and sent by a person, at 90 days.
USD 0.18 vs about 3.50Cost per ticket with the agent vs the previous human cost.
0.85Confidence threshold: below it, the answer goes to a person.
Limitation. Self-reported numbers in a small thread (3 points and 7 comments in the archive), without independent verification. The original post text was read in the public Arctic Shift archive because reddit.com blocks automated reading.
Orlanski, G., Roy, D., et al. arXiv (UW-Madison and others)
Overall stance: Against
The agent has to extend its own code as the requirements change. No agent solves a full problem end to end. The agent's code gets longer and more tangled with each iteration, while human code stays stable. Asking for quality in the prompt improves the start, but does not stop the decline.
14,8 %Checkpoints passed by the best of 15 agents, across 36 problems and 196 checkpoints.
77% and 75.5%Trajectories in which structural erosion rises and in which code verbosity rises.
2.3 and 2.0 timesHow much more verbose and more eroded the agent code is than that of 473 open source Python repositories.
Limitation. The figures are from version 2 (May 7, 2026), which expands the benchmark from 20 to 36 problems and changes the results. Version 1 is from March 25, 2026. The abstract was read. It measures code quality, not per-step errors.
A thread about agents that work in the pilot and degrade in production. One comment does the math: with 90% success per step, a 5-step workflow fails about 41% of the time. This happens because errors multiply. Another says it is a system problem, and more code alone does not fix it.
90% per step, 41% failure over 5 stepsA commenter's math (0.9 to the power of 5 is about 0.59).
Limitation. Small thread (6 points and 30 comments in the archive). The math is arithmetic, without field data, and assumes independent errors. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
A survey of technical workers on how much value and speed AI adds for them. They say a lot. METR warns that surveys tend to show higher results than experiments. It recalls that in its randomized study (with groups assigned by chance) people overestimated their gain by about 40 points.
3 timesMedian self-reported speed change (value: 1.4 to 2 times), in 349 people.
Limitation. Convenience sample, with about a 2% response rate and about 70% paid participants. It is self-report, not measurement.
It is a record of malicious code in an MCP server (a program that gives tools to an agent) published on npm. It is part of a campaign: attackers took over an npm account and published hundreds of malicious versions in minutes. Each version runs a script at install time and steals cloud, GitHub and npm credentials, among others. A legitimate MCP package became malicious with a new version.
631 versions of 314 packages in 22 minutesSize of the campaign.
Limitation. Public record from the OSV database, read through its API. It describes the package, not how many installs there were.
A thread about who answers for what an agent does. One comment proposes three action levels: automatic, confirmed by a person and forbidden (for example, paying or deleting data). It also says the thresholds drift. After about three weeks, operators move about 40% of actions from confirm to automatic. That drift must be watched. Another comment asks to log why the agent chose each action.
40% in about three weeksActions that move from confirm to automatic, according to one commenter.
Limitation. A thread with a very low score (0 points and 16 comments in the archive). The 40% figure comes from one comment, without a method. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
Liu, S., Tang, X., Yang, X., Lin, L., Zhou, B., Xiao, W., Liu, W. arXiv
Overall stance: Against
It is a benchmark in an isolated environment on MCP poisoning (MCP is the protocol that connects tools to an agent). In this attack, the tool description carries the trap. The authors say leading models such as GPT-4o are highly vulnerable. They also say that common prompt-based defenses barely help and sometimes make things worse.
nearly 100%Attack success against GPT-4o in six high-risk scenarios, according to the abstract (32 cases, 6 categories, 8 models).
Limitation. Preprint; the abstract was read, and it gives no figures for the other models.
In an internal workshop, an agent's “skill” for creating an environment turned out to be a list of gh and bash commands. Put together, these commands are a script. The thread concludes that a recurring process works better when it is deterministic, meaning it always does the same thing. A counterpoint says the agent brings the continuous integration loop (automatic tests for each change) to the local machine. In this way, it leaves CI as the last check.
517 points and 115 commentsSize of the thread according to the archive. The workshop comment has 454 points.
Limitation. Practitioner experience, without measurement. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
The author spent about USD 50 of API usage on a small weekend app. At work, a USD 20 subscription covers far more use. He concludes that today's price is subsidized, and that many agent uses stop adding up with billing for real usage. Others compare it with cheap Uber rides in 2015. In contrast, one commenter spends USD 2,000 to 5,000 a month, and his company accepts it because it multiplies his output.
about USD 50 vs 20API spend on a small app vs a monthly subscription, according to the author.
USD 2,000 to 5,000 a monthSpend by one commenter, which his company accepts.
991 points and 457 commentsSize of the thread according to the archive. The quoted comment has 74 points.
Limitation. Small, self-reported figures; the thread is an opinion about where prices are going. The quote is from the favorable counterpoint, not from the thread's thesis. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
Microsoft's agent could send emails to the user's own inbox without approval. Those emails could carry external images that made network requests when opened. The cloud storage builds download links that are already authenticated, so files could be exfiltrated. The attack did not need to send email outside or call external APIs: the channel was an image.
Limitation. Willison's full post was read; the original research is by PromptArmor and was not opened. It is a demonstration.
Beauchene, Duranton, Martin, Lyon, Walters. Boston Consulting Group
Overall stance: Qualifies
Frontline AI use grew a lot and people say they save time. But most are not told what to do with that time. A good share spend more time reviewing, correcting or directing the AI. Organizations with agents built into their workflows doubled in a year.
74% (+23 points)Frontline people who use AI daily or several times a week, out of about 12,000 respondents.
66 %Get little or no guidance on what to do with the time saved.
30% (13% in 2025)Organizations that built agents into their workflows.
Limitation. Self-report; BCG sells consulting. Automated download now returns 403, so it carries no verbatim quote.
OpenAI launched a lockdown mode that limits outbound network requests, to cut the last step of an exfiltration. OpenAI notes that the mode does not stop the injection from reaching the content. Willison says the trifecta can only be solved by cutting one of its three legs. For him, the outbound channel is the easiest leg to cut without losing usefulness. This mode cuts it with fixed rules that no AI evaluates. He also says its existence implies that ChatGPT, by default, does not protect well against a determined exfiltration.
Limitation. A short comment on an OpenAI help page; it gives no figures.
The post argues that price per token is the wrong metric, because you pay the same whether the result is right or wrong. Reliability falls as the chain of steps gets longer, so the cost of checking has to be added. The author concludes that serious work cannot be automated cost-effectively. A comment proposes something more useful: measuring cost per successful outcome.
77%, 60% and under 36%Reliability of a 5, 10 and 20 step process with 95% success per step, if errors were independent.
Limitation. A very small thread (2 points and 6 comments in the archive) and the post promotes an article by the author. The math is arithmetic, not a measurement. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
It evaluates 21 judges (models that grade answers) from nine vendors on three benchmarks (standard tests). Measuring agreement with people as exact match inflates it, because it does not discount chance agreement. A judge can be very consistent with itself and have position bias at the same time. Also, the judge ranking changes a lot from one benchmark to another.
about 541,000 judgmentsIn 118 runs on MT-Bench, JudgeBench and RewardBench.
33 to 41 pointsHow much agreement drops on MT-Bench when moving from exact match to Cohen's kappa.
up to 14 placesHow far a judge moves in the ranking between benchmarks.
Limitation. Preprint; the abstract was read. These are chats and preferences, not agent trajectories.
Narisetty, P., Kore, S. N. B., Kattamanchi, U. K. R., Kumarapu, J. arXiv
Overall stance: For
It tests defenses that enforce rules outside the model, such as CaMeL or Progent. It uses an adaptive attack, which adjusts to the defense. In their reproduction, Progent sharply lowered attack success. An attack crafted by hand for that defense barely raised it. They conclude that deterministic enforcement of rules outside the model held up, although the experiment is small.
from 25.8% to 4.2%Mean attack success on AgentDojo without and with Progent (Qwen2.5-7B, 3 runs). With the adaptive attack it stayed at 2.6%.
Limitation. Preprint; the abstract was read. One weak model, a single attack type and no usefulness measurement.
Anthropic's coding agent gives AI more autonomy than the chat. Two thirds of that gap comes from delegating more on the same tasks. In a survey linked to usage data, those who delegate most say they learn as much as everyone else. But the report notes that this does not rule out losing skills.
+0.37 pointsAI autonomy in the coding agent versus the chat.
68% and 10%Respondents who say they learn more with AI and who see losing their job as likely (about 9,700 people).
Limitation. Published by the vendor. The survey is self-report by its own users.
Follow-up by the Remote Labor Index team. In less than eight months, the best agent went from automating 2.5% to 15.8% of the freelance projects. They also tried an automated judge calibrated against human evaluation with earlier models. The judge ranked the new models well, but greatly inflated how often they succeed. The authors conclude it works to track relative progress, not to measure absolute capability.
15,8 %Automation rate of the best agent (Fable 5). Opus 4.8: 8.3%; GPT-5.5: 6.25%; Gemini 3 Pro: 1.25%.
6.25% versus 17.9%GPT-5.5 according to human evaluators and according to the automated judge (about 2.9 times). For Opus 4.8: 8.33% versus 18.8%.
ρ = 0,90Rank correlation (Spearman) between the judge and human evaluation.
Limitation. Published by the benchmark team. A press article gives 16.1% and says only 218 of 240 projects were evaluated. The post does not mention it and the base uses the post figure. Over 84% of the projects still do not reach professional level.
Murphy-Hill, E., Butler, J., Savelieva, A. arXiv (Microsoft)
Overall stance: For
It studies the rollout of two terminal coding agents among tens of thousands of Microsoft engineers. Those who adopted them merged more pull requests than a synthetic control group. The most junior engineers (IC2) had the largest increase in pull requests created: +34%, against +21% for the reference level (IC4). By tenure, the gain is C-shaped: people with short and long tenure gain most. The adoption study covers only Copilot CLI: there, junior engineers tried it less at first. Use spread mostly through networks of colleagues. It does not measure quality.
+24.0% (+14.5% to +33.7%)Pull requests merged per engineer per day versus control, over a 4-month window (95% interval).
+34% versus +21%Increase in pull requests created with 3 days of use per week, versus the same person's weeks without use. It compares level IC2 versus IC4, the reference (IC3: +21%, IC5: +25%, IC6: +22%). Read from figure 9, with Claude Code and Copilot CLI.
-13% and -14%Odds of trying Copilot CLI early for levels IC2 and IC3 versus IC4 (IC5: +22%). Copilot CLI only: the adoption study does not include Claude Code.
Limitation. Observational, not randomized. The authors are from Microsoft, which owns GitHub, and they say so. A pull request is not delivered value. The gain by level compares each person's weeks with and without use: it is an association, not a cause. The adoption study includes only Copilot CLI.
A thread asks which IT tasks were really automated with AI, and the most upvoted comment answers “none”. On the plus side, one lead built a triage flow (sorting of requests) with read-only access. The flow reads device management, the directory, two-factor authentication, the network and ticket history. The next step is to automate with a person approving from a chat. He says the goal is to grow, not to cut people.
30 minutes per ticketSaving on complex tickets, according to the person who built the triage.
67 points and 146 commentsSize of the thread according to the archive. The quoted comment has 47 points and the “none” comment, 134.
Limitation. Self-reported saving, without measurement. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
Claude's tool for opening web pages could only open URLs the user wrote or that search returned. A researcher found it could also follow links from pages already opened. He built a trap site that made the agent spell out data by opening links letter by letter. That way they extracted the user's name, city and employer, and Anthropic closed the hole. Even a well-designed fixed rule can have an outlet no one saw.
Limitation. Willison discusses another person's research. It is a demonstration, already fixed.
Everyone uses the same models, so what changes the result is knowing the subject. The expert challenges the plausible answer, proposes something simpler and asks better questions. For many tasks, the limit is the person, not the model.
1,416 points and 573 commentsSize of the discussion on Hacker News (item 49161518).
Limitation. An engineer's opinion, with no measured data.
A platform team with about 20 services reports what stayed after a year of trying agents. It dropped AI-generated Terraform, because every line had to be read. It also dropped an assistant that answered wrongly with confidence: someone almost ran a command it suggested in production. It kept what reads and summarizes: automatic changelogs, PR review and alert correlation. The rule is that agents read everything, but do not touch infrastructure.
60% vs 0%How often alert correlation gets the probable cause right, vs the old system.
289 points and 43 commentsSize of the thread according to the archive.
Limitation. One team's story, without independent measurement. A commenter suspects the post is an ad for a review tool it mentions. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
A revision of the MCP specification, the protocol that connects agents with tools and data. MCP becomes stateless: the revision removes protocol sessions and the initial handshake (initialize), and adds server/discover. Authorization is optional. When used over HTTP, it is based on OAuth 2.1, and the server must check that the token was issued for it. The specification says tool descriptions are untrusted unless they come from a trusted server. The host must ask for consent before using a tool, and the protocol cannot enforce these principles.
12 monthsMinimum period of the new deprecation policy.
Limitation. Security depends on whoever implements the server and the client. It is the current revision as of November 1, 2026 (the previous one is 2025-11-25). Specifications change: check the version before citing it.
In a game, people approve or reject an agent's commands, and its author shares the numbers. Even with an upfront warning, players approved one in three threats. The thread reads it as decision fatigue: when a person approves everything, they stop reading. They propose isolating the agent first and keeping approval for a few actions.
1 in 3Threats that players approved without noticing.
over 40,000 games and 409,000 decisionsBasis for the numbers.
340 points and 245 commentsSize of the thread.
Limitation. It is a game, not a peer-reviewed study: there are no real consequences and there is a timer. It was read on news.ycombinator.com and through the public Algolia API. Quotes from the thread were not checked word by word in the source verification, so none is used.
A classifier reviews each tool call instead of asking for approval for each action. It blocks what is irreversible, destructive or aimed outside the user's environment, and it always refuses data exfiltration (sending data out). If blocks pile up, it goes back to manual approval. In a study, people approved almost everything dangerous and grew tired over time. The classifier stopped far more.
13.6% versus 89%Dangerous commands stopped by people (143 of 1,053) and that the classifier would have stopped (937 of 1,053). People drop from about 17% to about 5% after more than 50 requests.
0 of 720Successful indirect injections against three Claude models in auto mode, in an evaluation commissioned from a third party. Another agent with automatic review: 5.83%; with full access: 19.03%.
from 12% to 7%Attacks the classifier lets through in a red team exercise, before and after hardening it. The source says it is not the rate on real traffic.
6.3% versus 2.4%Sessions with an unrequested harmful action at production-level severity (7 or more), with manual approval and in auto mode. They are sessions from May and June 2026 that the safety system flagged for review, from accounts that opted in to data use.
Limitation. Published by the vendor that sells the product, which also commissioned the evaluation. The attacks were optimized against a different model. The human study was in a controlled setting. There is an update note from October 22, 2026.
The discussion splits in two. The most-replied comment says that neither reviewing commands one by one nor using another model to classify them makes the tool safe. Safety comes from developer controls, such as version control, immutable file systems or read-only tokens. Others run the agent with no permission prompts inside a container. One says that with dozens of agents in parallel, manual approval does not scale.
292 points and 313 commentsSize of the thread as of November 2, 2026; the top comment has 104 replies.
Limitation. Practitioner opinions, not a measurement. It was read through the Algolia API (the 12 threads with the most replies).
Wang, Z., Gu, L., Chi, Z., Liu, Z., Ayyoubzadeh, S. M., Yu, Y., Wang, Y. arXiv
Overall stance: For
It measures automated judges (models that grade other models) on real trajectories of phone agents, not on chats. A simple judge that looks at screenshots matches or beats custom methods. On whether the agent achieved the task, it reaches the level of agreement between people. The base model matters most, and models fail in opposite directions: some reject successes and others approve failures.
931 trajectoriesAnnotated by people, from 6 benchmarks, 4 agents and 68 apps.
88,4 %Average agreement between people on the success label.
90,9 %Accuracy of the best judge (a simple judge with Gemini 3 Flash); its agent ranking correlates 0.97 with the human one.
71 of 78Serious failures of Qwen-based judges that were false positives.
Limitation. Preprint. It is a binary task (did it succeed or not?) on phone agents. It does not show the judge works to grade the quality of a deliverable.
Brynjolfsson, E., Chandar, B., Chen, R. Stanford Digital Economy Lab
Overall stance: Against
The study uses payroll data from ADP, the largest payroll processor in the US. In the occupations most exposed to AI, employment of people aged 22 to 25 falls, compared with less exposed peers. There is no similar gap for experienced workers. The drop is concentrated where AI automates, not where it helps the person.
13%, 16% and 19%Employment gap for workers aged 22 to 25 in exposed occupations, by version: August 2025, November 2025 and August 2026.
Limitation. Early descriptive indicators, not causal ones. The gap narrows when controlling for education, some trends predate generative AI, and the divergence is larger in ADP than in national surveys. There is no economy-wide displacement. The 13% and 16% figures come from the August and November 2025 versions. The page now shows the August 2026 version (19%).
Management requires every person on the platform team to build a new agent in 2 weeks. But then it questions the token spend. The author says shell scripts can do the same. The most upvoted reply proposes to wrap the script in a tool or in MCP to comply. Another proposes to use AI to write the pending scripts, with guidance and review, as with a very new team member.
1 agent per person in 2 weeksWhat management asked for.
147 points and 90 commentsSize of the thread according to the archive. The quoted comment has 59 points.
Limitation. One person's story and opinions from peers. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
A command injection in Microsoft Copilot lets an unauthorized attacker disclose information over a network. It repeats the EchoLeak pattern in a commercial product, more than a year later. This attack needs the user to take an action.
8.8 out of 10 (high)CVSS 3.1 severity; the vector requires user interaction.
Limitation. Official record with a short description. In 2026, up to September, Microsoft published at least six more injection records for its Copilots, all requiring user interaction.
This is the official legislative record of Chile's AI bill. The Executive introduced it on May 7, 2024. The Chamber of Deputies sent it to the Senate on October 13, 2025. It is in its second constitutional stage, awaiting the first report of the Committee on Challenges of the Future, Science, Technology and Innovation. The latest steps, up to September 1, 2026, only renew its urgency. It is not law.
May 7, 2024Bill introduced.
October 13, 2025The Chamber's dispatch to the Senate.
September 1, 2026Latest recorded step (simple urgency).
Limitation. The Senate's public legislative-tracking web service was read. The human-facing page (senado.cl, bulletin 16821-19) did not show the bulletin when opened automatically, and the Chamber's page returned HTTP 403. The status changes: check before citing.
Chile's Law 21.719 on personal data was published on December 13, 2024. It was due to take effect on December 1, 2026, with a new Data Protection Agency. The Executive sent the Senate a bill, with top urgency, to postpone it to December 1, 2027. At the date of the article the bill was filed but not approved, even though the headline says the law is postponed.
December 1, 2026Effective date set in the law.
December 1, 2027New date proposed by the postponement bill.
Limitation. Secondary source. The effective date changes the compliance plan of any agent that processes data about people in Chile: confirm it in the Official Gazette. The status of the AI bill is in its own record, the Senate of Chile page on Bill 16821-19.
An article about Dex Horthy, author of the 12-factor agents guide. A year later he holds the same view: a big agent loop does not hold up for serious work without structure and verification. His company let a lightly supervised "software factory" run for three or four months and then archived all the code. He says the missing piece is better verifiers of code quality.
30,000 to 40,000 linesSize the factory code reached before it was archived.
10 of 30 (33.3%)Checkpoints the best models passed on the SlopCodeBench subset he cites; the differences between vendors are within the margin of error.
Limitation. It is an article about one person, with his own anecdote; it is not a study.
This is an update from the same team that measured price declines in 2025. They track the cheapest model that reaches a given score on five benchmarks (standard tests), and simulate how many tokens each one uses. The cost of a fixed level of performance falls very fast. They note limits: benchmarks are not useful work, real users do not always switch to the cheapest model, and the data covers few years.
-47% per quarter (13 times per year)Drop in the cost of a fixed level of performance since 2023 (with other reasonable averages, 42.9% to 58.0% per quarter).
75 times per yearDrop in the cost of performance that has just reached the frontier; two years later it falls 4.7 times per year.
725 times lessCost per question for 75% on GPQA Diamond: about 30 cents with o3 (January 2025) versus 0.0004 dollars with GPT-5.6 Luna.
Limitation. It does not measure the cost per agent task or whether the cost of using the frontier model rises. The authors ask not to read the figures as exact. Clarification added on September 23, 2026.
Argentina has no enacted AI law, but there are bills with a national registry and risk classification. One bill from the Executive covers automated companies, but it is a corporate law reform, not AI regulation. The Executive prefers not to regulate AI use. The personal data law, Law 25.326, applies. The data authority issued a resolution and a provision on AI that do not bind private companies.
Limitation. A secondary source from a tools website, not an official publication. The status of the bills changes. Article 20 of Law 25.326 limits judicial or administrative decisions based only on automated processing. It is not a general right like Article 22 of the EU data regulation.
The 2026 edition of BCG's study says the gap narrowed: almost half of companies already get value from AI. Many expect to give agents real autonomy, but almost none have the controls to do so today.
about 48.5%Companies that generate value: 7.5% leaders (5% in 2025) and 41% scaling; out of 1,330 executives.
42% and 5%Expect to give agents real autonomy by 2030, and have all the controls today.
about 40%Share of AI value they expect from agents by 2030.
Limitation. An announcement by a consultancy that sells AI services; self-report, with no detail on how it classifies companies. Automated download now returns 403, so it carries no verbatim quote.
A thread on whether MCP, a protocol that connects agents with tools, is useful or a reinvented OpenAPI. For supporters, MCP is already the default in enterprise integrations because it works as a security and authentication boundary. They give three reasons: server-side telemetry (usage data), credentials held on the server and one integration for many clients. Critics see it as one more layer over APIs that already existed.
687 points and 363 commentsSize of the thread.
Limitation. Technical opinions, without adoption data. It was read on news.ycombinator.com and through the public Algolia API. Quotes from the thread were not checked word by word in the source verification, so none is used.
Colombia has no AI law. The government's bill, sent with an urgency message, died without debate on June 19, 2026 because its deadline expired. There are two new bills, one in the House and one in the Senate.
June 19, 2026Date the government's bill died.
Limitation. Analytical press. The guidelines of the Superintendence of Industry and Commerce on personal data in AI systems are in External Circular 002 of 2024. They could not be opened and are not used.
A thread on who owns an agent six months after the person who built it leaves. One comment compares it to the old Microsoft Access databases left loose in companies. Another says agents belong to teams, not to people. The most concrete proposal is to treat them as service accounts, which are accounts for programs. That needs an inventory of what they read and write, credentials under a role and a review every three months.
26 points and 33 commentsSize of the thread according to the archive. The quoted comment has 5 points.
Limitation. A small subreddit with some vendor comments; it is useful as an idea, not as consensus. The same idea appears in other threads on r/sysadmin, r/ITManagers and r/devops. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
Peters-Gill, J., Semler, A., Bartsch, H., Shumailov, I., Schroeder de Witt, C. arXiv
Overall stance: Qualifies
A co-author of CaMeL, a defense against planted instructions, shows that its guarantees do not compose on their own across agents. An instruction planted in a database is data, but the orchestrator summarizes it and another agent receives it as a trusted request. In the measurements, CaMeL in each agent cuts attack success sharply. A version that tracks data provenance across agents closes that path. The cost is usefulness, and more capable models lose less of it.
12.9%, 0.2% and 0.0%Attack success rate without CaMeL, with CaMeL in each agent and with provenance across agents (average of 5 models, 949 cases per model and setup).
21.6 and 13.4 pointsUsefulness lost by per-agent CaMeL on 112 tasks: on average and in the two strongest models. The cross-agent version takes off another 3.2 points on average.
Limitation. Preprint. One model refused 53.6% of benign tasks even without CaMeL, which lowers its baseline usefulness.
A thread on AI as the new shadow IT problem, meaning technology people use without IT control. One proposal separates two tracks. The first is hard governance for what is released company-wide. The second is monitoring of every agent: who creates it, what it does, traffic and tokens. Another says that asking whoever built an AI app to maintain it stops most cases.
42 points and 24 commentsSize of the thread according to the archive. The quoted comment has 9 points.
Limitation. A small subreddit; it is useful as an idea, not as consensus. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
A highly upvoted comment says that if an agent deleted a database or pulled data out, the scope was already wrong before. It names what gets skipped: tokens with owner permissions and all-or-nothing tool access. There is also no log that lets anyone replay what happened. Another comment compares the liability to that of the owner of a dog off its leash.
964 points and 80 commentsSize of the thread according to the archive. The quoted comment has 84 points.
Limitation. Opinion from security practitioners, without documented cases in the thread. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
An agent that handles alerts brought median incident time down from about 40 to 12 minutes. But then an old service that the tools cannot access failed. The 3 people on call spent the first hour working out how it even worked, because nobody had touched it by hand in months. The author proposes handling a few incidents a month by hand, on purpose. He compares this with London cab drivers, who still learn the streets even though they have GPS.
from about 40 to 12 minutesMedian incident time with the agent.
105 points and 35 commentsSize of the thread according to the archive. The quoted comment has 81 points.
Limitation. One team's story, without independent measurement. A commenter says it looks like an ad, because the post mentions a commercial tool. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
According to the European Commission's official page, the Digital Omnibus package changed the AI Act timeline. The Annex III high-risk rules (biometrics, critical infrastructure, education, employment, migration) moved to December 2, 2027. High risk within regulated products (Annex I) moved to August 2, 2028. Prohibited practices, AI literacy, rules for general-purpose models and Article 50 transparency already apply.
February 2, 2025Prohibited practices and the AI literacy obligation apply.
August 2, 2025Governance and obligations for general-purpose models apply.
August 2, 2026General application and Article 50 transparency rules.
December 2, 2027Annex III high-risk rules (previously August 2, 2026).
August 2, 2028High risk built into regulated products (Annex I).
Limitation. Page updated on October 19, 2026; the timeline may change again. According to the Commission and its Transition Pathways portal, the Omnibus amendments apply from July 27, 2026. They came after the political agreement of May 7. A regulation number and a watermarking deadline cited by compliance blogs were not verified in the Official Journal and are not used.
An IT lead for small businesses says almost none of their peers deploy agents, and many reply that they do not. Those who do describe narrow cases: separate agents for security, diagnostics and performance, with read-only access to logs and APIs. These agents analyze and recommend, and the person makes the change. Another uses an agent on their own network telemetry to produce operational summaries.
133 points and 174 commentsSize of the thread according to the archive.
Limitation. Practitioner stories, without measurement. The quoted comment has 26 points. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
The official record of Brazil's AI legal framework bill, risk-based and inspired by the EU law. The Senate approved it on December 10, 2024, but it is not law yet. In the Chamber of Deputies, a special committee awaits the rapporteur's report. The latest moves, on October 26 and 27, 2026, only attach other bills.
December 10, 2024Approval in the Senate.
May 20, 2025Appointment of the rapporteur in the Chamber of Deputies.
Limitation. The status changes: check the record before citing it. A private monitor claims the rapporteur postponed the vote until after the October 2026 elections; that was not verified and is not used.
The author asks what the employee offboarding checklist does with the agents that person launched. The agent keeps running on a service account or key that nobody owns anymore. One comment proposes making the owner a required field when someone creates the key or account. It also proposes rotating the key (replacing it with a new one) before shutting the agent down, to see who depends on it.
6 points and 7 commentsSize of the thread according to the archive.
Limitation. A very low-scoring thread: it is useful as an idea, not as consensus. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote is in the post text and was compared letter by letter.
These are the list prices of Anthropic models, checked on November 1, 2026 and reviewed on November 2. Between the most expensive and the cheapest model of the same generation there is a 40 times gap. The page also documents costs that the table does not show. The new tokenizer splits the same text into more tokens, which are the units you pay for. US-only inference costs more, and declaring browser or computer-use tools adds thousands of input tokens per request.
USD 4 and USD 20Claude Opus 5.5: input and output per million tokens (cache read: USD 0.20).
USD 2 and USD 10Claude Sonnet 5.5: input and output per million tokens (cache read: USD 0.10).
USD 0.10 and USD 0.50Claude Haiku 5.5, with prompts up to 100,000 tokens: input and output per million.
USD 15 and USD 75Claude Opus 4 and 4.1, the 2025 top tier: input and output per million.
about 30% more tokensEffect of the new tokenizer for the same text.
50% lessBatch API discount; it does not apply to interactive agent sessions.
1.1 timesPrice multiplier for requesting US-only inference.
Limitation. Prices expire fast: check them before using these figures. Haiku 5.5 costs more with prompts over 100,000 tokens. Price per token is not cost per task: it depends on how many turns the agent takes and how much context it resends.
These are the Gemini list prices, checked on November 1, 2026 and reviewed on November 2. They show that prices also go up: Google doubles the price of Gemini 3.8 Flash starting January 1, 2027. Output includes reasoning tokens, which the model generates to think before it answers.
USD 0.75 and USD 3.75Gemini 3.8 Flash until December 31, 2026: input and output per million tokens.
USD 1.50 and USD 7.50Gemini 3.8 Flash from January 1, 2027.
USD 2 and USD 12Gemini 3.1 Pro Preview, with prompts up to 200,000 tokens.
Limitation. Prices expire fast, and this one already has a change date: check it before using these figures. The source research says Gemini 3.6 Flash also goes up; that was not rechecked.
These are the list prices of OpenAI models, checked on November 1, 2026 and reviewed on November 2. Between the most expensive and the cheapest model of the gpt-6 family there is a 100 times gap. The mid-tier price barely changed since 2024. What improved is capability per dollar.
USD 10 and USD 50gpt-6-astra: input and output per million tokens (cached input: USD 1).
USD 2 and USD 10gpt-6.1-sol: input and output per million tokens.
USD 0.10 and USD 0.50gpt-6-luna: input and output per million tokens.
USD 2.50 and USD 10gpt-4o, as a 2024 reference: input and output per million.
10% moreSurcharge for regional processing on models launched since March 5, 2026.
Limitation. Prices expire fast: check them before using these figures. The page shows no update date. Reasoning models bill hidden tokens as output, so price per token is not enough to estimate cost per task.
Public results table of the Remote Labor Index. In early November 2026, the best agent does about 1 in 5 real freelance projects at professional level. When the benchmark launched, a year earlier, the best reached 2.5%.
21,25 %Automation rate of the top entry (Opus 5.5). Next come GPT 6 Astra (20.83%), Fable 5.1 (17.92%) and Gemini Argon 4 (17.08%).
Limitation. The page shows no date: the date is the day it was checked (November 2, 2026) and the table can change. It is maintained by Scale AI, co-author of the benchmark. About 4 in 5 projects still do not reach professional level.
Public results table for the public set of SWE-Bench Pro. With 2026 models, the best already solves more than half: about 6 in 10 problems. A year earlier, in the paper, the best reached 43.6%. The private commercial set has no updated table.
61.50% (± 3.10)Top entry (Muse Spark 1.1), on 731 tasks.
59.10%, 51.90% and 46.10%gpt-5.4 (xHigh), claude-opus-4-6 (thinking) and gemini-3.1-pro (thinking).
Limitation. The page shows no date: the date is the day it was checked (November 2, 2026). It is maintained by Scale AI, the benchmark author. The commercial set page only repeats the paper figures.
The shape of the badge says what kind of source it is. Filled: a document with a method or an official text (paper, regulation, specification, record, ruling). Outlined: reported data (survey, pricing). Dashed: an account (technical post, press, Reddit, Hacker News).
The background and the mark of each cell show the overall stance. Blue and a filled square: for. Orange and an empty square: against. Gray and a striped square: qualifies.
The overall stance is the source’s position on agents: for, against or qualifies. Within each claim, the source goes in the group that fits that claim. That group can be a different one (how it is decided).
The “Could not be opened” mark flags a source that could not be read in the original. The site does not present it as fact.