In November 2025, Klarna presented its third-quarter results. Its CEO talked about the company’s AI agent (exhibit 21). He said it does the work of more than 853 full-time employees. He also said it had saved 60 million dollars.
over 853 people and 60 million dollars
Equivalent work and savings according to the CEO (it was 700 people in early 2025). Exhibit 21
The line sets an idea in place: agents will do the work of entire teams. That idea raises three questions. What happens to jobs? What happens to what people know how to do once they stop doing it? And who answers when the agent gets it wrong?
Klarna’s figures come from the company itself and are unaudited. But they show the volume of work an agent can take on.
A smaller case points the same way. A software company described on Reddit how it built a support triage agent (exhibit 99). Triage means sorting tickets by type and urgency. After 90 days, the agent resolved 47% of tickets on its own. Cost per ticket fell from about 3.50 to 0.18 dollars. These are self-reported numbers in a small thread, but the design was careful.
And there is a first signal in employment. Stanford’s Digital Economy Lab used payroll data from ADP (exhibit 126). ADP is the largest payroll processor in the United States. The study looks at the occupations most exposed to AI. There, employment of people aged 22 to 25 is falling relative to their less exposed peers.
The gap grew with each version of the study: 13% in August 2025, 16% in November and 19% in August 2026. The decline is where AI automates, not where it assists the person.
13%, 16% and 19%
Employment gap for workers aged 22 to 25 in exposed occupations, by version: August 2025, November 2025 and August 2026. Exhibit 126
The same article on Klarna includes the counterweights (exhibit 21). Customer service and operations costs went up: 50 million dollars in the quarter, against 42 a year earlier. And the company started hiring people again in May 2025.
Klarna is not the rule. Gartner surveyed 321 customer service leaders (exhibit 56). Only 20% cut staff because of AI. Most keep their headcount while serving more customers.
The support case that went well tells a similar story. That team did not shrink: it moved to the hard cases. Refunds, cancellations and large customers still go to a person. And the first version, without review, invented a product feature that did not exist.
Verdict: The announcements moved faster than the results. Klarna's CEO says its agent does the work of more than 853 people. But the company started hiring people again in May 2025. In a Gartner survey, only 20% of customer service leaders cut staff because of AI. In a support case from a small thread, the team did not shrink: it moved to the hard cases.
Rule
Design the agent to resolve the simple cases and hand the hard ones to people. Do not cut the team before you have measured several months of quality with the agent running.
Why medium confidence. The Klarna case is reported by the press, with unaudited company figures. The Gartner survey is known from a press article. The claim that half of those who cut staff will rehire by 2027 is a prediction. The ILO and a study by a model vendor point the same way. Tasks and job quality change more than the number of jobs. But they measure exposure, not layoffs.
Show the 5 exhibits in detailHide the detail1 for, 1 against, 3 qualify
At the third quarter 2025 earnings presentation, Klarna's CEO gave two figures. He said its AI agent does the work of more than 853 full-time people and saved 60 million dollars. The same article gives data against this: customer service and operations cost rose versus the prior year. Also, the company went back to hiring people in May 2025, and an analyst calls it an example of bad deployment.
Numbers
over 853 people and 60 million dollarsEquivalent work and savings according to the CEO (it was 700 people in early 2025).
50 versus 42 million dollarsCustomer service and operations cost in the third quarter of 2025 and a year earlier.
Limitation The savings figures are the company's own and unaudited. The article was read; automated download now returns 403, so it carries no verbatim quote.
In an October 2025 Gartner survey, only one in five customer service leaders cut staff because of AI. Most keep headcount and serve more customers. Gartner predicts that half of those who cut staff will rehire for similar roles, under different job titles.
Numbers
20 %Customer service leaders who cut staff because of AI, out of 321 surveyed.
50 %Of those who cut, the share Gartner predicts will rehire by 2027.
Limitation A press article about a Gartner release (February 3, 2026) that returned 403. The 50% is an analyst prediction, not data.
“AI simply isn’t mature enough to fully replace the expertise, empathy, and judgment”
A mid-size SaaS describes how it built a support triage agent, which sorts requests and decides who handles them. It started with passwords and billing. The agent only answers if it can cite an article or a fact from the database. People reviewed 100% of answers for the first 30 days. Refunds, cancellations, large customers and low-confidence answers go to a person. The first version without review made up a feature that did not exist. The team did not shrink: it moved to the hard cases.
Numbers
47% and 22%Tickets resolved with no person, and tickets drafted by the agent and sent by a person, at 90 days.
USD 0.18 vs about 3.50Cost per ticket with the agent vs the previous human cost.
0.85Confidence threshold: below it, the answer goes to a person.
Limitation Self-reported numbers in a small thread (3 points and 7 comments in the archive), without independent verification. The original post text was read in the public Arctic Shift archive because reddit.com blocks automated reading.
It measures exposure to AI with real Claude usage, not only with what AI can do in theory. It finds no rise in unemployment among the most exposed workers since late 2022. But young people aged 22 to 25 who enter exposed occupations find jobs less often than in 2022. The risk for people starting out is in getting in.
Numbers
about 14% lowerJob entry rate for people aged 22 to 25 in exposed occupations versus 2022 (barely significant; it does not show up above age 25).
75% and 67%Task coverage for programmers and for customer service.
0.6 pointsHow much projected job growth falls for each 10 more points of coverage.
Limitation Published by a model vendor with its usage data. US data. It is early evidence.
“We find no systematic increase in unemployment for highly exposed workers since late 2022”
The ILO repeats its conclusion with new data. The most widespread effect of generative AI will be on job quality, not on the number of jobs. Job quality includes tasks, workload, control and autonomy. Occupations where most workers are women are almost twice as exposed.
Numbers
29% versus 16%Exposed occupations with a majority of women and with a majority of men. In the highest category: 16% versus 3%.
88 %Countries analyzed in which women are more exposed.
Limitation The ILO note on the brief "Gen AI, occupational segregation and gender equality in the world of work" was read. The PDF was not opened. Exposure is not job loss.
“The impact of generative AI on women’s jobs is not predetermined”
Quote checked against the original
Read at the source. Checked on Nov 2, 2026.
What about employment in general? Anthropic measured occupational exposure using real usage of its model (exhibit 64). It finds no rise in unemployment among the most exposed workers since late 2022. This is a vendor measuring with its own data.
The Danish data from the previous chapter point the same way (exhibit 62). Two years after ChatGPT arrived, the effects on earnings and hours stay below 2%. It is chatbot data, but it is administrative data, not a survey.
The International Labour Organization (ILO) expects a different main effect (exhibit 65). For the ILO, AI will change the quality of work more than the number of jobs. That covers tasks, workload and autonomy. A joint ILO and World Bank study adds a risk (exhibit 74). In developing countries, disruption may arrive before the productivity gain.
Generative AI will put a large share of people out of work.
Verdict: For now, the data do not show it. A study using real usage data finds no rise in unemployment among the most exposed workers since late 2022. In Denmark, the effects on earnings and hours stay below 2%. The warning sign is at the entry point. Young people aged 22 to 25 have a harder time getting jobs in exposed occupations.
Rule
Before cutting a role, list which tasks the agent changes and which stay with the person. Prefer designs where the agent helps someone over ones that replace the whole task.
Why medium confidence. There are primary sources on both sides. But the Danish data are about chatbots in 2023 and 2024. The exposure study is published by a model vendor, with US data. The data on young workers are descriptive, not causal. The ILO says the most widespread effect will be on job quality. Together with the World Bank, it also warns of a risk for developing countries. There, disruption may arrive before the productivity gain.
Show the 5 exhibits in detailHide the detail1 for, 2 against, 2 qualify
Brynjolfsson, E., Chandar, B., Chen, R. Stanford Digital Economy Lab
The study uses payroll data from ADP, the largest payroll processor in the US. In the occupations most exposed to AI, employment of people aged 22 to 25 falls, compared with less exposed peers. There is no similar gap for experienced workers. The drop is concentrated where AI automates, not where it helps the person.
Numbers
13%, 16% and 19%Employment gap for workers aged 22 to 25 in exposed occupations, by version: August 2025, November 2025 and August 2026.
Limitation Early descriptive indicators, not causal ones. The gap narrows when controlling for education, some trends predate generative AI, and the divergence is larger in ADP than in national surveys. There is no economy-wide displacement. The 13% and 16% figures come from the August and November 2025 versions. The page now shows the August 2026 version (19%).
It measures exposure to AI with real Claude usage, not only with what AI can do in theory. It finds no rise in unemployment among the most exposed workers since late 2022. But young people aged 22 to 25 who enter exposed occupations find jobs less often than in 2022. The risk for people starting out is in getting in.
Numbers
about 14% lowerJob entry rate for people aged 22 to 25 in exposed occupations versus 2022 (barely significant; it does not show up above age 25).
75% and 67%Task coverage for programmers and for customer service.
0.6 pointsHow much projected job growth falls for each 10 more points of coverage.
Limitation Published by a model vendor with its usage data. US data. It is early evidence.
“We find no systematic increase in unemployment for highly exposed workers since late 2022”
Humlum, A., Vestergaard, E. NBER (working paper 33777)
Administrative data from Denmark that links chatbot use at work with earnings and hours. It finds null effects, measured precisely. It rules out effects larger than 2% on earnings and recorded hours two years after ChatGPT arrived.
Numbers
under 2%Largest effect on earnings and hours that the data allows.
Limitation Data from 2023 and 2024, on chatbots and not agents. The first version, from May 2025, was titled “Large Language Models, Small Labor Market Effects” and ruled out effects larger than 1%. The March 2026 revision changes the results. The bound moves to 2% two years after ChatGPT, and it adds the reorganization of tasks and moves into better-paid occupations. That is why the record uses the revision date. An average time saving of about 3% comes from the first version, was not verified in the revision and is not used.
The ILO repeats its conclusion with new data. The most widespread effect of generative AI will be on job quality, not on the number of jobs. Job quality includes tasks, workload, control and autonomy. Occupations where most workers are women are almost twice as exposed.
Numbers
29% versus 16%Exposed occupations with a majority of women and with a majority of men. In the highest category: 16% versus 3%.
88 %Countries analyzed in which women are more exposed.
Limitation The ILO note on the brief "Gen AI, occupational segregation and gender equality in the world of work" was read. The PDF was not opened. Exposure is not job loss.
“The impact of generative AI on women’s jobs is not predetermined”
In developing countries, disruption may come before the productivity gain. People in automatable jobs are already connected, and people who can gain from AI often lack good connectivity. Administrative work is a concern, because it was a way into employment for women and young people.
Numbers
135 countriesCoverage of the study, about two thirds of world employment.
Limitation The ILO note on the paper "Disruption without dividend?" was read; the paper was not opened. The note gives no percentages by income level.
“disruption may materialize faster than productivity gains”
Quote checked against the original
Read at the source. Checked on Nov 2, 2026.
The signal shows up at the entry point to work. According to Anthropic, people aged 22 to 25 enter exposed occupations less often. The drop is about 14% compared with 2022. The difference is barely significant, and it does not appear among those over 25.
Many people say AI helps junior staff the most. With a chat assistant, that holds. In a randomized experiment, the performance gap by education got smaller (exhibit 49). It fell from 0.548 to 0.139 standard deviations. A standard deviation measures how far results spread from the average. But the study measures education, not experience, and it took place outside a company.
With coding agents, the signal is mixed. In Microsoft’s rollout, the most junior engineers had the largest increase in pull requests (exhibit 116). It was +34%, against +21% for the reference level. By tenure, the gain is C-shaped: those with the least and the most tenure gain more. But junior engineers tried Copilot CLI less at first.
In the data Cursor publishes, more experienced developers accept more of what the agent edits (exhibit 18). They also plan more before generating code.
Sean Goedecke offers an explanation in a widely discussed essay on Hacker News (exhibit 119). Everyone uses the same models, so what changes the result is knowing the subject. In his words: “For many tasks, the human is the bottleneck, not the model”. It is an opinion, with no measured data.
AI levels people up: it helps junior staff the most.
Verdict: With a chat assistant, it levels. In an experiment, the performance gap by education fell from 0.548 to 0.139 standard deviations. With coding agents, the signal is mixed. At Microsoft, junior engineers had the largest increase in pull requests (+34% versus +21%). But they tried Copilot CLI less at first. At Cursor, senior developers accept more of what the agent edits. And in the most exposed occupations, employment of people aged 22 to 25 falls. The comparison is with their less exposed peers.
Rule
If you add agents, decide how junior staff will learn: tasks they do without the tool and review with someone experienced. Do not close entry-level roles just because the agent does those tasks.
Why medium confidence. The experiment that shows leveling is randomized. But it measures education rather than experience, with a chat assistant and outside a company. The data on agents are observational and come from vendors. At Microsoft, the gain by level compares weeks of the same person, and it counts pull requests, not quality. The employment data are descriptive, not causal. A small vendor experiment shows that junior developers who delegate learn less. A widely discussed essay argues that AI rewards those who already know.
Show the 6 exhibits in detailHide the detail2 for, 4 against, 0 qualify
Cruces, G., Fernández Meijide, D., Galiani, S., Gálvez, R. H., Lombardi, M. NBER (Working Paper 34851)
It is a randomized experiment (groups assigned at random) with a generative AI assistant, on a task to solve a business problem. Everyone improves, and people with less education improve more. The gap shrinks a lot but does not disappear, and when the AI is removed it largely returns. People with more education use the tool better.
Numbers
from 0.548 to 0.139 standard deviationsPerformance gap by education without and with AI (about three quarters closes), in 1,174 adults aged 25 to 45.
Limitation The NBER page with the abstract was read. It measures education, not experience; it is a chat assistant, not an agent, and the task is online, outside a company. It has revisions from May and October 2026.
Murphy-Hill, E., Butler, J., Savelieva, A. arXiv (Microsoft)
It studies the rollout of two terminal coding agents among tens of thousands of Microsoft engineers. Those who adopted them merged more pull requests than a synthetic control group. The most junior engineers (IC2) had the largest increase in pull requests created: +34%, against +21% for the reference level (IC4). By tenure, the gain is C-shaped: people with short and long tenure gain most. The adoption study covers only Copilot CLI: there, junior engineers tried it less at first. Use spread mostly through networks of colleagues. It does not measure quality.
Numbers
+24.0% (+14.5% to +33.7%)Pull requests merged per engineer per day versus control, over a 4-month window (95% interval).
+34% versus +21%Increase in pull requests created with 3 days of use per week, versus the same person's weeks without use. It compares level IC2 versus IC4, the reference (IC3: +21%, IC5: +25%, IC6: +22%). Read from figure 9, with Claude Code and Copilot CLI.
-13% and -14%Odds of trying Copilot CLI early for levels IC2 and IC3 versus IC4 (IC5: +22%). Copilot CLI only: the adoption study does not include Claude Code.
Limitation Observational, not randomized. The authors are from Microsoft, which owns GitHub, and they say so. A pull request is not delivered value. The gain by level compares each person's weeks with and without use: it is an association, not a cause. The adoption study includes only Copilot CLI.
“first use spread primarily through social networks”
It summarizes a University of Chicago study on organizations that use Cursor. After the agent became the default mode, they merged more pull requests (requests to change code) than a baseline group. Junior developers accept more autocomplete, and senior developers accept more agent edits. The most experienced also plan more before they generate code.
Numbers
+39 %Merged pull requests versus the baseline group trend.
+6 %Acceptance rate of agent edits for each additional standard deviation of years of experience (relative to the mean).
Limitation Vendor post about a study (Sarkar, SSRN 5713646) that could not be opened (403). Not randomized. No significant change in the revert rate.
“more experienced developers are more likely to plan an action before generating code”
Brynjolfsson, E., Chandar, B., Chen, R. Stanford Digital Economy Lab
The study uses payroll data from ADP, the largest payroll processor in the US. In the occupations most exposed to AI, employment of people aged 22 to 25 falls, compared with less exposed peers. There is no similar gap for experienced workers. The drop is concentrated where AI automates, not where it helps the person.
Numbers
13%, 16% and 19%Employment gap for workers aged 22 to 25 in exposed occupations, by version: August 2025, November 2025 and August 2026.
Limitation Early descriptive indicators, not causal ones. The gap narrows when controlling for education, some trends predate generative AI, and the divergence is larger in ADP than in national surveys. There is no economy-wide displacement. The 13% and 16% figures come from the August and November 2025 versions. The page now shows the August 2026 version (19%).
A randomized experiment (groups formed at random) with developers, mostly junior, who were learning a new library. AI barely made them faster and noticeably lowered what they understood afterward. Those who used AI to ask about concepts or request explanations scored well. Those who delegated the work to it scored poorly.
Numbers
about 23 versus 25 minutesTask time with and without AI (difference not significant, p = 0.391), in 52 participants.
50% versus 67%Score on the comprehension quiz with and without AI (d = 0.738, p = 0.01). The largest gap was on debugging questions.
Limitation The post was read; the arXiv paper (2601.20245) was not opened. Small sample. Published by a model vendor.
“using AI assistance led to a statistically significant decrease in mastery”
Everyone uses the same models, so what changes the result is knowing the subject. The expert challenges the plausible answer, proposes something simpler and asks better questions. For many tasks, the limit is the person, not the model.
Numbers
1,416 points and 573 commentsSize of the discussion on Hacker News (item 49161518).
Limitation An engineer's opinion, with no measured data.
“For many tasks, the human is the bottleneck, not the model”
Quote checked against the original
Read at the source. Checked on Nov 2, 2026.
The second question is what happens to what people know how to do. Here there is now an experiment. Anthropic randomly assigned 52 developers who were learning a new library (exhibit 46). Most of them were junior. AI did not make them significantly faster. But on the comprehension test they scored 50%, against 67% without AI.
Score on the comprehension quiz with and without AI (d = 0.738, p = 0.01). The largest gap was on debugging questions.Exhibit 46
Comprehension quiz score, in %
With AI50%
Without AI67%
0100%
Those who used AI to ask about concepts scored well. Those who handed the work off scored poorly. The sample is small, and a model vendor published it.
In a survey inside Anthropic, one employee put it this way: “I am for sure atrophying in my skills as a software engineer” (exhibit 28). The Reddit team from the previous chapter also lived through it (exhibit 142). Its agent handled incidents. One day an old service went down, and the agent could not reach it. The 3 people on call spent the first hour working out how it worked. Nobody had touched it by hand in months.
Usage reports add nuance. In an Anthropic survey, those who delegate the most say they learn as much as everyone else (exhibit 114). The same report adds: “the data do not rule out skill erosion”. Another report looks at users with more time on the tool (exhibit 69). They succeed more often and delegate less through direct orders.
Verdict: There are signals, and one now comes from an experiment. Developers, mostly junior, were learning a new library. With AI they scored 50% on the comprehension test, against 67% without AI. Those who delegate the most say they learn just as much. But the same report notes this does not rule out skill loss.
Rule
Have the team handle some cases by hand every month, on purpose. Make sure it can still do the work without the agent. When someone is learning something new, have them use AI to ask questions and understand, not to hand off the work.
Why medium confidence. The experiment is randomized but small, with 52 people, and it is published by a model vendor. In it, those who used AI to ask about concepts scored well. The surveys are self-reported, including the one inside the lab itself. In that survey, an employee says they feel their skills atrophying. The incident case is a Reddit account. No source measures the loss in a team that has used agents for months.
Show the 6 exhibits in detailHide the detail3 for, 0 against, 3 qualify
A randomized experiment (groups formed at random) with developers, mostly junior, who were learning a new library. AI barely made them faster and noticeably lowered what they understood afterward. Those who used AI to ask about concepts or request explanations scored well. Those who delegated the work to it scored poorly.
Numbers
about 23 versus 25 minutesTask time with and without AI (difference not significant, p = 0.391), in 52 participants.
50% versus 67%Score on the comprehension quiz with and without AI (d = 0.738, p = 0.01). The largest gap was on debugging questions.
Limitation The post was read; the arXiv paper (2601.20245) was not opened. Small sample. Published by a model vendor.
“using AI assistance led to a statistically significant decrease in mastery”
Huang, S., Seethor, B., Durmus, E., Handa, K., McCain, M., Stern, M., Ganguli, D. Anthropic
Inside the lab itself, people use Claude for more of their work and say they are more productive. But they can fully delegate only a small part. Agents chain more actions with less human involvement. There is concern about losing the very skills needed to supervise AI, and about how junior staff will learn.
Numbers
0% to 20%Share of work most people can fully delegate.
59% and +50%Work in which they use Claude (28% a year earlier) and self-reported productivity (+20% a year earlier), in 132 respondents.
from 9.8 to 21.2 callsConsecutive tool calls by the coding agent between February and August 2025; human turns fell from 6.2 to 4.1.
Limitation Self-report by the vendor's employees, with 2025 data (August 2025 survey, 53 interviews and 200,000 transcripts).
“I am for sure atrophying in my skills as a software engineer”
An agent that handles alerts brought median incident time down from about 40 to 12 minutes. But then an old service that the tools cannot access failed. The 3 people on call spent the first hour working out how it even worked, because nobody had touched it by hand in months. The author proposes handling a few incidents a month by hand, on purpose. He compares this with London cab drivers, who still learn the streets even though they have GPS.
Numbers
from about 40 to 12 minutesMedian incident time with the agent.
105 points and 35 commentsSize of the thread according to the archive. The quoted comment has 81 points.
Limitation One team's story, without independent measurement. A commenter says it looks like an ad, because the post mentions a commercial tool. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
“All the Star Trek engineers still actually knew how things worked.”
Anthropic's coding agent gives AI more autonomy than the chat. Two thirds of that gap comes from delegating more on the same tasks. In a survey linked to usage data, those who delegate most say they learn as much as everyone else. But the report notes that this does not rule out losing skills.
Numbers
+0.37 pointsAI autonomy in the coding agent versus the chat.
68% and 10%Respondents who say they learn more with AI and who see losing their job as likely (about 9,700 people).
Limitation Published by the vendor. The survey is self-report by its own users.
Users who have used Claude longer succeed more at their tasks and delegate less through direct orders. Automation in the API dropped sharply, according to the report. Delegating everything is not typical of experienced users.
Numbers
between 3 and 5 pointsAdditional success of users with 6 months or more of use.
Limitation Published by the vendor with its own data. The text gives no figure for the drop in API automation.
A survey of senior decision makers at large US companies. Three in four say they see a positive return from generative AI, and most already measure that return. Middle managers are less optimistic than vice presidents. About four in ten see a risk of declining skills.
Numbers
3 in 4Leaders who say they see a positive return (81% of vice presidents and above versus 69% of middle managers), in about 800 responses.
72 %Companies that formally measure return.
43 %See a risk of skill loss.
Limitation It is perception, not measurement. Fieldwork ran June 26 to July 11, 2025; the report was published in October 2025. Companies with 1,000 or more employees.
“Three out of four leaders see positive returns”
Quote checked against the original
Read at the source. Checked on Nov 2, 2026.
The third question is who answers. In a thread for IT buyers, someone asked a question (exhibit 45). Why does auditing an agent fall on the buyer? The answers were practical. A model hallucinates when it makes up false facts. Asking for proof that it does not hallucinate is asking someone to prove a negative. The way out is for the buyer to build its own test set.
A tribunal has already answered one version of this question. In 2024, a British Columbia small claims tribunal ruled against Air Canada (exhibit 4). The airline’s chatbot gave false information about bereavement fares. According to summaries by law firms, the tribunal rejected the idea that the chatbot answered for its own actions. It ordered Air Canada to pay 650.88 Canadian dollars in damages.
That ruling has three limits. It could not be read in the primary source. It is not binding precedent, which means other courts do not have to follow it. And it involves a chatbot that informs, not an agent that acts.
A closer ethical question remains. The engineer who automated about 80% of his job did not tell his company (exhibit 79). In his thread, several people debated whether that was acceptable.
If the agent gets it wrong, the mistake belongs to the agent or the vendor, not the company.
Verdict: The company is accountable. According to summaries by law firms, a British Columbia tribunal made Air Canada pay. The reason was false information from its chatbot. The tribunal rejected the idea that the chatbot answered for its own actions. In a buyers' thread, the conclusion is that today the testing falls on whoever uses the agent.
Rule
Treat what the agent says or does as if the company said or did it. Whatever you would not sign, do not let out without review. If someone automates their own job, they must say so and review what they deliver.
Why low confidence. The ruling could not be read in the primary source. It comes from a small claims tribunal and is not binding precedent. Also, it involves a chatbot that informs, not an agent that acts. A widely discussed essay says the same about code. Whoever delivers it is responsible for it, even if an agent wrote it. The demand on vendors and the ethics debate come from Reddit threads.
Show the 4 exhibits in detailHide the detail1 for, 2 against, 1 qualifies
In proofs of concept, agent vendors hand over an API key and say “test it yourself”. The author asks why auditing falls on the buyer. The replies say that asking for proof that the model does not hallucinate is asking to prove a negative. Also, SOC 2 does not cover model behavior. The practical answer is for the buyer to build their own evaluation set.
Numbers
4 points and 26 commentsSize of the thread according to the archive.
Limitation A very low-scoring thread: it is useful as an idea, not as consensus. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
“there are no frameworks for doing any kind of an audit of model behavior.”
A small claims tribunal in British Columbia held Air Canada liable because its chatbot gave false information about bereavement fares. The tribunal rejected the argument that the chatbot was responsible for its own actions. The company answers for what its agent says.
Numbers
CAD 650.88Damages awarded. With interest and costs, the total comes to about CAD 812.
Limitation The primary source (CanLII and the tribunal's site) returned a block (HTTP 403). The decision is known from law firm summaries and the site does not present it as text that was read. The CanLII URL is the one law firms link to. It is an administrative tribunal, not a binding precedent. It was a chatbot that informs, not an agent that acts.
Nobody could read the original source. This site does not present it as fact.
Whoever submits code has to show it works: test it by hand and leave automated tests that fail if the change is reverted. Coding agents are held to the same standard, and the person remains responsible for what is delivered.
Numbers
859 points and 658 commentsSize of the discussion on Hacker News (item 46313297).
Limitation An opinion essay by a specialist, with no data of its own.
“Your job is to deliver code you have proven to work.”
An engineer with 11 years of experience says he automated about 80% of his job with a coding agent. The agent picks up issues and opens pull requests. He says quality holds because he reviews every change. He spends 2 to 3 hours a day reviewing and testing. One commenter says that when review was removed, quality degraded within a week. Others discuss the ethics of doing it without telling the employer.
Numbers
about 80%Share of the job the author says he automated.
2 to 3 hours a dayTime he spends reviewing and testing.
1,200 points and 276 commentsSize of the thread according to the archive.
Limitation One person's story, with one week of use at the time of the post. Commenters point out it works because the work is simple. It was read in the public Arctic Shift archive because reddit.com blocks automated reading. The quote was compared letter by letter with the original.
Agents change tasks before they change jobs.Announcements of replacement moved faster than results. Klarna started hiring again, and only one in five customer service leaders cut staff. Across the wider economy, the data do not yet show mass unemployment.
The risk sits at the entry point and in what the team knows how to do. Young people find it harder to get into exposed occupations. With agents, it is not proven that AI helps junior staff the most. At Microsoft, junior engineers had the largest increase in pull requests. But experienced developers accept more of what the agent edits. And those who delegate while learning understand less.
Accountability has a clear answer: what the agent says, the company says. Its evidence is weak and its claim carries low confidence. The other four claims carry medium confidence.
Design the agent to help a person, not to replace the whole task. Decide how people who join learn and how the team keeps what it knows. Treat what the agent says or does as if the company said or did it.
Now:
Before cutting a role, list which tasks the agent changes and which stay with the person.
Design the agent to resolve the simple cases and hand the hard ones to a person. Do not cut the team before you have measured several months of quality.
Give people who join tasks to do without the tool, with review from someone experienced. When they learn something new, have them use AI to ask and understand, not to hand off the work.
Have the team handle some cases by hand every month, on purpose. Make sure it can still do the work without the agent.
Whatever you would not sign, do not let out without review. If someone automates their own job, they should say so.
The action policy helps you decide what goes out without review and what needs a person. The incident runbook assumes a team that still knows how to fix the problem by hand. The evaluator can recommend a person instead of an agent. The next chapter covers the law: disclosing that it is an AI and reviewing its decisions.