A pattern repeats often enough to be worth naming. A company builds an AI system. It works in testing. It launches. The dashboard shows good numbers for a quarter. Then, quietly, people stop using it, and nobody announces this, because no single person decided it. The system just stops being the path of least resistance, and work routes around it.
Nothing dramatic happened. There was no outage, no scandal, no moment where the project was declared dead. That is what makes this failure mode so expensive. A system that fails loudly gets fixed. A system that fails quietly gets renewed.
The industry has produced plenty of statistics about this. Handling them carefully turns out to be part of the lesson, so we will start there rather than treating the numbers as settled fact.
First, Read the Failure Statistics Properly
You have probably encountered the claim that 95% of AI pilots fail. It comes from a report published in 2025 by a group at MIT, The GenAI Divide: State of AI in Business 2025 , and it went viral because it was a startling number attached to a trusted institution.
It deserves scrutiny rather than repetition. The report drew on roughly 150 executive interviews, a survey of a few hundred employees, and analysis of about 300 public deployments. It was preliminary and not peer-reviewed. Critics, including academics who examined it closely, argued that the headline 95% figure was not clearly supported by the underlying exhibits, and noted that the report concluded by promoting a project associated with its own authors. Several called for the full data to be released or the report withdrawn.
None of that means the finding is wrong. Other evidence points in a similar direction. Gartner predicted in June 2025 that over 40% of agentic AI projects would be cancelled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls.
McKinsey’s State of AI survey, published in November 2025, found that while 88% of organisations regularly use AI in at least one function, only 23% were scaling an agentic system anywhere, and in any given business function no more than 10% reported scaling agents. Separate McKinsey research found that nearly two-thirds of enterprises had experimented with agents while fewer than 10% had scaled them to deliver tangible value.
The honest summary is this.
The direction of the evidence is consistent and the precise magnitude is not.
Failure rates are high. Whether the number is 40%, 60%, or 95% depends heavily on what counts as a project, what counts as failure, and how long you wait before measuring.
That distinction matters practically, not just academically. If you take 95% as gospel, the rational response is not to try. If you understand that most failures cluster around a small number of avoidable causes, the rational response is to avoid them. This article is about those causes.
The Failure Modes That Actually Matter
1. Automating a process that should have been deleted
This is the most expensive mistake and the least discussed, because identifying it requires questioning why work exists rather than how it is done.
A report gets produced monthly. Automating its production is straightforward and demonstrably saves eight hours. The question nobody asks is who reads it. Sometimes the answer is nobody, and has been for two years. You have just spent money making an unnecessary artefact appear faster.
The same applies to approval steps that exist because of an incident in 2019, forms that collect fields nothing consumes, and handoffs that survive because the org chart changed but the process did not. Automation preserves all of it in amber, and makes it harder to remove later because now there is a system depending on it.
The test: before automating any step, ask what would break if it simply stopped. If the honest answer is “nothing immediately,” investigate before you build.
2. Starting from the technology
A team is told to find AI use cases. Note the direction of that instruction. It starts with a solution and searches for a problem, which reliably produces projects that are technically interesting and commercially marginal.
Projects that start from a business problem have a natural advantage: someone already cares about the outcome, and there is an existing baseline to improve on. Projects that start from the technology have to manufacture their own stakeholder, and the stakeholder they find is usually whoever was most enthusiastic in the meeting rather than whoever owns the cost.
3. Choosing the impressive demo over the boring workflow
The demo that gets applause is customer-facing, conversational, and visible. The workflow that pays for itself is usually internal, repetitive, and dull.
Invoice matching does not impress a board. It also has a natural error-catching mechanism, occurs thousands of times a year, and replaces expensive human attention with something measurable. Meanwhile the conversational customer assistant, the one that photographs well, operates in the highest-risk environment you have.
There is a selection effect worth understanding here. Projects chosen for visibility are chosen by people who will not maintain them.
4. Data problems, which are the actual bottleneck
McKinsey’s research on scaling agentic AI found roughly eight in ten companies citing data limitations as a roadblock. This is consistent with what practitioners report, and it is the least glamorous finding in the field.
The specific problems recur: the authoritative version of a policy exists in three places with three different answers; product data has gaps that humans silently work around; the CRM contains duplicates that nobody has time to merge; historical records use categories that changed meaning in 2021.
Humans absorb all of this without noticing. They know the spreadsheet is more current than the intranet. A system does not know, and will confidently serve the wrong answer.
This is why data preparation is usually the largest line item in an honest project plan, and why the projects that skip it fail in ways that look like AI problems but are not.
5. Mistaking prompts for a strategy
A meaningful number of “AI strategies” consist of a well-crafted prompt, tested manually a dozen times, and then deployed.
Prompts are the surface. Beneath them sit the questions that determine whether the system survives: where the data comes from, how retrieval is scoped, what happens when a tool call fails, what the system does when confidence is low, how outputs are logged, how regressions get caught. A prompt is a configuration detail. Treating it as the architecture is like treating a menu as a restaurant.
The tell is a project with no evaluation set. If nobody can produce fifty real cases with known correct answers, the system’s quality is a matter of opinion, and opinions drift.
6. Evaluating on averages, which hides the risk
This deserves the most attention because it is where the money is lost.
Suppose a support system resolves 80% of tickets. That number will be presented as an unambiguous win. Now ask which tickets constitute the other 20%.
They are not a random sample. Easy questions resolve. What remains is billing disputes, cancellations, complaints, edge cases, and customers who are already unhappy. The residual is systematically enriched with your highest-value and highest-emotion conversations. So the headline says 80% of volume automated. The reality may be 80% of volume and a much smaller share of the risk, with a new layer of friction sitting in front of the conversations that determine whether customers stay.
Klarna’s experience is the clearest public illustration. In February 2024, Klarna and OpenAI announced that Klarna’s AI assistant was handling roughly two-thirds of customer service chats, doing work equivalent to about 700 agents, with resolution times falling from around eleven minutes to under two. In May 2025, chief executive Sebastian Siemiatkowski said the company had over-indexed on cost at the expense of quality and would invest in human support again.
The nuance is usually stripped out in retelling, so it is worth keeping. Klarna publicly disputed the “reversal” framing, saying it continued to invest heavily in AI, that the assistant’s workload had grown, and that hiring humans reflected a dual-track approach rather than a retreat.
Both things can be true. The efficiency was real. The conclusion that optimising purely for cost degraded something that mattered was also real.
The transferable lesson is about measurement, not about AI.
The average-case metrics described the wins while the damage accumulated in the tail, and the tail was not on the dashboard.
7. Ignoring consistency, not just accuracy
There is good research on this. The τ -bench benchmark, built to test agents on realistic customer service tasks with tool access and written policies, introduced a metric called pass^k : whether the agent solves the same task correctly on all k attempts rather than at least one. In the original paper, a leading function-calling agent that succeeded on more than 60% of single attempts dropped below 25% at pass^8 in the retail domain.
Translate that into operations. A system can be right most of the time and still be unreliable for any specific customer, because “most of the time” is measured across cases, not within them. Two customers with identical situations can receive different treatment.
There is also compounding arithmetic that people consistently underestimate. A ten-step workflow at 95% reliability per step is roughly 60% reliable end to end. At 99% per step it is about 90%. Chains multiply errors.
8. Granting too much autonomy too early
The pattern is predictable. The pilot works. Confidence rises. Someone removes the human review step to capture the full saving, because the review step is where the remaining cost sits.
That review step was doing more than catching errors. It was generating the signal that told you the system was working. Remove it and you lose your only early warning system at exactly the moment your exposure increases.
Autonomy should be earned with evidence, one increment at a time, and each increment should be reversible.
9. No human fallback, or a fallback nobody can find
Every system encounters cases outside its competence. The question is whether it handles that gracefully.
Failing well means recognising uncertainty, saying so plainly, and routing to a person quickly. Failing badly means guessing confidently, or trapping the user in a loop with no visible exit. Customers forgive a system that says it cannot help and connects them to someone who can.
They do not forgive one that wastes eight minutes of their time and then disconnects.
10. No named owner
Committees do not maintain systems. When something drifts, the fix requires someone with the authority to change it and the accountability to notice.
Projects without a named owner degrade on a predictable schedule. A policy changes and the source documents are not updated. The model vendor ships an update and behaviour shifts subtly. An API deprecates. Each individual event is small. Cumulatively they render the system untrustworthy, and once trust goes, usage goes.
11. Nobody monitoring production
Traditional software fails visibly. It throws errors. AI systems degrade silently, producing fluent, plausible, wrong output that looks exactly like correct output.
Without deliberate monitoring, the first signal is a customer complaint, and by then it has been happening for weeks. Monitoring means running your evaluation set on a schedule, tracking escalation rates and human override rates over time, and sampling real outputs for review. Override rate is particularly useful, because it is the cheapest available proxy for quality: when your team starts changing more of the drafts, something has shifted.
12. Never defining an acceptable error rate
“It has to be accurate” is not a specification. Every system has an error rate, including the manual one you are replacing.
Most organisations have never measured their human error rate, which means AI systems get held to an implicit standard of perfection that the status quo does not meet. Before deployment, define the acceptable rate per error type, because not all errors are equal. A misfiled document and an incorrect refund are not the same event and should not share a threshold.
13. Security and privacy treated as a later phase
Two categories of risk are routinely underestimated.
The first is data exposure: which systems the AI can read, whether it respects existing permissions, and whether one user can retrieve another’s data through it. Retrieval systems that ignore the permission model of the underlying source are a common and serious flaw.
The second is prompt injection, which is specific to this technology and genuinely difficult. Any system that reads untrusted content, such as incoming emails, web pages, or uploaded documents, may encounter instructions embedded in that content aimed at the model. If the system can also take actions, the consequence is not a bad answer but an unauthorised action. There is no complete defence available today.
The practical mitigations are to limit what the system can do, require approval for consequential actions, and never let a system that reads untrusted input hold broad write permissions.
14. Vendor dependence and hidden costs
Two specific exposures are worth naming.
Model dependence means your system’s behaviour can change when your provider updates a model. Without an evaluation set, you will not detect this until someone complains.
Cost scaling is the other. Agentic systems consume tokens in proportion to usage, so costs grow with adoption. A pilot’s economics can look excellent and production’s can look considerably worse, precisely because it succeeded.
15. Employees routing around the system
If a system is slower, less reliable, or more annoying than the old way, people revert. They will not say so, because saying so is awkward when leadership has invested in it. Usage statistics will look adequate because people log in and then do the work elsewhere.
This is worth watching for directly. Falling usage among your most experienced staff is the earliest reliable signal of a quality problem, because they are the ones best able to tell that the output is not good enough.
16. Automation that creates more work than it removes
The system produces drafts. Someone reviews each draft. The review takes nearly as long as writing would have, because verifying unfamiliar text is cognitively harder than producing familiar text.
This is a real and common outcome, and it does not mean the technology failed. It means the task was poorly chosen. Good candidates are tasks where checking is much faster than doing. Assembling research is a good candidate: reading a briefing is far quicker than compiling one. Writing nuanced client correspondence is a poor candidate, because the review is nearly the whole job.
How Successful Projects Are Actually Built
The sequence below is deliberately unexciting. That is the point.
Find the bottleneck.
Not the most interesting process. The one costing real money or capping growth. Ask where work queues up and where people complain about their week.
Measure the current process.
Volume, time per instance, loaded cost, and current error rate. Without a baseline you cannot prove improvement and you will end up arguing about impressions. The error rate matters most, because it sets the fair standard.
Choose a narrow workflow.
One workflow, one team, one clearly defined scope. Narrower than feels ambitious.
Define success numerically, in advance.
“Reduce processing time from 15 minutes to under 5 for at least 70% of cases, with an error rate below 3%.” Numbers written down before you start, because numbers written afterwards adapt to the result.
Map the exceptions before you build.
Ask the people doing the work what the weird cases are. Their answer tells you what the real scope is, and it is always larger than the process documentation suggests.
Fix the data.
Establish which source is authoritative, remove superseded material from the index, and resolve the duplicates. This is the longest step and it is where projects are actually won.
Set permissions narrowly.
Start with the minimum access required. Broad permissions granted during a pilot rarely get tightened later.
Build human checkpoints into the design.
Decide before building what requires approval and what does not, and set thresholds by consequence rather than by convenience.
Build an evaluation set.
Fifty to a hundred real cases with known correct outcomes. This is the single most valuable artefact the project produces, and the one most often skipped.
Run in parallel, not in place.
Both systems running, outputs compared, humans still doing the work. Expensive for a few weeks and dramatically cheaper than discovering the problems in production.
Collect every correction.
Each time a human changes the output, that is a labelled failure case. Feed them into the evaluation set.
Deploy gradually.
A subset of volume or a single team first. Keep the ability to switch back instantly, and make sure someone other than the builder knows how.
Monitor continuously.
Scheduled evaluation runs, tracked escalation and override rates, sampled outputs.
Expand only on evidence.
More scope or more autonomy when the current level has data supporting it, never because the roadmap said so.
The Pre-Investment Checklist
Ten questions to answer before committing budget. Any “no” is worth resolving first. 1.
| # | Ask this before you spend anything |
|---|---|
| 1 | Can you state the business problem without naming an AI product? 2. |
| 2 | Do you know the current volume, time, cost, and error rate? 3. |
| 3 | Have you confirmed the process should exist at all? 4. |
| 4 | Is there a single named owner accountable for the outcome? 5. Is the data accurate enough to act on, and do you know which source is authoritative? 6. |
| 5 | Do the systems involved have APIs that support the actions required? 7. |
| 6 | Can you write down the rules and the exceptions? 8. |
| 7 | Have you defined an acceptable error rate per error type? 9. |
| 8 | Do you have, or can you build, an evaluation set of real cases? 10. Have you defined what happens when the system is unsure, and can a user reach a human easily? A useful eleventh question: if this stopped working silently tomorrow, how long before someone noticed? If the answer is measured in weeks, build the monitoring before you build the system. |
The Uncomfortable Conclusion
Read back through the failure modes. Very few are about artificial intelligence.
Automating unnecessary work is a process problem. Bad data is a data governance problem. No owner is an organisational problem. No evaluation set is an engineering discipline problem. Measuring averages while the risk sits in the tail is a management problem.
This aligns with what the research attributes failures to. Gartner’s cancellation forecast cited costs, business value, and risk controls, not model capability. McKinsey’s framing points at data foundations and operational discipline.
That reading is an interpretation rather than a settled fact, and a reasonable person could argue the technology is simply not mature enough for many of the tasks being attempted. There is something to that, particularly regarding consistency, where the benchmark evidence is genuinely sobering. But it does not explain why some organisations get durable value from the same models that others cannot deploy at all.
The practical implication is more encouraging than the failure statistics suggest. If failures came from model limitations, you would have to wait. Because they mostly come from scoping, data, measurement, and ownership, they are within your control.
The projects that succeed tend to look similar and tend to look boring. One narrow workflow. Measured before and after. Data cleaned first.
A human in the loop at the point of consequence. An evaluation set run on a schedule. One person accountable. Expansion only when there is evidence.
That is not what the market is selling. It is what tends to still be running in three years.
Frequently Asked Questions
What is the single most common cause of failure?
Data quality, by most accounts, with unclear ownership close behind. Neither is an AI problem, which is why neither gets fixed by choosing a better model.
How long before we know if a project is working?
Define the success criteria first, then give it a defined window, typically one to three months of parallel running with real volume. If you cannot tell whether it worked, the problem is that success was never defined numerically.
Should we build internally or buy?
Buy anything standardised. Build only where the workflow is genuinely specific to your business and represents real cost. Building to avoid a subscription fee is usually a false economy once maintenance is counted.
Is it worth attempting given the failure rates?
The rates describe how projects are typically run, not what is achievable. The failure modes are concentrated and mostly avoidable, which is a considerably better situation than one where the technology itself was the limit.
Sources and Further Reading
Gartner, Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027 (press release, 25 June 2025).
McKinsey & Company, The State of AI in 2025: Agents, Innovation, and Transformation (5 November 2025).
McKinsey & Company, Building the Foundations for Agentic AI at Scale . Data limitations cited by roughly eight in ten companies.
MIT NANDA, The GenAI Divide: State of AI in Business 2025 (2025), together with published methodological criticism of it. Preliminary and not peer-reviewed; treat the 95% figure with care.
Yao, S. et al., τ -bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains (arXiv:2406.12045). The pass^k consistency metric.
Klarna and OpenAI joint announcement, February 2024; Bloomberg interview with Sebastian Siemiatkowski, May 2025; and Klarna’s published response disputing the reversal characterisation.
The projects that fail are rarely the ones with the weakest technology. They are the ones where nobody asked whether the process should exist at all.
The GosAI
If you would like an honest assessment of whether a workflow in your business is worth automating — and the reasons it might not be — get in touch with The GosAI.
