Most accounts of AI implementation failure list the same causes: poor data, weak sponsorship, unclear goals, too few skills. They are all true somewhere. They also do not explain the pattern we see most often, where a pilot works, the sponsor is pleased, the system goes live, and within two months the people who were meant to be relieved of work are doing more of it than before.
The pattern has a specific cause. Nobody decided who handles the cases the automation cannot.
A pilot is run on a set of cases someone chose. The sponsor picks a representative month, or the consultant picks a clean sample, or the first fifty items are used because they are to hand. The automation handles ninety-something percent of them and everyone concludes it works.
The sample is not wrong. It is incomplete. It is drawn from the cases that were easy to find, and the easy cases are the ones with every field present, one clear owner and no history. The cases that were awkward, the order with two customers' names on it, the invoice with a line nobody recognizes, the request that does not fit any of the categories, tend to be set aside early because they slow the demonstration. The pilot's number is then a number about the easy cases.
Production brings the whole distribution. A field is blank because a person was in a hurry. Two records disagree because two systems were updated on different days. A customer asks for something that has no category. A supplier changes the format of an invoice without telling anyone. Each of these is rare. Together they are a large share of the work.
An automation meeting one of these has three choices: guess, stop, or hand it to a person. Guessing is how errors reach customers. Stopping with no one told is how a process quietly halts. Handing it to a person works only if a person is there to receive it, with enough context to decide, and a reason to do it today.
An exception queue is not a mailbox that everyone can see. It has three parts, and a queue missing any of them will fail.
An owner. One named person, with a named deputy, whose job description includes clearing it. Not a team, not "operations".
A clock. Every case in the queue has an age, and there is a limit on how old it may become before it is escalated. A case with no clock waits indefinitely, and the customer on the other end is the one who notices.
A rule for what happens next. The person clearing the queue can fix the case, reject it, or send it back with a reason, and each of those outcomes is recorded. The recorded reasons are how the automation improves, because the most common reason becomes next month's change.
A queue with these three parts turns exceptions from an interruption into an input. We cover how to find the exceptions before go-live in writing down the exceptions rather than the happy path. This piece is about what happens after, when the queue has real traffic.
Four figures tell you whether the automation is working, and none of them is the headline automation rate.
The exception rate, by reason. A flat rate with a changing mix is healthy. A rising rate means the world has changed and the automation has not.
Time in the queue, median and oldest. The oldest case is the one that hurts.
Minutes per exception. An exception costs more than a normal case, because a person has to rebuild the context the automation had.
Rework. How often a handled exception comes back.
The headline rate is a number about the easy cases again. The four above are numbers about the work that is left to people, and they decide whether the project saved anything.
Imagine a company that matches supplier invoices to receipts, about two hundred a week. In the pilot, the automation matches 92 percent without help, so sixteen invoices a week need a person. Each takes about twelve minutes, which is a little over three hours a week, and the project is approved on that basis.
In production the automation matches 71 percent. The gap is six kinds of exception that the sample never contained: partial deliveries, a unit of measure that differs between order and invoice, credit notes, duplicate submissions, a supplier whose invoices carry a different name from the one on file, and invoices for services with no receipt at all. That is fifty-eight exceptions a week, about eleven and a half hours of work, against a plan that staffed three.
The queue grows by roughly forty cases a week. Within a month the oldest is five weeks old, a supplier has been paid twice and another is chasing a late payment. The team concludes that AI does not work. The automation does what it was shown. What was missing was a person with eleven hours, a clock and the authority to change the rules.
The natural owner is the person who did the work before the automation arrived, because they already know what a strange case looks like. That is also why the choice is often skipped: the person is busy, and the automation was supposed to give their time back.
Be honest about this at the start. The automation gives time back on the ordinary cases and asks for it again on the unusual ones, and the second ask is smaller but harder, because exceptions need judgment. The queue owner should be someone whose judgment the business already trusts, with enough standing to change a rule when the same exception arrives for the fourth week running. If the only person who can change the rule is the consultant, the queue will grow every time the consultant is out.
Before approving a pilot, ask for the answer to one question in writing: when this automation cannot handle a case, who handles it, how soon, and what do they see?
If the answer is a name, a limit and a screen, the pilot is a pilot. If the answer is "we will work that out once we see the volume", the project is a demonstration, and its result will not predict production. This is the practical form of the foundations argument in your AI is not underperforming, your business context is: context that is not recorded cannot be handed to a person or to an agent when the case gets hard.
Take the process you most want to automate and pull last month's cases. Sort them by whether a person had to make a judgment, and list the reasons in plain words. Do not try to be exhaustive. The six or seven reasons that cover most of the judgment cases are your exception list, and the count of cases under each is your first estimate of the queue. If nobody can say who would handle it, you have found the project's first task.
We start with a no-risk discovery: we sort a month of your cases by exception reason, estimate the queue and the hours it needs, and show you who would own it, and you pay for the discovery only if you go on. The build comes with a guaranteed estimate, so if we estimate low, we absorb the difference. After go-live, Managed Technical Operations works with your owner on the queue's clock and on the reasons, so the automation keeps pace with the business.
An automation is only as good as the plan for what it cannot do.
The AI Readiness Review is a 90-minute working session plus a written scorecard across data hygiene, documented process, permissions and ownership. Fixed scope, no obligation. Or see how we approach it.
More on the same problem:
"Something isn't working." "We're outgrowing our tools." "We're afraid to make the wrong move." No-Risk Discovery is a short, practical conversation that gets you clarity before you commit to anything big. We'll tell you if we're a fit. If we're not, we'll tell you that too.
Say who you are and ask. A person replies within one business day.
