What should an AI do when it cannot find the customer's order?

Written by Tobias Riis Christensen, August 2026

Order lookups are one of the most common things an AI handles. But they are also one of the first places where it visibly fails. Over the past year we have watched our engine manage this in a way that was technically correct but practically wrong. This is what we learned from fixing it.

So, let me get into it. What should an AI do when it cannot find the customer's order?
My argument would be: Hand the conversation to a person, with the possible answers already prepared.

A person can find the order more easily, as they have more places to look and more ways to look in them. The reason the order number was not clear to the AI right away is usually just small and human, like a misspelled email or a different name on the order. Asking the customer to fix that is the wrong move. It invites unnecessary back and forth as well as frustration.

A common thing our engine did was ask for something it should have found itself
It asked well. Correct grammar, an apology, the right tone for the market. Read one in isolation and you would sign it off. Then you read 40 of them in a week and notice that the customer had already given us what we were asking for – at least in a way.

I want to be precise about why this is worse than it looks. A support agent in the same position does not ask. They try the surname. They search the postcode. They see that the email address has "cristian" without an H, and try adjusting the spelling. They check whether the order is sitting in a different market's system under a different ID. That is six systems, six ways, none of it written down anywhere, all of it obvious to anyone who has done customer service themselves.

But not for an AI that's seeing this information for the first time and only has an elaborate prompt to follow. So the AI's polite question is giving the customer homework they should not have to do.

Every one of those six things is individually not worth building
I assumed this was a prompting problem. Tell the model to try harder, tell it not to ask, give it more tools to search with.

Then you start mapping what it would take. Searching on a surname is one with its own edge cases. Recognising a misspelled email address is a different one, and it carries a judgement call about how wrong is too wrong to guess at. Checking whether the order sits in another market's system is a third, and it needs a rule for which markets are plausible for this particular customer.

Taken one at a time, almost none of them are worth the work. Each one wins back a handful of conversations a year, against hours of build-time and a permanent maintenance cost for all the edge-cases you want to handle. Any product manager looking at one of them in isolation declines it, and is right to.

So we are not building most of them, and I have stopped thinking anyone should. A support agent can already do all of it, and nobody had to build any of it.

They picked it up by doing the job. So the capability is already on your payroll.

Building it again in software means paying for it twice, and the software version still would not replace the agent.

Which does not make the engineering job smaller. It moves it. The engine does not need to search better. It needs to know that it failed, hand the case over, and have the next step ready when it does.

There is a second cost that pushed me the same way. Every tool you hand the model is another decision it has to make correctly, so more search paths means more routes to being confidently wrong, meaning the tenth tool can make it worse at using the first nine.

Handing over should not mean handing over a blank page
This is where the work went instead. If the agent finds the order, which they usually do, the conversation has a small number of possible endings, and our knowledge articles already decide what each of them looks like. Refund, reship, wait, escalate.

So the engine prepares an answer for each of those endings and hands the case over with all of them attached. The agent picks the one that turns out to be the right way to service that customer. They still do not write the reply.

If they cannot find the order either, we are back to asking. That is fine, because by then the question is genuinely the only move left rather than the first one we reached for.

An automation rate that counts this is measuring sentence production
This is the part I would argue with anyone about.

The industry quotes automation rates. So do we. If the denominator includes every reply the AI managed to write, and the numerator counts a request for information the customer already supplied, then the number describes the model's ability to produce a fluent sentence. It says nothing about whether a conversation closed – or rather, whether the AI actually helped the customer.

Ours did that until we split it out. I do not know what anyone else's rate does, because nobody publishes their definition. That seems like the more useful thing to compare than the percentage.

The version we now watch: a failed lookup holds the case for a person and does not count as anything. A genuine miss can still ask, because in that situation an agent asks too.

What I still cannot do
Everything in the first section. The engine cannot (out of the box) search on a surname, cannot guess at a misspelled email, cannot check the other market's system on a hunch. I have made my peace with most of that staying human. Every automation project has a point where the next fix costs more than it saves. Knowing where yours sits matters more than pretending it is not there.

Which is a strange thing to find at the end of chasing one bad reply. The bottleneck was never the model's reasoning. It was that a two-year agent knows sixty small things about where data hides, and none of them were ever written down. Writing all of that down and instructing an AI on it is probably not worth it. Making the human agent's work on the case super easy is.