Why most enterprise AI pilots stall before production

I have sat in a lot of rooms where an AI demo went well. The room warms up, somebody says “this changes everything”, and the energy is real. Six months later the same organisation is still running the same process by hand.

That gap is not a technology problem, and it is not a talent problem. I have built an AI practice from zero, put agents into production, and sat on both sides of the table — vendor side at Atlassian, partner side before and after. The pattern is consistent enough now that I can usually tell at the demo which pilots will survive.

Three things stop them, and none of them are the model.

1. The demo runs on data you were handed. Production runs on data you have to go and get.

Every demo dataset has the same properties: small, clean, curated, and already exported by someone who wanted the demo to work. Production data has none of them. It sits in systems with owners, access reviews, retention rules, and a person who will quite reasonably ask why you want it.

I get reminded of this every time I build something of my own. Naatil looks like a simple directory until you are keeping twenty-nine thousand listings accurate across fourteen districts in two languages, with business owners editing their own records. The interesting engineering was never the search. It was everything around the data staying true.

So before anyone builds, trace the actual path from source system to model, and name the human who approves each hop. If nobody can name that person, you do not have a pilot. You have a demo with a roadmap attached.

2. Nobody agreed what “working” means.

Ask a room how they will know the pilot succeeded and you will usually get an adjective. Faster. Better. More consistent. Adjectives cannot be defended in a budget conversation nine months later, and they cannot tell you whether last week’s change helped.

The fix is unglamorous: write the evaluation before you write the feature. Thirty examples with expected outputs, agreed by the people who own the process, beats any amount of intuition. It is the difference between “the assistant seems good” and “the assistant handles 27 of our 30 known cases, and here are the three it does not”.

When I built BestCredit I made the ranking formula public, because a comparison you cannot check is just an opinion with a table around it. The same standard applies to an AI feature inside a business. If the only evidence is that it impressed people in a room, you have not measured anything.

3. After the demo, nobody’s name is on it.

A pilot needs a champion. Production needs an owner. Those are different jobs and often different people, and the second one is much harder to recruit — because a champion gets the credit, and an owner gets woken up when it breaks at month four.

This is the most common quiet death. The pilot works, everybody agrees it works, and then it simply has nowhere to live. No team’s roadmap has a line for it. No runbook mentions it. No budget line survives the next planning cycle. It does not get cancelled; it gets forgotten, which is worse, because nobody learns anything.

It is why managed-service models exist for this work at all. Not because the engineering is beyond the client, but because “who owns this in six months” is a real question that deserves a real answer before you start.

What a pilot that survives looks like

It is deliberately unexciting:

• One workflow, not a platform. Narrow enough that success is unambiguous.
• A named owner who will still be accountable in month six.
• A data path that already exists, with the access question settled before the build.
• A measurable before and after, agreed in advance by the people who do the work today.
• An answer to “who runs this when it breaks”, written down.

None of that is about AI. That is rather the point.

The organisations getting this into production are not the ones with the best models — everyone has access to much the same models now. They are the ones who treated the pilot as the start of an operations problem rather than the end of an evaluation.