Why most AI pilots stall before production (and how to fix it)

Every AI pilot we've been asked to rescue had a great demo. Somebody typed a question, the model gave a smart answer, and the room nodded. Then nothing happened for three months.

The model was almost never the reason. These were.

Nobody agreed what "working" means

A demo works if it impresses the people in the room. A production system works if it does a specific job better than whatever happens today. Those are different targets.

Before we build anything, we ask for twenty real examples of the task: real tickets, real emails, real documents. Then we write down what a good answer looks like for each. That list becomes the test suite. Without it, every review turns into an argument about vibes.

The data lives somewhere awkward

The demo used a clean PDF. Production needs the CRM, three shared drives and a spreadsheet one person updates on Fridays. Getting reliable, permission-aware access to that data is usually most of the work, and it's work nobody budgeted for.

The agent can do too much, or nothing

Teams either give the model a single "do anything" tool and get scared, or wrap it in so many rules it can't act. The pattern that works sits in the middle: a small set of narrow tools, each with checked inputs and server-side permissions, and a human approval step for anything expensive or customer-facing. Remove the approvals one at a time when the logs show it's safe.

Nobody owns it after launch

Models change. Prompts drift. Someone adds a new product line and the assistant starts quoting last year's prices. A live AI system needs an owner, monitoring and a habit of re-running the test set before every change.

Five signs your pilot has stalled

1. Only the project team has used it. Three months in, no customer or front-line employee has touched it.
2. Every review is a debate. People argue about whether an answer is "good" because nobody wrote down what good looks like.
3. The demo data is still the only data. It runs on a sample export, and the live connection is "next sprint".
4. Nobody can say what it costs to run. There's no number per thousand tasks, so nobody can say whether it's worth scaling.
5. The best prompt lives in one person's notebook. If they went on holiday, the pilot would stop.

Two or more of these and you're stuck. The good news: the fix is usually smaller than the pilot itself.

A 30-day rescue plan

This is roughly what we do when a client brings us a stuck pilot:

Days	What we do
1 to 5	Read the code and the logs, sit with the intended users, collect twenty real examples and write the expected result for each
6 to 10	Score the current pilot against those examples, so everyone sees the same number
11 to 20	Fix data access properly, narrow the agent's tools, add approval for risky actions and wire the examples into every release
21 to 30	Put it in front of five to ten real users, measure, fix what they hit, and name the owner who runs it from here

At the end of the month you have a number to argue about instead of an opinion, and a decision: scale it, change it or stop it. Stopping is a fine outcome too. It's cheaper than three more months of hoping.

What it looks like when it works

Altrun, our own multi-channel assistant, is built the way we'd rebuild a stalled pilot: each capability is a narrow tool with structured input and output, every step is logged and retried on its own, and patterns are proven there before they go into client agents. The same habits run through our AI agent development work.

What to do instead

• Pick one workflow, not a platform.
• Collect real examples before writing a prompt.
• Budget for the integration, not just the model.
• Ship to a few real users within weeks.
• Decide who owns it on day one.

None of this is glamorous. It's also why some AI projects make it to production and most don't. If you have a pilot that's stuck, we can help unstick it.