Everyone has a theory for why AI pilots stall. The data’s messy. The model isn’t good enough yet. The rollout moved faster than the training did. Pick a theory and you’ll find a vendor selling the fix for it.

The actual research says something less obvious. Most AI pilots aren’t failing because of the AI.

The scale of the problem

RAND found that AI projects fail at more than twice the rate of ordinary IT projects, over 80 percent. Project NANDA’s preliminary findings, out of MIT, put the share of generative AI pilots showing no measurable return at 95 percent. Gartner expects more than 40 percent of agentic AI projects to be canceled by the end of 2027. S&P Global found the share of companies abandoning most of their AI initiatives jumped from 17 percent to 42 percent in a single year.

Ordinary IT projects have never had a great track record either: Standish’s long-running research puts outright failure at roughly one in five projects, with close to half landing over budget or over schedule. AI projects failing at all isn’t surprising, then. What’s alarming is that they’re failing at several times that baseline rate, not a modest step up from it.

There are three failure reasons underneath all of that, and none of them is the model.

Reason one: nobody defined the problem

RAND asked the people who build these systems why their projects failed, then ranked the causes. The most common one was business leadership misunderstanding how to set the project up to succeed in the first place. Nobody defined the problem clearly enough before the work started.

That’s not new, and it’s not really about AI. PMI’s research on project failure found that inaccurate requirements were the primary cause of failure in 37 percent of unsuccessful projects, going back over a decade before anyone was running an agentic pilot. Every project, AI or otherwise, needs a clear problem statement and a real success metric. That part is table stakes. Get it right and you’ve earned the right to build. You haven’t fixed anything yet.

Reason two: automating a process nobody redesigned

Deloitte’s explanation for its own failure data is that organizations are automating broken processes instead of redesigning operations. “Broken” isn’t quite right, though. Most of these processes aren’t broken. The company is still running, which means the process works well enough to get by. Some of it is genuinely inefficient. But the bigger issue is that AI lets you do the underlying work differently, not just faster. A step that used to need its own approval, its own handoff, its own person, existed because that’s what the available tools permitted at the time, not because the work actually required it. Automate that step exactly as it stands, and you take a constraint that no longer applies and build it permanently into the new system.

Deloitte’s own numbers show how often this gets missed: nearly half of organizations introduced AI without redesigning the workflow it sits inside, and just 12 percent report redesign at scale, with a new operating model behind it. Redesign, in the sense process people actually mean it, isn’t reshuffling boxes on a flowchart. It’s asking, for every step, whether it should be eliminated, combined with another step, or shifted somewhere else, the same three moves that have defined process redesign since business process reengineering became a discipline in the 1990s. 

Here’s where it usually breaks down for an AI pilot specifically: the subject matter experts are typically in the room, most teams get that part right. What doesn’t get questioned is the tooling underneath them. A system of record, a productivity tool everyone already knows how to use, gets treated as fixed infrastructure rather than one more thing worth asking about, so the redesign happens around it instead of including it. Nobody sits down and asks whether that tool, or the step it requires, is still earning its place, or whether it’s just familiar. Redesign doesn’t get skipped on purpose. It gets skipped because the existing system was never actually on the table. And redesign isn’t just rearranging the same steps in the same order. It’s asking whether the process needs all of them: fewer handoffs, fewer tools, fewer places where a request sits waiting on someone else’s calendar before anyone even looks at it. Automate it exactly as it stands, same steps, same handoffs, same queue, and you get the same inefficiency. Just faster, and executed with more confidence.

Reason three: the business rules were never code

Here’s what the research doesn’t say loudly enough: plenty of pilots start with a genuinely well-defined problem, and a real process redesign, and still fail. The team interviews the right people. They write down the actual requirements. They redesign the process instead of automating it as-is. And the pilot still stalls, because the AI can’t return the same result twice, or hit the accuracy a given step actually requires.

A requirements document describes what the process should do. It rarely contains the actual rules the process runs on: the eligibility logic, the pricing exceptions, the compliance thresholds, the judgment calls a fifteen-year employee makes without thinking about it. That knowledge lives in application code nobody wants to touch, in a compliance folder nobody opens except at audit time, and in the heads of the two or three people who’ve been doing the job the longest. IDC estimated years ago that Fortune 500 companies lose at least $31.5 billion a year because knowledge like that never gets written down before someone leaves.

Even when the knowledge does exist somewhere, an agent still has to find the right version of it. Ask an agent for the current expense policy and it doesn’t fail because it can’t read. It fails because it pulls whatever document SharePoint or OneDrive ranks as most relevant, which might be last year’s policy, a draft that was never approved, or a regional variant that doesn’t apply to the group asking. Nothing in a shared drive tells the agent which version is actually in effect. A person doing the same search has the same problem; a person just has more context to catch the mistake before it turns into a decision.

Companies have tried to fix this before. Business rules engines have existed for two decades specifically to pull logic out of code and into one governed place. They’ve been costly, and they’ve mostly failed for the same reason: nobody got all the rules into one place that was actually accessible to everyone who needed it, the people running the process, the systems executing it, and now the agents automating it. A rules engine gets built for one team, then another team builds a second one, and a third keeps its logic in a spreadsheet nobody else can see. Large companies routinely end up running two or three rules systems that don’t talk to each other.

There’s a more basic reason they never became the single source of truth. They were built for IT and systems to execute, not for the humans who actually own the policy to read. A business owner couldn’t open the rules engine and confirm it said what they meant, so they kept their own version, a spreadsheet, a policy document, a page in someone’s head, and the two drifted apart the moment either one changed. That drift never went away. It’s compounded. The same rule can now live in a rules engine, in application code, in a policy document, in an AI agent’s prompt, in a skill definition, and in the head of whoever’s been doing the job the longest, all at once, with nothing keeping them in sync.

If your AI pilot is trying to fix this right now, it’s probably reaching for an ontology, a knowledge graph, or a semantic layer, the standard fixes everyone reaches for. Each one can genuinely help. An ontology gives the model a map of how things relate. A semantic layer keeps a term or a metric consistent. A knowledge graph connects facts that used to sit in separate silos. What none of them do is codify the actual business rule. They’re probabilistic, every one of them, which means none of them can guarantee the answer. The model is still inferring, still weighing likelihoods, still capable of being wrong in a new way instead of the old one. Stack probabilistic tools on top of a probabilistic model and you get a better guess. You don’t get a deterministic one.

An AI agent running on top of that fragmentation doesn’t fix it. It has to guess at the eligibility rule, the discount exception, the compliance threshold nobody wrote down, and it guesses with total confidence. That’s how a pilot produces a fabricated eligibility decision or an invented discount that sounds right and isn’t.

Business rules, business logic, company policy, the law and industry standards that apply to you, none of it is real to an agent until it exists as structured, machine-readable rules instead of a document the agent has to interpret, a governed source of truth compiled into a form software can actually execute. That’s rules as code. Policy as code, law as code: the same idea wearing different names.

We call the source of truth Canon and the compiled, machine-readable version SIGN. The name doesn’t matter. What matters is that the rule gets decided once, in one place, instead of guessed at differently by every agent and every person that touches it.

That’s the difference between a pilot that gives a different answer every time it’s asked and one that gives the same answer every time, and can show exactly where that answer came from.

How do I save my AI pilot?

You need a governed enterprise knowledge layer. Not an ontology, and not a knowledge graph, though if you already have one, it becomes an input to build from rather than a separate system to maintain on top of it. We call it Canon, and the name means what it says: a single place, in the true sense of the word, your source of truth, where the policies and business rules that actually run your business get written down once and kept current.

Canon by itself isn’t enough. It gets compiled into SIGN, a language your agents can execute directly instead of interpreting from a document. That compilation is what makes agent behavior deterministic: the agent isn’t inferring the rule from whatever context it happened to find, it’s following the rule it was handed, so the same input produces the same response, every time.

• Pick one use case, process, or stage. Not the whole pipeline, one scope you can actually finish and test.
• Collect the policies and business rules that govern it. This is where you build your Canon. This collection can often be automated from documents, code, AI prompts, and skills.
• Review the process for optimization if you haven’t already. Don’t just carry forward the inefficiency you inherited; ask whether a step should be eliminated, combined, or shifted before you lock it in.
• Make the governed knowledge available to your agents. The rule they reason over should come from the Canon you just built, not from whatever document they can find.
• Measure the result: same input, same output, every time. That consistency is the test, not just accuracy.

You’ve invested significantly in your AI pilot. Let’s save it.