From Answer to Action: Why Most AI Pilots Stop Before the Work Is Done

Most enterprise AI pilots succeed at the part that was never the bottleneck. They produce a good answer, and then a person still does the work.

The demo is genuinely impressive. Someone pastes in a customer email, the model reads it, identifies the issue, cites the relevant policy, and drafts a reply. Everyone in the room agrees this is going to save a lot of time.

Then it goes to the team who handle those emails, and the time saved turns out to be smaller than expected. The representative still opens the billing system to check whether the charge was actually duplicated. Still checks whether this account is inside the refund threshold. Still asks a supervisor, because it is over the limit. Still updates the CRM, still sends the confirmation, still closes the case.

The model did the reading. The organization still did the work.

Six reasons pilots stall at the demo

This pattern is common enough to have shown up in forecasting. In June 2025, Gartner predicted that over 40% of agentic AI projects will be canceled by the end of 2027, drawing on a January 2025 poll of 3,412 webinar attendees, 61% of whom reported significant or conservative investment in the technology. The reasons it gave were not model capability: escalating costs, unclear business value, and inadequate risk controls.¹

The same analysis named a second problem worth keeping in mind while evaluating vendors — agent washing, the rebranding of existing assistants, chatbots, and RPA as agentic. Gartner estimated only around 130 of the thousands of vendors claiming agentic capability were genuine.¹

First decide whether you need an agent at all

There is a useful distinction here that gets lost in the marketing. Anthropic's engineering guidance separates two architectures that are routinely conflated: workflows are “systems where LLMs and tools are orchestrated through predefined code paths,” while agents are “systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.”²

That difference decides most of your engineering. A workflow is predictable, testable, and cheap to reason about, because the sequence is code. An agent is flexible and can handle situations you did not anticipate — and correspondingly harder to evaluate, secure, and debug, because the sequence is a runtime decision.

The guidance is blunt about which to reach for: “find the simplest solution possible, and only increase complexity when needed,” and “consider adding complexity only when it demonstrably improves outcomes.” For many applications, “optimizing single LLM calls with retrieval and in-context examples is usually enough.” Workflows suit well-defined tasks where predictability matters; agents earn their cost when flexibility and model-driven decisions are genuinely required at scale.²

Most enterprise processes that people describe as needing an agent are actually workflows with two or three genuinely ambiguous steps. Structuring them that way — deterministic code for the path, model calls for the judgment — usually produces something you can ship, test, and defend. Reaching for full autonomy first is how projects end up in the 40%.

The anatomy of a workflow that finishes the job

1. Convert the request into something a system can route

Work arrives as an email thread, a call, a form, a PDF, or an event. The first job is turning that into structured fields: what is being asked, which entities it concerns, what is missing, and how risky it looks. If required information is absent, the correct output is a request for it — not a confident guess.

2. Ground the decision in what the organization actually knows

A recommendation built only on model memory is a liability. Before proposing anything, the workflow should retrieve the account record, the applicable policy version, the entitlement, and the prior history — and validate that the requester is who they claim to be and permitted to ask.

3. Give it narrow tools, not broad access

The unit of control is the tool contract. billing.create_credit(account_id, amount, reason_code) with validated parameters and a server-side ceiling is safe in a way that database access never is. Separate read from write. Make destructive and financial operations distinct tools with their own authorization.

This layer is standardizing faster than most teams realize. The Model Context Protocol, introduced by Anthropic in late 2024 as an open standard for connecting AI applications to external systems, was donated on December 9, 2025 to the Agentic AI Foundation — a directed fund under the Linux Foundation co-founded with Block and OpenAI, with AWS, Bloomberg, Cloudflare, Google, and Microsoft among its supporting organizations.³ Betting on an open tool interface is now a considerably safer architectural position than it was a year ago.

4. Put the human where the consequence is

Autonomy should be configured per action, not per project. A classification that routes a ticket can run unattended. A credit above a threshold should not. The reviewer's job is to approve or correct a prepared recommendation with its evidence attached — not to reconstruct the analysis from scratch, which would eliminate the saving entirely.

5. Finish, verify, and keep the evidence

The workflow is not complete when the model produces output. It is complete when the system of record is updated, the customer has been told, the result has been checked, and there is a retained trace: the original request, what was retrieved, which policy version applied, what was recommended, who approved it, which tools were called with which parameters, and what finally happened.

Match autonomy to consequence

Measure the work, not the model

Benchmark scores tell you little about whether a workflow is doing its job. The measures that matter are operational, and most of them are unglamorous.

• Completion rate: what share of runs reach a finished outcome without human rescue.
• Correct-action rate: of the runs that completed, how many took the action a competent employee would have taken.
• Escalation precision: whether the cases sent to a human were genuinely the ones that needed one — over-escalation quietly destroys the business case.
• Reviewer edit rate: how often approvers change the recommendation before accepting it, and in which direction.
• Time to resolution end to end, not model latency.
• Cost per completed unit of work, including retries, review time, and failures.

Build the evaluation set from real historical cases with known-correct outcomes, including the awkward ones, and re-run it whenever a prompt, model version, policy, or tool changes. Without that, an upgrade is an uncontrolled experiment on live customers.

Choosing a first workflow that will actually ship

None of this is an argument against ambition. AI is already carrying real load in service organizations — in Salesforce’s 2025 State of Service research, service professionals estimated AI was resolving roughly 30% of cases and expected that to reach 50% by 2027.⁵ The organizations that get there will not be the ones with the best demo. They will be the ones that did the unglamorous engineering: defining the outcome, wiring the systems, bounding the tools, placing the human, and keeping the receipts.