The Signal

A biweekly read for operators thinking seriously about AI. Less noise, more signal.

The Lead

The pilot is the easy part

The numbers arrived in a cluster this summer and they all say the same thing. Nearly two-thirds of enterprises have experimented with AI agents; fewer than one in ten have scaled them into anything that delivers value, per McKinsey. Deloitte’s August survey puts scaled, orchestrated adoption at 15% of US enterprises. Gartner now forecasts that more than 40% of agentic projects will be cancelled by 2027, and names the causes: unclear ROI and weak risk controls. Only 12% of CEOs in PwC’s survey say AI has produced both revenue gain and cost reduction.

Meanwhile 80% of enterprise applications shipped this year embed an agent. The agents are everywhere. The value is in a few places. The space between those two facts is where most of this year’s budget went, and where most of the disappointment is being recorded.

Here is what most post-mortems get wrong. They treat the failed pilot as a technology story: the model wasn’t good enough, the integration was harder than expected. In the pilots we have watched fail, the technology worked. What was missing was everything around it. Nobody baselined the process before the agent touched it, so nobody could prove it got better. Nobody set the kill criteria up front, so the pilot could not end, only fade. Nobody owned it after launch, so the first drift went unnoticed. Nobody budgeted for monitoring, so “production” meant “the pilot, but nobody is watching anymore.”

A pilot is easy because it has a champion, a deadline, and low stakes. Production is hard because it has none of those. The transition is not a technical milestone. It is an operating-model change, and it has to be designed before the pilot starts, not negotiated after it succeeds. The pilots that ship are the ones built to be measured, killed, or handed over from day one.

The median payback on agents that do reach production is about five months, per BCG and Forrester. That is a good return. It is available only to the teams who design the ninety days after the pilot as carefully as the pilot itself.

This issue’s Field Manual is the 90-day structure we use. It is not a technology plan. It is the plan for everything the technology plan leaves out.

Three Things Worth Knowing

Orchestration doubled in a quarter. Governance did not

KPMG’s Q2 survey found the share of large US organizations orchestrating multiple agents doubled in a single quarter, from 9% to 18%. Multi-agent pipelines are also becoming a default in developer tools, with the newest coding environments shipping one-command multi-agent modes for hard tasks. The operator problem: every additional agent in a chain multiplies the hand-offs where a bad output becomes the next agent’s trusted input. If your single-agent pilot had no monitoring plan, your multi-agent one needs a monitoring plan and a map of who checks whom. Scale the oversight before you scale the agents.

The cancellations are already scheduled

Gartner’s forecast that 40% of agentic projects will be cancelled by 2027 is not a prediction about bad technology. The stated causes are unclear ROI and inadequate risk controls, both of which are decided before the first line of code. A project without a baseline cannot show ROI. A project without gated actions cannot show control. If your current pilot could not answer “what did this workflow cost before, and what does it cost now” in one sentence, it is on the 2027 list already. The fix is a spreadsheet and an afternoon, and it has to happen now.

Task routing is the production model, not the pilot model

The agents reaching production fastest share a shape: bounded, high-volume, verifiable work — extraction, drafting, first-pass review, ticket triage — where output can be checked and errors are cheap. Sales-development agents pay back in under four months; finance and operations agents closer to nine, per the same surveys, and the difference is verifiability. Pilots often pick the impressive workflow. Production should pick the checkable one. Route the bounded work to the agent, keep the judgment with the human, and let the routing table — not the demo — decide what ships first.

From the Practice

The day-90 question

Every pilot we run has one question written at the top of the plan on day one: on day 90, what would make us kill this? Not what would make us proud of it. What would end it. A specific number: error rate above X, cost per task above Y, adoption below Z.

Writing it down changes the pilot in two ways. The team stops optimizing for the demo and starts optimizing for the metric, because the metric is the thing that decides. And the executive sponsor stops having to be the bad guy, because the kill criterion was agreed before anyone was attached.

Most pilots die of ambiguity. They never quite fail, so they never quite end, and they consume budget and attention for a year. A pilot with a written kill criterion either ships or dies on schedule. Both outcomes are wins. The one that fades for a year is the only loss.

The Field Manual

The 90-day pilot that actually ships

This is the structure we use to take an agent from pilot to production. Ninety days, three phases, and the decisions most pilots leave until it is too late are made before day one.

Seven steps. The first three happen before anyone builds anything, and they are the ones that decide the outcome.

Before day one: pick the checkable workflow

Choose a task that is bounded (a clear start and end), high-volume (enough repetitions to measure), and verifiable (a human can tell in under a minute whether the output is right). Extraction, triage, first-pass drafting. Not the flagship; the one you can score.

Before day one: baseline it

Measure the current process for two weeks before the agent touches it: time per task, error rate, cost per task, and who does it. Write the numbers down. Without this you cannot prove improvement, and a pilot that cannot prove improvement is a cancellation with a delay.

Before day one: write the kill criterion and the ship criterion

One sentence each, with numbers. “Ship if error rate is under 3% and cost per task is under $0.80 by day 90. Kill if error rate is above 8% at day 60.” Get the sponsor’s signature. This is the single most important step in the plan.

Days 1 to 30: shadow

The agent runs on real work but a human does the task as usual. You compare the two outputs and log every disagreement. No customer sees the agent. The goal is a disagreement log, not a demo. Commitment level: Recommend.

Days 31 to 60: stage

The agent prepares, a human releases. Every output goes live only after a one-click human approval. Measure the approval rate and the time to approve. If humans approve 95% without edits, the agent is ready for the next level. If they are editing half of them, it is not, and the disagreement log tells you why. Commitment level: Stage.

Days 61 to 90: execute with audit

The agent acts; a human reviews a sample and can reverse within a window. Now measure the real numbers against the baseline from step two and the criteria from step three. Name the owner who keeps watching after day 90, and the monitoring budget — as a line item, not an intention. Commitment level: Execute with audit.

Day 90: ship or kill, on schedule

Compare to the criteria. Ship it with its owner and budget, or kill it and write the one-paragraph post-mortem. Do not extend. An extended pilot is the fade that costs a year.

The meta-rule

Design the handover before the pilot. The pilot is the easy part because it has a champion, a deadline, and low stakes. Production has none of those unless you build them in: an owner, a budget, a kill switch, and a number that decides.

Save this. Run the three before-day-one steps on your current pilot this week. If any is missing, that pilot is already on the 2027 cancellation list.

Have a pilot that has been “almost ready” for a while? Bring it to a Discovery call and we’ll run the day-90 question on it together, baseline and kill criterion included.

Book a Discovery Call

Call or no call, I’d like to hear how this lands. If you have shipped an agent to production and the handover looked different from this, or you think the 90-day frame is too slow or too fast, just reply. The sharpest pushback tends to end up in a future Field Manual.

Or read this issue on LinkedIn.

Applied AI strategy, training, and implementation.

Ryan Rezeika · Founder · Vancouver, BC
ryan@resonai.ai · resonai.ai · LinkedIn

You’re receiving this because you subscribed at resonai.ai/signal. · Unsubscribe