# Measuring and reporting results > **What you will learn** > Why most adoptions never prove their effect, what to measure and when, and > which numbers an executive can actually read. ## This is where most of them fail In [what to do first](/guide/ax-choosing-tasks) you picked whether to attack quality, cost, or revenue. But **picking and proving are different jobs.** The reason this chapter exists is a number from MIT NANDA's 2025 study of over 300 initiatives and 153 executives. > **$30 billion went in, and 95% reached no measurable financial impact.** One of the three traits shared by the 5% that did was **"evaluation based on business outcomes rather than technical benchmarks."** The other two were deep integration into a specific workflow, and continuous learning. Other surveys from the same period sharpen the gap. ``` Executives reporting they see productivity gains 79% Executives confident they can measure AI ROI 29% ``` **That 50-point gap** is the defining problem of enterprise AI right now. Everyone knows it got better; nobody can say so in numbers. ## The baseline is everything The cause is usually one thing. **Nobody measured before deployment.** ``` After launch someone asks: "So how much faster is it?" → "How long did it take before?" → "Well... half a day, maybe?" → That is where it ends ``` Miss it once and it is unrecoverable. Past cycle time, error rate, and rework rate cannot be measured retroactively. A baseline needs four things. | Element | Detail | |---|---| | **Target workflow** | named precisely. Not "customer service" but "first-line triage through draft reply" | | **Metrics that workflow already produces** | numbers somebody actually counts today | | **A long enough measurement window** | enough to average out weekday and seasonal variance — usually 2–4 weeks | | **An owner** | someone who will track **the same metrics** after deployment | **The last row is the one that goes missing.** If a consultant measures before and nobody measures after, there is nothing to compare against. ## Five layers What you measure comes in layers, and **which layer you stop at decides whether the report lands.** ```mermaid graph TD A["1. Technical performance
accuracy, latency"] --> B["2. User adoption
active users, rework rate"] B --> C["3. Operational KPIs
cycle time, defect rate, cost per transaction"] C --> D["4. Strategic outcomes
customer satisfaction, retention"] D --> E["5. Financial impact
cost, revenue, margin"] ``` | Layer | Who reads it | Stopping here means | |---|---|---| | 1. Technical performance | the build team | **executives cannot read it** | | 2. User adoption | line managers | ends at "people do use it" | | 3. Operational KPIs | operations leaders, CFO | **budget attaches from here** | | 4. Strategic outcomes | executives | needs a causal story | | 5. Financial impact | CFO, board | the destination | **Reporting only layer 1 is the most common failure.** "92% accuracy" means nothing to an executive. "40 minutes to 12 minutes per item" means something. ## Start with cycle time Among layer-3 metrics, **cycle time** reads best. It connects directly to throughput and labour efficiency, so operations leaders and CFOs understand it without translation. ``` "One invoice: 40 min → 12 min (70% faster)" → 2,000/month × 28 min = 933 hours → the next question is where those hours went ``` **That last line is decisive.** Time saved that simply evaporates never reaches the financial statements. You have to be able to say whether it went into **higher throughput**, **redeployed headcount**, or **less overtime** before you can climb to layer 5. ## Measure the hidden costs too Measuring only the gains is half a picture. | Item | Why | |---|---| | **Rework rate** | share of AI output a human had to fix. High means the time was not really saved | | **Override rate** | share of suggestions a human reversed. A proxy for trust | | **Review time** | checking is working time too | | **Token and API cost** | pay-per-use. Set by model tier and conversation length | | **Operational headcount** | people on maintenance and improvement | **At a 40% rework rate, "40 min → 12 min" is a lie.** It is really 12 minutes plus the fixing. ## Set expectations here > **As of August 2026.** These figures change. Take the **ratios and the > principle**, not the numbers. From analyses of enterprise deployments: ``` Median 3-year ROI : 210% Average payback period: ~16 months ``` **The 16 months matters.** Demand financial impact in three months and nearly every initiative looks like a failure. Reporting to layer 3 in year one and reaching layer 5 in year two is the realistic plan. ## Not the same as evals An easy confusion. | | [Evals](/guide/glossary) | Results measurement | |---|---|---| | Asks | **is the answer right?** | **is the business better?** | | Cadence | before every release, daily | monthly, quarterly | | Audience | the build team | executives | | Without it | quality erodes silently | funding stops | **You need both.** Evals live at layer 1; results measurement at layers 3–5. How to build evals is covered in the technical part — for now it is enough to know they are a separate thing. ## The one-page report ``` [Initiative] Automated invoice entry [Period] 2026-03 → 2026-08 (6 months) ■ Baseline (2026-02, 4-week measurement) 40 min per item / 3.1% error rate / 2,000 items per month ■ Current 12 min per item / 1.4% error rate / 2,300 items per month 11% rework rate / 180 human approvals per month ■ Operational effect 933 hours/month saved → 15% more throughput + 210 fewer overtime hours ■ Cost Model and API 1.8M won/month / 0.3 FTE maintenance ■ Next Top rework cause (missing supplier code) returned to Phase 2 as a data-quality item ``` **That last section is what makes it a living report.** Numbers with no next action stop being read by the following quarter. ## Common misconceptions ### "Everyone says it's better — why is there no number?" **Because the saved time had nowhere assigned to go.** Ten people saving 30 minutes a day is five hours, but if those five hours turn into no output, the financial statement says zero. **Saving time and spending it are separate decisions**, and the second is [change management](/guide/ax-change-management)'s job. ### "We have no time to measure, we have work to do" **Three metrics are enough.** One cycle time, one quality, one rework rate. Three tracked consistently for six months beats ten attempted and none completed. --- ## Check yourself **1. Why must the baseline be measured before deployment?**
Answer **Because it cannot be measured later.** Past cycle time, error rate, and rework rate have no retroactive measurement. Miss it once and the initiative is stuck at "it feels better" forever.
**2. Why is "92% accuracy" the wrong thing to report to executives?**
Answer **It is a layer-1 metric and doesn't translate into business language.** Budget attaches from layer 3 onward. Say it as cycle time — "40 minutes to 12 minutes per item" — and it lands.
**3. How can cycle time drop 70% with zero financial effect?**
Answer **The saved time was never converted into other output.** You have to be able to say whether it went into throughput, redeployment, or reduced overtime to reach the financial layer. That is a change-management problem, not a measurement one.
--- Do not postpone this one. Measure today or never → [Design a baseline yourself](/guide/ax-try-baseline)