How to measure the ROI of AI: a plain guide
Usage counts and staff surveys do not tell you whether AI paid off. A baseline, a fair comparison and full costs do. Here is how to set that up, step by step.
Most companies now spend on AI. Far fewer can show what it returned. A September 2026 report of McKinsey’s survey has 37% of organisations attributing some earnings impact to AI, and only about 6% attributing 5% or more (FM Magazine). The gap is rarely that AI did nothing. It is that nobody set up a measurement that could tell.
Why the usual numbers mislead
Usage is not impact. Licences, logins and prompts tell you people opened the tool, not that the work got better or cheaper.
People misjudge their own speed. In a randomised trial by METR, 16 experienced open-source developers took 19% longer on tasks where AI was allowed, while believing it had sped them up by about 20% (METR). Self-reported time saved is a hypothesis, not a measurement.
Executives’ estimates run high. In a survey of about 750 US finance executives, the productivity gain they reported was three times what their own revenue and staffing figures implied. We covered that study here.
Checking AI output is a cost. In one survey of 270 finance leaders, teams spent 26% of their time verifying AI output (CFO Dive). If review time is left out, the return is overstated.
Five steps to a number you can defend
1. Write down the estimand
State exactly what you are measuring: the effect of which tool, on which outcome, for which teams, compared with what, over what period. “Did Copilot save time?” is not measurable. “Did access to the tool change tickets closed per agent-week in the support team over six months, compared with teams that got it later?” is. More on estimands.
2. Record a baseline before rollout
Measure the outcomes you care about, such as volume, time per task, error rate, cost and quality, for long enough before launch to see their normal ups and downs. Without a baseline, any trend that was already under way gets credited to AI.
3. Build a fair comparison
The best comparison is a randomised pilot: some teams or people get the tool now, others later. When that is not possible, roll it out in stages and compare teams that have it with teams that do not yet, using difference-in-differences. Measure both groups from the same data, the same way.
4. Count every cost
Include licences, integration and data work, training, the time staff spend checking and correcting output, and the management time to run the change. The return is what is left after all of it.
5. Report a return with a range
Give the effect on each outcome with its uncertainty, then the return and payback period with a range, and name the assumptions that move it most. A single precise figure invites false confidence. Our ROI and payback calculator shows how the range changes the decision.
What a good answer can look like
Sometimes the answer is a clear gain in one team and nothing in another. Sometimes it is no measurable effect. A study of linked Danish records found precise null effects of AI chatbots on earnings and hours, ruling out average gains above 2% (NBER w33777). Both answers are useful. They tell you where to expand, where to change how the tool is used, and where to stop paying.
Sources
- FM Magazine, "Companies' financial value from AI holds firm in 2026", September 2026.
- METR, "Measuring the impact of early-2025 AI on experienced open-source developer productivity", July 2025.
- CFO Dive, report on a Datarails survey of finance leaders, 2026.
- Humlum and Vestergaard, "Large language models, small labor market effects", NBER working paper 33777.