Is AI Actually Saving Your Business Time? How to Measure the Return
To know whether AI is saving your business time, measure it the way you would measure any process change:
- pick one task and a unit, such as an invoice, an email or a report;
- record how long it takes and how often it goes wrong before AI;
- measure the same things with AI for long enough, including the time people spend checking and fixing its output;
- put a value on the difference, and subtract every cost.
Don’t rely on how it feels. In a 2025 randomised trial, experienced software developers took 19% longer on tasks when they were allowed AI tools, yet believed afterwards that AI had made them 20% faster. Perception and measurement can point in opposite directions.
The short answer
- Baseline first: time per unit, error and rework rate, cycle time, cost.
- Measure with AI on the same basis, including review, corrections and exceptions.
- Compare fairly: the same task mix, a long enough period, and timestamps from systems rather than estimates where possible.
- Count all costs: licences or usage fees, setup and integration, oversight, training and maintenance.
- Value the freed time, and decide in advance what it will be used for.
- Set decision rules before the pilot: scale, fix or stop.
Why perception misleads
METR, an AI research organisation, ran a randomised controlled trial with 16 experienced open-source developers working on 246 real tasks in projects they knew well. Tasks were randomly assigned to allow or disallow AI tools. With AI, tasks took 19% longer. Before starting, the developers had expected a 24% speed-up. After finishing, they still believed AI had sped them up by 20%.
The result is about one group of experts at one point in time. Tools have improved since. METR’s February 2026 update, with 57 developers and more than 800 tasks, judged it “likely that developers are more sped up from AI tools now”, but called its data “only very weak evidence for the size of this increase”. One reason the update gave: some developers declined to take part because they did not want to work without AI.
For a business, the lesson is general. Felt speed is not measured speed, and the effect depends on the task, the person and the tool. The only reliable answer is your own measurement.
What to measure
| Measure | What it tells you | How to collect it |
|---|---|---|
| Time per unit | The direct effect on effort | System timestamps where possible; otherwise timed samples |
| Error and rework rate | Whether faster is also right | Corrections logged during review; downstream complaints |
| Cycle time | How long the work waits, not just how long it takes | Start and finish timestamps |
| Review and exception time | The hidden cost of AI output | Logged separately from the AI step |
| Cost per unit | Whether the economics work | All costs divided by volume |
| Adoption | Whether people actually use it | The share of eligible tasks done with AI |
| Quality outcomes | The effect on customers and audits | Complaints, audit findings, returns |
A measurement sheet you can copy
One row per unit of work, kept for the baseline period and the AI period alike:
| Column | Example entry |
|---|---|
| Date and role | 12 November, accounts clerk |
| Unit and reference | Invoice SUP-1042 |
| With or without AI | With |
| Start and finish time | 09:14 to 09:16 |
| Corrections made | VAT number corrected |
| Exception? | No |
| Notes | New layout from this supplier |
Where your systems record timestamps, take them from there and keep the sheet for corrections and notes.
A fair comparison
- Same unit, same task mix. Don’t compare AI on easy invoices against people on hard ones.
- Long enough. At least four to eight weeks, and through a full cycle such as a month-end close.
- System data over self-reports. Timestamps and logs beat estimates.
- A comparison where you can. Alternate weeks with and without AI, or compare a pilot team with a similar team.
- Watch the novelty effect. Early enthusiasm, or early frustration, fades. Measure after the learning period too.
- Watch selection. If only the keenest staff join the pilot, results won’t transfer to everyone.
Signs your numbers are misleading
- Only the enthusiasts were measured.
- Review and correction time is missing.
- The task mix changed between periods, such as a quiet month against a busy one.
- Errors were found later, downstream, and never counted.
- Volume grew, and the hours saved were absorbed without anyone deciding how.
Count every cost
| Cost | Often forgotten because… |
|---|---|
| Licences or usage fees | Usage fees grow with volume |
| Setup and integration | It is paid once, then ignored in the monthly picture |
| Review and correction time | It sits inside “using AI” rather than as its own line |
| Oversight and maintenance | Prompts, tests and owners need time each month |
| Training | Every new joiner needs it too |
| Data preparation | Clean data was a prerequisite nobody budgeted for |
An illustrative calculation
The figures below are invented to show the method. They are not benchmarks.
A finance team enters 600 supplier invoices a month.
Before AI:
- 5 minutes per invoice: 50 hours.
- 3% need rework at 10 minutes each: 3 hours.
- Total: 53 hours.
With AI extracting the data into drafts:
- 2 minutes per invoice, including review: 20 hours.
- 10% go to manual handling as exceptions, at 6 minutes each: 6 hours.
- 2% need rework at 10 minutes each: 2 hours.
- Total: 28 hours.
Time saved: 25 hours a month. At a fully loaded cost of OMR 6 an hour, that is worth OMR 150 a month.
Costs:
- Running costs of OMR 58 a month: OMR 40 for the tool, plus 3 hours of oversight.
- A one-off setup cost of OMR 1,200.
- The setup pays back in about 13 months.
The example makes two points. Saving 25 hours a month can still be a modest financial return at small volumes. And the answer changes with volume: at several thousand invoices a month, the same setup cost is recovered far faster. Do the arithmetic for your own case before you commit.
Freed time is only valuable if it is used
Hours saved don’t appear on the profit and loss account unless something changes:
- more work handled by the same team;
- a faster month-end close;
- quicker replies to customers;
- time moved to work that was being neglected.
Decide in advance what freed time is for, and check whether it happened. The International AI Safety Report 2026 found early evidence of no effect on overall employment. Gains at company level come from reorganising work, not from the tool alone.
Decide with rules set in advance
Before the pilot starts, write down what will happen:
- Scale if time per unit falls by an agreed margin and the error rate does not rise.
- Fix and re-test if the time saving is real but quality slipped, or review takes too long. Often the prompt, the data or the validation needs work; checking AI output and from prompts to workflows cover the usual fixes.
- Stop if the net value stays negative after a fair attempt to fix it.
Writing the rule down first protects you from your own enthusiasm, and from the vendor’s.
Frequently asked questions
How do we measure whether AI saves time? Take a baseline of time per unit and error rate for one task before AI, then measure the same task with AI for four to eight weeks, including review and correction time. Use system timestamps where possible, and compare the same mix of work.
Why not just ask employees whether AI helps? Because perception is unreliable. In METR’s 2025 trial, experienced developers believed AI had sped them up by 20% when it had slowed them by 19%. Ask people too, but decide on measured results.
What costs should an AI business case include? Licences or usage fees, setup and integration, the time people spend reviewing and correcting output, oversight and maintenance, training, and any data preparation.
How long should an AI pilot run? At least four to eight weeks, and through a full business cycle such as a month-end close, so the novelty effect wears off and you see normal volumes and exceptions.
What if people save time but the numbers don’t improve? Freed time only creates value if it is redeployed. Decide in advance what it is for, such as more volume, faster turnaround or neglected work, and check whether that happened.
When should we stop an AI project? When it misses the decision rules you set before the pilot, and a fair attempt to fix the prompt, data or validation doesn’t change that. Stopping a project that doesn’t pay is a good result of measurement, not a failure.
Sources: METR — Measuring the impact of early-2025 AI on experienced open-source developer productivity · METR — We are changing our developer productivity experiment design, February 2026 · International AI Safety Report 2026 — executive summary · Anthropic — Define success criteria and build evaluations
Want to know whether AI is paying off in your business? ZAWAT builds AI and automation with measurement designed in from the start: baselines, logs and dashboards that show time, quality and cost. Book a call.