You know it is paying off when you can name the baseline, the full running cost and the decision it changes. If any of the three is missing, you are not measuring return. You are measuring enthusiasm.
Why the invoice does not answer the question
An AI tool or data platform arrives with a clear price. The price is the smallest honest number. Around it sit integration work, support requests, the people who need training, the incidents when it fails, and the manual steps that remain because the tool did not quite fit. Add the business hours spent working around it and the real cost is often a multiple of the licence.
Return has the opposite problem. Vendors quote hours saved in the abstract. Nobody checks whether those hours were redirected to anything, or whether the same people are still doing the work next to the new tool.
Four questions before and after any purchase
1. What is the baseline today? How many hours, at what loaded cost, does the work take now? Without a before number there is no after number. Estimate it, label it as an estimate and write it down.
2. What is the total running cost? Licence, infrastructure, support, incidents, changes and the business hours that remain. I call the per-product version of this PCU-V: the verified monthly cost of keeping something alive, not the sticker price.
3. Which decision does it change? Name one specific decision that gets better, faster or cheaper. If the answer is "general productivity," ask for the hours and who will spend them differently.
4. What do we stop if it does not work? Set a review date and an agreed threshold before you sign. Deciding the exit after the money is spent is how pilots become permanent costs.
Check it ninety days later
At the review date, compare against the baseline you wrote down. Did the manual hours fall? Did the people who used to do the work move to something else, or are they doing both? Is the tool used by the share of the intended audience you expected? If hours are flat and usage is low, the purchase is not paying off, whatever the vendor dashboard says.
Why AI makes this more urgent, not less
AI sits on top of the data you already have. If the underlying numbers are inconsistent or unowned, an AI layer scales the inconsistency and adds a new cost to check its output. Fixing the foundation first often returns more than the AI purchase itself. I wrote more about this in when AI makes your data problem worse.
A first step you can take this week
Pick the last data or AI purchase your company made. Try to answer the four questions in writing. The gaps in your answers show what you would need to measure next. For a practical walkthrough of one such decision, read the $50,000 purchase case, or put a first number on the work around it with the FTE Debt calculator.