Measure AI Success by Business Outcomes, Not Usage
A TechRadar commentary argues that organizations should evaluate AI through outcomes such as product quality, delivery cycle time, rework rates and cost rather than token consumption, GPU utilization, feature counts or license adoption. It recommends defining a target outcome, establishing a baseline before AI enters the workflow, and pairing AI usage with measures such as cycle time, rework rate and cost per change. The author suggests using DORA or SPACE frameworks to assess engineering work and choosing a relevant outcome such as response time, pipeline conversion or forecast accuracy for other roles.
The commentary offers leaders a practical way to connect AI use to business results: establish the desired outcome and baseline, then assess whether AI improved delivery speed, quality, rework or cost rather than rewarding usage and consumption.
Imagine a NASCAR team where leadership decides the winner is the car that burns the most fuel. The crew chief tunes every engine to run rich and wasteful. Drivers ride the gas and the brake at once. The telemetry shows record fuel consumption. The car finishes dead last.
For a stretch this year, wasteful consumption metrics drove AI measurement programs. Companies stood up internal leaderboards ranking employees by how many tokens they burned. A Meta employee built one, crowned the top "token legends," then quietly took it after the numbers leaked externally. By spring the trend had a name: tokenmaxxing.
By the start of summer the inevitable backlash arrived, with companies like Uber acknowledging they’d already spent their entire AI budget for the year in a matter of months.
More recently, the inevitable swing toward using the minimal number of AI tokens (tokenminning) leads to the same problem. If leadership defines winning as using the least amount of fuel, the winning car would never leave the starting line.
Burning or saving tokens should never be the end goal. The objective is shipping more products, at higher quality, with the same or fewer people. The job of a leader is to define the most strategic path toward the finish line, and based on that definition, build metrics that support it.