AI success depends on business outcomes, not usage vanity metrics

Imagine a NASCAR team where leadership decides the winner is the car that burns the most fuel. The crew chief tunes every engine to run rich and wasteful. Drivers ride the gas and the brake at once. The telemetry shows record fuel consumption. The car finishes dead last.

For a stretch this year, wasteful consumption metrics drove AI measurement programs. Companies stood up internal leaderboards ranking employees by how many tokens they burned. A Meta employee built one, crowned the top "token legends," then quietly took it after the numbers leaked externally. By spring the trend had a name: tokenmaxxing.

By the start of summer the inevitable backlash arrived, with companies like Uber acknowledging they’d already spent their entire AI budget for the year in a matter of months.

More recently, the inevitable swing toward using the minimal number of AI tokens (tokenminning) leads to the same problem. If leadership defines winning as using the least amount of fuel, the winning car would never leave the starting line.

Burning or saving tokens should never be the end goal. The objective is shipping more products, at higher quality, with the same or fewer people. The job of a leader is to define the most strategic path toward the finish line, and based on that definition, build metrics that support it.

Why leaderboards miss the mark

A vanity metric is any number that looks great on a slide, leaderboard or dashboard but doesn't change how you run your business. In the AI space, even highly technical and complex metrics can become vanity metrics.

Here’s a couple I’ve seen lately, each a different way to look busy GPU utilization rates that brag about how much raw compute you’ve hoarded, regardless of whether it's solving real problems. AI features shipped per quarter, where leadership slaps a sparkle-emoji text summarizer on every legacy product to pad the release notes.

Metrics like these can feel rigorous, but they're detached from the end work itself. When leaders don’t relate metrics to end goals, engineers may race toward the nearest countable item.

The missing race map

Work completed only matters if it’s the right work. When leadership celebrates "95% adoption" of GitHub Copilot because the licenses are deployed, they can mistake superficial access for deep engagement. They assume productivity is skyrocketing. They're measuring who adopted the tool, not who is effectively using it to improve outcomes.

Developers might be generating more code faster, but if the tool isn't deeply integrated into a refined process, PR sizes inflate and the bottleneck simply shifts to human code review. Cycle time stays stagnant, or worse, bug rates go up because humans are rubber-stamping AI hallucinations.

Close the disconnect: tie depth of use directly to outcomes and business goals. If you have 95% "adoption" but your cycle time hasn't dropped by 30% or your rework hasn't decreased, you don't have high use. You just have an expensive new software distribution.

How to set the finish line

A single number rarely tells a story. If I can produce twice as much code but it takes twice as long to ship, I haven't gained efficiency. I've created more work. The right question isn't how much AI you're using — it's whether the work is actually getting better.

For engineering teams with more established workflows, start with the outcome you want to move, then use a framework like DORA or SPACE to check whether you got there. The framework is the instrument, not the goal. DORA tracks the health of your delivery pipeline — how often you deploy, how long changes take, how quickly you recover from failures, how often changes cause incidents.

SPACE tracks the full picture of developer productivity across five dimensions, including outcomes and satisfaction. Neither was designed for AI, which is exactly why they're useful: they’ll help you measure whether AI made the work better, not whether AI got used.

Pair one of those frameworks against your AI tool usage and your business outcomes like cycle time, rework rate, cost per change. A 25% increase in AI adoption that correlates with a drop in delivery stability isn't a win. The 2025 DORA report found that's exactly what many teams got when they deployed AI into underprepared systems. The tools amplified what was already there. Good engineering foundations got better, fragile ones got more fragile.

For roles outside engineering — marketing, finance, operations — build from first principles. Start with one outcome you're already trying to move: response time, pipeline conversion, forecast accuracy. Establish the finish line before AI enters the workflow. Measure toward that same outcome after. If it moved, figure out whether AI caused it and whether you can repeat it. If it didn't, you've learned something too. Picture a support team that wants to cut first-response time.

They baseline it at nine hours, then let agents draft replies with an AI assistant. Two months later response time is down to three hours, and reopen rates haven’t budged — so the speed isn’t costing them quality. That’s an outcome they can name, measure and repeat. It never needed a leaderboard.

Keeping the finish line in sight

The goal of a metric is to manage toward an outcome. If you're doing more work on the metric than on the outcome, you're upside down. You're looking at the gas tank, not the finish line. So before you stand up your next AI metric, ask one question: what outcome would move if this number did? If you can’t name it, you’re not measuring progress — you’re watching the fuel gauge. Define the finish line first; everything else is just telemetry.

We've featured the best AI website builder.

This article was produced as part of TechRadar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.

The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit