Beating the benchmark was the easy part
When Microsoft released its MAI-Code-1-Flash coding model this month, it beat Claude Haiku 4.5 across the standard coding benchmarks. SWE-Bench Verified, 71.6 to 66.6. SWE-Bench Pro, 51.2 to 35.2. Terminal Bench 2, 54.8 to 41.6. Good numbers, the kind a launch is usually built on.
What stood out was the number sitting next to them. Microsoft also reported the average tokens each task took, something labs usually leave off the slide. On SWE-Bench Verified, MAI-Code-1-Flash reached the answer in 10.8K tokens to Haiku’s 27.3K, about 60% fewer. The pitch was the efficiency, not the score.
From whether to how
For a few years, a model launch answered one question: who scores highest. The benchmark was the scoreboard and the headline was a capability claim. We beat the number.
Most models clear the standard benchmarks now, so the score on its own has stopped settling much. The next question is the one Microsoft put on the slide: not whether a model clears the bar, but how. Same answer, fewer tokens. Less latency. A lower cost per task.
A score tells you the answer was right. It doesn’t tell you what it cost.
None of this takes anything away from a high score. It is what happens when a capability stops being rare. Once most models can clear the bar, the question worth asking is what clearing it costs.
Efficiency is a capability, not a discount
Token efficiency sounds like a discount, a line for procurement. It is more than that. Getting the same answer in fewer tokens means the model took a more direct route to it, with less wasted work on the way. That is a real improvement, not a coupon.
It is also the kind that holds up in production. On a benchmark, tokens are a column. In a running system, every token is billed, and billed again each time an agent spins up another copy of itself to work in parallel. A model that scores a few points higher at three times the tokens can be the wrong choice for the actual job. The cost is where that shows up, before the invoice does.
Why now
Reasoning models changed the math. They get to better answers by thinking at length, which can run to ten times the output tokens of a direct reply. Once the same benchmark can be cleared at wildly different token counts, a score with no cost beside it is half a number. It tells you how well a model did, not how much work it took to do it.
So this is the benchmark catching up to what the person paying the bill already knew. Expect it to spread. Labs will report tokens per task the way they report accuracy, and the number that matters will quietly become accuracy per token. A leaderboard ranked by a single score stops making sense once cost is a second axis. What replaces it is a curve, and where a model sits on it depends on what you will pay to run it.
One caution. If the field leans too hard on tokens per result, it will favor the cheap, sure win over the expensive, uncertain one, and much of the real progress has come from work nobody could justify on cost at the time. Efficiency is worth measuring. The breakthroughs rarely showed up efficient.