Satish Vutukuru

← All writing

Beating the benchmark was the easy part

· 3 min read

Beating the benchmark was the easy part

When Microsoft released its MAI-Code-1-Flash coding model this month, it beat Claude Haiku 4.5 across the standard coding benchmarks. SWE-Bench Verified, 71.6 to 66.6. SWE-Bench Pro, 51.2 to 35.2. Terminal Bench 2, 54.8 to 41.6. Good numbers, the kind a launch is usually built on.

What stood out was the number sitting next to them. Microsoft also reported the average tokens each task took, something labs usually leave off the slide. On SWE-Bench Verified, MAI-Code-1-Flash reached the answer in 10.8K tokens to Haiku’s 27.3K, about 60% fewer. The pitch was the efficiency, not the score.

From whether to how

For a few years, a model launch answered one question: who scores highest. The benchmark was the scoreboard and the headline was a capability claim. We beat the number.

Most models clear the standard benchmarks now, so the score on its own has stopped settling much. The next question is the one Microsoft put on the slide: not whether a model clears the bar, but how. Same answer, fewer tokens. Less latency. A lower cost per task.

A score tells you the answer was right. It doesn’t tell you what it cost.

None of this takes anything away from a high score. It is what happens when a capability stops being rare. Once most models can clear the bar, the question worth asking is what clearing it costs.

Efficiency is a capability, not a discount

Token efficiency sounds like a discount, a line for procurement. It is more than that. Getting the same answer in fewer tokens means the model took a more direct route to it, with less wasted work on the way. That is a real improvement, not a coupon.

It is also the kind that holds up in production. On a benchmark, tokens are a column. In a running system, every token is billed, and billed again each time an agent spins up another copy of itself to work in parallel. A model that scores a few points higher at three times the tokens can be the wrong choice for the actual job. The cost is where that shows up, before the invoice does.

Why now

Reasoning models changed the math. They get to better answers by thinking at length, which can run to ten times the output tokens of a direct reply. Once the same benchmark can be cleared at wildly different token counts, a score with no cost beside it is half a number. It tells you how well a model did, not how much work it took to do it.

So this is the benchmark catching up to what the person paying the bill already knew. Expect it to spread. Labs will report tokens per task the way they report accuracy, and the number that matters will quietly become accuracy per token. A leaderboard ranked by a single score stops making sense once cost is a second axis. What replaces it is a curve, and where a model sits on it depends on what you will pay to run it.

One caution. If the field leans too hard on tokens per result, it will favor the cheap, sure win over the expensive, uncertain one, and much of the real progress has come from work nobody could justify on cost at the time. Efficiency is worth measuring. The breakthroughs rarely showed up efficient.


Related

Scientific coding is the frontier

AI has mostly solved everyday coding, the kind where answers are cheap to check. Scientific coding is the harder and more consequential frontier, and history says the tools science forces into existence end up belonging to everyone.

The world has to grade itself

The scarce input to AI progress is no longer data. It is a world worth practicing in. But a world is far harder to come by than a dataset, because the property that makes an RL environment valuable pulls against the property that makes it trainable, and that tension is now the rate limit on how fast AI improves.

Everything is an eval now

Today an eval is something a specialist runs: a model scored against a benchmark. As agents do more of the producing, that word escapes the lab. Every answer you get becomes a small eval you are running, and the durable skill of the era turns out to be evaluation, not prompting.

← All writing