You need to run an AI model. Fifteen companies sell access to the same one, and their prices differ by 8.6 times.
You pick the cheapest. Depending on what you are building, you may have just chosen the most expensive option available at any price, and nothing on your invoice will ever tell you.
Here is how that happens, using only public data.
The objection first, because it is the right one
Nobody building a real-time product picks a provider off a price list. They check the speed column too. So what is this actually saying?
That the buyer is not uninformed. The buyer is unequipped.
The engineer checking the speed column is doing a calculation in their head, for one provider, on one afternoon. That calculation cannot be written down in any form that survives the meeting. It does not go into a budget, because it is not a number anyone agrees on. It does not go into a contract, because there is no standard unit to write it in. It does not go into a procurement comparison, a vendor scorecard, or a lender’s file. It gets re-derived from scratch by every engineer at every company, and then it is thrown away.
The claim is not that people choose wrong. It is that the right answer exists, can be calculated from public information, and has nowhere to live.
One term, and then we are done with jargon
Cost of useful work is what you spent, divided by the output that actually arrived in time. A token is the unit a model produces, roughly three quarters of a word.
Output that showed up too late still cost you money. It goes in the top of the fraction. It does not go in the bottom, because you cannot use it. That is the whole idea.
Two buyers, same model, opposite answers
Buyer one is building a voice assistant. Tokens have to be generated faster than the speech is played back, or the audio stalls. That sets a floor: at least 100 output tokens per second. Below it the product does not work. This is not a preference. It is the condition under which the output is worth anything.
Buyer two is summarising documents overnight. Same model, same prompts. It needs to finish by morning. Nobody is waiting, so there is no speed floor at all.
Priced per million tokens, both buyers see one ranking, and the cheapest provider wins by more than five times.
For buyer two, that ranking is right. Every provider finishes overnight. Everything they paid for, they can use. Cheapest wins, and paying a premium for speed would buy nothing.
For buyer one, the ranking is upside down.
What the cheapest provider actually delivers
The cheapest provider produces 15.2 tokens per second. The floor is 100.
It is not close, and it is not an occasional bad day. The typical request is 6.6 times too slow. So for this buyer, almost nothing that provider produces can be used.
That changes the arithmetic in a way worth pausing on. The cost per unit of usable output is not high. It is undefined, because you are dividing by nothing. A high price can be traded off against something else. An undefined one cannot. There is no discount that makes this provider correct for this buyer, because it is not delivering the thing being bought.
The cheapest option that actually works is the fastest provider, at 329.6 tokens per second, for roughly 64 cents per million usable tokens.
Same model. Same input. Opposite answers. A factor of five between the two.
Why no current metric catches this
Price per GPU-hour cannot see it. It prices the machine, not the work.
Price per million tokens gives both buyers the same ranking, and is wrong for one of them. It counts every token produced, whether or not it arrived in time to matter.
Speed benchmarks report how fast, and say nothing about cost.
Uptime guarantees promise the service is switched on, and nothing about whether the work is usable.
Each of these is reasonable on its own. None of them can be combined with the others. And none of them can express what just happened above.
The weakness in our own example
The cheapest provider is running a compressed version of the model. The expensive ones are not.
That cuts against us as much as against the market. If the cheap option is a squeezed-down copy, then part of the 8.6-fold price gap is explained by the product being different, and this is not a clean comparison.
So the honest version is narrower than the headline. Within the compressed version, the reversal holds on speed alone. Across versions, the buyer is also comparing two different products and is not entitled to assume they are equally good until someone tests it.
Today nobody is asked to. Same model name, same token count on the bill, different output. That is a second problem, not a rescue from the first.
What we cannot see, stated plainly
The public speed figures are typical-day numbers, not worst-case numbers.
For a question about whether output arrives in time, the worst case is what matters, and it is not published. We are using what exists.
The direction of that gap is knowable even if the size is not. Real systems are slower on their bad days than on their typical ones, never faster. So the true worst-case picture is worse than the one above at every provider, and the reversal we are describing is the conservative version. On a bad day the cheap provider does not become usable.
We would rather publish the conservative version and name the instrument than publish the dramatic version and hide it.
Where these numbers come from
Prices come from published rate cards. Speeds come from a public benchmarking service that tests providers continuously.
Three of the fifteen providers have fully published figures. The other twelve are reconstructed estimates. The two-buyer comparison above uses only real, published providers. Any statistic computed across all fifteen is partly an estimate, and we label it that way, because a number built partly on estimates is an estimate. That rule applies to us before it applies to anyone else.
Sweeping the deadline across all fifteen, the winner changes about four times, and at the worst single step a small change in the deadline moves the cost of useful work by around 178 percent. That figure is partly estimated. What holds on published data alone is smaller and still enough: among the real providers, the ranking under a real-time deadline is not the ranking by price.
What would prove this wrong
If the price ranking and the deadline-aware ranking turn out to match in practice, this post is wrong.
The test is cheap and anyone can run it. Take your own workload. Write down your deadline. Look up your providers’ published speeds. Check whether your cheapest option is still cheapest once you stop counting output that arrives too late.
If it is, we want to know. That result belongs in the record alongside the ones that went the other way.
First in a series drawn from The Missing Meter, a paper proposing a standard unit for AI compute: one unit of output delivered inside its deadline and above an agreed quality bar. Full paper, specification, and the code behind every figure:
https://themissingmeter.org