Take one model - Llama 4 70B, open weights, same parameters at every vendor that serves it. Then read its price sheet across the providers that host it: roughly 0.65 per million tokens on a self-serve H100-class endpoint at 100-150 tokens a second, up to about 4.20 on a fast-ASIC endpoint at 750 tokens a second. Same weights. Six price points and seven speeds across the shelf. This is not a pricing inconsistency to be reported to a distributor. It is a standard market structure with a name, and the name comes from oil: it is a refining market.
The crack-spread view
In oil, the same crude is refined at different plants, at different margins, into products at different prices. The difference between the crude and the products is the crack spread - it varies by plant, by region, by how fast the refinery can run, and by how specialized the product is. It is not a mispricing. It is the price of throughput.
Open-weight inference has the same anatomy exactly:
- The weights are the crude. Public, fungible, priceless at the source. No one owns the input.
- The silicon is the plant. An H100 on a self-serve cluster refines slowly; a Cerebras or Groq-class config refines fast. The same model through different plants is the same product at different throughput, and throughput is the product.
- The spread is the margin. A 6x price spread and a 5-7x P50 latency spread across the same model class is the market telling you what speed is worth, to whom, right now. The spread is the public record of the latency premium - and it is a number you can quote.
The extremes make the curve visible. On the cheap/slow end, the self-serve H100 tier is the refinery run at low throughput - the buyer is in no hurry and the plant is the commodity. On the fast end, the 750-tps ASIC tier is the specialty product - the buyer is a user waiting, and the product is time. And the outlier that shows you where the curve bends: GPT-OSS-120B on a Cerebras CS-3 running at around 2,700 tokens per second and 280-millisecond time to first token, a different open-weights model on a different fast plant, demonstrating that the speed ceiling of the open-weights class is not the H100's speed ceiling.
What the spread means for buyers
The single most valuable shift: you are not buying a model, you are buying a model-throughput pair. "Llama 4 70B" is not a product the way "Llama 4 70B at 150 tps for 0.65" is. Every time a post says "use Llama for this," the missing half of the sentence is "at what speed, and what do you pay for it."
Three rules that fall out:
- Match the tier to the waiting party. If a human is waiting on the response, the latency premium is a UX purchase - buy the fast plant, up to the point where the user's wait stops being the bottleneck. If a machine is waiting (batch, overnight, enrichment), the fast plant is paying for a speed no one is consuming - buy the cheap plant.
- Treat the spread as a variable, not a constant. The crack spread moves with silicon supply. The specialty spreads in 2024-25 are wider than the 2025-26 spread, because the fast supply used to be scarce; every new Cerebras or Groq-class deployment compresses the top of the spread. Budget at the spread shape, not at a capture.
- Keep the weights as your exit option. The one structural power of the open-weights market is that the crude is public: any plant becomes your seller, and your cost-ceiling is "run it myself," which is exactly the crossover in local vs. API math. You negotiate with the spread by staying close enough to the self-serve tier that it keeps being a credible threat.
What to do
- Benchmark your real tasks across the fast/medium/cheap tiers of one model and record cost-per-task and p50/p95 latency per tier, in cheapest API by task style.
- Route user-waiting traffic to the latency tier and machine-waiting traffic to the throughput tier, and keep the split visible in the dashboard, not buried in a config.
- Re-benchmark the spread quarterly - the crack moves with supply, and a stale capture of the spread is a stale budget.
- Keep a self-serve or local path on one hot model class, even at 10% volume, so the weights-as-exit-option stays real and not theoretical.