The batch tier is the most under-read line on the LLM rate card. The majors run it at roughly half the standard rate, with a completion window (24 hours is the standard class), and the standard explanation is "cheaper because it is less urgent." The explanation is right about the price and wrong about the mechanism. The batch tier is a forward contract on tokens: you commit the payload now, you accept delivery within the window, and the discount is the price of the immediacy you sold. The provider is not giving you a loyalty discount. It is selling you its spare capacity, at the price of your flexibility, in the same shape a forward market sells power at the price of the delivery window.
What a forward contract prices
A forward on tokens prices two things, and the discount is the sum:
- The immediacy. The standard tier is a spot market: the token is delivered now, at the price of the now, and the "now" is the scarce asset, because the real-time capacity is the capacity that answers the user who is waiting. The batch tier is the delivery within the window, and the discount is the price of the window, the same as the forward-power discount is the price of the delivery window against the spot.
- The utilization. The batcher's real cost to the provider is the marginal cost of the spare capacity - the fleet time that would otherwise be idle, priced at the marginal-cost floor. The discount is the gap between the spot price and that marginal cost, and the gap is the provider's idle-capacity value, priced to the buyer who will take the window.
The 50 percent figure is the market's current read of that gap. It is not a constant: it is the spread between the spot tier and the provider's marginal batch cost, and the spread moves with the provider's real-time utilization. A provider at tight real-time utilization has a smaller gap and a smaller batch discount; a provider with spare capacity has a larger gap and a larger one. The discount is a price signal, and it is the one that tells you the provider's utilization, per tier, without the provider saying so.
The buyer's two decisions, reframed
The forward frame changes the two standard batch decisions:
- "Should I batch this?" is "am I selling the right timing, at the right price?" The batch discount is a price on your deadline. If your deadline is outside the window (a nightly job, a 24-hour enrichment run), you are selling flexibility you have, and the 50 percent is the price you get for it. If your deadline is inside the window (a same-day report), you are selling flexibility you do not have, and the batch tier is not an option, it is a product that will miss your deadline, at a price that does not pay for the miss. The forward is only worth the discount when the window fits the deadline, and the fit is the whole question.
- "What do I batch?" is "which of my traffic is a forward, not a spot?" The forward is the traffic with a real deadline and slack to the deadline - the traffic that can be delivered late and does not break. A user-facing request is a spot, always; a scheduled enrichment is a forward, always; the gray zone (a report that is "by EOD" but is actually "by EOD + 6 hours") is the forward that the standard tier prices as a spot, and the delta is the money.
The 24-hour window is the contract's delivery term, and the term is the thing to price, not the discount: a 24-hour delivery at 50 percent is a different product from a 1-hour delivery at 50 percent, and the rate card does not always itemize the window per model class. The window is the term, the discount is the price, and the product is the pair.
What the discount reveals about the provider
The batch discount, read per provider, is the provider's utilization signal, and the signal is the one that does not require a status page:
- A widening batch discount (the gap between the batch and the spot is opening) is the provider's real-time utilization falling - the provider is paying more for the flexibility, because the spare capacity is less valuable, and the provider is pricing the surplus. It is a soft signal of demand, and it is the one to watch alongside the price movements.
- A narrowing batch discount is the opposite: the real-time capacity is tighter, the spare is more valuable, and the provider is paying less for the window. It is a demand signal in the same direction the repricing would be, and it leads the repricing, because the batch discount adjusts faster than the list price does.
The forward frame also explains the router's batch advantage: an aggregator pooling batch volume across buyers is a forward-market participant with a larger book, and the larger the book, the better the clearing price, which is why the pooled batch can clear below the vendor's own promotional spot.
What to do
- Classify your traffic as spot or forward, by deadline, and route the forward to the batch tier at the window that fits the deadline - not the default window, the one that fits.
- Read the batch discount per provider as a utilization signal: the spread between batch and spot, per provider, is the provider's real-time utilization, and it is the one price signal that leads the repricing.
- Price the window, not just the discount: the 24-hour term and the 1-hour term are different products, and the rate card's window-per-class is the line to check before you route a forward at the wrong term.
- Keep the spot tier for the deadline-tight traffic and the batch tier for the deadline-slack traffic, with the split visible in the ledger, so the "effective rate" is a portfolio of two markets, not a blur of one.