The standard mental model is the one the code carries: get a 429 or a 5xx, back off, retry, continue, and the error handling is plumbing and the bill is the bill. The math does not work out that way, because the retry is a priced event with three components, and the three components are invisible to the error-handling frame.
The first is the re-bill. The retry resends the full prompt, and the input tokens bill again on every attempt - at the cache rate when the prompt is still warm in cache, at list price when the TTL lapsed while you were backing off, at the write fee on top when the cold path must re-cache (the 1.25x-2x class of the write, per the prompt-caching billing). The second is the latency, billed in waiting: a 1-2-4-8-16 second backoff chain is 31 seconds before the fifth attempt, and a ten-retry chain crosses 17 minutes, and the user gets the wait without being offered a fare class for it. The third is the correlation. Retries fire as a herd, at the same failure, across clients, at the same instant, and the herd is the pricing event, because at that instant everyone bids for the same residual capacity, and the capacity goes to whoever arrived, and the rest of the herd buys at worse terms or not at all.
The fare grid: what the airline actually sells
The airline's answer to demand exceeding capacity is not a single higher price. It is a grid of fare classes, each a configuration of boarding probability, cost, and flexibility: the changeable full fare, the premium refundable, the discount fare, and the standby with its no-show distribution, calibrated per route and per daypart. The 429 is the sold-out slot, and the retry policy is one fare class in that grid. The yield-management read of the backoff parameters:
| The backoff parameter | What it is in the yield frame | What the buyer gets |
|---|---|---|
| The retry count | The offered success probability: 1 - (1 - p) to the n, over the attempts | Completion probability, at the re-bill cost per attempt |
| The initial delay | Which window you enter: short is the surge residual, long is the queue | Wait time, or position |
| The multiplier | A forecast of when demand clears, as a bet on the price path | Worse terms on the same event, when the forecast is wrong |
| The delay cap | The maximum commitment | The bounded tail wait, on the longest task |
| The jitter | Decorrelation from the herd, per client | The herd's success holds, the individual's barely moves |
The read of the grid: the retry is the discount fare for the availability, not a reliability feature, and as a discount fare it sits inside the provider's capacity management - and inside yours, because the 429 discipline across providers works on the assumption that failure modes are independent, and the independence is the fare difference between providers' residual supply.
The price of a retry, in concrete numbers
Take a 15,000-token input prompt at the industry-standard mid-tier pair - $0.60 per million input, $2.40 per million output, the 4-to-1 spread - with a 2,000-token output. The base attempt is roughly $0.009 of input and $0.005 of output. A retry with the cache warm, read at the ~0.1x read-rate class, is roughly $0.001 of input. A retry with the cache cold - the TTL lapsed during the backoff - is the full $0.009 again, plus the cache write at the 1.25x-2x class, roughly $0.011 to $0.018 to re-enter.
The per-retry numbers are under a cent, and that is where the point is easy to misread. The point is not the per-retry price; it is the fleet. At one million tasks a month and a 1-2 percent global retry rate - the range a busy fleet hits against a saturated provider - the retried input is 10,000 to 20,000 resends: $90 to $180 a month on input re-bills alone at mid-tier rates, and $750 to $1,500 at the frontier $5/$25 pair, before the re-billed output, which is four times the input line at the same spread. The retry line is small against the base bill, which is exactly the misread: the retry's price is not only the fee. It is the latency and the first-attempt failure, and the fee is the visible part of it - and at frontier rates the fee stops being trivial.
The deeper yield-management problem is yours, not the provider's: you charge the customer at the first-attempt fare and deliver at the retry-fare latency, with no fare class for the difference. The wait is the availability that was sold, and it was sold at the wrong price.
The grid beyond the retry
Yield management is the whole grid, and the retry is one class of it. The other classes, in the order the price falls:
- The downgrade, the cheapest fare on the same route. Reroute the task to the cheaper model, at the provider where the 429 is not an event. The downgrade is acceptable only when the class survives your quality bar, and the quality bar is why it is the last class, not the first.
- The batch, the literal standby. The industry-standard batch channel - roughly 50 percent of the list price, the 24-hour completion window - is the no-show option for work that is not interactive: the capacity the interactive fleet leaves standing, sold at the discount to the buyer who can wait. The async workload's default should be the batch (the forward-market read of the discount), not the hot retry loop.
- The cross-provider backoff, the second exchange office. The residual at provider A's failure is provider B's headroom, and the hedge works only when the failure modes are actually independent, and the independence is the question the 429 playbook settles per provider pair.
What to do
- Log the cost of the retry, not just the count: the input tokens billed, at cache or full rate, and the wall clock from first attempt to success. The count is the plumbing view and the cost is the yield view, and the two diverge exactly when the TTL lapses mid-backoff.
- Alert at a retry rate above 2 percent of requests, and read the alert as the pricing event that it is - the residual you are buying has become expensive - not only as a reliability event, because the reliability fix (provider capacity) is the provider's, while the pricing response (the downgrade, the batch, the reroute) is yours.
- Set the retry ceiling per task-criticality class: 2-3 for interactive, and the batch channel (the forward market on tokens) for async, instead of the hot backoff loop.
- Match the backoff chain against your cache TTL: if the chain outlasts the TTL, every retry pays full price, and either the TTL extends or the retry sequence shortens, so the cheap retry is the one that actually happens.
- Put the downgrade before the fifth retry in the decision tree, because the cheap model that answers now is the availability the premium model's backoff loses, and in the yield frame the fare class that lands is the fare class that wins. The policy lives in the gateway layer - at a gateway like Mappace, as a route parameter - and in the yield frame it is a fare schedule, which gets fare-schedule reviews.