Back to blog

The prompt cache is inventory, not a discount: write fees, TTLs, and the working-capital frame

The prompt cache charges you a write fee (1.25x to 2x) to hold a copy, sells reads at 0.1x while the copy is fresh, and expires the copy on a TTL. That is inventory: restocking fee, shelf life, write-off. The frame changes when the cache pays and when it quietly costs you money.

Mappace Team · Research2026-07-067 min read
BillingResearch

The prompt cache is usually explained as a discount: matching prefix, cheaper tokens, that is the whole story. The explanation is wrong, and the wrongness is where the money is. The cache is inventory. The write fee (1.25x to 2x the standard input rate, depending on the TTL class) is the restocking fee - you pay to prime a copy of the KV into the provider's memory. The read rate (on the order of 0.1x) is the price of drawing against stock that is already on the shelf. The TTL (a 5-minute to 1-hour class, depending on the provider) is the shelf life, and when the stock expires the shelf is empty and the next request is a full restock.

The inventory frame is the correct one, because it has the same failure modes inventory has, and the "discount" frame hides all of them.

The working-capital line

The provider is carrying the cache as working capital: HBM held against a TTL, at a cost the provider books. The price structure is the cost of that capital, priced down to the buyer:

  • The write fee is the cost of priming the copy - a fixed charge per cache-entry, paid once per TTL window, whether or not the entry is read ten times or zero. It is the restocking fee, and it is the line that makes the cache a working-capital decision rather than a free optimization.
  • The read rate is the cost of drawing against the copy while it exists - 0.1x of the read-side, which is why a hot cache is a large discount and a cold cache is a surcharge on the first request of each window.
  • The TTL is the shelf life, and it is the variable the frame is built around: the cache only wins if your request pattern re-touches the same prefix inside the TTL, more often than not.

The buyer's decision is the same decision a retailer makes about a fast-moving SKU: carry the stock if the turnover is high enough that the restocking fee amortizes, and do not carry it if the turnover is inside the shelf life. The restocking fee is the write fee, the shelf life is the TTL, and the turnover is your request pattern on the shared prefix.

The working-capital arithmetic

The cache pays when three numbers line up, and the arithmetic is the whole story:

  1. The prefix is large enough that the 0.1x read rate, times the prefix length, is a genuine saving against the 1x uncached rate - a 50-token system prompt cached is a rounding error, a 20K system prompt cached is a line item. The prefix is the SKU, and small SKUs do not carry inventory.
  2. The TTL is long enough against your request cadence. A cache hit requires a re-request inside the TTL window, and the TTL is the provider's choice of shelf life, not yours. A steady stream of requests every 30 seconds is inside the 5-minute class; a batch that touches the prefix once an hour is not, and the first request of each hour is a full restock at the write fee, whether or not the rest of the hour is a draw against the copy.
  3. The write fee amortizes across the draws. A write at 1.25x with 20 reads at 0.1x is a net saving of roughly 1x per read minus the 0.25x restocking, spread across the window. A write at 2x (the 1-hour TTL class) with 3 reads is a loss, and the loss is the working capital the provider charged you to carry stock you did not turn.

The three numbers are all visible, and the mistake is reading them as a single "cache discount" rather than the three separate charges they are - restocking, draw, and shelf-life - each with its own number and its own failure condition.

The silent failures, and why the frame catches them

The "discount" frame misses the two standard inventory failures; the inventory frame catches both:

  • The expiry that looks like a cache. The cache-silently-breaks failure mode - the prefix changes by one token, the cache misses, the read rate is the uncached rate, and nothing on the invoice says the cache stopped working - is the inventory version of a shelf-life expiry: the stock is technically there, the SKU changed, and the draw is a restock. The frame that catches it is the turnover check: your read rate should be the 0.1x number for the prefix length you cached, and the delta to the uncached rate is the write-off.
  • The TTL that is not the one you think. The 5-minute class and the 1-hour class are different shelf lives at different write fees, and a workloads that straddles the boundary (a prefix warm for an hour, then cool, then warm again) is paying the 1-hour write fee for a 5-minute turnover. The frame that catches it is the shelf-life against turnover check: the TTL you have priced must be the TTL your cadence is inside, not the TTL you want.

What to do

  • Model the cache per prefix, not per model: prefix length, write-fee class, TTL class, and your request cadence on that prefix. The four numbers are the inventory file, and the restocking-to-draw ratio is the one that tells you the cache is working.
  • Watch the read rate on your cached prefixes in the usage data, not just the cost. A read rate that drifts toward the uncached rate is a write-off in progress, and it is the silent-break signal before the cost shows it.
  • Re-check the TTL class against your cadence at every change: a cadence that moved outside the 5-minute class is a 5-minute class that is quietly a 1-hour class, at the 1-hour write fee.
  • Budget the cache as working capital, per prefix, with a write-off line for the expiries - the restocking fee is recurring, and the write-off is the line the "discount" frame never shows you.