Track LLM costs to the token without a $2k/mo observability platform

LLM cost observability is sold as a platform. It does not need to be one. The total number of questions a cost ledger must actually answer is three: who spent (per key, per workload), what it was spent on (per model, per task class), and when the shape went wrong (hourly anomaly, trend, variance between tasks). The $2k/month platforms start from these same three questions and attach dashboards to them. This is the version that starts with the three answers and adds dashboards later — if they ever earn their cost.

The ledger: one row per request

The minimal schema is a handful of fields: when, which key, which task class, which model, in/out/cached tokens, dollars. One row:

08:14:02  prd-chat   triage     sonnet-mid   in=3120  out=410   cached=2900   $0.0061
08:14:31  prd-chat   triage     gpt-mid      in=3410  out=120   cached=0      $0.0115   <- failover hop
08:15:19  agent-ov   refactor   gpt-mid      in=18400 out=880   cached=0      $0.0615

The row is written by the layer that knows the answer — the app, an SDK wrapper, or the endpoint that served the request. A compatible response with a usage block carries the token counts; some responses also carry a dollar value; the ledger's job is to attach the keys the platform does not know: your task class, your key naming. That is the entire delta between a provider's billing console (which sees the account, aggregated by provider's model name) and a ledger (which sees your attribution questions, per key and per task).

The three queries, and what each one has to answer

1. Who spent. Group dollars by key, per day. The answer has to be per workload. If the answer is one number for everything, you have a meter, not attribution — and the next conversation is "whose key was that," which a ledger should make unanswerable.
2. What was it spent on. Group by model, split the token columns into input, output, cached. This is the mix query, and the mix is the cost driver, not the model name: two "the same" workloads at different input/output ratios can differ 2-3x in dollars on the same model.
3. When did the shape go wrong. Dollars per hour, compared to a rolling baseline by hour of day. Not a threshold ("alert at $500") but a shape alert: 3-5x the baseline for the same hour is the signature of an unattended loop, a retry storm, or a task class that silently moved to an expensive model. The operational side of that signature is what separates "spend is high" from "spend is anomalous" — a difference measured in days between the two.

Caps: trip, don't just notify

The ledger feeds the stop. A notification is a log line; a tripped cap is a load-bearing wall. If the shape anomaly is the signature of an unattended run, then by definition no human is at a keyboard, and the control must not require one: per-key (and ideally per-task) spend caps that trip, returning a clean 4xx that the client does not auto-retry. The notification arrives after the wall, not instead of it.

Sketch of the aggregation and trip logic — the shape matters more than the library:

def on_hour_close(key: str, hour_spend: float, baseline: float) -> None:
    if hour_spend > baseline * 3 and hour_spend > 5.0:
        page(f"{key} at ${hour_spend:.2f} in one hour (baseline ${baseline:.2f})")
    if hour_spend >= KEY_CAP:
        trip_key(key)  # clean 4xx from here on; no auto-retry

When a full platform actually pays

The honest version: at some scale, the platform answer flips. Three triggers, visible in discussions of what teams outgrow CSVs for and in what per-key tracking is used for at scale: multiple teams on multiple keys where nobody owns the "who spent" answer; quality signals (evals, acceptance rate) that only make sense on the same row as the cost — "dollars per accepted answer" needs both columns; and org-level audit or residency requirements a ledger CSV does not meet. A minimal per-key cost tracker done well — an issue log against an open-source router is the archetype for what "enough" looks like — is the middle path. Until one of the three triggers is real, the CSV answers the three questions and the tooling bill is $0.

A metered endpoint exposes the ledger row per request by default — per token, per key — so the ledger starts as a tail -f on the API's usage feed instead of a sidecar in the app. The exact cap semantics, per key and per task, are in the docs; what an endpoint charges, including the answer to "what's the markup?" (none), is in pricing.