The full answer
Split the bill until the growth sits in one place. Cost is requests times tokens per request times the price of each kind of token, so ask which of those grew and for which feature, model and customer. When traffic rose 20% and cost rose 200%, the money usually went to more tokens per request, a pricier model, broken prompt caching, retries and loops, or traffic that is not your users.
1. Get the numbers to attribute cost
Tag every model call with the feature, customer or tenant, model, prompt version and environment, and log its usage: input tokens, cache write tokens, cache read tokens and output tokens. With those, the bill can be broken down like any other cost report. Without them, you are guessing, so add them first.
2. Break it down
| Question | What it catches |
|---|---|
| Which feature or API key grew? | One new feature, a batch job, a test environment left running |
| Which model? | A switch to a larger model, or a fallback that kept serving after an incident |
| Input or output tokens? | Longer prompts and histories, or longer answers |
| Cache reads fell, cache writes rose? | Caching broke: you pay more than the base price for writes and get no cheap reads |
| Calls per user task? | Agent loops, retries, a client that resends on timeout |
| Top customers by spend? | One tenant, a script, or abuse |
3. The usual causes
- Growing context. A chat that resends the whole conversation every turn costs more on each turn. Average input tokens per call creep up as conversations get longer, with no change in traffic.
- More retrieved text. Someone raised the number of chunks from 5 to 20 to fix a quality issue.
- Broken prompt caching. On the Claude API a cache write costs 1.25 times the normal input price (5-minute cache) and a read about a tenth or less. If something now changes the start of the prompt on every request (a timestamp, a reordered tool list), you pay for writes and never read them back.
- Model choice. Prices between models of the same family differ many times over; Opus 5.5 lists $4 per million input tokens against Haiku 5.5 from $0.10. A config change or a fallback path can move traffic to the expensive one.
- Agents. A loop that retries a failing tool, or an agent that reads whole files, can use many times the tokens of a normal run.
- Abuse. A public endpoint without per-user limits can be used as a free model by someone else.
4. Fix and keep it fixed
- Fix the cause you found: trim or summarise history, restore the stable prompt prefix, cap retrieved text, route simple requests to a smaller model, put a step and token budget on agent runs.
- Move work that does not need an instant answer (nightly classification, backfills) to the Batch API, which is half price on the Claude API.
- Add per-user rate limits and quotas, and limits on input size.
- Set budgets and alerts per feature on daily spend and on tokens per request, so a change shows up in a day, not on the invoice.
What interviewers want: cost as requests times tokens times price, attribution by feature and model, the specific suspects (history growth, caching, model switches, loops, abuse), and alerts that catch the next spike early.
Last updated 16 October 2026
Sources