How to reduce LLM API costs, in order of effort

To reduce LLM API costs, take 4 steps in order of effort: cache the part of the prompt that repeats, send work no one is waiting for as a batch at half price, ask for less output, and move to a smaller model where your eval says it holds. The first 2 cannot change an answer, so they need no new testing. The last 2 can, so test them before they ship.

On a support assistant answering 5,000 questions a day on Claude Opus 5.5, the 4 steps take the bill from $6,780 a month to $33. The first step alone takes it to $2,221.

First, find where the money goes

Every response reports its tokens: input, output and, on Claude, the tokens written to and read from the cache. Log them with the feature that made the call, then multiply by the prices. Most bills have 2 big lines:

• The repeated start. A chat or an agent sends its system prompt, tools and documents on every request. In our example that is 8,000 of the 8,300 input tokens.
• The output. On the current Claude models output costs 5 times as much as input: $20 per million tokens on Opus 5.5 against $4.

The example: an 8,000-token start, a 300-token question and a 600-token answer, 5,000 times a day over 10 busy hours, on Opus 5.5. Each request costs about 4.5 cents, $6,780 a month. Choose the steps below and watch the bill after each one.

Try itCut an LLM bill, one step at a time

Model now

·Claude Opus 5.5·Claude Sonnet 5.5

Steps

·All·Cache the start·Batch what can wait·Half the output·Smaller model

Requests a day

5Krequests

over 10 busy hours

1001K10K100K

Share that can wait

30%of requests

no one is waiting for these answers

0%50%100%

1. Today$6,780
2. 1Cache the fixed start$2,221
3. 2Send what can wait as a batch$1,888
4. 3Ask for half the output$1,123
5. 4Move to Claude Haiku 5.5$33.17

$33.17a month, down from $6,780

1. Cache the part that repeats

Prompt cachingPrompt CachingLetting the API keep the processed start of a prompt for a few minutes, so later requests that begin the same way are cheaper and quicker. keeps the processed start of a prompt for a few minutes, and the next request that begins with the same tokens reads it at a fraction of the price: 5% of the input price on Opus 5.5 and Sonnet 5.5, 10% on most other Claude and OpenAI models. In the example the start is read from the cache on almost every request, and the bill falls by two-thirds, to $2,221.

To make it work, put what never changes first, in the order tools, system prompt, messages, and keep anything that changes (a timestamp, a name) at the end. OpenAI caches by default. On Claude, add cache_control to the request. Then check cache_read_input_tokens: 0 on every call means the cache is not working. Cached tokens also stop counting against Claude's input-tokens-per-minute limit on most models, so the same limit carries more traffic.

The prompt caching calculator prices your own prompt with the 5-minute and the 1-hour cache, and prompt caching vs KV caching explains what is kept.

2. Send what can wait as a batch

Anthropic's Message Batches API and OpenAI's Batch API both charge half the normal price. Most Claude batches finish within an hour, and both allow up to 24. Anything no one is waiting for qualifies: tagging yesterday's tickets, running an eval, writing a weekly report, enriching a catalogue.

In the example 30% of requests can wait, and batching them takes the bill from $2,221 to $1,888. Caching still works inside a batch, on a best-effort basis. The production module of the free AI engineering course sends a backlog as a batch with a custom_id on each request, because results can come back in any order.

3. Ask for less output

Output is the expensive half, so a shorter answer saves more per token than a shorter prompt:

• Set max_tokens to what a good answer needs.
• Ask for the length you want in the prompt: "2 sentences", "a list of at most 5".
• Ask for structured output (a label, a JSON object) where code reads the answer.
• Lower the effort setting on simple tasks. It controls how many tokens the model spends on an answer, its thinking included.

Halving the 600-token answer takes the example from $1,888 to $1,123. A shorter answer can also be a worse one, so this step goes through your eval.

4. Move to a smaller model where the eval holds

Claude Haiku 5.5 costs $0.10 per million input tokens and $0.50 per million output tokens, a fortieth of Opus 5.5, and Anthropic aims it at high-volume tasks such as classification, extraction and routing. Moving the whole example to it takes the bill to $33.

Whether that is safe is a question for your eval, not your budget. Run the same test cases on both models and compare. If the small model fails on a share of requests, send only those to the large one. That is model routingModel RoutingSending each request to the cheapest model that handles it well, by rules, a classifier or a cascade, and checking with an eval that quality holds., and RouteLLM found routers that cut costs by more than half in some cases with no loss in quality. Our model picker starts from the job and the score a model needs.

Also: send less context

Every token of a long prompt is paid for on every request that is not cached. Retrieval that sends the 3 passages a question needs, instead of the whole handbook, cuts the input before any of the steps above. Trimming a conversation's oldest turns does the same for long chats, at the cost of what the model can remember.

Check the bill each month

Prices change, and a feature's traffic grows. Price each feature on the price table or the cost calculator before you build it, and compare with what you log once it runs. An organization admin can read the totals by model and workspace from Anthropic's Usage and Cost API. If the estimate and the bill drift apart, the usage fields tell you which line moved.