What is streaming in an LLM API?

A model writes its answer one TokenTokenThe small piece of text a language model reads and writes, often part of a word. Prices, limits and the context window are all counted in tokens. at a time. Without streaming, your code waits until the last token is written and then gets the whole answer in one response. With streaming, the API sends each piece as soon as it exists. The total time is about the same. What changes is when the first words arrive.

Why it matters

There are 2 reasons, and the second is the one that bites engineers.

The first is the wait a person sees. Someone watching a chat window can start reading as soon as the first words arrive, so streaming cuts the blank wait to the time the first token takes, while the answer itself takes as long as before. Measure both: the time to the first token and the time to the last.

The second is that long answers can fail without it. An HTTP request that sits silent for minutes may be cut by a network along the way, and the Claude Python SDK gives up on a non-streaming request after 10 minutes by default. It also refuses a non-streaming request it expects to run longer than that and tells you to stream instead. With streaming, data keeps flowing, so the connection stays alive.

What arrives, in order

A stream is a series of server-sent events. In the Claude API they come in a fixed order:

1. message_start, with the message's id and model.
2. For each block of the answer, content_block_start, a run of content_block_delta events carrying the text, then content_block_stop.
3. message_delta, carrying the stop reason and the token counts so far.
4. message_stop.

ping events can turn up at any point to keep the line open. On models that think before answering, the thinking blocks come first; when their text is hidden, which is the default on Claude Opus 5.5, nothing of them is streamed and the answer text starts sooner.

The catch: errors after the 200

Without streaming, an error comes back as an HTTP status, such as 529 for an overloaded server. With streaming, the request has already succeeded with a 200 when the first event arrives, so a failure halfway through comes as an error event inside the stream. Code that only checks the status code will show half an answer as if it were whole. Handle the error event, and decide what the person sees when an answer stops early.

In code

You rarely read the events by hand. The SDKs collect them for you:

import anthropic

client = anthropic.Anthropic()
with client.messages.stream(
    model="claude-opus-5-5",
    max_tokens=16000,
    messages=[{"role": "user", "content": "Explain what a context window is."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
    final = stream.get_final_message()
print(final.usage.output_tokens)

text_stream gives the text as it comes, and get_final_message gives the same complete message a non-streaming call returns, with its stop reason and usage, so logging and cost tracking work the same either way.