# What is streaming in an LLM API?

Getting a model's answer in small pieces as it is written, instead of all at once at the end, so people see text start sooner and long answers do not hit a timeout.

- Page: https://tecyfy.com/concepts/streaming
- Subject: How AI works (https://tecyfy.com/concepts)
- Level: intermediate
- Also called: Streaming Responses, Token Streaming, Server-Sent Events

A model writes its answer one [Token](https://tecyfy.com/concepts/token) at a time. Without streaming, your code waits until the last token is written and then gets the whole answer in one response. With streaming, the API sends each piece as soon as it exists. The total time is about the same. What changes is when the first words arrive.

## Why it matters

There are 2 reasons, and the second is the one that bites engineers.

The first is the wait a person sees. Someone watching a chat window can start reading as soon as the first words arrive, so streaming cuts the blank wait to the time the first token takes, while the answer itself takes as long as before. Measure both: the time to the first token and the time to the last.

The second is that long answers can fail without it. An HTTP request that sits silent for minutes may be cut by a network along the way, and the Claude Python SDK gives up on a non-streaming request after 10 minutes by default. It also refuses a non-streaming request it expects to run longer than that and tells you to stream instead. With streaming, data keeps flowing, so the connection stays alive.

## What arrives, in order

A stream is a series of server-sent events. In the Claude API they come in a fixed order:

1. `message_start`, with the message's id and model.
2. For each block of the answer, `content_block_start`, a run of `content_block_delta` events carrying the text, then `content_block_stop`.
3. `message_delta`, carrying the stop reason and the token counts so far.
4. `message_stop`.

`ping` events can turn up at any point to keep the line open. On models that think before answering, the thinking blocks come first; when their text is hidden, which is the default on [Claude Opus 5.5](https://tecyfy.com/prices/claude-opus-5-5), nothing of them is streamed and the answer text starts sooner.

## The catch: errors after the 200

Without streaming, an error comes back as an HTTP status, such as 529 for an overloaded server. With streaming, the request has already succeeded with a 200 when the first event arrives, so a failure halfway through comes as an `error` event inside the stream. Code that only checks the status code will show half an answer as if it were whole. Handle the error event, and decide what the person sees when an answer stops early.

## In code

You rarely read the events by hand. The SDKs collect them for you:

```python
import anthropic

client = anthropic.Anthropic()
with client.messages.stream(
    model="claude-opus-5-5",
    max_tokens=16000,
    messages=[{"role": "user", "content": "Explain what a context window is."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
    final = stream.get_final_message()
print(final.usage.output_tokens)
```

`text_stream` gives the text as it comes, and `get_final_message` gives the same complete message a non-streaming call returns, with its stop reason and usage, so logging and cost tracking work the same either way.

## Related

- [Token](https://tecyfy.com/concepts/token): The small piece of text a language model reads and writes, often part of a word. Prices, limits and the context window are all counted in tokens.
- [Latency](https://tecyfy.com/concepts/latency): How long a user waits for a model's answer, measured as the time to the first token and the time to the last, and judged by the slowest requests, not the average.
- [Inference Latency](https://tecyfy.com/concepts/inference-latency): The time a model takes to produce an answer, split into reading the prompt (prefill, which sets the time to first token) and writing the answer one token at a time (decode).
- [API](https://tecyfy.com/concepts#api): Application Programming Interface - a set of protocols and tools for building software applications

## Sources

- [Streaming messages](https://platform.claude.com/docs/en/build-with-claude/streaming)
- [Python SDK](https://platform.claude.com/docs/en/cli-sdks-libraries/sdks/python)
- [Thinking](https://platform.claude.com/docs/en/build-with-claude/thinking)