The full answer
Temperature controls how random a language model's choice of the next token is. Low temperature makes the model pick the most likely tokens almost every time, giving focused, consistent output; high temperature spreads the choice to less likely tokens, giving more varied, creative and sometimes less accurate output.
How it works At each step the model produces a score (logit) for every possible next token. Before turning scores into probabilities, they are divided by the temperature:
- Temperature below 1 sharpens the distribution: likely tokens become even more likely.
- Temperature of 1 leaves the model's distribution unchanged.
- Temperature above 1 flattens it: unlikely tokens get a bigger share.
- Temperature near 0 approaches greedy decoding, always picking the top token.
Effects on output
| Temperature | Behaviour | Typical uses |
|---|---|---|
| Low (0 to 0.3) | Consistent, conservative, can be repetitive | Extraction, classification, code, factual Q&A |
| Medium | Balanced | General assistant answers, explanations |
| High | Varied, surprising, more errors and drift | Brainstorming, fiction, alternative phrasings |
Things that are often misunderstood
- Temperature 0 is not fully deterministic. Hosted APIs can still give different outputs for the same input, because of how computations are batched and run.
- Low temperature doesn't make answers true. A model can be consistently wrong; accuracy comes from good context and instructions.
- High temperature isn't the only route to creativity. Asking for many distinct ideas, or giving unusual constraints, often works better than turning randomness up.
- Ranges differ by provider: for example 0 to 1 in Anthropic's API and 0 to 2 in OpenAI's.
- Some models don't let you change it. Many newer models, especially reasoning models, fix temperature and reject other values; prompt instructions then do the work.
Related settings Top-p (nucleus) and top-k sampling limit which tokens can be chosen at all, while temperature reshapes the probabilities. Providers generally advise adjusting temperature or top-p, not both.
Choosing a value Start with the provider's default, then test a lower value for tasks needing consistency and a higher value for creative ones, scoring outputs on a test set. Running each test case several times shows how much the variation actually matters for your task.
Last updated 16 October 2026