What is logit bias?

Before a language model writes each TokenTokenThe small piece of text a language model reads and writes, often part of a word. Prices, limits and the context window are all counted in tokens., it gives every token in its vocabulary a raw score, called a logit. Higher means more likely. Those scores are turned into probabilities and one token is picked. Logit bias lets you reach in at that moment and add your own fixed amount to the scores of tokens you choose.

How it is set

In OpenAI's Chat Completions API it is the logit_bias field: a map from token IDs to a number from -100 to 100. The number is added to that token's logit at every step.

• -100 effectively bans the token.
• 100 effectively forces it whenever it is allowed.
• Small values, between -1 and 1, gently lower or raise the chance.

The effect of a given value depends on the model, so anything between the extremes needs testing.

The catch: it works on tokens, not words

The map takes token IDs, not words. You first have to run the word through the model's own TokenizerTokenizerThe part that cuts text into the tokens a model reads, and turns the model's tokens back into text. Each model family has its own. to find its IDs, and the same word can be several different tokens. "Hello", " hello" with a leading space, and "HELLO" are usually separate tokens, and a long word may be split into pieces. Ban one and the model may simply write another form. Banning a common piece can also break unrelated words that share it.

So logit bias is good at small, closed jobs and poor at "never mention this topic".

What it is good for

• Forcing a label. For a classifier that must answer "yes" or "no", push both tokens up strongly and keep the answer to 1 token.
• Removing a habit. Lower a token the model overuses, such as a filler word or a particular character.
• Keeping out a token that breaks a downstream parser.

Better tools for most jobs

Most of what people once did with logit bias now has a cleaner tool. Structured outputStructured OutputA setting that makes a model's answer follow a JSON schema you give it, so your code can read fields such as a category or an order number without parsing free text. with a JSON schema or an enum restricts the answer to the allowed values directly. A clear instruction in the prompt handles style. Presence and frequency penaltiesPresence PenaltyA penalty applied to tokens already present in the context to encourage new topics. cut repetition without naming tokens. And not every API offers logit bias at all, so check your provider's reference before building on it.