What is classifier-free guidance?

A diffusion modelDiffusion ModelA generative model that iteratively denoises data starting from random noise makes an image by starting from pure noise and removing a little of it at each of many steps, guided by your prompt. Left to itself, though, it follows the prompt only loosely. Classifier-free guidance is the reason "a red bicycle against a blue wall" actually comes out red and blue. Nearly every text-to-image model uses it, and the "guidance scale" or "CFG scale" setting in image tools controls it.

The idea in 3 steps

At every denoising step the model makes 2 predictions:

1. What the noise looks like with your prompt.
2. What it looks like without any prompt.

The gap between them is the effect of the prompt: the direction in which the prompt pulls the image. Guidance takes the prompt-free prediction and moves from it along that direction, and then further, multiplied by the guidance scaleGuidance ScaleA hyperparameter controlling the strength of conditional guidance in diffusion sampling.. At a scale of 1 you get the plain prompted prediction. Above 1 the prompt's influence is exaggerated.

Why "classifier-free"

An earlier method steered images with a separate classifier model that judged, at each step, how much the noisy image looked like the target class. That meant training an extra model that could read noisy images. In 2022 Jonathan Ho and Tim Salimans at Google showed you could skip it. During training, the prompt is simply dropped some of the time, so one model learns both the prompted and the unprompted prediction. At generation time you ask it for both and combine them.

Turning the dial

• Low scale: more varied, natural images that may drift away from what you asked.
• Medium scale: what most tools default to, a balance of following the prompt and looking natural.
• High scale: follows the prompt closely, but colours oversaturate and odd artefacts appear.

It is a trade between faithfulness and quality, much as TemperatureTemperatureA setting that controls how much chance goes into each word a model picks. Low gives focused, predictable answers; high gives more varied ones. is for text.

Negative prompts ride on it

A negative promptNegative PromptText describing what the model should avoid generating, used in diffusion or text‑to‑image pipelines. replaces the "without any prompt" prediction with a prediction for the things you do not want, such as "blurry, extra fingers". Guidance then pushes away from those as well as towards your prompt. That is why negative prompts only work in models that use this kind of guidance.

What it costs

2 predictions per step means about twice the compute per image. Some newer models are trained ("distilled") to give guided results in a single prediction, which is why their tools ask you to keep the guidance scale at a low, fixed value.