What is self-consistency prompting?

Asking a model the same question several times with some randomness, letting it reason each time, and taking the answer it reaches most often.

intermediate · Also called Self-Consistency Prompting, Majority Voting

Ask a model a tricky maths question once and it may slip on a step and give a wrong answer with full confidence. Ask it 10 times and the slips tend to land in different places, while the correct reasoning tends to arrive at the same number. Self-consistency uses that: sample several answers and take a vote.

How it works

Prompt the model to reason step by step, a chain of thought.
Run the same prompt several times, 5 to 40 is typical, with Temperature above 0 so each run takes a different path.
Pull out the final answer from each run.
Return the answer that appears most often.

The reasoning text itself is thrown away. Only the final answers are counted.

Why it works

There are many ways to reason badly, and they scatter: one run drops a minus sign, another misreads the question. Sound reasoning paths tend to converge on the same result. So when several independent attempts agree, that answer is more likely right than any single attempt.

A Google team, with Xuezhi Wang as first author, tested this in 2022. Over plain chain-of-thought prompting, voting raised accuracy by 17.9 percentage points on GSM8K, a set of school maths word problems, and by smaller but clear margins on other reasoning tests.

What it costs

Every sample is a full answer, so 10 samples cost about 10 times the output tokens and, unless run in parallel, 10 times the wait. It is a direct trade of money for accuracy.

Where it fits, and where it does not

It needs answers that can be compared: a number, a label, a choice from a list, a yes or no. For an essay or a summary there is nothing to count, since every run is different. Variants for open-ended text ask a model to pick the most consistent of the drafts instead, which is close to using an LLM as a judge.

Today's reasoning models already explore several paths inside their own thinking, so the gain from voting on top of them is usually smaller than in 2022. It is still worth trying for high-stakes extraction or classification: if 5 runs disagree, that disagreement is a useful signal to send the case to a person.

Sources

Where each fact comes from

Inference
Using a trained model to produce an answer. Training builds the model once; inference is every time someone asks it something, and it is what you pay for per token.
Chain of Thought
Having a model write out the steps of its reasoning before it gives an answer. Once a prompting trick, now built into "thinking" models.
Temperature
A setting that controls how much chance goes into each word a model picks. Low gives focused, predictable answers; high gives more varied ones.
Prompt Engineering
Writing and testing the instructions, context and examples you give a model so it does a job reliably, and measuring each change instead of guessing.
LLM as a Judge
Using a language model to grade another model's output against a rubric, for qualities code cannot check. It scales to thousands of answers, but must be checked against a person's verdicts before it is trusted.