The full answer
An LLM judge is fast and cheap and, done well, agrees with people about as often as people agree with each other. But it has known biases: it favours the first answer shown (position bias), longer answers (verbosity bias) and its own model's style (self-enhancement bias), and it can miss errors it cannot itself reason through. Correct for them with clear rubrics, swapped orders, a different judge model, and regular checks against human labels.
The evidence
Zheng et al., "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena" (NeurIPS 2023), found strong judges matched human preferences over 80% of the time, the same level at which humans agree with each other. They also documented position, verbosity and self-enhancement bias, and weak judging of maths and reasoning questions.
The weaknesses and their fixes
| Weakness | What happens | Fix |
|---|---|---|
| Position bias | In A-vs-B comparisons, the first (or second) answer wins more often | Judge both orders; count a win only if it holds in both, otherwise a tie |
| Verbosity bias | Longer answers look more thorough | Rubric that rewards concision; length limits; score criteria separately |
| Self-preference | A judge rates its own model's outputs higher | Use a different model family to judge |
| Limited reasoning | It misses a wrong calculation or code bug | Give it the reference answer; check facts with code; run the code |
| Vague scores | "Rate 1 to 10" drifts between runs | Rubric with defined levels; pass/fail per criterion; ask for reasoning before the score |
| Prompt sensitivity | Small wording changes move scores | Fix the judge prompt and version it like code |
How to set up a judge well
- One criterion per question. "Is every claim supported by the sources? yes/no" beats an overall quality score.
- Give it what it needs: the input, the output, the sources and, when you have one, the reference answer.
- Ask for reasoning, then the verdict, in a fixed format your code can parse.
- Calibrate: have people label 50 to 100 cases, and measure how often the judge agrees with them. Re-check whenever you change the judge model or prompt.
- Watch it over time: track the judge's agreement with people on a sample, not just the scores it gives.
When not to use one
When the check can be done in code (format, exact values, does the test pass?), or when the decision is high-stakes and needs a person. A judge also costs tokens: a judge call for every production request can double your bill, so sample.
A good closing line
"I treat the judge as a measuring instrument: I calibrate it against people before trusting it, and I re-calibrate when anything about it changes."
Last updated 16 October 2026
Sources