LLM-as-a-judge

What is LLM-as-a-judge?

LLM-as-a-judge is the practice of using a language model to grade the output of another language model. You give the judge the question, the answer, a description of what a good answer looks like, and it comes back with a score and usually a short explanation.

The reason this exists is a practical problem. For a classification task you can check the answer against a known label and count how often it is right. For an open-ended answer you cannot. There is no single correct summary of a report, no single correct reply to a customer, and no single correct way to explain a policy. If the only reliable judge is a person, then you can evaluate a few dozen answers a week, which is nowhere near enough to tell whether a change to your prompt made things better.

The researchers who put the approach on the map framed it as a scalable and explainable way to approximate human preference, and that phrasing is worth holding on to. You are not measuring truth. You are approximating what a careful human reviewer would have said.

How you set up a judge

There are three shapes, and picking the right one matters more than picking the right model.

Pairwise comparison. You give the judge two answers to the same question and ask which one is better. Of the three shapes this one is the most reliable, because comparing is a much easier job than assigning an absolute score. Use it when you are deciding between two prompts, two models or two versions of a system.

Single answer grading. You give the judge one answer and a scale, and it returns a score. You need this shape for production monitoring, since there is nothing to compare against. It is also the shape where scores drift most, so keep your scale short. Three or four levels with a written description of each is far more stable than a score out of ten.

Reference-guided grading. You give the judge a correct answer alongside the one being scored. This is much stronger than it sounds, especially for anything with a factual core, because the judge no longer has to know the answer, only to compare.

Whichever shape you use, the quality of your instruction to the judge decides almost everything. "Score this answer out of ten" gives you noise. "Score this answer as good, adequate or poor, where good means every claim is supported by the source material provided" gives you something you can act on.

The biases you are designing around

The original research on this identified a set of failure patterns, and they show up in practice with tedious regularity.

Position bias. In a pairwise comparison, the judge favours whichever answer it sees first. The standard fix is simple: run every comparison twice with the order swapped, and only count it as a win when both runs agree.

Verbosity bias. Longer answers score higher, whether or not the extra length adds anything. If you are comparing a concise system to a wordy one, expect the wordy one to look better than it is.

Self-preference. A judge tends to rate output from its own model family more highly. Using a different model as judge than the one you are evaluating is a cheap way to reduce this.

Limited reasoning. On tasks that need actual reasoning, maths, multi-step logic, careful reading of a contract, the judge has the same weaknesses as the model it is judging. It cannot reliably catch an error it would have made itself.

How far you can trust the score

Here is the honest ceiling. A strong judge model agrees with human preference about as often as two human reviewers agree with each other. That sounds like a limitation, and it is, but it is also the point: human review is not a gold standard either. Two careful people reading the same customer reply will often disagree about whether it was good enough.

What follows from that is a rule you can work with. An automated score is good enough to spot a change, to rank two versions against each other, and to flag answers worth a human look. It is not good enough to be the last word on an individual case. If a score determines whether a customer gets a refund or whether a document goes out unreviewed, put a person in the loop.

The practical way to build confidence is to calibrate. Take fifty answers, have a person grade them, have the judge grade them, and see where they disagree. Those disagreements tell you more about your instruction to the judge than any amount of tuning does. Repeat it whenever you change the judge model or the instruction.

What to watch out for with LLM-as-a-judge

Never let the judge grade against a vague standard. A criterion you cannot explain to a new colleague in one sentence is a criterion the judge will interpret differently every run.

Watch out for the judge grading its own homework. If the same model writes the answer and scores it, you have built a machine that tells you it is doing well.

A score with no explanation is hard to act on. Ask the judge to say why. Even if you do not read every explanation, you need them the day the scores go strange.

You are paying for a second model call. Judging is not free, and on a high volume application the evaluation bill can approach the production bill. This is a good reason to sample rather than to score everything.

Changing the judge invalidates your history. A new judge model or a rewritten instruction gives you a new scale. If you compare this month against last month across that change, you are measuring your evaluator and not your application.