LLM-as-a-Judge: Can a Model Grade Another Model's Homework? (How Eval Pipelines Do It)

You have 4,000 support-bot answers and no defensible way to grade them. Human reviewers would take weeks you do not have, so you set an LLM as a judge over them, ask a frontier model to read every answer, and immediately feel uneasy: you have just asked a language model to mark a language model's homework.

Here is the honest answer this article defends: LLM-as-a-judge works better than most engineers expect and worse than the vendors selling it claim, and the difference between those two outcomes is whether you validated the judge. What follows is the working manual: how a judge prompt is built, the three scoring patterns, the biases with the research and measured magnitudes behind them, the mitigations that survive production, and how to check your evaluator model against human labels before trusting a single score.

What Is LLM-as-a-Judge?

The snippet answer, no preamble:

LLM-as-a-judge is an evaluation technique in which a large language model scores or ranks another AI system's outputs against a written rubric, instead of a human doing it. It is used to grade open-ended text, such as answers, summaries and agent steps, where exact-match metrics cannot work.

Three moving parts matter for everything below: the candidate output (what is being graded), the rubric (what "good" means, in words), and the judge model plus its prompt (the thing doing the grading). Every failure in this article is a failure of one of those three.

Three things readers routinely confuse, in one line each. A judge is not a benchmark: benchmarks like MMLU rank models on fixed academic tasks, while a judge grades your application's outputs on your task (model-shopping via LLM benchmarks is a different article). A judge is not a string metric: BLEU and ROUGE count word overlap with a reference, which is why they fail below. And a judge is not a replacement human: it is an approximation of a human rater, and H2-6 depends on that distinction. The technique also has a formal literature, including a widely cited survey on LLM-as-a-judge; this is a studied method, not a tweet.

Why String-Matching Metrics Broke (and What Replaced Them)

Here is the failure in one worked contrast. Reference answer: "You can return an item within 30 days of delivery." Candidate A: "Returns are accepted up to a month after your order arrives." Correct, and near-zero BLEU, because almost no words overlap. Candidate B: "You can return an item within 30 days of purchase." Subtly wrong, because purchase is not delivery, and high BLEU, because eight of ten words match. The metric punished the right answer and rewarded the wrong one.

What you actually want to measure is semantic: is this grounded, is it relevant, is it complete, is it safe? Those are judgement calls, which is why modern LLM evaluation converged on using models as raters; a survey of the field now catalogues the variants. (Eval fundamentals and dataset construction are covered in our sibling evals guide.) Deterministic scoring, the precision-and-recall world of how classical model evaluation metrics work, still applies wherever answers are checkable; judges enter exactly where it stops applying.

How Closely Does an LLM Judge Agree with Humans?

The headline number, from the primary source: in Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (Zheng et al., NeurIPS 2023), GPT-4 matches human preferences, controlled expert votes and crowdsourced votes alike, at "over 80% agreement, the same level of agreement between humans".

Two things every second-hand citation drops. First, the context: human experts agree with each other at roughly that rate, so the question was never "is the judge perfect" but "is it as consistent as the humans it replaces". Second, the caveat: that figure comes from open-ended chat preference on one benchmark, with one judge model, in 2023. It does not transfer to your domain, your rubric, or your model. Nobody's published number tells you anything reliable about your pipeline, which is why the validation section of this article exists.

How an LLM Judge Actually Works

Before the tooling, the truth most vendor pages skip: a judge is a prompt, not a product. Every tool in the tools section wraps a prompt like the one you will write below, sends it to a model, and parses the reply. Understanding the prompt is understanding the technique.

Build an AI-First Career, Master the Complete Skillset

Choose from our industry-leading programs designed for career success

NSDC Certified

Modern Software and AI Engineering Program

Master full-stack development with AI integration

12 MonthsDuration

AI-LedCurriculum

Career SupportSupport

+1000 more

Go to Program

NSDC Certified

Modern Data Science and ML with specialisation in AI

Advanced data science techniques with AI specialization

12 MonthsDuration

AI-LedCurriculum

Career SupportSupport

+1000 more

Go to Program

NSDC Certified

Advanced AIML with Specialisation in Agentic AI

Deep dive into AIML with focus on Agentic systems

12 MonthsDuration

AI-LedCurriculum

Career SupportSupport

+1000 more

Go to Program

NSDC Certified

DevOps, Cloud & AI Platform Engineering

Build and manage AI-powered cloud infrastructure

12 MonthsDuration

AI-LedCurriculum

Career SupportSupport

+1000 more

Go to Program

NSDC Certified

AI Engineering Advanced Certification by IIT-Roorkee

Premier AI engineering certification from IIT-Roorkee

3 MonthsDuration

AI-LedCurriculum

Career SupportSupport

Go to Program

NSDC Certified

AI Forward Deployed Engineer Program

Full-stack engineering, production AI and client-facing consulting

12 MonthsDuration

AI-LedCurriculum

Career SupportSupport

+1000 more

Go to Program

The Five Parts of a Judge Prompt

• Role and task framing. "You are evaluating a customer-support answer for groundedness." A vague role produces a vague grader.
• One narrowly defined criterion. "Is this answer good?" is unanswerable; "is every factual claim in this answer supported by the provided context?" is answerable. This line is the single highest-leverage sentence in the prompt.
• A scoring scale with every point defined in words. A bare "score 1 to 5" lets the model reinvent the rubric on every call.
• The inputs, clearly delimited. Candidate output, plus context or a reference where the pattern needs it, wrapped in XML tags or backticks so the judge cannot confuse content with instructions.
• A fixed output format. Reasoning first, then the score, in strict JSON with fixed keys. Order matters, and the mitigations section explains why.

One habit that separates working judges from decorative ones: give the LLM judge the same context the candidate system had. A judge asked to assess groundedness without the retrieved documents is guessing, and quietly rewards confident hallucinations.

Choosing a Scoring Scale (Binary, Likert, or Continuous)

Among LLM evaluation metrics, scale choice is the least discussed and most consequential decision. The practitioner recommendation: prefer binary or three-point scales for anything you intend to act on. Binary criteria are easier for humans to label (which makes validation tractable), judges are far more self-consistent on binary calls, and a 1 to 10 score has no defensible meaning, since nobody can say what separates a 6 from a 7, including the judge.

Watch for score clustering: on integer scales, judges pile most outputs onto one or two values (typically 3 and 4 on a 1 to 5 scale), destroying the resolution you wanted. The fix is decomposition: replace "rate helpfulness 1 to 10" with four binary checks (answers the question, grounded, no unsupported claims, right tone) and sum them; every point of the resulting 0 to 4 score means something specific.

What G-Eval Does Differently

G-Eval (Liu et al., EMNLP 2023) has the model generate its own chain-of-thought evaluation steps from a short task description, then fill in a scoring form. The clever part: it takes a probability-weighted average over the score tokens instead of the single emitted integer, which recovers fine-grained scores and directly attacks score clustering. With GPT-4 as backbone it reached a 0.514 Spearman correlation with humans on summarisation, well ahead of prior automated metrics. Two knowing-this-helps-you details: the weighting needs access to token probabilities, which means how large language models actually generate text matters here; and the authors themselves flag that LLM evaluators may be biased toward LLM-generated text, an early warning of the self-preference problem in H2-4.

The Three Judging Patterns

LLM as a judge evaluation comes in exactly three structural patterns, formalised in the MT-Bench paper most judge tooling descends from: single answer grading, pairwise comparison, and reference-guided grading. Pattern choice depends on whether your system is a retrieval pipeline, an agent, or a single-turn generator, which is also the dividing line in production-grade RAG and LLM systems curricula.

Single-Output Scoring (Direct Assessment)

The judge sees one output and a rubric, and returns a score. It costs one judge call per candidate, so it scales linearly and is the only pattern practical for scoring live production traffic. You get an absolute number you can plot, threshold and alert on, which is why it powers regression dashboards. The weaknesses follow you home: absolute scores are poorly calibrated, drift across model versions, and suffer the leniency and clustering problems of the next two sections. Run it on a sampled slice of production, and stop there; monitoring LLM quality in production is its own discipline beyond this page.

Pairwise Comparison

The judge sees two candidate answers to the same input and picks the better one, or a tie. Relative judgement is what LLMs are best at, it mirrors how humans express preference, and it posts the highest agreement with humans of the three patterns, as the arena-style leaderboards show. But you learn that B beat A, not whether either was acceptable. Cost also grows quadratically: a full ranking of n candidates means O(n²) pairs, which is why arenas sample pairs and fit an Elo or Bradley-Terry rating. And this is the pattern where position bias bites hardest, as H2-4 shows with numbers. Use it for the questions that genuinely are relative: prompt A/B decisions, model selection, "did this change help".

Reference-Based Grading

The judge sees the candidate answer and a gold reference, and grades semantic agreement with it, not string overlap. This exists because judges kept failing math and reasoning tasks, and the fix is the single strongest number in this article after the 80% figure: on a 10-question math set graded in both presentation orders, the paper's authors measured judge failure rates of 14/20 with a default prompt, 6/20 with chain-of-thought, and just 3/20 once a reference answer was supplied (a failure here means the judge marked a wrong answer correct). Wherever a defensible reference exists, this is the most reliable pattern by a distance. The catch is human-priced: someone has to write the references, and only tasks with a knowable right answer qualify.

The Biases That Make LLM Judges Unreliable

Name the framing first, because it sets expectations correctly: these are not bugs you patch out. They are properties of using a next-token predictor as a rater. You manage them; you do not eliminate them. Each bias below gets the same four beats: what it is, why it happens, what the research measured, and how to detect it in your own pipeline today.

Position Bias: Candidate Order Changes the Verdict

What: in pairwise comparison, swap which answer appears first and the verdict can flip with nothing else changed. Why: the judge conditions on a sequence, and attention over a long prompt is not order-neutral; sequence position and how attention and transformer architectures shape model behaviour are the mechanistic grounding for why this is structural. Measured: across MT-Bench pairs graded in both orders, the paper's bias table shows Claude-v1 was consistent only 23.8% of the time and leaned first-position in 75%; GPT-3.5 was 46.2% consistent (50% first-position lean); GPT-4 was the most consistent at 65%, still biasing first 30% of the time. Detect it yourself: run 50 pairs twice, once in each order. The share whose verdict flips is your judge bias rate; above roughly 10 to 15%, do not trust pairwise results without the swap mitigation.

Verbosity Bias: Longer Answers Score Higher

What: judges systematically prefer longer, more elaborate answers, whether or not the extra words add correctness. Why: in the human preference data these models trained on, length correlates with thoroughness, and the correlation got learned as preference. Measured: the paper's "repetitive list" attack restructured 23 answers to be longer without adding any information, and the weaker judges were fooled 91.3% of the time (Claude-v1 and GPT-3.5 alike), while GPT-4 failed only 8.7%. The effect is large enough to bend public leaderboards: Length-Controlled AlpacaEval exists precisely to remove it, and controlling for length raised the metric's correlation with Chatbot Arena rankings from 0.94 to 0.98. Detect it yourself: plot judge score against output token count; a positive slope on a task that does not demand length is bias, not quality. The product consequence deserves plain language: if your judge rewards length and you pick prompts by judge score, you will quietly evolve toward wordier answers your users never asked for.

Self-Enhancement Bias: Models Favour Their Own Outputs

What: a judge scores text it generated, or that its model family generated, above equivalent text from another family. Why: the current mechanism story is self-recognition. Panickssery et al. found a linear correlation between an evaluator's ability to recognise its own generations and the strength of its self-preference; more simply, a model also rates text partly by how probable it finds it, and its own outputs are probable by construction. Measured, with the authors' own caution: in the MT-Bench data, GPT-4 favoured itself with a 10% higher win rate and Claude-v1 with 25%, but the authors stress that with limited data and small differences they cannot confirm the effect. Report it that way; overstating this finding is exactly the citation sloppiness this article exists to fix. Detect it yourself: if you grade GPT-4 outputs with GPT-4, re-score a sample with a different-family judge and compare rankings. And the blunt version: using the same model as generator and judge is the most common mistake in this field, and the one that most flatters your metrics.

Leniency Drift and Rubric Decay

What: absolute scores creep upward over time, across long batches, prompt revisions and judge-model upgrades, without quality actually improving. Why: judges are agreeable by default (RLHF trains for agreeableness), vague rubric points get reinterpreted on every call, and when the provider silently updates the judge behind the same API name, your entire history shifts under you. The practitioner's version, as an illustrative composite: a team watches its eval dashboard improve 8% overnight and celebrates, until someone notices the judge model was upgraded that morning and the product never changed. Detect it yourself, and this is the cheapest insurance you will ever buy: freeze an anchor set of 20 to 30 examples with fixed human labels and re-score it on every judge run. If anchor-set scores move, the judge changed, not your product.

Hiring Partners:

GoogleAmazonMicrosoftFlipkartAdobe1200+ more

The Judge Can't Grade What It Can't Solve

What: on math, multi-step reasoning, domain code or specialist knowledge, a judge confidently marks wrong answers right. Why: grading is not an easier task than answering; verification-is-cheaper-than-generation holds for provable problems, not open-ended reasoning. Measured: the 14/20 to 3/20 failure-rate drop from H2-3 is the paper documenting this limitation and its reference-guided fix at once. The rule: if your judge model cannot reliably perform the task, it cannot reliably grade it either; supply a reference, or route the task to humans.

Mitigations That Actually Work

None of these make a judge unbiased. Each measurably reduces one named failure, and every item maps back to a bias so this reads as engineering, not a listicle.

• Swap the order and average. Run every pairwise comparison in both orders; count a win only when both agree, else call it a tie. The MT-Bench paper prescribes exactly this, and Wang et al.'s Balanced Position Calibration formalises it. Doubles pairwise cost; buys the most reliability per rupee in this list.
• Few-shot calibrate with your own labels. Put 3 to 5 human-graded examples in the judge prompt, spanning the scale and including one borderline case. The strongest single fix for leniency and clustering; the paper notes longer prompts made its calls 4× more expensive, so budget for the tokens.
• Reasoning before the score, never after. Require reasoning first, score last; a score-first judge writes post-hoc rationalisations, which are worthless. The reasoning text doubles as your debugging trail during validation. This is ordinary prompt design patterns that make model outputs more reliable applied to evaluators.
• Constrain the output format. Demand strict JSON with fixed keys and validate it with a schema, using structured-output modes where offered. Kills unparseable replies and invented out-of-scale scores. Cheap and non-negotiable.
• Use a different model family as judge. Never grade a model's output with itself. Directly targets self-enhancement bias and costs a config change; if you must use the same family, disclose it wherever the numbers appear.
• Ensemble judges and take the majority. Two or three judges from different families, majority verdict or averaged scores. Reliability rises, and judge disagreement becomes a free triage signal for human review. Cost scales per judge, so reserve it for high-stakes evals.
• Decompose the rubric into binary checks. Back to H2-2; listed here because it is genuinely one of the strongest fixes for clustering.
• Keep a frozen anchor set. Back to H2-4; listed so the checklist is complete. Re-score it on every run to catch drift.

How to Validate Your Judge Against Humans

State the hard rule first, because the whole SERP skips it: an unvalidated judge is a vibe check with extra steps and a bigger API bill. Beyond general agreement numbers, the only question that matters is how well your judge agrees with humans on your task, measured on your own LLM as a judge metrics for that agreement. And honesty about evidence is part of the method: what follows is practitioner guidance, not a derived statistical result, and we say so where it applies.

Build a Human-Labelled Calibration Set

Sample 50 to 150 real outputs from your actual system, not synthetic examples, and not only the easy ones. Stratify deliberately: if 95% of your outputs pass, a random sample hides almost every failure, and a judge that always says "pass" scores 95% raw accuracy while agreeing with nobody. Oversample known failures until the set is closer to balanced, because agreement statistics die on class imbalance. Have two humans label independently where you can afford it and reconcile disagreements; their agreement is your ceiling, since a judge cannot meaningfully out-agree the humans defining "correct". On sample size, the honest guidance: below ~50 labels, the confidence interval on your agreement number is too wide to act on; 100 to 150 stratified labels is where practitioners typically get a stable ship/don't-ship estimate; more helps, but marginally versus spending the effort on a better rubric. This human-labelled set is the golden dataset for LLM evaluation on your task; deeper dataset construction lives in the sibling evals guide.

Pick the Right Agreement Metric

Raw percent agreement is the naive one, and it lies. On an imbalanced set, a judge that always says "pass" looks perfect; the worked numbers below show why you use it only as a sanity check.

Cohen's kappa (κ) is for two raters on categorical labels, judge versus human on pass/fail. In words: how much better than chance the pair agrees, where 1.0 is perfect and 0 is chance. Worked example, with the arithmetic shown: you label 100 examples and the judge agrees with you on 90. Say the breakdown is 80 you both passed, 10 the judge passed and you failed, 0 the judge failed and you passed, 10 you both failed, so your own pass rate is 80% and the judge's is 90%. Expected agreement by chance is the sum of matching marginals: (0.80 × 0.90) + (0.20 × 0.10) = 0.74. Then κ = (0.90 − 0.74) / (1 − 0.74) = 0.615 ≈ 0.62. Read the lesson before the number: 90% agreement sounded shippable; κ = 0.62 is merely "substantial". That correction is why raw agreement is never enough.

Krippendorff's alpha (α) generalises kappa: more than two raters, ordinal or interval scales (so it handles 1 to 5 scores, which κ does not), and missing labels. Reach for it with three graders or a Likert scale. Spearman's ρ and Kendall's τ handle ranked or continuous scores, when what you care about is whether the judge orders outputs as a human would rather than matching exact values; most score-based eval research reports Spearman.

What Agreement Level Is Good Enough to Ship?

Use the attributed scales, not folk wisdom. The kappa bands come from Landis and Koch (1977): 0.21 to 0.40 fair, 0.41 to 0.60 moderate, 0.61 to 0.80 substantial, 0.81 up almost perfect; a convention, not a law. For alpha, the Krippendorff guidance sets α ≥ 0.80 for conclusions you rely on and α ≥ 0.667 as the floor for tentative ones.

The practitioner's answer beats the textbook's, though: the threshold depends on the decision the score drives. A judge at κ ≈ 0.4 is useless for gating a release but perfectly good for triage, like surfacing the worst 5% of production traffic for a human to read, where a false positive costs a minute. A judge gating a customer-facing deploy needs κ ≥ 0.6 and a human reviewing anything near the threshold. And the check that closes the argument: measure your judge against human–human agreement on the same set. If two annotators only reach κ = 0.5 themselves, demanding 0.8 from the judge is incoherent; your rubric is the problem, not the model.

Re-Validate When Anything Changes

Re-run validation whenever the judge model or version changes, the rubric changes, your output distribution shifts, or a new use case lands. The frozen anchor set from H2-4 is the continuous, nearly free version of this same check. Being able to run this loop is also becoming a career skill; what AI engineers are expected to know in India's hiring market increasingly includes exactly this kind of eval literacy.

A Worked Example: Grading a Support Bot's Answers

The setup, deliberately concrete: a RAG-powered support assistant for an Indian e-commerce app, answering order, refund and delivery questions from retrieved policy text, and the criterion is groundedness: is every factual claim in the answer supported by the retrieved context? Here is the complete judge prompt, built to demonstrate every rule this article has argued for:

SYSTEM:
You are evaluating the groundedness of answers from an e-commerce support bot.

CRITERION (binary, both values defined):
- grounded = true -> every factual claim in ANSWER is directly supported by CONTEXT.
- grounded = false -> at least one factual claim appears nowhere in CONTEXT,
or contradicts it. Opinion, empathy and phrasing do NOT count as factual claims.

Always write your reasoning FIRST, then the verdict. Reply with ONLY this JSON:
{"reasoning": str, "unsupported_claims": [str, ...], "grounded": true|false}

FEW-SHOT EXAMPLES (2 of 2 shown in production prompt):
[pass] CONTEXT: "Returns are accepted within 30 days of delivery."
ANSWER: "You can return it within 30 days of delivery."
-> {"reasoning": "The only claim mirrors the policy clause.",
"unsupported_claims": [], "grounded": true}

[borderline fail] CONTEXT: "Delayed orders may be eligible for compensation."
ANSWER: "You will get compensation because the order was late."
-> {"reasoning": "Policy says MAY be eligible; answer upgrades it to WILL get,
an unsupported certainty.", "unsupported_claims":
["You will get compensation"], "grounded": false}

USER TEMPLATE:
CONTEXT: <retrieved policy text>
ANSWER: <bot answer>

Note what the prompt embodies: one narrow criterion, both scale points defined in words, delimited inputs, reasoning before verdict, strict JSON, and few-shot examples that include a borderline case. Now three scored outputs:

The third row is the lesson. A strict human reviewer might reject the bot for not telling the user to check seller eligibility first, while the judge graded only what was claimed. That is not a judge error or a human error; it is a rubric ambiguity, and finding three or four of these per validation run is what calibration actually feels like. If your RAG debugging instincts need sharpening before you can read verdicts like a grader, this walkthrough helps:

Now close the loop: run this judge over your 100 human-labelled examples, compute κ per H2-6, tighten the rubric wherever the judge's own reasoning shows it misreading the criterion, and repeat. One practitioner observation worth more than any model choice: two or three rubric revisions typically move agreement further than upgrading to an expensive judge model. Fix the grader's instructions before shopping for a better grader.

When NOT to Use an LLM Judge

A judge is the tool of last resort, not first. Use the cheapest reliable method that answers your question, and reach for a judge only when nothing deterministic will do. Four cases cover most of the demand behind "is an LLM necessary for judge duty here".

When You Have Ground Truth, Use Exact Match

Classification labels, extracted fields, numeric answers, multiple choice, SQL outputs, unit-testable code: use ==, F1, or the test suite. Deterministic, free, instant and exactly correct, versus probabilistic, costly and approximately correct. The same holds for structural validity: JSON schema validation, regex, a compiler, a linter. If you want hallucination detection without an LLM judge on checkable content, this is the entire answer: if you can write an assertion, write the assertion.

When the Stakes Are Safety-Critical

Medical, legal, financial-advice, child-safety and regulated-decision contexts. A judge can triage, flagging outputs for review or ranking a queue by risk, but it must never be the final arbiter without human sign-off. Treat this as a governance position rather than a technical one: a wrong pass is invisible and unauditable at scale. External frameworks say the same in institutional language; the NIST AI Risk Management Framework and ISO/IEC 42001 both codify human oversight of AI systems as an expectation, not a preference.

When Judgement Needs Domain Expertise the Model Lacks

Clinical accuracy, legal reasoning, specialist engineering correctness, or your organisation's tacit standards: brand voice, escalation etiquette, internal policy nuance. The judge has no grounding for what "good" means here, and per H2-4 it will grade confidently anyway. The pragmatic middle path beats a flat "don't": convert tacit expertise into an expert-written rubric with expert few-shot examples, validate hard, and let the judge handle only the criteria that survive validation. Partial automation of checkable criteria is worth far more than abandoning evaluation entirely.

When the Economics Don't Work

A judge call costs roughly one generation call, so scoring every production request can approach doubling your inference spend, before order-swapping doubles it again or ensembling triples it. Sample 1 to 5% of traffic for monitoring and run full suites only at release gates. The cheaper options are honest ones: smaller or distilled judge models, open-weight evaluators fine-tuned for grading (the Prometheus line of work), and code-based graders for everything checkable.

Tools That Run LLM Judges for You

Every entry below is, at heart, a wrapper that sends a judge prompt and stores the result. The tool is the easy part of an LLM evaluation pipeline; the rubric and the validation are the work. A reader who absorbed H2-6 could build this in an afternoon with the raw API; what tools buy is trace linking, dataset versioning, storage and a UI. Alphabetical, no pick, no affiliates:

Choose on whether it links judge scores back to traces, versions your eval datasets, supports pairwise and order-swapping natively, lets you self-host, and lets you swap judge model families without rewriting evaluators. The surrounding lifecycle work, deployment, CI, registries, is its own discipline.

Judging Agents and Multi-Step Outputs

Frontier note, because the field is moving here: grading an AI agents run means grading a trajectory, not a string. Was the right tool called, with the right arguments, in a sensible order, and did the run terminate? Step-level rubrics ("was this tool choice reasonable given the state?") are judgement calls no assertion can express, so they go to a judge, while the mechanically checkable parts (did the API call succeed, was the schema valid) stay code-based. That hybrid, deterministic where possible, judged where necessary, is the pattern to copy. The OpenAI Evals framework remains the reference point for model-graded checklists at this level.