Corrective RAG (CRAG): How It Improves RAG Accuracy
Ask your RAG assistant something your documents don't actually cover, and watch what happens. It doesn't say "I don't know." It grabs the five least-wrong chunks it can find, hands them to the model, and comes back with a fluent, confident, completely made-up answer. With citations, no less. That's the part that gets people.
The problem isn't generation. It's that standard retrieval has no idea when it's failed. Top-k retrieval always succeeds in the sense that it returns k results, whatever their quality. There's no null case, no "nothing useful here" flag. Whatever comes back gets treated as gospel.
Corrective RAG (CRAG) is one of the more sensible fixes for this. It bolts a lightweight evaluator onto your pipeline that grades what got retrieved, then decides what to do about it: keep it, throw it out and look elsewhere, or hedge between the two. Full name, for the record: Corrective Retrieval Augmented Generation. We'll use both through this piece.
This article walks through the actual mechanism, a worked example across all three paths, how CRAG differs from Self-RAG, what implementing it really looks like, and what it costs in latency and money. No hand-waving on that last part, promise.
What Is Corrective RAG (CRAG)?
Corrective RAG is a RAG architecture that adds a small, dedicated evaluator to score retrieved documents before generation happens. Based on that score, it picks one of three moves: refine what it has, discard everything and search elsewhere, or do a bit of both. The point is simple, don't let the model generate off garbage context.
Two things make it worth paying attention to, both straight from the original paper:
• It's plug-and-play. Wraps around whatever RAG pipeline you already have.
• No fine-tuning needed. The generator stays exactly as it is.
Quick housekeeping before we go further: CRAG is also the name of Meta's KDD Cup 2024 benchmark dataset for evaluating RAG systems. Different thing entirely. This article is about Yan et al.'s architecture, not Meta's benchmark. Sorted.
Where CRAG Came From
It showed up in Corrective Retrieval Augmented Generation (Yan et al., January 2024), part of a wave of self-correcting RAG papers that landed around the same time as Self-RAG and Adaptive-RAG. All three were reacting to the same realisation, that retrieval quality, not generation quality, was quietly wrecking RAG systems in production. If you haven't already got a handle on how a standard retrieval pipeline fits together with agents, this piece on retrieval, agents and MCP is worth a read first.
Why Standard RAG Has No Idea When It's Wrong
A top-k retriever is a ranking function, not a relevance function. It returns the k nearest items no matter what, so there's no built-in "I found nothing" state. Whatever gets pulled goes straight to the generator like it's gospel.
This tends to fail in three distinct ways:
• The answer just isn't in the corpus. Retrieval grabs the closest available noise anyway, and the model runs with it.
• The right document exists but is buried. A ten-page policy PDF gets retrieved for one relevant clause, and nine and a half pages of filler dilute the context.
• Retrieval is half right. Some chunks are relevant, some aren't, and the model blends them into an answer that's impossible to fact-check from the outside.
Before you reach for CRAG, worth saying plainly, better chunking, reranking, and hybrid search are cheaper fixes and you should try them first. None of them tell the pipeline when to stop and go look somewhere else, though. That's the specific gap CRAG fills. Adding a decision step on top of retrieval also starts pushing your system toward agentic behaviour, and if that line feels blurry, this breakdown of RAG versus agentic AI clears it up.
If you want to actually diagnose whether your problem is retrieval quality before adding a correction layer on top, this video walks through the diagnostic process in reasonable depth. Worth twenty minutes before committing to the added cost.
The Retrieval Evaluator: CRAG's Core Piece
The evaluator is a small model that takes a query-document pair and spits out a relevance confidence score. In the paper, it's a fine-tuned T5-based model, roughly 0.77B parameters. Small on purpose, because if grading cost as much as generating, the whole idea falls apart economically. "Lightweight" here isn't a marketing adjective, it's the design constraint everything else hangs off.
Two thresholds decide what happens next, an upper bound and a lower bound:
• Above the upper threshold: at least one document is confidently relevant.
• Below the lower threshold: everything retrieved is confidently useless.
• In between: genuine uncertainty, no clean call either way.
Here's the bit most write-ups skip: the paper's threshold values are tuned to its own benchmark datasets. They will not transfer to your corpus, no matter how much you'd like them to. Set aside time to tune these properly. A reasonable starting method: grade a hundred real query-document pairs by hand, label them yourself, and pick thresholds where precision and recall settle where you're comfortable with your tolerance for false confidence. If precision-recall tradeoffs feel fuzzy, this refresher on evaluation metrics is genuinely useful here, not a detour.
One more thing worth flagging early: most production builds swap the fine-tuned evaluator for an LLM grader with structured output. Cheaper to build, pricier to run per call. More on that trade-off later.
Build an AI-First Career, Master the Complete Skillset
Choose from our industry-leading programs designed for career success
Modern Software and AI Engineering Program
Master full-stack development with AI integration
+1000 moreModern Data Science and ML with specialisation in AI
Advanced data science techniques with AI specialization
+1000 moreAdvanced AIML with Specialisation in Agentic AI
Deep dive into AIML with focus on Agentic systems
+1000 moreDevOps, Cloud & AI Platform Engineering
Build and manage AI-powered cloud infrastructure
+1000 moreAI Engineering Advanced Certification by IIT-Roorkee
Premier AI engineering certification from IIT-Roorkee
AI Forward Deployed Engineer Program
Full-stack engineering, production AI and client-facing consulting
+1000 moreThe Three Action Paths: Correct, Incorrect, Ambiguous
Every write-up on this topic names the three labels. Almost none explains what actually happens inside each one. Quick version first, expanded below.
| Action | Trigger | What happens |
|---|---|---|
| Correct | At least one document scores above the upper threshold | Refine the good documents, discard the noise inside them, then generate |
| Incorrect | Every document scores below the lower threshold | Throw it all out, rewrite the query, search externally, refine, generate |
| Ambiguous | Scores fall between thresholds | Refine what you have and combine it with external results |
Correct: Refine What You Already Have
Triggered when at least one retrieved document clears the upper threshold. Here's the part almost everyone skips: the documents don't just get passed through untouched. A relevant document can still be mostly noise.
CRAG runs a decompose-then-recompose step. Each document gets split into fine-grained knowledge strips, usually a sentence or two each. Every strip gets scored by the same evaluator. Weak strips get dropped. What survives gets stitched back together, in order, into the context the generator actually sees.
This turns "here are five documents broadly on-topic" into "here are the eleven sentences that actually answer the question." Skip this step, which most homegrown implementations do, and you'll wonder why your CRAG pipeline performs about the same as plain RAG. Now you know why.
Incorrect: Discard Everything and Look Elsewhere
Triggered when every retrieved document falls below the lower threshold, meaning the corpus genuinely doesn't have the answer. Everything gets discarded, not down-weighted, fully binned. Naive RAG would never do this. It doesn't know how to say no.
The pipeline then goes looking for external knowledge. The paper uses web search, but first rewrites the user's natural-language question into keyword form, because search engines and dense retrievers want very different phrasing. Whatever comes back gets refined the same way, strip by strip, before generation.
The part nobody tells you: web search is one example of "external knowledge," not a requirement baked into the architecture. If you're working in Indian BFSI, healthcare, or government contexts, an open internet call might be a flat compliance no. The architecture works identically with a secondary internal index, a different retriever over the same data, a database lookup, or even just an honest "I don't have this, routing you to a human." That last option is often the right production behaviour, and it's almost never mentioned anywhere else.
Ambiguous: Hedge and Combine Both
Triggered when scores land between the two thresholds, meaning the evaluator genuinely can't decide. The pipeline combines refined internal documents with external results rather than forcing a binary call.
This exists because real corpora produce partial matches constantly. A strict keep-or-discard rule either throws away useful context or generates from too little of it. It's also the path most tutorials quietly drop when they collapse the evaluator to a yes-or-no grader, more on that in the implementation section. How a knowledge-based agent decides what to do with what it knows is a decent primer if you want the classical AI framing behind this kind of decision procedure.
One Query, Three Paths: A Worked Walkthrough
Abstract flowcharts only get you so far. Let's run one scenario through all three paths: an internal HR policy assistant at an Indian IT services company, corpus made up of leave policy, travel policy, deputation guidelines, and the employee handbook. Scores below are illustrative, not measured off a real model.
Path 1, Correct: "How many casual leaves do I get in a year?"
Retrieval pulls the leave-policy section, evaluator scores it well above the upper threshold. Refinement strips out the document header, the revision-history table, and the sick-leave paragraph, keeping just the three sentences and the entitlement table that actually answer the question. Generator produces a tight, grounded answer.
The quiet win here: even on the easy path, refinement noticeably improves the answer by removing distractors. People assume Correct is a no-op step. It isn't.
Path 2, Incorrect: "What are the income tax slabs for FY 2026-27?"
Retrieval returns whatever's closest, maybe the travel reimbursement policy or a payroll FAQ, because top-k always returns something. Everything scores below the lower threshold. All of it gets binned. Query gets rewritten, external knowledge gets fetched and refined, generation happens with a clear citation to the outside source.
Now the enterprise twist: in a BFSI deployment where open web access is a hard no, this same path should end in "This is outside the HR policy corpus" plus a routing action. That's a correct, safe outcome, and honestly a lot better than the confident, invented answer naive RAG would have handed you.
Path 3, Ambiguous: "Can I carry forward leave while on client deputation abroad?"
Retrieval turns up the general carry-forward clause, genuinely relevant but not specific, and the deputation guideline, partially relevant. Neither directly addresses the overlap. Scores land in the middle band. Pipeline refines what it has, queries the external source, and generates from the combination, ideally flagging that internal policy covers carry-forward generally but not this specific overseas case.
This is, in most enterprise corpora, the most common real path, not the edge case. Which is exactly why dropping it from your implementation hurts more than people expect.
CRAG vs Self-RAG vs Adaptive RAG: What Actually Differs
| Dimension | Naive RAG | Self-RAG | Corrective RAG (CRAG) |
|---|---|---|---|
| Core question asked | None, just retrieve and generate | Should I retrieve, and can I trust what I got? | Was retrieval good, and if not, where else do I look? |
| Who evaluates | Nobody | The generator itself, via reflection tokens | A separate, lightweight retrieval evaluator |
| Needs fine-tuning? | No | Yes, the generator learns to emit reflection tokens | No, plug-and-play on an existing pipeline |
| Can skip retrieval entirely | No | Yes, decides on demand | No, always retrieves first, then judges |
| Action on bad retrieval | None, generates anyway | Down-weights or critiques the passage while writing | Discards it and goes looking for external knowledge |
| External fallback | No | No | Yes, web search or a secondary source |
| Filters inside a document | No | Partially, passage-level critique | Yes, strip-by-strip refinement |
| Added latency | Baseline | Moderate | Moderate to high on the worst-case path |
| Best fit when | Corpus reliably covers queries | You control the model and can fine-tune it | You can't fine-tune, and out-of-corpus questions are common |
The short version, the one that actually matters when you're deciding: Self-RAG makes the model a better consumer of retrieval. CRAG makes the retrieval itself better, with an escape hatch when it can't be fixed. Self-RAG's check lives inside the model and has to be learned. CRAG's check is bolted on from outside. That's precisely why Self-RAG needs fine-tuning and CRAG doesn't, and for most teams who can't fine-tune a generator, that one fact settles the decision on its own.
They're not rivals, by the way, they're composable. The CRAG paper itself evaluates a combined Self-CRAG setup, applying CRAG on top of a Self-RAG generator. People routinely assume it's an either-or choice. It isn't.
Where Adaptive RAG Fits (a Different Axis Entirely)
Adaptive-RAG (Jeong et al.) routes by query complexity, deciding between no retrieval, single-step retrieval, or multi-step iterative retrieval before anything even gets pulled. It isn't correcting bad retrieval at all, it's deciding how hard to try in the first place. Different axis, different job.
Put simply: Adaptive RAG decides how hard to try. CRAG decides whether what came back was good enough. Self-RAG decides how much to trust it while writing. Three different questions people mix up constantly, and once you've got a solid handle on how LLMs get trained and fine-tuned in the first place, this trio stops being confusing pretty fast.
How Scaler Transformed Careers in Different Fields
Scaler learners achieved 2.5x salary growth with average post-Scaler CTC reaching ₹23L.
Google
Amazon
Microsoft
Flipkart
Adobe1200+ moreImplementing CRAG: LangGraph vs What the Paper Actually Says
The most popular reference implementation runs as a LangGraph state graph: retrieve, grade documents, hit a conditional edge, then either transform the query and search the web or go straight to generate. The conditional edge is where the action path gets chosen. LangGraph's branching model suits this well, CRAG is fundamentally a decision node wearing a workflow costume, which starts to look a lot like agent orchestration once you squint at it, see also how agent frameworks orchestrate multi-step workflows.
Here's where it quietly diverges from the paper, and it's worth knowing before you build on top of it:
• Different evaluator. The paper fine-tunes a small T5 model. The tutorial uses an LLM with structured output returning a plain relevant or not-relevant. Faster to build, no training pipeline, but you pay LLM inference on every single grading call, and you lose the graded confidence score that made three thresholds possible to begin with.
• Three paths often become two. With a binary grader there's no middle band, so the common setup is "anything relevant, generate, nothing relevant, search the web." The ambiguous path just vanishes. If your corpus produces frequent partial matches, and most enterprise corpora do, reinstate a middle band using a graded score instead of a boolean.
• Knowledge refinement usually goes missing entirely. Most tutorials pass whole documents through on the Correct path with no strip-level filtering. It's cheap to add and it's where a meaningful chunk of the paper's actual gain lives.
If you're only fixing one thing, add the grading step first. Even a plain binary gate that refuses to generate when nothing scores well kills the worst failure mode, confidently answering from irrelevant context, before you've built any web-search plumbing at all.
Also worth knowing about: the authors' own reference implementation on GitHub, and LlamaIndex ships a Corrective RAG pack too if you'd rather not build the graph from scratch.
On the portfolio-project front, a working CRAG pipeline with a documented threshold-tuning experiment and an honest before-and-after evaluation on the same question set beats yet another basic RAG chatbot by a mile. It shows you can measure things, not just wire an API call together and call it a day.
What CRAG Actually Costs
Nobody counts this properly, so let's do it. For a five-document retrieval: five evaluator calls to grade documents. If you're doing refinement at four strips per document, that's up to twenty more scoring calls. On the Incorrect path, add one query-rewrite call, an external search round trip, and refinement of whatever comes back. Then generation, same as always. Against naive RAG's single retrieve-and-generate call, this is not a rounding error.
| Path | Extra model calls | Rough added latency | When it fires |
|---|---|---|---|
| Correct | One grading call per document, plus strip-level calls if you refine | Sub-second with a small grader | At least one document scores above the upper threshold |
| Incorrect | Grading calls, one query-rewrite call, one search round trip, refinement of new results | A few seconds, dominated by the search call | Everything scores below the lower threshold |
| Ambiguous | Grading calls plus both refinement and external search | Similar to Incorrect, sometimes worse | Scores land between the two thresholds |
Latency is dominated by the fallback path, not the happy path. Correct stays fast because it's just grading. Incorrect and Ambiguous both drag a network round trip into the mix, which is realistically a few seconds, and that matters a lot for a user-facing chat product, less so for a nightly batch job.
A few practical mitigations, none of them exotic:
• Use an actually small grader model, not a frontier one, for grading calls.
• Batch document grading into a single call where quality holds up.
• Cache grades for repeated query-document pairs, you'll see more repeats than you'd expect.
• Short-circuit refinement when confidence is already very high.
• Set a hard latency budget on the fallback path that degrades gracefully to "I don't have this" instead of hanging forever.
The failure mode nobody mentions: web search can be worse than your own corpus. The open internet has SEO spam, stale pages, and content that flatly contradicts your authoritative internal policy. A CRAG pipeline that falls back to the web can confidently answer from a genuinely bad source, a different flavour of wrong than the one you built this whole thing to fix. Domain allow-lists and clear source labelling in the answer help.
Before adopting any of this, run thirty to fifty real questions, deliberately including some out of corpus, through both the naive pipeline and the CRAG version. Compare answer quality, p95 latency, and cost per query together, not in isolation. Measure first, decide after.
When to Use Corrective RAG
Use it when:
• Users regularly ask questions your corpus doesn't cover.
• A wrong answer costs more than a slow one or an honest refusal.
• You can't fine-tune the generator.
• Retrieved documents run long and noisy, think policy PDFs and contracts.
• You need to show which source an answer actually came from.
• You've already measured a retrieval quality problem chunking and reranking alone can't fix.
Skip or defer it when:
• Your corpus reliably covers what people ask.
• Latency budgets are tight and it's user-facing.
• You haven't fixed chunking, retrieval, and reranking yet, do that first, it's cheaper.
• Query volume is very high and per-query cost is the actual constraint.
• You can fine-tune the generator and Self-RAG's tighter integration fits better.
If you do go ahead, adopt it in order: grading gate first, refinement second, external fallback last, only after the compliance question is settled. Most of the reliability gain shows up at step one, for the least cost. No need to build the whole thing on day one.
Worth knowing if you're prepping for interviews: "how would you stop a RAG system from hallucinating when retrieval fails" is basically a standard GenAI system-design question now. "Grade the retrieval, then act on the grade, refine, replace, or combine" is a complete, senior-sounding answer on its own. Mentioning why CRAG skips fine-tuning while Self-RAG can't is the detail that separates an okay answer from a good one. If you want structured practice building and evaluating retrieval systems end to end, these generative AI courses cover the ground.
Turn Learning into Career Growth
FAQs
What is Corrective RAG (CRAG)?
It's a RAG architecture that adds a lightweight evaluator to score retrieved documents before generation, then takes one of three corrective actions, refine, discard and search elsewhere, or combine both, so the model doesn't generate off bad context.
How does CRAG decide whether retrieval was good?
A lightweight evaluator scores each retrieved document's relevance and produces a confidence score. Two thresholds, upper and lower, map that score to one of the three action paths.
What are the three actions in Corrective RAG?
Correct, at least one document is confidently relevant, refine and use it. Incorrect, nothing is relevant, discard everything and fetch external knowledge. Ambiguous, the evaluator is unsure, combine refined internal documents with external results.
What's the difference between CRAG and Self-RAG?
Self-RAG trains the generator to critique its own retrieval internally, using reflection tokens, which needs fine-tuning. CRAG uses a separate external evaluator, needs no fine-tuning, and can throw out retrieval entirely and search elsewhere. They can also be combined.
Does Corrective RAG need web search?
No. Web search is the paper's example of external knowledge, not a hard requirement. A secondary internal index, a different retriever, a database lookup, or an explicit "not in my corpus" refusal all work, and are often safer in regulated environments.
Does CRAG require fine-tuning the LLM?
No. It's plug-and-play, wraps an existing pipeline, leaves the generator untouched. The paper fine-tunes only the small evaluator, and a lot of implementations swap even that for a prompted LLM grader.
How much latency and cost does CRAG add?
The Correct path adds grading only, typically sub-second with a small model. The Incorrect path adds query rewriting plus an external search round trip, realistically a few seconds. Cost scales with how many documents and strips get graded.
Is Corrective RAG worth implementing in production?
Usually yes if out-of-corpus questions come up often and wrong answers are costly. Adopt it incrementally, grading gate first, then refinement, then fallback. Most of the reliability gain comes from that first step alone.
Wrapping Up
Naive RAG's real flaw isn't that retrieval sometimes fails, it always will, on some fraction of queries. The flaw is that the pipeline never notices. CRAG's whole contribution is a cheap, external check that turns retrieval quality into something the system can actually act on instead of just shipping whatever came back.
Grade first. Refine second. Fall back last. Let real questions on your own corpus decide how far down that list you actually need to go.
Building retrieval systems that can evaluate and correct themselves, and knowing when the added cost is worth it, is the kind of applied engineering Scaler's AI & Machine Learning program is built around.
