Most RAG teams eventually add evals.

That is progress.

But it creates a new problem.

Teams start treating eval scores like truth.

They are not.

Most RAG evals measure whether the answer looks right. Production cares whether the system did the right thing.

This is where teams get false confidence.

A RAG system can pass the eval and still fail in production. The answer may match the expected answer but come from the wrong source. It may be correct but use old data. It may sound helpful but ignore a permission boundary. It may give a confident answer when the correct behavior was to stop.

The eval says green. The product is still wrong.

This is not an argument against evals. If you do not have evals, build them. But do not confuse having evals with understanding the system.

A weak eval suite gives you confidence faster than it gives you truth. That is the dangerous part.

Your eval set is made from questions you already understand#

An eval set tells you how the system behaves on questions you remembered to test.

That sounds obvious, but this is where many teams fool themselves.

Most eval sets start from clean questions: FAQ questions, demo questions, questions from internal docs, questions the product team already knows and questions someone wrote while looking at the corpus.

That is still useful.

But it is not production.

Production users do not ask questions the way your eval file asks questions.

Your eval asks:

What is the refund policy for annual plans?

A real user asks:

I paid for the yearly thing last month. Can I still get my money back if I cancel today?

Same intent, very different shape.

The eval question is clean. The production question is messy. It has vague words, missing context and time sensitivity.

Your eval passes the clean version, then the product fails on the real one.

A clean eval set can make the system look much better than it really is. Rewrite those same questions in real user language and the score can move quickly.

That gap is the point.

This is why a high eval score can mislead you. The system is not being tested against production. It is being tested against the team’s memory of production.

And that memory is usually too clean.

It misses strange phrasing, half formed questions, old product names and questions with two intents mixed together. It misses questions where the answer is not in the corpus. It misses permission related cases until something breaks.

This is not because people are careless. It is because teams write evals from what they understand.

Production finds what they forgot.

A useful RAG eval set needs dirty questions. Real user wording, bad phrasing, missing context, old terminology, ambiguous intent, questions that should not be answered and questions where two sources disagree.

The goal is not to make the eval set huge. The goal is to make it less polite.

If your eval only tests clean questions, it will reward a clean demo.

Production is not clean.

A correct answer can still be wrong system behavior#

Diagram of the RAG evaluation path. Seven boxes in a row: user question, retrieval, context building, permission check, generation, citation, final answer. A callout points to the final answer box labeled 'what the eval sees.' A bracket spans all seven boxes labeled 'what production depends on.'

The eval can pass while failure hides in the six stages before the answer.

This is the part most answer evals miss.

They grade the output. They do not grade the path.

And in production RAG, the path matters.

Take a simple example.

User asks:

Does the Enterprise plan support audit log export?

Expected answer:

Yes.

The system answers:

Yes, Enterprise supports audit log export.

Eval passes. Looks good.

But what actually happened?

Maybe retrieval found the right feature table, the model used the right evidence, the source was current and the user had permission to see that answer. Good. That is correct system behavior.

But maybe something else happened.

Maybe the model guessed from prior knowledge. Maybe it used an old pricing page. Maybe it retrieved a sales deck that mentioned Enterprise and audit logs somewhere. Maybe the citation pointed to a page that did not prove the answer. Maybe the user was not allowed to see the document behind the answer.

Same final answer. Very different system.

That is the problem with final answer accuracy.

A correct answer is not always a correct system.

If the system is right for the wrong reason, the eval should not celebrate. It should catch it, because luck does not scale.

A production RAG answer has a path behind it:

question retrieval context permissions generation citation response

If your eval only checks the response, it is ignoring most of the system.

You need to know whether retrieval found the right source, whether the final context had the actual evidence, whether the model used that evidence, whether the source was current, whether the user was allowed to access it and whether the system should have answered at all.

Each of these can fail separately.

Retrieval can work but context construction can drop the useful chunk. Context can be right but the model can ignore it. The model can use evidence but overstate what the evidence supports.

This is why “answer matches expected output” is too weak as the main signal.

It is not useless. It is just late.

By the time you are only looking at the final answer, many important failures have already happened behind the curtain.

This is also why citations are tricky. A citation is not automatically grounding. Sometimes the answer is correct and the citation is weak. Sometimes the citation is relevant but does not prove the claim. Sometimes the source is right but stale. Sometimes the model cites the nearest official looking document because it needs to cite something.

If your eval does not check the evidence path, it can reward a system that looks grounded but is only performing groundedness.

That is not production quality.

That is theater.

Your eval rewards answers when the right behavior is refusal#

Many evals carry one hidden assumption: a good system answers the question.

Production is not that simple.

Sometimes the best answer is not an answer. It is “I do not have enough information.” Or “which region do you mean?” Or “you do not have access to that document.” Or “this needs human approval.”

If your eval mostly rewards completion, it will push the system toward answering too much.

That looks good in a test set. It looks helpful in a demo. It is risky in production.

Imagine a user asks:

Can I permanently delete this customer’s data?

If the retrieved context is incomplete, the system should not guess. It should not combine two weak chunks and act confident. It should ask for missing information or escalate.

A final answer eval may mark a confident answer as good if it sounds reasonable. Production may treat the same answer as a serious failure.

In production, “I don’t know” is not always a failure. Sometimes it is the correct behavior.

A RAG eval that cannot reward refusal is measuring helpfulness. It is not measuring judgment.

And production systems need judgment.

Averages hide the failures users actually feel#

A single eval score is comfortable.

92% accuracy.

Looks good. Easy to report. Easy to compare. Easy to put in a dashboard.

But users do not experience your average score. They experience the one answer in front of them.

And the failures that matter are often buried inside the average.

The system may be 92% overall, but what is it on refund questions? What is it on legal questions? What is it on enterprise policy? What is it on permission related data? What is it when the corpus has stale documents? What is it when the system should refuse?

That 8% failure bucket may contain the cases where trust actually breaks.

This is why average accuracy is not enough. You need to slice the eval by things that matter in the product: topic, risk level, source type, user permission, freshness, refusal case and customer impact.

A support answer and a security answer should not be treated as the same unit of accuracy. A wrong answer about button copy is annoying. A wrong answer about data deletion is a different category.

One score hides that difference.

It makes the system look simpler than it is.

That is how teams end up with a dashboard that looks healthy while users are losing trust in the product.

One number is not an eval strategy. It is a dashboard number.

Measure behavior, not just answers#

Evals are necessary.

The problem is not that teams run evals. The problem is that they trust the wrong signal.

A RAG eval should not only ask whether the answer looked right. It should ask whether the system behaved correctly.

Did it retrieve the right source? Did it use current data? Did it respect permissions? Did it stay inside the evidence? Did it refuse when the evidence was missing? Did it fail in the cases that actually matter?

That is the real test.

Production RAG is not judged by one clean answer. It is judged by the path that produced that answer.

If your eval cannot see that path, it is measuring the wrong thing.