RAG Evaluation: How to Measure Retrieval Quality Before

Table of Contents:

• Why RAG Evaluation Starts with Retrieval
• Retrieval Metrics That Actually Matter
• Building a Labeled Evaluation Dataset
• Automated RAG Evaluation Pipeline in Python
• Answer-Level Metrics: Faithfulness and Relevance
• Production Monitoring and Continuous Eval
• Common Evaluation Mistakes
• Frequently Asked Questions

Why RAG Evaluation Starts with Retrieval

Short answer: RAG evaluation must measure retrieval quality separately from generation quality — because when the wrong chunks reach the LLM, no amount of prompt engineering produces correct answers.

Teams come to HinterBuild convinced their LLM is broken. They run LLM evaluation suites, compare GPT-4o against Claude, tune system prompts for weeks — and answers are still wrong. In 80% of cases, the problem is upstream: retrieval returns irrelevant chunks, and the model faithfully summarizes garbage. This is the same root cause we document in why RAG pipelines return garbage.

Key Takeaways:

• Measure retrieval recall@k before tuning LLM prompts or swapping models
• Build eval sets from real user queries — synthetic benchmarks lie about production performance
• Target recall@5 ≥ 0.85 on domain queries before investing in generation optimization
• Separate retrieval metrics (recall, MRR, nDCG) from answer metrics (faithfulness, relevance)
• Run eval continuously in CI and sample production queries weekly

If you understand embeddings and chunking but skip retrieval eval, you are flying blind. This guide shows how to measure retrieval quality with production-grade Python tooling.

Retrieval Metrics That Actually Matter

Short answer: The four retrieval metrics that predict RAG answer quality are recall@k, precision@k, MRR, and nDCG — measured against a labeled set of query-document pairs from your actual domain.

Metric Definitions

Why Recall@k Is the Gate Metric

Recall@k answers: "If a user asks this question, does the right source document appear in the retrieved set?"

If recall@5 is 0.45, the LLM never sees the correct context 55% of the time. No prompt fixes that. Fix chunking, embeddings, or add reranking first.

Precision@k matters after recall is acceptable. High recall + low precision means the right doc is buried in noise — the LLM gets distracted by irrelevant chunks.

MRR vs nDCG

MRR cares only about the rank of the first relevant document. Use MRR when users need one definitive source (policy lookup, error code resolution).

nDCG (normalized Discounted Cumulative Gain) rewards having multiple relevant documents ranked highly. Use nDCG when answers synthesize across sources (incident postmortems, research summaries).

import math
from dataclasses import dataclass

@dataclass
class EvalQuery:
    query: str
    relevant_doc_ids: set[str]

def recall_at_k(retrieved_ids: list[str], relevant_ids: set[str], k: int) -> float:
    top_k = set(retrieved_ids[:k])
    if not relevant_ids:
        return 0.0
    return len(top_k & relevant_ids) / len(relevant_ids)

def precision_at_k(retrieved_ids: list[str], relevant_ids: set[str], k: int) -> float:
    top_k = retrieved_ids[:k]
    if not top_k:
        return 0.0
    hits = sum(1 for doc_id in top_k if doc_id in relevant_ids)
    return hits / len(top_k)

def mrr(retrieved_ids: list[str], relevant_ids: set[str]) -> float:
    for rank, doc_id in enumerate(retrieved_ids, start=1):
        if doc_id in relevant_ids:
            return 1.0 / rank
    return 0.0

def dcg_at_k(relevance_scores: list[float], k: int) -> float:
    scores = relevance_scores[:k]
    return sum(rel / math.log2(i + 2) for i, rel in enumerate(scores))

def ndcg_at_k(retrieved_ids: list[str], relevance_map: dict[str, float], k: int) -> float:
    gains = [relevance_map.get(doc_id, 0.0) for doc_id in retrieved_ids[:k]]
    ideal_gains = sorted(relevance_map.values(), reverse=True)[:k]
    dcg = dcg_at_k(gains, k)
    idcg = dcg_at_k(ideal_gains, k)
    return dcg / idcg if idcg > 0 else 0.0

Run these metrics after every pipeline change — embedding model swap, chunk size adjustment, ColBERT migration, or Graph RAG addition.

Building a Labeled Evaluation Dataset

Short answer: A useful RAG eval set contains 50-200 real user queries, each labeled with the document IDs that should be retrieved — built from support tickets, search logs, and SME annotations.

Dataset Construction Process

Step 1 — Collect real queries. Export the last 90 days of user questions from your RAG application. Filter to unique phrasings. Target 100-200 queries for statistical significance.

Step 2 — Label relevant documents. For each query, have a domain expert identify which document(s) contain the answer. Use a simple labeling UI or spreadsheet:

query_id | query_text                              | relevant_doc_ids       | difficulty
---------|----------------------------------------|------------------------|----------
q001     | What is the refund window for annual?  | doc_policy_refund_v3   | easy
q002     | Who approved vendor X for EU region?   | doc_vendor_42, doc_eu  | hard

Step 3 — Tag query difficulty. Single-hop queries ("What is X?") vs multi-hop ("Who manages the team that owns Y?"). Segment metrics by difficulty — multi-hop failures indicate you need Graph RAG or self-querying retrieval.

Step 4 — Include negative cases. Add queries where the answer is NOT in the corpus. Your system should say "I don't know" — not hallucinate. Track false-positive retrieval rate.

Dataset Size Guidelines

Avoiding Labeling Pitfalls

Pitfall 1 — Labeling at document level when answers live in chunks. If you chunk documents, label at chunk ID level. A relevant document with bad chunking produces irrelevant chunks — your eval should catch this.

Pitfall 2 — Using only easy queries. Include paraphrases, typos, and ambiguous phrasings that real users submit.

Pitfall 3 — Stale labels. When documents update, refresh labels. Version your eval set alongside your corpus.

Store eval datasets in your backend API repo or a dedicated eval database — not in notebooks that nobody reruns.

Automated RAG Evaluation Pipeline in Python

Short answer: An automated eval pipeline runs your retrieval function against the labeled dataset, computes metrics, compares against baselines, and fails CI if regression exceeds thresholds.

Eval Runner

import json
from dataclasses import dataclass, field
from typing import Callable

@dataclass
class EvalResult:
    query_id: str
    query: str
    retrieved_ids: list[str]
    relevant_ids: set[str]
    recall_at_5: float
    precision_at_5: float
    mrr: float
    ndcg_at_10: float

@dataclass
class EvalReport:
    results: list[EvalResult] = field(default_factory=list)

    @property
    def avg_recall_at_5(self) -> float:
        return sum(r.recall_at_5 for r in self.results) / len(self.results)

    @property
    def avg_mrr(self) -> float:
        return sum(r.mrr for r in self.results) / len(self.results)

    @property
    def avg_ndcg_at_10(self) -> float:
        return sum(r.ndcg_at_10 for r in self.results) / len(self.results)

def run_retrieval_eval(
    eval_queries: list[dict],
    retrieve_fn: Callable[[str], list[str]],
    k_recall: int = 5,
    k_ndcg: int = 10,
) -> EvalReport:
    report = EvalReport()

    for item in eval_queries:
        query = item["query"]
        relevant_ids = set(item["relevant_doc_ids"])
        retrieved_ids = retrieve_fn(query)

        relevance_map = {doc_id: 1.0 for doc_id in relevant_ids}

        result = EvalResult(
            query_id=item["query_id"],
            query=query,
            retrieved_ids=retrieved_ids,
            relevant_ids=relevant_ids,
            recall_at_5=recall_at_k(retrieved_ids, relevant_ids, k_recall),
            precision_at_5=precision_at_k(retrieved_ids, relevant_ids, k_recall),
            mrr=mrr(retrieved_ids, relevant_ids),
            ndcg_at_10=ndcg_at_k(retrieved_ids, relevance_map, k_ndcg),
        )
        report.results.append(result)

    return report

Baseline Comparison

Always compare against a baseline before declaring improvement:

def compare_reports(baseline: EvalReport, candidate: EvalReport) -> dict:
    return {
        "recall_at_5_delta": candidate.avg_recall_at_5 - baseline.avg_recall_at_5,
        "mrr_delta": candidate.avg_mrr - baseline.avg_mrr,
        "ndcg_at_10_delta": candidate.avg_ndcg_at_10 - baseline.avg_ndcg_at_10,
        "regressed_queries": [
            r.query_id for r in candidate.results
            if r.recall_at_5 < 1.0 and any(
                b.query_id == r.query_id and b.recall_at_5 == 1.0
                for b in baseline.results
            )
        ],
    }

Run baseline comparison when testing:

• New embedding model (embeddings guide)
• Different chunk sizes
• Reranker addition
• Hybrid search vs dense-only

CI Integration

def assert_eval_thresholds(report: EvalReport, thresholds: dict):
    assert report.avg_recall_at_5 >= thresholds["recall_at_5"], (
        f"recall@5 {report.avg_recall_at_5:.3f} below threshold {thresholds['recall_at_5']}"
    )
    assert report.avg_mrr >= thresholds["mrr"], (
        f"MRR {report.avg_mrr:.3f} below threshold {thresholds['mrr']}"
    )
THRESHOLDS = {"recall_at_5": 0.85, "mrr": 0.70}

def test_retrieval_quality():
    eval_queries = json.load(open("eval/queries_v3.json"))
    report = run_retrieval_eval(eval_queries, my_retrieve_fn)
    assert_eval_thresholds(report, THRESHOLDS)

Gate deployments on eval thresholds. Store eval reports as artifacts in your CI/CD pipeline.

Per-Query Failure Analysis

When eval fails, export the worst queries for manual inspection:

def export_failures(report: EvalReport, output_path: str, recall_threshold: float = 1.0):
    failures = [
        {
            "query_id": r.query_id,
            "query": r.query,
            "recall_at_5": r.recall_at_5,
            "retrieved": r.retrieved_ids[:5],
            "expected": list(r.relevant_ids),
            "missed": list(r.relevant_ids - set(r.retrieved_ids[:5])),
        }
        for r in report.results
        if r.recall_at_5 < recall_threshold
    ]
    with open(output_path, "w") as f:
        json.dump(failures, f, indent=2)
    return failures

Failure exports reveal systematic issues: all failures on table-heavy docs → fix chunking; all failures on acronyms → add hybrid search; all failures on filtered queries → fix self-querying retrieval.

Answer-Level Metrics: Faithfulness and Relevance

Short answer: After retrieval metrics pass thresholds, measure answer faithfulness (is the answer grounded in retrieved context?) and answer relevance (does the answer address the query?) using LLM-as-judge or human evaluation.

The Two-Stage Eval Model

Stage 1: Retrieval Eval (recall@k, MRR, nDCG)
    ↓ passes threshold
Stage 2: Answer Eval (faithfulness, relevance, completeness)

Skipping Stage 1 and jumping to answer eval wastes time — you cannot distinguish retrieval failure from generation failure.

LLM-as-Judge for Faithfulness

from pydantic import BaseModel
from openai import OpenAI

client = OpenAI()

class FaithfulnessScore(BaseModel):
    score: float  # 0.0 to 1.0
    reasoning: str
    unsupported_claims: list[str]

FAITHFULNESS_PROMPT = """Rate whether the ANSWER is fully supported by the CONTEXT.
Score 1.0 if every claim in the answer appears in the context.
Score 0.0 if the answer contains claims not found in the context.
List any unsupported claims."""

def evaluate_faithfulness(context: str, answer: str) -> FaithfulnessScore:
    response = client.beta.chat.completions.parse(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": FAITHFULNESS_PROMPT},
            {"role": "user", "content": f"CONTEXT:\n{context}\n\nANSWER:\n{answer}"},
        ],
        response_format=FaithfulnessScore,
    )
    return response.choices[0].message.parsed

Target faithfulness ≥ 0.85. Lower scores indicate hallucination — often caused by the LLM ignoring retrieved context or retrieval returning partial context.

Answer Relevance

class RelevanceScore(BaseModel):
    score: float
    reasoning: str

RELEVANCE_PROMPT = """Rate whether the ANSWER directly addresses the QUERY.
Score 1.0 if the answer fully addresses the question.
Score 0.5 if partially addressed.
Score 0.0 if the answer is off-topic."""

def evaluate_relevance(query: str, answer: str) -> RelevanceScore:
    response = client.beta.chat.completions.parse(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": RELEVANCE_PROMPT},
            {"role": "user", "content": f"QUERY:\n{query}\n\nANSWER:\n{answer}"},
        ],
        response_format=RelevanceScore,
    )
    return response.choices[0].message.parsed

Low relevance with high faithfulness means retrieval found on-topic docs but the LLM answered the wrong aspect. Tune the system prompt. Low faithfulness with high retrieval recall means the LLM is ignoring context — tighten grounding instructions.

RAGAS Framework Integration

For teams wanting standardized metrics, RAGAS provides context_recall, context_precision, faithfulness, and answer_relevancy out of the box:

from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevancy, context_recall, context_precision
from datasets import Dataset

def run_ragas_eval(queries: list, answers: list, contexts: list, ground_truths: list):
    dataset = Dataset.from_dict({
        "question": queries,
        "answer": answers,
        "contexts": contexts,
        "ground_truth": ground_truths,
    })
    return evaluate(
        dataset,
        metrics=[context_recall, context_precision, faithfulness, answer_relevancy],
    )

RAGAS context_recall correlates with your manual recall@k labels. Use it for rapid iteration; validate against your labeled set quarterly.

Production Monitoring and Continuous Eval

Short answer: Production RAG monitoring logs retrieval scores, samples queries for human review, and reruns offline eval weekly against frozen query sets to catch regressions from corpus updates.

What to Log on Every Request

import time
from dataclasses import dataclass, asdict
import json

@dataclass
class RAGRequestLog:
    request_id: str
    query: str
    retrieved_doc_ids: list[str]
    retrieval_scores: list[float]
    reranked_doc_ids: list[str]
    answer: str
    retrieval_latency_ms: float
    generation_latency_ms: float
    total_latency_ms: float
    embedding_model: str
    llm_model: str

def log_rag_request(log: RAGRequestLog, sink):
    sink.write(json.dumps(asdict(log)) + "\n")

Ship logs to your observability platform. Dashboard:

• Retrieval latency P50/P95
• Average top-1 similarity score (drops indicate embedding drift or corpus pollution)
• Queries with similarity score < 0.5 (likely retrieval failures)
• User thumbs-down rate correlated with retrieval scores

Weekly Sampling Workflow

1. Sample 50 random production queries
2. Have SME label whether retrieved docs were relevant
3. Compute weekly recall@5 on the sample
4. Alert if weekly recall drops > 5% from baseline

Automate sampling with a cron job on your cloud infrastructure:

def weekly_sample_review(production_logs: list[dict], sample_size: int = 50) -> list[dict]:
    import random
    sample = random.sample(production_logs, min(sample_size, len(production_logs)))
    return [
        {
            "request_id": log["request_id"],
            "query": log["query"],
            "retrieved_doc_ids": log["retrieved_doc_ids"],
            "needs_review": True,
        }
        for log in sample
    ]

Eval on Corpus Updates

Every corpus update (new documents, re-embedding, chunk strategy change) triggers:

1. Full offline eval against frozen query set
2. Comparison report vs last passing eval
3. Block deployment if recall@5 regresses > 3%

This prevents the common failure mode: a bulk re-ingestion silently degrades retrieval for 30% of query types.

Cost of Eval

LLM-as-judge eval costs ~$0.01-0.03 per query with GPT-4o-mini. A 200-query eval run costs $2-6 — trivial compared to the engineering time wasted tuning prompts on broken retrieval. Budget eval runs into your LLM cost planning.

Common Evaluation Mistakes

Short answer: The most damaging eval mistake is optimizing answer quality metrics while retrieval recall remains below 0.70 — you are polishing outputs built on wrong inputs.

Mistake 1: Evaluating Only End-to-End

End-to-end eval conflates retrieval and generation failures. A wrong answer could mean bad retrieval OR bad generation. Always eval retrieval independently first.

Mistake 2: Using Generic Benchmarks

MTEB scores and MS MARCO rankings do not predict your domain performance. A model scoring 65 on MTEB may score 40 on your legal corpus. Build domain-specific eval sets.

Mistake 3: Too Few Queries

Ten queries prove nothing. Statistical noise dominates. Minimum 50 queries for directional signal; 100+ for confident decisions.

Mistake 4: Ignoring Query Segments

Aggregate metrics hide failures. Segment by:

• Query length (short vs long)
• Query type (factual vs multi-hop vs filtered)
• Document type (FAQ vs PDF vs code)
• Language (if multilingual)

Multi-hop failures at 0.30 recall while single-hop hits 0.95 means you need Graph RAG, not a new embedding model.

Mistake 5: No Baseline

Every change needs a before/after comparison. "Recall improved" means nothing without knowing the starting point and whether the improvement is statistically significant.

Mistake 6: Evaluating Once and Forgetting

Corpus drift, model updates, and prompt changes degrade quality over time. Schedule weekly automated eval and monthly human review.

Primary references: official documentation, official documentation, official documentation.

Frequently Asked Questions

What is RAG evaluation?

RAG evaluation measures how well a retrieval-augmented generation system finds relevant documents (retrieval metrics) and produces accurate, grounded answers (answer metrics). It separates retrieval quality from LLM generation quality.

What is a good recall@5 for production RAG?

Target recall@5 ≥ 0.85 on your domain-specific eval set before optimizing LLM prompts. Below 0.70, fix chunking and embeddings first. Above 0.90, invest in reranking and generation tuning.

How many eval queries do I need?

Minimum 50 for early development, 100-200 for production confidence. Include query type diversity: single-hop, multi-hop, filtered, and out-of-corpus queries.

Should I use RAGAS or custom eval?

Use RAGAS for rapid prototyping and standardized metrics. Build custom eval with domain-labeled queries for production decisions. Validate RAGAS scores against your labeled set — they correlate but are not identical.

How do I eval retrieval without labeled data?

Start by logging production queries and having SMEs annotate 50-100 samples. For bootstrap eval, use LLM-generated queries against known document summaries — but replace with real user queries as soon as possible.

What is the difference between faithfulness and relevance?

Faithfulness measures whether the answer is supported by retrieved context (no hallucination). Relevance measures whether the answer addresses the user's question. An answer can be faithful but irrelevant, or relevant but unfaithful.

How often should I rerun RAG eval?

Run full eval on every pipeline change (embedding model, chunk size, reranker). Run weekly sampled production eval for drift detection. Run human review monthly on 50+ production samples.

Can I eval RAG in CI/CD?

Yes — store a frozen eval query set in your repo, run retrieval eval in CI, and gate merges on recall@5 and MRR thresholds. Answer-level LLM-as-judge eval is too slow and costly for every commit — run it nightly or pre-release.

Conclusion

RAG evaluation is the discipline that separates production systems from demos:

• Measure retrieval recall@k, MRR, and nDCG before touching LLM prompts
• Build eval sets from real user queries with SME-labeled relevant documents
• Automate eval in CI with threshold gates on every pipeline change
• Add answer faithfulness and relevance eval only after retrieval passes
• Monitor production continuously — corpus drift breaks pipelines silently

At HinterBuild, we eval retrieval first on every RAG & LLM systems engagement:

• RAG & LLM Systems
• Backend API Engineering
• Cloud Infrastructure & DevOps
• AI Agent Development

Schedule a consultation to build your RAG evaluation framework.