Obsidian Metadata
| type | source |
| url | https://jxnl.co/writing/2025/05/19/there-are-only-6-rag-evals/ |
| authors | Jason Liu |
| published | 2025-05-19 |
Source: There Are Only 6 RAG Evals (Jason Liu)
URL: https://jxnl.co/writing/2025/05/19/there-are-only-6-rag-evals/
With three variables (Question, Context, Answer) there are exactly six conditional relationships:
- Tier 1 (before RAG evals): retrieval precision/recall, MAP@K, MRR@K. Fast, no LLM.
- Tier 2: C|Q context relevance · A|C faithfulness/groundedness · A|Q answer relevance.
- Tier 3: C|A context support coverage · Q|C question answerability · Q|A self-containment.
Domain emphasis: medical → A|C; customer service → A|Q; technical docs → Q|C. “Ask which of the 6 relationships a 20-metric framework is actually measuring.”
The FAQ’s caveat: implement Tier 2/3 with validated judges, not off-the-shelf prompts, and expect domain-specific failures beyond the six. → RAG Evaluation