Obsidian Metadata
Criteria Drift
“To grade outputs, people need to externalize and define their evaluation criteria; however, the process of grading outputs helps them to define that very criteria. We dub this phenomenon criteria drift, and it implies that it is impossible to completely determine evaluation criteria prior to human judging of LLM outputs.” — Shankar et al., Who Validates the Validators? (2024)
The paper also found that some criteria are dependent on the specific outputs observed. They can’t be defined a priori. This undercuts any approach that assumes evaluation can be separated from looking at outputs.
Consequences for how you work
- Don’t write the rubric first. Treat initial expectations (reference answers, rubrics) as a starting point you’ll revise. Reviewers focused on a checklist miss problems outside it. Example: you may only discover that a policy-compliant refund escalation frustrates users after reading a few interactions. See Reference Answers and Rubrics.
- Do Error Analysis before designing evaluators, and repeat it periodically so the rubric stays current.
- Treat evaluation criteria as living documents. The teams that keep trust in their evals accept drift instead of fighting it (see Maintaining Trust in Evals).
- Re-review earlier traces after you discover new failure modes. You’ll spot things you missed the first time.
- Don’t hand evaluation to an optimizer too early. Automated prompt optimization hill-climbs a fixed metric. It can’t discover new criteria (see Prompt Optimization).
- Fully automated error analysis misses product-context failures, because “a team cannot provide context it has not discovered yet” (see Source - Do Automated Evals Work).
- Writing critiques surfaces drift. Honeycomb’s domain expert: “Seeing how the LLM breaks down its reasoning made me realize I wasn’t being consistent about how I judged certain edge cases.” (see Honeycomb Query Assistant).
Related
Error Analysis · Critique Shadowing · Benevolent Dictator · Source - Who Validates the Validators