Obsidian Metadata

type	concept
sources	Source - AI Evals FAQ, Source - Demystifying Evals for AI Agents

Eval-Driven Development

Eval-driven development (EDD) means writing evaluators before implementing features, like TDD.

Hamel & Shreya: “Generally no.”

• In traditional software, failure modes are predictable. LLMs have infinite surface area for failure, so you can’t anticipate what will break.
• Writing evals for imagined errors leads to blocked teams and wasted effort on metrics with no impact on quality.
• Better: start with Error Analysis and write evaluators for errors you discover, not errors you imagine.
• Exception: specific constraints where you know exactly what success looks like, e.g. “never mention competitors”. Writing that eval early is fine.
• Always do a cost–benefit analysis first.

Anthropic: “We recommend practicing eval-driven development”

Anthropic’s agent-evals post takes a different emphasis:

• Build capability evals that define planned capabilities before agents can fulfill them, then iterate until the agent performs well. These often start at a low pass rate and act as bets on future models. When a new model ships, run the suite to see which bets paid off.
• Early on, evals force the team to specify success. Two engineers reading the same spec can interpret edge cases differently, and an eval suite resolves the ambiguity.
• Start with 20–50 simple tasks drawn from real failures, beginning with what you already test manually and what’s in the bug tracker or support queue.

Reconciling the two views

Both agree on the essentials:

• Tasks should come from real failures and real usage (bug tracker, support queue, traces), not brainstorming.
• Unambiguous success criteria (Anthropic: “two domain experts would independently reach the same verdict”) = explicit Pass/Fail definitions.
• Read the transcripts (Traces).

The disagreement is mostly about when. Anthropic frames capability evals for agents with well-specified outcomes (coding, tool use). Hamel warns against front-loading evals for open-ended LLM product behavior before you’ve seen outputs (Criteria Drift). A practical synthesis: write evals upfront only for known, crisp requirements and hard capability targets. Everything else emerges from error analysis.

Related

When to Build an Automated Evaluator · Capability vs Regression Evals · Criteria Drift