Obsidian Metadata

type	source
url	https://hamel.dev/blog/posts/eval-smell/
authors	Hamel Husain
published	2026-06-29

Source: “It’s Hard to Eval” Is a Product Smell

URL: https://hamel.dev/blog/posts/eval-smell/ · Hamel Husain

“Our product is hard to eval” is a product smell: what’s hard for you to verify is hard for users too. Design for verification before building evals.

Three before/after sketches: AI data agent (provenance, assumptions, unverifiable items, a notebook view), PE lesson planner (anchor to a vetted plan, diff of edits with reasons), workers’-comp report (research assistant surfacing facts, contradictions, open questions with page links before generating the report).

Guiding questions: what must the user check · what trusted thing to compare against · what expert heuristics apply · what smaller units can be accepted, edited, or rejected. Themes: provenance and progressive disclosure, adapting to trust. Rooted in needfinding and sensemaking. “With AI, verification is the bottleneck.”

→ Full notes: Designing for Verification