Thought UmweltThought Umwelt

thought umwelt

Principles and Opinions

4 min read

Obsidian Metadata

typereference

Principles and Opinions (cheat sheet)

The FAQ describes these as “sharp opinions about what works in most cases. They are not universal truths. Use your judgment.” Each line links to the note with the details.

Data and process

  1. Always look at the data. It’s the highest-ROI activity, and the one most often skipped. → Error Analysis
  2. Error analysis decides which evals to write. Evals target errors you observe, not errors you imagine. → Eval-Driven Development
  3. You can’t define the criteria before you see outputs (criteria drift). → Criteria Drift
  4. Record every problem, including ones that aren’t the model’s fault. Defer the “why”. → Error Analysis
  5. Annotate ≥ 30 traces yourself before any AI suggestions; stop at saturation (≥ 100). → Theoretical Saturation
  6. Stop at the first upstream failure in a trace. → Traces
  7. One domain expert (benevolent dictator) defines quality. → Benevolent Dictator
  8. Don’t outsource error analysis, to people or to LLMs. → Annotation Outsourcing, What LLMs Can and Can't Automate in Evals
  9. Use LLMs to scale what you’ve learned, not to avoid looking at data.

Metric design

  1. Binary pass/fail plus a written critique. No Likert scales. → Binary Pass-Fail vs Likert Scales
  2. Generic metrics (helpfulness, coherence, ROUGE, BERTScore) aren’t quality measures. They’re OK as exploration signals. → Generic Metrics
  3. Application-specific metrics look like “Failure to Escalate To Human”.
  4. One evaluator per failure mode. No holistic judges.
  5. Code beats judges whenever code can decide. → Code-Based Evals
  6. Fix spec gaps in the prompt before building evals. → When to Build an Automated Evaluator
  7. Build expensive evals only for failures you’ll iterate on repeatedly.
  8. A 100% pass rate is a red flag. ~70% may be healthier. → Source - 99 Percent of Executives Are Misled by AI Advice
  9. Prefer a pass-all rate with drill-down over weighted composite scores. → Combining Evals into a Single Metric

Judges

  1. The only way to trust a judge is to measure it against human labels. → Judge Validation
  2. Report TPR/TNR (or precision/recall), not accuracy.
  3. Keep train, dev, and test separate. Test once.
  4. Give each judge only the context it needs. → Judge Context and Context Rot
  5. Using the same model for the product and the judge is usually fine. Alignment is what counts.
  6. Correct aggregate rates for judge error, and report CIs. → Judge Bias Correction
  7. “It’s not the judge that created value.” The value is in the looking. → Critique Shadowing

Tools and product

  1. Build a custom annotation tool. It’s the most impactful investment (~10× faster iteration). → Annotation Tools and Review Interfaces
  2. Remove ALL friction from looking at data.
  3. “It’s hard to eval” is a product smell. Design for verification. → Designing for Verification
  4. No favorite vendor. Choose on support; build custom on top. → Eval Tooling and Vendors
  5. Be skeptical of tools where AI writes the rubric and scores against it.
  6. Store prompts in Git; experiment in notebooks or integrated prompt environments. → Prompt Management
  7. Look at the prompt any framework sends. → Source - Show Me The Prompt
  8. Standardize processes, not metrics, across teams. → Internal Eval Platforms

Strategy and org

  1. Evals are part of development, like debugging. Expect 60–80% of dev time on error analysis and eval.
  2. Sell results, not “evals”. → Making the Case for Evals
  3. Roadmaps should count experiments, not features. → Capability Funnel and Experiment Roadmaps
  4. Check whether error analysis points at the model before switching models. → Model Selection
  5. RAG isn’t dead; naive retrieval is. Measure retrieval approaches. → Is RAG Dead
  6. These methods will outlast today’s prompt tricks.

Where sources disagree (worth knowing)

  • Eval-driven development: Hamel & Shreya say “generally no”. Anthropic recommends it for agent capabilities. → Eval-Driven Development
  • Partial credit / weighted scoring: Anthropic allows partial credit across components. Hamel prefers decomposing into several binary checks and discourages weighted composites.
  • TPR convention: the FAQ treats “positive” as failure; the validate-evaluator skill treats “positive” as Pass. → Judge Validation

Graph View