thought
umwelt
Search
Search
Dark mode
Light mode
Explorer
Tag: evals
61 items with this tag.
Sep 27, 2026
Capability Funnel and Experiment Roadmaps
evals
strategy
management
Sep 27, 2026
Eval Tooling and Vendors
evals
tooling
vendors
Sep 27, 2026
Transition Failure Matrix
evals
agents
analysis
Sep 27, 2026
When to Build an Automated Evaluator
evals
strategy
cost-benefit
Sep 27, 2026
Annotation Outsourcing
evals
annotation
people
Sep 27, 2026
Annotation Tools and Review Interfaces
evals
tooling
annotation
ux
Sep 27, 2026
Benevolent Dictator
evals
annotation
people
Sep 27, 2026
Binary Pass-Fail vs Likert Scales
evals
metric-design
core
Sep 27, 2026
Combining Evals into a Single Metric
evals
metrics
reporting
Sep 27, 2026
Diverse Query Evaluation
evals
strategy
Sep 27, 2026
Error Analysis
evals
error-analysis
core
Sep 27, 2026
Eval Maintenance and Cadence
evals
maintenance
process
Sep 27, 2026
Eval-Driven Development
evals
strategy
debate
Sep 27, 2026
Generic Metrics
evals
anti-pattern
metrics
Sep 27, 2026
How Many Examples Do You Need
evals
data
sample-size
Sep 27, 2026
Internal Eval Platforms
evals
platform
organization
Sep 27, 2026
Judge Context and Context Rot
evals
llm-judge
context-engineering
Sep 27, 2026
Judge Validation
evals
llm-judge
statistics
core
Sep 27, 2026
LLM-as-Judge
evals
llm-judge
automated-evals
core
Sep 27, 2026
Minimum Viable Eval Setup
evals
getting-started
Sep 27, 2026
Prompt Optimization
evals
prompting
automation
Sep 27, 2026
Team Collaboration on Evals
evals
people
process
Sep 27, 2026
Three Levels of Evaluation
evals
framework
Sep 27, 2026
Trace Sampling Strategies
evals
sampling
data
Sep 27, 2026
pass@k and pass^k
evals
agents
statistics
non-determinism
Sep 27, 2026
Eval Playbook
playbook
evals
moc
core
Sep 27, 2026
Abstention and Uncertainty
evals
calibration
hallucination
Sep 27, 2026
Agentic Workflow Evaluation
evals
agents
tool-calls
Sep 27, 2026
Human Handoff Evaluation
evals
handoffs
agents
Sep 27, 2026
Multi-turn Conversation Evaluation
evals
multi-turn
conversations
Sep 27, 2026
RAG Evaluation
evals
rag
retrieval
Sep 25, 2026
Critique Shadowing
evals
llm-judge
process
Sep 25, 2026
Reference Answers and Rubrics
evals
annotation
rubric
Sep 25, 2026
Coding Agent Evaluation
evals
coding-agents
agents
Sep 25, 2026
Document Processing and Chunk Size
evals
document-processing
chunking
context
Sep 25, 2026
Multi-Step Workflow Evaluation
evals
workflows
pipelines
Sep 25, 2026
Error Discovery Skill Workflow
evals
tooling
agents
error-analysis
Sep 25, 2026
Model Selection
evals
strategy
models
Sep 25, 2026
Prompt Management
evals
prompting
tooling
Sep 25, 2026
Capability vs Regression Evals
evals
agents
strategy
Sep 25, 2026
Maintaining Trust in Evals
evals
trust
process
Sep 25, 2026
Making the Case for Evals
evals
management
communication
Sep 25, 2026
CI Evals vs Production Monitoring
evals
production
ci-cd
monitoring
Sep 25, 2026
Guardrails vs Evaluators
evals
production
guardrails
Sep 25, 2026
Designing for Verification
evals
product-design
verification
Sep 25, 2026
Evals with Sensitive Data
evals
privacy
data
Sep 25, 2026
Reviewing Large Traces
evals
annotation
agents
Sep 25, 2026
What LLMs Can and Can't Automate in Evals
evals
automation
ai-assistance
Sep 25, 2026
Synthetic Data
evals
data
synthetic-data
Sep 25, 2026
Code-Based Evals
evals
automated-evals
assertions
Sep 25, 2026
Judge Bias Correction
evals
statistics
llm-judge
Sep 25, 2026
Criteria Drift
evals
research
rubric
Sep 25, 2026
Product Evals vs Model Benchmarks
evals
fundamentals
Sep 25, 2026
Theoretical Saturation
evals
error-analysis
qualitative-research
Sep 25, 2026
Traces
evals
fundamentals
observability
Nov 10, 2025
I did a deep study on AI Evals, sharing my learning and open for discussion
clippings
evals
ai/strategy
Nov 10, 2025
huggingface/evaluation-guidebook
clippings
ai
evals
Nov 10, 2025
The One Skill Every AI PM Needs (That Nobody Taught You)
clippings
ai
evals
Nov 10, 2025
AI Evaluations Clearly Explained in 50 Minutes (Real Example) | Hamel Husain
ai
evals
clippings
youtube
Nov 10, 2025
What I learned from "AI Evals for Engineers and PMs"
clippings
course
ai
evals
Nov 01, 2025
LangWatch AI Agent Testing and LLM Evaluation Platform
product
evals
AI
open-source
