Obsidian Metadata

type	guide
sources	Source - AI Evals FAQ

Human Handoff Evaluation

Capture the complete user journey, including the handoff. The trace continues until the user’s need is resolved or the session ends, not when the AI hands off.
Log: the handoff decision, why it happened, context transferred, wait time, human actions, final resolution, and whether the human had enough context.
Many failures happen at handoff boundaries: handing off too early, too late, or without proper context.
In Error Analysis, treat handoffs as potential failure modes. Ask: Was the handoff necessary? Did the AI provide adequate context?
Track both handoff quality and handoff rate. Sometimes the best improvement is fewer handoffs, not better-executed ones.
Real examples: “Handoff failures (not recognizing when to transfer to humans)” was one of Nurture Boss’s top 3 issues, and “missed required handoffs” was one class of failure that every automated error-analysis tool missed (see Nurture Boss, Source - Do Automated Evals Work).

thought umwelt