Thought UmweltThought Umwelt

thought umwelt

Agentic Workflow Evaluation

3 min read

Obsidian Metadata

typeguide
sourcesSource - AI Evals FAQ, Source - Demystifying Evals for AI Agents

Agentic Workflow Evaluation

Two phases

1. End-to-end task success (black box)

Did the agent meet the user’s goal? Define a precise success rule per task and measure it with human review or validated judges. During Error Analysis, record the first upstream failure.

Once error analysis shows which workflows fail most, move to…

2. Step-level diagnostics

With Traces logged, score components:

  • Tool choice: right tool selected?
  • Parameter extraction: inputs complete and well-formed?
  • Error handling: empty results, API failures?
  • Context retention: earlier constraints preserved?
  • Efficiency: steps, seconds, tokens.
  • Goal checkpoints: key milestones in long workflows.

Testing tool calls

Test name, arguments, result, and resulting state as separate checks, with code assertions when behavior is objective.

Example: the agent selected cancel_order, passed the correct order ID, received a successful response, and the order status changed, before telling the user it was canceled.

Also test authorization and preconditions. A valid tool call is still wrong if the user didn’t approve the action or a required check was skipped.

Decompose into checkpoints: “Find Berkeley homes under $1M and schedule viewings” becomes parameters extracted → relevant listings retrieved → availability checked → calendar invites sent. Each passes or fails independently, which makes debugging tractable.

Find hotspots with a transition failure matrix

Rows = last successful state, columns = where the first failure occurred. See Transition Failure Matrix.

Test cases for agent failures

Same as Multi-turn Conversation Evaluation: reproduce with the simplest test that still fails, and use multi-turn tests only when the failure depends on conversation context.

Anthropic’s agent-eval vocabulary and advice

  • Task (inputs + success criteria), trial (one attempt; run several), grader (with checks/assertions), transcript, outcome (final environment state), evaluation harness, agent harness / scaffold (you evaluate harness + model together), suite.
  • Grade outcomes, not paths. Rigid tool-sequence checks are brittle; agents find valid alternatives. Use partial credit for multi-component tasks.
  • Grader mix: code (string match, fail-to-pass tests, static analysis, outcome/state verification, tool-call params, transcript stats), model (rubrics, natural-language assertions, pairwise, reference-based, multi-judge), human (SME review, spot checks, A/B, IAA). Combine as weighted, all-must-pass, or hybrid.
  • Stable, isolated environments per trial.
  • Well-specified tasks with reference solutions. Two experts should reach the same verdict. A 0% pass@100 usually means a broken task.
  • Balanced sets: test when a behavior should and shouldn’t happen (e.g., search vs don’t search).
  • Make graders hack-resistant, and read transcripts so failures look fair.
  • Handle non-determinism with pass@k and pass^k.

Example task spec (conversational support agent)

graders:
  - type: llm_rubric
    rubric: prompts/support_quality.md
    assertions:
      - "Agent showed empathy for customer's frustration"
      - "Resolution was clearly explained"
      - "Agent's response grounded in fetch_policy tool results"
  - type: state_check
    expect:
      tickets: {status: resolved}
      refunds: {status: processed}
  - type: tool_calls
    required:
      - {tool: verify_identity}
      - {tool: process_refund, params: {amount: "<=100"}}
      - {tool: send_confirmation}
  - type: transcript
    max_turns: 10
tracked_metrics:
  - type: transcript
    metrics: [n_turns, n_toolcalls, n_total_tokens]
  - type: latency
    metrics: [time_to_first_token, output_tokens_per_sec, time_to_last_token]

Agent types

  • Coding: see Coding Agent Evaluation.
  • Conversational: verifiable end state + interaction-quality rubric; usually a simulated user LLM (τ-bench, τ²-bench).
  • Research: groundedness checks, coverage of key facts, source quality, exact match where possible; calibrate rubrics against experts often (BrowseComp).
  • Computer use: real or sandboxed environments, checking URL/page state and backend state (WebArena, OSWorld); also check tool choice (DOM vs screenshots).

Multi-Step Workflow Evaluation · Human Handoff Evaluation · Transition Failure Matrix

Graph View