Obsidian Metadata

type	source
url	https://hamel.dev/blog/posts/eval-tools/
authors	Hamel Husain
published	2025-10-01

Source: Selecting the Right AI Evals Tool

URL: https://hamel.dev/blog/posts/eval-tools/ · Hamel Husain (panel with Shreya Shankar & Bryan Bischof)

LangSmith (Harrison Chase), Braintrust (Wayde Gilliam), and Arize Phoenix (SallyAnn DeLucia) each completed the same homework assignment (the course’s recipe-chatbot repo) on recorded video.

Criteria: workflow/DX (failure → iteration time; notebook and SDK ergonomics) · human-in-the-loop (axial coding often missing) · transparency vs magic (beware an AI that writes the rubric and scores) · ecosystem integration (no proprietary DSLs; export).

Hamel personally uses the tools as a backend data store plus Jupyter and custom annotation UIs. → Eval Tooling and Vendors