Obsidian Metadata
| type | source |
| url | https://github.com/ai-evals-course/evals-skills |
| blog | https://hamel.dev/blog/posts/evals-skills/ |
| authors | Hamel Husain, Shreya Shankar |
| published | 2026-03-02 |
Source: Evals Skills for Coding Agents (ai-evals-course/evals-skills)
Repo: https://github.com/ai-evals-course/evals-skills · Blog: https://hamel.dev/blog/posts/evals-skills/
A set of agent skills (Claude Code, etc.) that encode the course methodology and guard against common footguns seen at 50+ companies.
npx skills add https://github.com/ai-evals-course/evals-skillsPrompt: “Run the evals-start skill from the evals plugin and follow the skill it picks. If it picks eval-audit, investigate each diagnostic area using a separate subagent in parallel, then synthesize the findings into a single report.”
| Skill | Vault note |
|---|---|
| evals-start (router) | Error Discovery Skill Workflow |
| eval-audit | Eval Audit Checklist |
| error-discovery (+ review-loop) | Error Discovery Skill Workflow, Error Analysis |
| generate-synthetic-data | Synthetic Data, Synthetic Data Prompts |
| write-code-eval | Code-Based Evals |
| write-judge-prompt | LLM-as-Judge, Judge Prompt Template |
| validate-evaluator | Judge Validation, Judge Bias Correction |
| evaluate-rag | RAG Evaluation |
| build-review-interface | Annotation Tools and Review Interfaces |
“These skills are only a starting point. Tune them to your data and domain.”