Obsidian Metadata
| type | concept |
| sources | Source - AI Evals FAQ, Source - Evals Flashcards |
Guardrails vs Evaluators
| Guardrails | Evaluators | |
|---|---|---|
| Where | Inline, synchronous, in the request/response path, before anything reaches the user | After a response is produced, usually async or batch |
| Speed | Milliseconds, deterministic | Can afford heavy compute (e.g., LLM judge) |
| Form | Regex, keyword block-lists, schema/type validators, lightweight classifiers | Assertions, LLM-as-Judge, human review |
| Targets | Clear-cut, high-impact failures: PII leaks, profanity, disallowed instructions, SQL injection, malformed JSON, invalid code syntax | Qualities rules can’t capture: factual correctness, completeness, tone, etc. |
| On trigger | Redact, refuse, regenerate / retry, fall back | Feed dashboards, regression tests, improvement loops. Doesn’t block the answer |
| Data | 100% of live traffic | Curated sets (CI) or sampled production traffic |
- Guardrail firings are user-visible, so false positives are production bugs. Teams version guardrail rules, log every trigger, and monitor rates to keep them conservative (very low FP rate).
- Together they form layered protection. Guardrails give immediate protection against objective failures; evaluators handle monitoring and improvement on nuanced criteria.
Can an evaluator be used to auto-fix outputs in production?
Yes, but only a subset. Two decision criteria:
- Latency and cost. Can it run fast and cheaply enough in the critical path without degrading UX?
- Error-rate trade-offs. False positives block good outputs and frustrate users; false negatives let harm through. In medical advice, FNs may cost more than FPs. In creative apps, FPs that block creativity may be worse than occasional quality misses.
Most guardrails are fast and have a very low false-positive rate, so you’d almost never use a slow, non-deterministic LLM judge as a synchronous guardrail. Exception: a cascade where a slow judge only handles the minority of borderline cases. Your trade-offs may differ.
Don't use off-the-shelf LLM guardrails blindly
Always look at the prompt they send (see Source - Show Me The Prompt). For example, the
guardrailslibrary made two LLM calls with XML-schema prompts to get structured output.
Fine-tuning small specialized classifiers as guardrails can be worthwhile (Hamel avoids fine-tuning judges, but not guardrails).
Related
CI Evals vs Production Monitoring · LLM-as-Judge · Code-Based Evals