Obsidian Metadata
| type | source |
| url | https://hamel.dev/notes/llm/rag/p6-context_rot.html |
| research | https://research.trychroma.com/context-rot |
| authors | Kelly Hong (Chroma), annotated by Hamel Husain |
| published | 2025-09-10 |
Source: P6: Context Rot (Kelly Hong, Chroma)
URL: https://hamel.dev/notes/llm/rag/p6-context_rot.html
18 frontier models tested. Performance degrades with input length even on simple tasks. NIAH is lexical and misleading; semantic matching and distractors degrade sharply; shuffled haystacks do slightly better; focused ~100-token memory beats 120k of padded history (LongMemEval); even repeating words degrades; no U-shaped position effect; no model is best everywhere. Under distractors, Claude tended to abstain while GPT tended to hallucinate.
Takeaways: performance isn’t uniform with length · presentation matters · context engineering is critical (orchestrator plus clean-context subagents). Diagnose by comparing focused vs bloated context outputs.
→ Judge Context and Context Rot
Video: https://youtu.be/3s_N60u0jEY (31 min). The annotated post above already covers it slide by slide.