Obsidian Metadata
| type | source |
| subtype | video |
| url | https://www.youtube.com/watch?v=aqKUwPKBkB0 |
| speakers | Vincent Warmerdam (marimo), Hamel Husain |
| duration | 1:13:45 |
Video: Build Your Own Eval Tools With Notebooks! (marimo)
Linked from the FAQ’s minimum viable eval setup (“use a notebook”). Vincent shows how a reactive notebook turns into a data-exploration and annotation tool.
”Staring” vs “looking” at data
The classic R ChickWeight example: a summary chart says “diet 3 is best”. Interacting with the chart (pan and zoom) reveals chickens losing weight and some dying early, which means the averages are misleading. Lesson: you can only chase a smell if you take the time to sniff. Spend real minutes with a chart, and use tools that make interaction cheap.
marimo in brief
- A reactive DAG of cells: change a cell and its dependents rerun (a lazy mode exists for heavy compute). No hidden state. Delete a variable and dependents error immediately, unlike Jupyter’s stale memory.
- Stored as a plain .py file (good git diffs, PEP 723 inline dependencies,
--sandbox, runnable as a script or app). Outputs aren’t stored, but you can export to HTML or ipynb, snapshot automatically, or host via WASM (Pyodide) on GitHub Pages. - UI elements (sliders, etc.), column layouts, an AI assistant that sees in-memory dataframe schemas, and custom rules.
Exploration → annotation workflow
- Embed texts (e.g., customer-support messages) and UMAP to 2D. UMAP preserves local clusters, while PCA preserves global structure.
- Type a query to color points by similarity, which shows which clusters light up (e.g., “delivery” is much bigger than “club card”).
- Wrap the Altair chart so a lasso selection returns a dataframe reactively, then read the selected texts.
- Feed the selection into an annotation widget (anywidget-based
mo-label): a card rendered by your own function (FastHTML works well), yes / no / skip, back, hotkeys, gamepad support, and browser speech-to-text notes. The labels are a Python list you can save however you like.
- anywidget is a cross-notebook widget standard (Jupyter, Colab, marimo). You write a bit of JS once and use it from Python everywhere.
- The broader message: most annotation boils down to binary yes/no/skip. Make it fun and frictionless so you’ll do a lot of it. “We’re trying to upgrade your imagination.”
→ Annotation Tools and Review Interfaces · Minimum Viable Eval Setup · Trace Sampling Strategies