“Regression test. Those live in ci, those have to pass a certain percent, you know, within some stochastic error rate. Then we have, as you’re building a product, evals of these aren’t passing right now, and this is launch quality. So we have a report card and we need to, on these categories, you know, be it 80 or 90% of all of these user journeys to launch, and then what we have what we call frontier or headroom evals, where we actively wanna be at 30% pass rate.”
Sarah SachsManages Notion's core AI capabilities and infrastructure team · Leader, Notion
Sarah Sachs, Manages Notion's core AI capabilities and infrastructure team. Head of AI at Notion, which makes the Custom Agents she discusses; Notion sells AI features and has reason to present them favorably.
Notion sells AI or agent tools or services, so it has a reason to publish flattering numbers about AI. The passage stays in the archive; read it with that in mind.
How this was checked
Checked against the source by a first reader, then confirmed independently by a second reader who fetched the source without seeing the first reader's notes (Independent review agent (Claude Sonnet), blind second pass from passage text and citation only, no repository access, 2026-11-03).
Transcript '[00:24:23] Sarah Sachs:' contains the passage. The passage starts at 'Regression test.' (sentence 'We have the equivalent of unit test.' precedes it). Exact match through 'we actively wanna be at 30% pass rate.'
How to cite this passage
Sarah Sachs (Notion), “Notion's Token Town: 5 Rebuilds, 100+ Tools, MCP vs CLIs and the Software Factory Future (Latent Space)”, Latent Space, 2026-04-15, at 00:24:23. https://www.latent.space/p/notion Archived at https://the-ai-record.com/passages/notion-regression-launch-quality-frontier-evals-30-2026
Notion