The AI Record
Notions last exam is a big thing inside the company and we have people, full-time staff to it exclusively. Mm. We have a data scientist, a model behavior engineer, and an full-time, um, evals engineer just dedicated to the evals that we pass 30% of the time.

Sarah SachsManages Notion's core AI capabilities and infrastructure team · Leader, Notion

Podcast: Notion's Token Town: 5 Rebuilds, 100+ Tools, MCP vs CLIs and the Software Factory Future (Latent Space), Latent Space, April 15, 2026, at 00:24:23. Reported by Swyx.

Checked against the source [ Grade C ] Said in the last year Sells AI tools

Filed under

Who is speaking

Sarah Sachs, Manages Notion's core AI capabilities and infrastructure team. Head of AI at Notion, which makes the Custom Agents she discusses; Notion sells AI features and has reason to present them favorably.

Notion sells AI or agent tools or services, so it has a reason to publish flattering numbers about AI. The passage stays in the archive; read it with that in mind.

How this was checked

Checked against the source by a first reader, then confirmed independently by a second reader who fetched the source without seeing the first reader's notes (Independent review agent (Claude Sonnet), blind second pass from passage text and citation only, no repository access, 2026-11-03).

Sarah Sachs, with 00:24:23 as the last timestamp before the text. The page has an interjected 'Mm.' that the passage keeps, and the wording matches exactly. The page spells it 'an full-time, um, evals engineer', as the passage does.

How to cite this passage

Sarah Sachs (Notion), “Notion's Token Town: 5 Rebuilds, 100+ Tools, MCP vs CLIs and the Software Factory Future (Latent Space)”, Latent Space, 2026-04-15, at 00:24:23. https://www.latent.space/p/notion Archived at https://the-ai-record.com/passages/notion-notions-last-exam-full-time-staff-2026

More from the same source

basically it just means can you create sort of like a, as automated as possible, a workflow for developing debugging. Mm-hmm. Merging, reviewing, and maintaining a code base

Simon LastCo-founder and head of AI · Executive, Notion

Interview: Notion's Token Town: 5 Rebuilds, 100+ Tools, MCP vs CLIs and the Software Factory Future (Latent Space), Latent Space (independent AI engineering podcast), April 15, 2026.

Checked against the source [ Grade C ] Said in the last year Sells AI tools Passage page

we’ll all sit down every Friday and look at the P 99 of like the most token exhaustive custom agent transcript and just look at why it didn’t do well and cut a bunch of tasks.

Sarah SachsManages Notion's core AI capabilities and infrastructure team · Leader, Notion

Podcast: Notion's Token Town: 5 Rebuilds, 100+ Tools, MCP vs CLIs and the Software Factory Future (Latent Space), Latent Space, April 15, 2026, at 00:09:31. Reported by Swyx.

Checked against the source [ Grade C ] Said in the last year Sells AI tools Passage page

in the latest version of the agent, I, it’s not in custom agents yet, but it will be like, like next week, a week after or so, um, there’s now like over a hundred tools. Just for all, all the crazy notion stuff.

Simon LastCo-founder and head of AI · Executive, Notion

Podcast: Notion's Token Town: 5 Rebuilds, 100+ Tools, MCP vs CLIs and the Software Factory Future (Latent Space), Latent Space, April 15, 2026, at 00:54:57. Reported by Swyx.

  • 100 tools agent tools

Checked against the source [ Grade C ] Said in the last year Sells AI tools Passage page

Regression test. Those live in ci, those have to pass a certain percent, you know, within some stochastic error rate. Then we have, as you’re building a product, evals of these aren’t passing right now, and this is launch quality. So we have a report card and we need to, on these categories, you know, be it 80 or 90% of all of these user journeys to launch, and then what we have what we call frontier or headroom evals, where we actively wanna be at 30% pass rate.

Sarah SachsManages Notion's core AI capabilities and infrastructure team · Leader, Notion

Podcast: Notion's Token Town: 5 Rebuilds, 100+ Tools, MCP vs CLIs and the Software Factory Future (Latent Space), Latent Space, April 15, 2026, at 00:24:23. Reported by Swyx.

  • 30% eval target pass rate

Checked against the source [ Grade C ] Said in the last year Sells AI tools Passage page

Something wrong? Report a correction. JSON: /api/v1/passages/notion-notions-last-exam-full-time-staff-2026.json