How do organizations keep AI spending in check?

“And the last thing is CPUs over GPUs. We've recently launched something at Notion called Workers. I don't think that the GPU is necessary for every job. A lot of the jobs that we have are actually serving discrete pieces of code. Like you don't need an LLM to turn a CSV into a PDF. You don't need an LLM to talk to Notion tool calls if we have a CLI. You definitely don't need an LLM to do deterministic SQL queries. This is where people become token poor very quick.”

Sarah SachsLeads engineering teams for AI, Notion · Leader, Notion

“And not all traffic is equal. It is a huge miss to send all of these to the latest Opus model. Some of these absolutely. Large scale data analysis, when you do it on Notion, will recommend Opus, right? When you triage an email inbox, if we're charging you to do that on Opus, we're ripping you off, and ourselves.”

Sarah SachsLeads engineering teams for AI, Notion · Leader, Notion

“Multi-model strategy works: Using cheaper models for simple checks and expensive ones for complex reasoning cut costs by 80%”

JuspayPublished by the organization itself (post), no named author

“And we have this notion of spend tiers. So by default, you get certain … tier, and then we have many levels of tiers. Once you're close to that tier, then you can raise a request to go to the next tier. And you just need a simple manager approval. … But we don't want to restrict engineers … they should be able to spend whatever they need because ultimately we want to … maximize the gain from this.”

Uday Kiran MedisettyDistinguished Engineer, Uber · Practitioner, Uber

“We expect this leverage to be partially offset by the ramp up in our use of third-party AI models. This reflects a deliberate choice to prioritize company-wide adoption over cost optimization at this stage. This investment is contributing to cost efficiency across many other expense lines (customer service, for example) as well as helping us to generate incremental revenue (in Acquiring and Advertising, for example).”

Mercado LibrePublished by the organization itself (regulatory filing), no named author

“GLOWC, an internal framework used by engineers at Airbnb, makes it easy to fine-tune open-weight language (and vision-language) models into fast, accurate classifiers. In this blog post, we put one of these classifiers head-to-head with open-weight and proprietary models and find that it matches frontier-level quality in under 100ms at roughly 1% of the cost.”

AirbnbPublished by the organization itself (post), no named author

“Proprietary models only win at low traffic: below roughly 15 to 30 RPM, Claude Sonnet 5.5 and GPT-6.1 Sol are cheaper than a dedicated GPU.”

AirbnbPublished by the organization itself (post), no named author

“With code quality at high effort, the average review now costs just $0.77, down 68% from the $2.43 it averaged over our two most expensive weeks before we started.”

Ryan SkidmoreAuthor of Cloudflare's "Orchestrating AI Code Review at scale" blog post · Practitioner, Cloudflare

“One trick that we do, which I think is really helpful, is while you're still within the warm cache period, you should probably do the compaction right now, because you have this long conversation … going to be much cheaper. So just go ahead, do the compaction, and now when the user comes back at 30 minutes or whatever, they're starting from this summary, and the summary now is going to be much cheaper for them to continue adding messages on.”

Lee RobinsonWorks on Grok Bot at SpaceX (formerly Cursor) · Practitioner, SpaceX (SpaceXAI)

“Running several agents in parallel burns tokens quickly, so set budgets and keep an eye on spending from day one.”

Noah BlauensteinerAuthor, Xebia articles · Practitioner, Xebia

“Notion's auto model does this really well. We have state-of-the-art models available always. But we also have an auto model there at the top that handles about 75% of our traffic, right? We have the ability to switch between models and our product, and we also offer it to our customers … they have access to these models without vendor lock-in.”

Sarah SachsLeads engineering teams for AI, Notion · Leader, Notion

“That expertise on use cases is also very valuable to Frontier Labs. We find that our evals and our Early Access Program partnerships actually help us a lot with Frontier Labs and it's something that we can exchange instead of extraordinary large commits. And I don't think the discount is ever worth the lost in optionality.”

Sarah SachsLeads engineering teams for AI, Notion · Leader, Notion

“91% of our employees were never hitting their usage caps, so instead of lowering caps and driving up alerts, we're moving to cheaper defaults.”

Brian ArmstrongCEO · Executive, Coinbase

“… speaking of costs, the cost of AI is too damn high. … since 2024, our costs have gone up at least 6x. … again, no question that it's had positive impact, but it's gone from something that I can self fund using my own budget to something that I need to ask permission from … the CFO.”

Anshu ChadhaDirector of Engineering, Dev Platform, Uber · Leader, Uber

“I’ve given him unlimited tokens because he’s going to refactor code… So I’m like, you can go, you can go nuts.”

Jeetu PatelPresident and Chief Product Officer · Executive, Cisco

“And how much does this get cost? I don't really have a clear answer to that because he's sharing my GitHub subscription. So I think I use most of it. But I think maybe it's been costing like $100 to $200 a month maybe. And that's not, that's almost for free. But I guess that will increase now. And also we're not using the most expensive models. So that cost could go up a lot. But it's enabled us to do this.”

Hallstein BrøtanSenior consultant, Novanet; developer on Nordic Corporate Bank's system · Practitioner, Nordic Corporate Bank

“And since all the requests go through one place, we could also add like audit logging, session traces, and a whole bunch of infrastructure around this. Every model request at Uber goes through this. So there's no model request that doesn't go through this. So that gives us like one single entry point for across all the models.”

Uday Kiran MedisettyDistinguished Engineer, Uber · Practitioner, Uber