The metrics organizations use to judge whether AI is working.
36 passages from 19 organizations; 29 checked against the source.
What the record shows Some measure time saved or backlog cleared rather than volume
Airbnb's CEO said what matters is not how many tokens you use but product throughput, design quality and how much ships, and Airbnb says it cut time from concept to delivery on key initiatives by as much as 60%. A Shopify engineer said a dependency backlog fell about 70% in the first 11 days of an agent workflow. Bank of America says EricaAssist cuts average call times by nearly one minute per interaction.
From Brian Chesky (Airbnb) , Airbnb , Erin Son (Shopify) and Bank of America .
Stripe compares the same sellers with and without its AI tool, and also cites an undefined figure
A Stripe engineering manager said that in weeks they use the internal tool Kai, account executives produce 2x the sales activity and close 39% more deals than the same sellers in weeks they do not. Separately, Stripe's president said seller productivity has increased by 20%, without defining the measure.
From Sharadh Krishnamurthy (Stripe) and Will Gaybrick (Stripe) .
Some headline gains come without a stated definition
Bank of America says 19,000 developers have seen efficiency gains of over 20%, and Meta says output per engineer rose 30% since the start of 2025; neither defines the measure in these passages.
From Bank of America and Meta .
These summaries are this site's, not quotations. Each was checked by a second reader, who confirmed it says no more than the passages it cites. The organizations' own words follow.
“ When Account Executives use Kai, they produce 2x the sales activity, create 17% more opportunities, generate 26% more revenue opportunities, and close 39% more deals when compared to the same sellers in weeks they don't use it.” Sharadh Krishnamurthy Engineering manager, Agent Foundation team · Leader ,
Post: Stripe developer blog , Stripe developer blog , July 30, 2026.
[ Grade B ] Said in the last year Passage page
“ One is you start measuring how many human touches you still do to have the agent do the right thing. That's supposed to go down.” Patrick Debois Product DevRel Lead · Practitioner ,
Context. Debois names the first of two metrics he has come to believe in for judging how productive a team is with coding agents.
Talk: Coding Agents Don't Scale Themselves. Neither Do Your Teams. The Rise of Agent Enablement , AI Engineer World's Fair 2026 (YouTube) , July 2, 2026, at 09:38 .
[ Grade B ] Said in the last year Sells AI tools Passage page
“ it's not about how many tokens you use, it's about the throughput of your product and the quality of your product designs and how much you're shipping.” Brian Chesky Co-founder and Chief Executive Officer · Executive ,
Earnings call: earnings call transcript via Platform Aeronaut (third-party transcript site) , earnings call transcript via Platform Aeronaut (third-party transcript site) , August 6, 2026.
[ Grade A ] Said in the last year Airbnb: $12.2 billion revenue, FY2025 Passage page
“ we are seeing doubling in the code output for engineers” Balaji Krishnamurthy Chief Financial Officer · Executive ,
Earnings call: earnings call transcript via Platform Aeronaut (third-party transcript site) , earnings call transcript via Platform Aeronaut (third-party transcript site) , August 5, 2026.
[ Grade A ] Said in the last year Uber: $52 billion revenue, FY2025 Passage page
“ Since the beginning of 2025, we've seen a 30% increase in output per engineer, with the majority of that growth coming from the adoption of agentic coding, which saw a big jump in Q4.” Meta Published by the organization itself (earnings call), no named author
Earnings call: earnings call transcript via Platform Aeronaut (third-party transcript site) , earnings call transcript via Platform Aeronaut (third-party transcript site) , January 28, 2026.
[ Grade A ] Said in the last year Meta: $201 billion revenue, FY2025 Passage page
“ We're seeing even stronger gains with power users of AI coding tools, whose output has increased 80% year over year.” Meta Published by the organization itself (earnings call), no named author
Earnings call: earnings call transcript via Platform Aeronaut (third-party transcript site) , earnings call transcript via Platform Aeronaut (third-party transcript site) , January 28, 2026.
[ Grade A ] Said in the last year Meta: $201 billion revenue, FY2025 Passage page
“ In the first 11 days of running the dependency workflow, the backlog of open issues fell by about 70%.” Erin Son Senior Engineer · Practitioner ,
Post: How River takes security work from a fix to merge , Shopify engineering blog , September 2, 2026.
[ Grade B ] Said in the last year Shopify: $11.6 billion revenue, FY2025 Passage page
“ the number of issues with a pull request attached by someone in an engineering, product, or design function has grown sevenfold” Linear Published by the organization itself (post), no named author
Post: Linear (own site) , Linear (own site) , August 26, 2026.
[ Grade B ] Said in the last year Linear: $100 million revenue, passed earlier in 2026 Passage page
“ And all of this led to twice the number of lines of code per engineer year over year.” Uday Kiran Medisetty Distinguished Engineer · Practitioner ,
Talk: Agentic SDLC at Uber: Building Blocks for Uber's Software Factory (talk, AI Engineer World's Fair 2026) , AI Engineer (conference organizer) YouTube channel , August 21, 2026.
[ Grade B ] Said in the last year Uber: $52 billion revenue, FY2025 Passage page
“ EricaAssist reduces average call times by nearly one minute per interaction” Bank of America Published by the organization itself (post), no named author
Post: Bank of America newsroom (the company's own press site) , Bank of America newsroom (the company's own press site) , July 21, 2026.
[ Grade B ] Said in the last year Bank of America: $113 billion revenue, FY2025 Passage page
“ Our PR throughput inside of Airbnb is 65% higher than it was before we introduced agentic AI.” Christopher Sanson Product Lead, AI Developer Experience · Leader ,
Social post: DX podcast page (DX sells developer-productivity measurement software) carrying a talk given at its DX Annual conference , DX podcast page (DX sells developer-productivity measurement software) carrying a talk given at its DX Annual conference , June 22, 2026.
[ Grade B ] Said in the last year Airbnb: $12.2 billion revenue, FY2025 Passage page
“ we're seeing a 76% increase in pull request frequency” Spotify Published by the organization itself (post), no named author
Post: Coding Is No Longer the Constraint: Scaling Developer Experience to Teams and Agents at Spotify , Spotify engineering blog , June 3, 2026.
[ Grade B ] Said in the last year Passage page
“ engineers using it ship about 70% more merged pull requests” OpenAI Published by the organization itself (post), no named author
Post: OpenAI forum (OpenAI-run) , OpenAI forum (OpenAI-run) , December 4, 2025.
[ Grade B ] Said in the last year OpenAI: $20 billion revenue, ARR, 2025 Sells AI tools Passage page
“ roughly corroborates the 67% increase in merged pull requests” Anthropic Published by the organization itself (paper), no named author
Paper: How AI is transforming work at Anthropic , Anthropic (own site) , December 2, 2025.
[ Grade B ] Said in the last year Anthropic: $47 billion revenue, run-rate, May 2026 Sells AI tools Passage page
“ More than 98% of users find the information they need, significantly decreasing call center volume” Bank of America Published by the organization itself (post), no named author
Post: Bank of America newsroom (the company's own press site) , Bank of America newsroom (the company's own press site) , August 20, 2025.
[ Grade B ] One to two years old Bank of America: $113 billion revenue, FY2025 Passage page
“ More than 98% of clients get answers they need from Erica within 44 seconds, on average.” Bank of America Published by the organization itself (post), no named author
Post: Bank of America newsroom (the company's own press site) , Bank of America newsroom (the company's own press site) , April 8, 2024.
[ Grade B ] Over two years old Bank of America: $113 billion revenue, FY2025 Passage page
“ seller productivity has increased by 20%” Will Gaybrick President, Technology and Business (also styled President of Product and Business) · Executive ,
Interview: Tokens Are the New Dollars | Stripe's Will Gaybrick & David George (podcast episode: "Stripe’s AI Strategy: Build More, Not Less") , The a16z Show (podcast and YouTube channel of Andreessen Horowitz, an investor in Stripe) , August 17, 2026. Reported by David George .
[ Grade C ] Said in the last year Passage page
“ Before I trusted this workflow, I back tested the scoring against months of clean up history. Then I ran it live. Seven for seven PRs with green CI. Now it runs daily on my laptop and hands work back for me to review. … $1.26 a pull request. A backlog of around 520 flags a year runs for under $700. If you had engineers working on this by hand, at least $26,000 that you're looking at.” Andrew Orobator Senior Software Engineer · Practitioner ,
Context. The speaker is a Reddit Android engineer. In the talk he describes this agent as running in his local workflow and on his laptop, and says some of his other practices are from his personal side projects. The $26,000 is his own estimate of manual cost, and the PR count is a small sample.
Talk: Scale the Judgment, Not the Model , AI Engineer (YouTube) , June 30, 2026, at 11:59 .
[ Grade B ] Said in the last year Passage page
“ it grew like 7X over, the last, I think it was, two months, three months, something like that. And then you see our engineering headcount growth. It’s, gone up by, 10% or something.” Walden Yan Co-founder and Chief Product Officer · Executive ,
Context. Walden Yan was describing an internal graph of Cognition's merged pull requests, which he said had grown 7X; the passage is the continuation of that answer.
Podcast: The Age of Async Agents: Cognition's Walden Yan and OpenInspect's Cole Murray , Latent Space , May 28, 2026, at 00:03:01 . Reported by Swyx .
[ Grade C ] Said in the last year Sells AI tools Passage page
“ And completely anecdotally based on my own experience, I've really only felt maybe 10 to 20% more productive with all of these phases that have come before. But now inside of Amazon, we've been running pilots with different teams across the company, and we've been seeing a median of 4.5x productivity improvement and sometimes more than 10x.” Clare Liguori Senior Principal Engineer, AWS · Practitioner ,
Context. Liguori is describing how pilots with Amazon teams compare with her own earlier experience of AI coding assistants.
Talk: From AI-Assisted to AI-Native: Building a Frontier Development Team , AI Engineer (YouTube) , August 28, 2026, at 1:11 .
[ Grade B ] Said in the last year Amazon: $717 billion revenue, FY2025 Sells AI tools Passage page
“ So another experiment that I want to talk about is an experimental sprint that was done in the Prime Video Organization. They took a 10-day sprint and they … experiment where they put, again, six engineers in a room and they let them go wild with Kiro. They brought down the project delivery time estimate from what was going to be 90 weeks down to 24, based on all of the progress they had made in this 10-day sprint.” Clare Liguori Senior Principal Engineer, AWS · Practitioner ,
Context. Liguori is describing an experimental sprint in Amazon's Prime Video organization.
Talk: From AI-Assisted to AI-Native: Building a Frontier Development Team , AI Engineer (YouTube) , August 28, 2026, at 4:05 .
[ Grade B ] Said in the last year Amazon: $717 billion revenue, FY2025 Sells AI tools Passage page
“ They watched 50 teams that were totally … distribution of early career folks, mid-career senior engineers, and that worked on existing systems. … And they watched them for the better part of last year, and they found something super interesting. They found that there was a big difference in the productivity gains that they saw between half of the teams and the other half. And in this case, they used a productivity metric of deployment velocity to production. … And they saw that for half of the teams, they achieved less than 3X increase.” Clare Liguori Senior Principal Engineer, AWS · Practitioner ,
Context. Liguori is describing a pilot run by Amazon Stores, which she says covers Amazon.com, its retail websites and its physical stores.
Talk: From AI-Assisted to AI-Native: Building a Frontier Development Team , AI Engineer (YouTube) , August 28, 2026, at 5:59 .
[ Grade B ] Said in the last year Amazon: $717 billion revenue, FY2025 Sells AI tools Passage page
“ [uReview] does … around 25,000 comments a week, and we get 10% of them actually get some feedback … We also saw that the overall address rate was around 67%, and almost three quarters of the high severity issues were usually addressed by the developers” Ameya Ketkar Speaker, AI Engineer World's Fair 2026 (job title not stated) · Practitioner ,
Context. Ameya Ketkar is reporting results for uReview, Uber's code-review system. The address rate is how often a developer acts on a comment, as he defined it earlier in the talk.
Talk: Building uReview, Uber's Multi-Agent Code Review Engine (talk, AI Engineer World's Fair 2026) , AI Engineer (YouTube) , August 28, 2026, at 10:13 .
[ Grade B ] Said in the last year Uber: $52 billion revenue, FY2025 Passage page
“ we saw that against like a very naive implementation, our costs were down by 60%, and our quality and our accuracy was up by around 70%.” Ameya Ketkar Speaker, AI Engineer World's Fair 2026 (job title not stated) · Practitioner ,
Context. Ameya Ketkar is summarizing the effect of the observability and evaluation work on uReview, Uber's code-review system.
Talk: Building uReview, Uber's Multi-Agent Code Review Engine (talk, AI Engineer World's Fair 2026) , AI Engineer (YouTube) , August 28, 2026, at 10:50 .
[ Grade B ] Said in the last year Uber: $52 billion revenue, FY2025 Passage page
“ requests and tokens tell us like usage, like merge requests and pipelines tell us about platform activity, but they do not tell us what value … providing. … the more meaningful long-term question really is like what is the future velocity and outcomes that we can measure with less friction, getting changes out safely.” Rajesh Bhatia Senior Director of Engineering, Cloudflare · Leader ,
Context. Asked how Cloudflare thinks about measuring return on AI spend.
Talk: From Assisted to Delegated: Cloudflare's AI Engineering Stack (interview, Background Agents Summit 2026) , Ona (YouTube) , May 6, 2026, at 20:49 .
[ Grade B ] Said in the last year Cloudflare: $2.2 billion revenue, FY2025 Sells AI tools Passage page
“ The impact on developer velocity internally is clear: we’ve never seen a quarter-to-quarter increase in merge requests to this degree. As AI tooling adoption has grown the 4-week rolling average has climbed from ~5,600/week to over 8,700.” Cloudflare Published by the organization itself (post), no named author
Post: The AI engineering stack we built internally, on the platform we ship , Cloudflare blog , April 20, 2026.
[ Grade B ] Said in the last year Cloudflare: $2.2 billion revenue, FY2025 Sells AI tools Passage page
“ reduced the time from concept to delivery by as much as 60%” Airbnb Published by the organization itself (regulatory filing), no named author
Regulatory filing: SEC filing (8-K exhibit: the company's own shareholder letter) , SEC filing (8-K exhibit: the company's own shareholder letter) , August 6, 2026.
[ Grade A ] Said in the last year Airbnb: $12.2 billion revenue, FY2025 Passage page
“ we've increased the number of features and improvements we shipped this year by nearly 80%.” Brian Chesky Co-founder and Chief Executive Officer · Executive ,
Earnings call: earnings call transcript via Platform Aeronaut (third-party transcript site) , earnings call transcript via Platform Aeronaut (third-party transcript site) , August 6, 2026.
[ Grade A ] Said in the last year Airbnb: $12.2 billion revenue, FY2025 Passage page
“ Employee engagement is measured through Amplifier, Klarna's AI-powered engagement survey.” Klarna Published by the organization itself (regulatory filing), no named author
Regulatory filing: SEC filing (the company's own regulatory report), furnished on Form 6-K , SEC filing (the company's own regulatory report), furnished on Form 6-K , May 27, 2026.
[ Grade A ] Said in the last year Klarna: $3.5 billion revenue, FY2025 Passage page
“ More than 80% of engineers are now using AI tools.” Brian Chesky Co-founder and Chief Executive Officer · Executive ,
Earnings call: earnings call transcript via Platform Aeronaut (third-party transcript site) , earnings call transcript via Platform Aeronaut (third-party transcript site) , February 12, 2026.
[ Grade A ] Said in the last year Airbnb: $12.2 billion revenue, FY2025 Passage page
“ 19,000 software developers have experienced efficiency gains of over 20% through real-time coding assistance” Bank of America Published by the organization itself (post), no named author
Post: Bank of America newsroom (the company's own press site) , Bank of America newsroom (the company's own press site) , October 14, 2026.
[ Grade B ] Said in the last year Bank of America: $113 billion revenue, FY2025 Passage page
“ We have like 50 users and I don't want to scare people, but the PR throughput for these folks is like 3X higher again.” Christopher Sanson Product Lead, AI Developer Experience · Leader ,
Social post: DX podcast page (DX sells developer-productivity measurement software) carrying a talk given at its DX Annual conference , DX podcast page (DX sells developer-productivity measurement software) carrying a talk given at its DX Annual conference , June 22, 2026.
[ Grade B ] Said in the last year Airbnb: $12.2 billion revenue, FY2025 Passage page
“ we complete about 189 tickets for this sprint, which is about four times as much or two to three times more than what we usually accomplish” Xiucheng Quek Software engineer · Practitioner ,
Talk: Background Agents for Genomics and Cloud Ops , YouTube video of a talk or interview (Xiucheng Quek, software engineer (Genentech)) , May 7, 2026.
[ Grade B ] Said in the last year Passage page
“ we're not saying, "Wow, we need fewer sellers." We're saying, "So, we need a lot more sellers."” Will Gaybrick President, Technology and Business (also styled President of Product and Business) · Executive ,
Interview: Tokens Are the New Dollars | Stripe's Will Gaybrick & David George (podcast episode: "Stripe’s AI Strategy: Build More, Not Less") , The a16z Show (podcast and YouTube channel of Andreessen Horowitz, an investor in Stripe) , August 17, 2026. Reported by David George .
[ Grade C ] Said in the last year Passage page
Stripe
Tessl
Airbnb
Uber
Shopify
Greptile
Boundary (BoundaryML)
Reddit
Cognition
Amazon
Cloudflare
Genentech