{
  "apiVersion": "v1",
  "topic": {
    "id": "testing",
    "title": "How is AI used for testing?",
    "summary": "AI that writes, runs or repairs tests, and what that does to quality.",
    "group": "Building software",
    "aliases": [
      "tests",
      "test coverage",
      "qa"
    ]
  },
  "passages": [
    {
      "id": "stripe-stripes-enormous-preexisting-battery-tests-over-three-2026",
      "text": "Stripe’s enormous preexisting battery of tests—over three million of them—can provide this feedback.",
      "speaker": "Alistair Gray",
      "role": "Software engineer, Leverage team",
      "organization": "Stripe",
      "aiVendor": false,
      "work": {
        "title": "Minions: Stripe’s one-shot, end-to-end coding agents—Part 2",
        "type": "post",
        "date": "2026-02-19",
        "publisher": "Stripe developer blog",
        "url": "https://stripe.dev/blog/minions-stripes-one-shot-end-to-end-coding-agents-part-2"
      },
      "grade": "B",
      "verification": {
        "status": "verified",
        "reviewedOn": "2026-10-31"
      },
      "topics": [
        "testing"
      ],
      "functions": [
        "engineering"
      ],
      "figures": [],
      "context": null,
      "url": "https://the-ai-record.com/passages/stripe-stripes-enormous-preexisting-battery-tests-over-three-2026"
    },
    {
      "id": "airbnb-reached-97-3-5k-original-enzyme-files-2025",
      "text": "we reached 97% of the 3.5K original Enzyme files",
      "speaker": "Charles Covey-Brandt",
      "role": "Author of the Airbnb engineering blog post on LLM test migration (role not stated on the page)",
      "organization": "Airbnb",
      "aiVendor": false,
      "work": {
        "title": "Accelerating Large-Scale Test Migration with LLMs",
        "type": "post",
        "date": "2025-03",
        "publisher": "Airbnb engineering blog",
        "url": "https://airbnb.tech/?p=366"
      },
      "grade": "B",
      "verification": {
        "status": "verified",
        "reviewedOn": "2026-10-31"
      },
      "topics": [
        "testing",
        "migrations"
      ],
      "functions": [
        "engineering"
      ],
      "figures": [],
      "context": null,
      "url": "https://the-ai-record.com/passages/airbnb-reached-97-3-5k-original-enzyme-files-2025"
    },
    {
      "id": "greptile-swarm-agents-every-pull-request-sandbox-2026",
      "text": "every time there's a pull request, we have a swarm of agents. And they go and look at every file that's changed, every file that's related to the files that change to figure out if there's bugs. And it also spins up your code in a sandbox, … dependencies, spins up local host, clicks around to try to break things, mocks inputs, and everything else to figure out if the code is broken.",
      "speaker": "Daksh Gupta",
      "role": "Co-founder",
      "organization": "Greptile",
      "aiVendor": true,
      "work": {
        "title": "AI-Generated Code Is Already Competing With Human Code",
        "type": "talk",
        "date": "2026-09-27",
        "publisher": "AI Engineer (YouTube)",
        "locator": "00:27",
        "url": "https://www.youtube.com/watch?v=474j-n1Ltxc"
      },
      "grade": "B",
      "verification": {
        "status": "verified",
        "reviewedOn": "2026-11-01"
      },
      "topics": [
        "code-review",
        "testing"
      ],
      "functions": [
        "engineering"
      ],
      "figures": [],
      "context": null,
      "url": "https://the-ai-record.com/passages/greptile-swarm-agents-every-pull-request-sandbox-2026"
    },
    {
      "id": "figma-playwright-agent-explore-code-big-win-productivity-2026",
      "text": "So for example, when … playwright … came out instead of having humans navigate the code now, the agent can explore the code. That was a big win unlock for productivity in a lot of our team.",
      "speaker": "Eyal Blum",
      "role": "Software Engineer",
      "organization": "Figma",
      "aiVendor": true,
      "work": {
        "title": "How to Get Your Org to Adopt Coding Agents (Without Shipping Garbage)",
        "type": "talk",
        "date": "2026-08-28",
        "publisher": "AI Engineer (YouTube)",
        "locator": "05:20",
        "url": "https://www.youtube.com/watch?v=5Bn0xro2ol8"
      },
      "grade": "B",
      "verification": {
        "status": "verified",
        "reviewedOn": "2026-11-01"
      },
      "topics": [
        "developer-productivity",
        "testing"
      ],
      "functions": [
        "engineering"
      ],
      "figures": [],
      "context": null,
      "url": "https://the-ai-record.com/passages/figma-playwright-agent-explore-code-big-win-productivity-2026"
    },
    {
      "id": "boundary-agents-run-baml-programs-inspect-transcripts-humans-triage-2026",
      "text": "What we did was we built a system that actually has agents constantly running and creating BAML programs. … We then look at the entire … transcripts, see what tools they use, see what happened, and obviously we as humans can inspect them. But more importantly, we can have agents go inspect them. And agents find what was good, what was bad. … And we can have humans collaborate with these issues to figure out which ones are real, which ones are hallucinations, which ones don't have taste as much as I hate to use that word. And then we can have agents go ahead and create fixes to these problems and go to address them.",
      "speaker": "Vaibhav Gupta",
      "role": "Co-founder and Chief Executive",
      "organization": "Boundary (BoundaryML)",
      "aiVendor": true,
      "work": {
        "title": "Fighting Slop with Slop",
        "type": "talk",
        "date": "2026-07-31",
        "publisher": "AI Engineer (YouTube)",
        "locator": "5:05",
        "url": "https://www.youtube.com/watch?v=AMiyLItEtLA"
      },
      "grade": "B",
      "verification": {
        "status": "verified",
        "reviewedOn": "2026-11-01"
      },
      "topics": [
        "autonomy-and-oversight",
        "bugs-and-incidents",
        "testing"
      ],
      "functions": [
        "engineering"
      ],
      "figures": [],
      "context": "Gupta is describing how the team tests its programming language when it does not rely on reading every line of code.",
      "url": "https://the-ai-record.com/passages/boundary-agents-run-baml-programs-inspect-transcripts-humans-triage-2026"
    },
    {
      "id": "cognition-big-shift-core-components-testable-purely-local-2026",
      "text": "We’ve also internally done a big shift to make a lot of our core, components of code testable as purely local dev without needing to actually, integrate with, any live services for this reason. And honestly, the older the company, the more you have to change to shift in this direction. But you can use AI to help you perform this migration nowadays.",
      "speaker": "Walden Yan",
      "role": "Co-founder and Chief Product Officer",
      "organization": "Cognition",
      "aiVendor": true,
      "work": {
        "title": "The Age of Async Agents: Cognition's Walden Yan and OpenInspect's Cole Murray",
        "type": "podcast",
        "date": "2026-05-28",
        "publisher": "Latent Space",
        "locator": "00:56:10",
        "url": "https://latent.space/p/cognition"
      },
      "grade": "C",
      "verification": {
        "status": "verified",
        "reviewedOn": "2026-11-01"
      },
      "topics": [
        "testing",
        "migrations"
      ],
      "functions": [
        "engineering"
      ],
      "figures": [],
      "context": null,
      "url": "https://the-ai-record.com/passages/cognition-big-shift-core-components-testable-purely-local-2026"
    },
    {
      "id": "shopify-over-roughly-six-weeks-weve-run-thousands-2026",
      "text": "Over roughly six weeks, we’ve run thousands of scans, produced over 300 findings ranging from defense-in-depth improvements to resolved security incidents.",
      "speaker": "Zack Deveau",
      "role": "Title not stated on Deveau's author page; the post is written from Shopify's Application Security team",
      "organization": "Shopify",
      "aiVendor": false,
      "work": {
        "title": "Building an agentic harness that outlasts the model",
        "type": "post",
        "date": "2026-07-29",
        "publisher": "Shopify engineering blog",
        "url": "https://shopify.engineering/building-an-agentic-harness-that-outlasts-the-model"
      },
      "grade": "B",
      "verification": {
        "status": "verified",
        "reviewedOn": "2026-10-31"
      },
      "topics": [
        "bugs-and-incidents",
        "testing",
        "security"
      ],
      "functions": [
        "security"
      ],
      "figures": [
        {
          "metric": "function-agent-volume",
          "value": 300,
          "unit": "count",
          "period": "about six weeks, as of 2026-07-29",
          "definition": "application security: findings produced by an agent harness that scans public-facing apps",
          "level": "unclear"
        }
      ],
      "context": null,
      "url": "https://the-ai-record.com/passages/shopify-over-roughly-six-weeks-weve-run-thousands-2026"
    },
    {
      "id": "genentech-allowing-bot-merge-directly-into-main-branch-2026",
      "text": "we are allowing the bot to merge that directly into our main branch because we have integration tests, unit tests, etc., canary tests.",
      "speaker": "Xiucheng Quek",
      "role": "Software engineer",
      "organization": "Genentech",
      "aiVendor": false,
      "work": {
        "title": "Background Agents for Genomics and Cloud Ops",
        "type": "talk",
        "date": "2026-05-07",
        "publisher": "YouTube video of a talk or interview (speaker in notes)",
        "url": "https://www.youtube.com/watch?v=GXDtw0EmoX0"
      },
      "grade": "B",
      "verification": {
        "status": "verified",
        "reviewedOn": "2026-10-31"
      },
      "topics": [
        "testing",
        "coding-agents"
      ],
      "functions": [
        "engineering"
      ],
      "figures": [],
      "context": null,
      "url": "https://the-ai-record.com/passages/genentech-allowing-bot-merge-directly-into-main-branch-2026"
    }
  ],
  "notice": [
    "Every passage is reproduced word for word from the cited source.",
    "Check the verification status: \"reported\" means not yet confirmed by an independent second reader.",
    "Organizations flagged aiVendor sell AI tooling and have a reason to publish flattering numbers.",
    "An empty result means the archive holds nothing on it, not that nobody has said anything."
  ]
}