In a later randomized split, Opus 5.5 at medium showed no measurable change in review outcomes or critical findings compared with Opus 5, so we switched.
Ryan SkidmoreAuthor of Cloudflare's "Orchestrating AI Code Review at scale" blog post · Practitioner, 
Cloudflare
Post: Cutting our inference bill by 68% by being obsessive about optimization, Cloudflare blog, October 19, 2026, at Changing models and effort.
Ryan Skidmore, Author of Cloudflare's "Orchestrating AI Code Review at scale" blog post. Wrote the engineering post describing the system Cloudflare runs internally, with its own usage data.
Cloudflare sells AI or agent tools or services, so it has a reason to publish flattering numbers about AI. The passage stays in the archive; read it with that in mind.
Checked against the source by a first reader, then confirmed independently by a second reader who fetched the source without seeing the first reader's notes (Independent review agent (Claude Sonnet), blind second pass from passage text and citation only, no repository access, 2026-11-07).
Exact match to the page text. The post is bylined to ryan-skidmore. Only apostrophe style was normalized.
Ryan Skidmore (Cloudflare), “Cutting our inference bill by 68% by being obsessive about optimization”, Cloudflare blog, 2026-10-19, at Changing models and effort. https://blog.cloudflare.com/obsessive-inference-optimization/ Archived at https://the-ai-record.com/passages/cloudflare-later-randomized-split-opus-55-medium-2026
Before the changes, authors acted on 34% of the code quality reviewer's serious findings. Now it's 59%.
Ryan SkidmoreAuthor of Cloudflare's "Orchestrating AI Code Review at scale" blog post · Practitioner, Cloudflare
Post: Cutting our inference bill by 68% by being obsessive about optimization, Cloudflare blog, October 19, 2026, at Review quality.
With code quality at high effort, the average review now costs just $0.77, down 68% from the $2.43 it averaged over our two most expensive weeks before we started.
Ryan SkidmoreAuthor of Cloudflare's "Orchestrating AI Code Review at scale" blog post · Practitioner, Cloudflare
Post: Cutting our inference bill by 68% by being obsessive about optimization, Cloudflare blog, October 19, 2026.
Important findings per review went from 0.28 before to 0.30 now, and the intervals overlap.
Ryan SkidmoreAuthor of Cloudflare's "Orchestrating AI Code Review at scale" blog post · Practitioner, Cloudflare
Post: Cutting our inference bill by 68% by being obsessive about optimization, Cloudflare blog, October 19, 2026, at Review quality.
Medium effort did lose real findings. In the randomized split, code quality at medium led to roughly a quarter as many acted-on findings as at high, which is why we're moving it to high.
Ryan SkidmoreAuthor of Cloudflare's "Orchestrating AI Code Review at scale" blog post · Practitioner, Cloudflare
Post: Cutting our inference bill by 68% by being obsessive about optimization, Cloudflare blog, October 19, 2026, at Review quality.
Before switching all reviews over from v1 to v2, we ran every review in every canary repo on both v1 and v2 in a comparative shadow mode. The two agreed on the approval decision in 98% of the merge requests we tested, so we moved everyone over in one go in early October and removed the v1 code.
Ryan SkidmoreAuthor of Cloudflare's "Orchestrating AI Code Review at scale" blog post · Practitioner, Cloudflare
Post: Cutting our inference bill by 68% by being obsessive about optimization, Cloudflare blog, October 19, 2026, at What else has changed since our last post?.
Something wrong? Report a correction. JSON: /api/v1/passages/cloudflare-later-randomized-split-opus-55-medium-2026.json