The configuration that catches the most of what our human reviewers catch is also the cheapest one we have run, and most of its budget goes to models that would be dismissed on sight as too weak for code review—which, individually, they are.
Dana DzikEngineer, Coval (author of the Coval blog post on its code reviewer) · Practitioner, 
Coval
Post: Notes on Building a Code Reviewer, Coval blog, September 10, 2026, at Where it goes.
Dana Dzik, Engineer, Coval (author of the Coval blog post on its code reviewer). Builds and runs the reviewer described in the post and reports its measurements; an insider.
Coval sells AI or agent tools or services, so it has a reason to publish flattering numbers about AI. The passage stays in the archive; read it with that in mind.
Checked against the source by a first reader, then confirmed independently by a second reader who fetched the source without seeing the first reader's notes (Independent review agent (Claude Sonnet), blind second pass from passage text and citation only, no repository access, 2026-11-07).
Exact match in the fetched page text, em dash included. Dana Dzik byline.
Dana Dzik (Coval), “Notes on Building a Code Reviewer”, Coval blog, 2026-09-10, at Where it goes. https://www.coval.ai/blog/how-we-built-our-code-reviewer/ Archived at https://the-ai-record.com/passages/coval-configuration-catches-most-what-our-2026
Asked to name sensitive risk domains in prose, the model claimed authorization in 38% of reviews, while about 18% of the changes touched anything authorization-shaped; required to quote the specific changed line instead, it produced 570 claims, of which 98% resolved to real code—the same model over the same pull requests, with only the form of the question changed.
Dana DzikEngineer, Coval (author of the Coval blog post on its code reviewer) · Practitioner, Coval
Post: Notes on Building a Code Reviewer, Coval blog, September 10, 2026, at The rebuild.
In August, people at Coval opened 2,578 pull requests and pushed roughly 3,400 further commits to them after they were first marked ready.
Dana DzikEngineer, Coval (author of the Coval blog post on its code reviewer) · Practitioner, Coval
Post: Notes on Building a Code Reviewer, Coval blog, September 10, 2026, at Introduction.
Reviews are capped at three findings, a partly arbitrary number kept because it measurably changes whether people read the output; and silence is the usual outcome, 922 of the 1,732 recorded runs of this design having published nothing, a mean of 0.70 findings per review.
Dana DzikEngineer, Coval (author of the Coval blog post on its code reviewer) · Practitioner, Coval
Post: Notes on Building a Code Reviewer, Coval blog, September 10, 2026, at The rebuild.
In its first week in production the roster brought the median cost per review from about thirty cents to eleven while publishing at roughly the same rate.
Dana DzikEngineer, Coval (author of the Coval blog post on its code reviewer) · Practitioner, Coval
Post: Notes on Building a Code Reviewer, Coval blog, September 10, 2026, at The roster.
Something wrong? Report a correction. JSON: /api/v1/passages/coval-configuration-catches-most-what-our-2026.json