Atlassian: “we conclude that RovoDev Code Reviewer is effective in gener…”

we conclude that RovoDev Code Reviewer is effective in generating code review comments that could lead to code resolution for 38.70% (i.e., comments that triggered code changes in the subsequent commits)

AtlassianPublished by the organization itself (paper), no named author

Paper: RovoDev Code Reviewer: A Large-Scale Online Evaluation of LLM-based Code Review Automation at Atlassian, arXiv (Atlassian researchers; accepted at ICSE 2026, SEIP track), January 5, 2026, at Abstract.

38.7% code review comment resolution rate, one-year evaluation. A person approves

Checked against the source [ Grade B ] Said in the last year Atlassian: $6.6 billion revenue, FY2026 Sells AI tools

Filed under

Atlassian sells AI or agent tools or services, so it has a reason to publish flattering numbers about AI. The passage stays in the archive; read it with that in mind.

How this was checked

Checked against the source by a first reader, then confirmed independently by a second reader who fetched the source without seeing the first reader's notes (Independent review agent (Claude Sonnet), blind second pass from passage text and citation only, no repository access, 2026-11-06).

The Abstract on the fetched arXiv page contains this text exactly, including 38.70% and the parenthetical about code changes in subsequent commits.

How to cite this passage

Atlassian, “RovoDev Code Reviewer: A Large-Scale Online Evaluation of LLM-based Code Review Automation at Atlassian”, arXiv (Atlassian researchers; accepted at ICSE 2026, SEIP track), 2026-01-05, at Abstract. https://arxiv.org/abs/2601.01129 Archived at https://the-ai-record.com/passages/atlassian-rovodev-code-reviewer-effective-generating-comments-resolution-2026

More from the same source

offers the promise of accelerating feedback cycles (i.e., decreasing the PR cycle time by 30.8%), alleviating reviewer workload (i.e., reducing the number of human-written comments by 35.6%), and improving overall software quality (i.e., finding errors with actionable suggestions).

AtlassianPublished by the organization itself (paper), no named author

30.8% pr cycle time reduction, one-year evaluation. A person approves
35.6% human written comments reduction, one-year evaluation. A person approves

Checked against the source [ Grade B ] Said in the last year Atlassian: $6.6 billion revenue, FY2026 Sells AI tools Passage page

In particular, this paper aims at answering the practical question: how can we design a review-guided, context-aware, quality-checked code review comment generation without fine-tuning?

AtlassianPublished by the organization itself (paper), no named author

Checked against the source [ Grade B ] Said in the last year Atlassian: $6.6 billion revenue, FY2026 Sells AI tools Passage page

Something wrong? Report a correction. JSON: /api/v1/passages/atlassian-rovodev-code-reviewer-effective-generating-comments-resolution-2026.json