TRANSMISSION // 2611.22793v1

When AI code review speeds up work—and adds more of it

Source published

A study of GitHub Copilot Reviewer finds faster merging for reviewed human pull requests, but rising submissions and a sustained increase in bot PRs may add to maintainer workload.

AI-generated conceptual illustration for When AI code review speeds up work—and adds more of it
AI-generated conceptual illustration; not documentary evidence.

AI review can make feedback faster, but does that leave open-source maintainers with less work—or more contributions to handle? A study of GitHub repositories finds signs of both: human pull requests reviewed by Copilot Reviewer were associated with faster merging, while overall submissions and bot-authored PRs rose after adoption.

The question behind the study

Open-source projects depend on maintainers to assess proposed changes, discuss revisions and decide what gets added to a codebase. An AI reviewer could help by providing quick feedback at scale. But faster feedback might also encourage more people—or automated accounts—to submit changes, potentially increasing the amount of work humans still need to oversee.

Feiyang Xu, Poonacha K. Medappa and Murat M. Tunc examine that trade-off around GitHub’s Copilot Reviewer, introduced in 2025. The tool comments on proposed code, while human maintainers retain responsibility for final decisions. The researchers ask how its adoption relates to maintainer workload, contributor behavior and the time it takes pull requests (PRs) to merge.

What the researchers did

The study draws on GitHub data from 3,937 repositories, mainly Python projects, and about 974,654 PRs. The repositories had to meet criteria including ongoing activity, at least 100 stars and PRs both before and after the reviewer’s announcement. The analysis covers March 2024 through February 2026.

To compare changes over time, the authors use a staggered difference-in-differences approach. They compare maintainers who began using Copilot Reviewer with maintainers who had not yet adopted it, accounting for differences between maintainers and shared changes across months. They also use survival analysis to examine how quickly PRs were merged, including PRs still open at the end of the observation period rather than treating them as completed cases.

The data distinguish human and bot accounts, not whether a person used AI to write code. A PR from a human account is counted as human-authored even if AI helped produce it. The study therefore measures observable account types, not the true degree of AI involvement in code creation.

More submissions, different patterns over time

After adoption, the authors estimate that maintainers reviewed about 10% more PRs and 14% more review comments per PR. Both human- and bot-authored submissions increased in the overall estimates. The time pattern, however, differs: human PRs initially rose, then fell back and eventually below the comparison group, while bot PRs showed a sustained increase.

The reported changes also differ in size. Human PRs became smaller on average by the study’s measures of code additions, files changed and commits. Bot PRs showed a significant increase in code additions, while the changes in files and commits were not statistically significant in the reported table. These are measures of PR size, not direct assessments of code quality.

For merging speed, the survival analysis finds a higher merge rate for human PRs reviewed by Copilot Reviewer than for PRs not reviewed by it. The pattern goes the other way for bot PRs: those reviewed by Copilot had a lower merge rate and took longer to merge. In a separate analysis, the estimated speed advantage associated with Copilot review was weaker for larger PRs.

Taken together, the results suggest that AI review may help move some human contributions through review more quickly while coinciding with more submissions to manage. The paper’s account of the workload change is not simply that AI review replaces human effort: the final decision remains with maintainers, and more PRs and comments can still require attention.

What the evidence cannot settle

The design compares adopters with maintainers who had not yet adopted the tool, but it is observational rather than a randomized experiment. The authors describe the rollout as a natural experiment; that label does not by itself prove that adoption caused every observed change. Differences in which repositories or maintainers choose to use the reviewer, or other changes in their workflows, may matter. The paper reports robustness checks, including alternative treatments of repository owners and changing levels of reviewer use, but these cannot remove every source of uncertainty.

The study also focuses on selected, active repositories, and its findings may not carry over to projects with different languages, sizes or review practices. Account labels do not reveal how much AI assistance human contributors used. Nor do PR size and merge timing capture every aspect of usefulness, correctness, security or long-term maintenance cost. A slower merge could reflect several things; it is not, on its own, proof of lower code quality.

Finally, this is an arXiv preprint, not evidence of peer-reviewed publication in the material available here. Its results are informative about one GitHub tool and setting, but should be read as provisional. The broader lesson is a question rather than a verdict: automating review can improve one part of a workflow without necessarily reducing the total work a community must do.

Reviewed cross-references

These connections were reviewed by a model, not independently fact-checked by a human. Similarity does not imply agreement; consult the original papers and the evidence below.

  • AI can prepare code migrations quickly. Getting teams to adopt them is harder.

    Supporting evidence: Both studies indicate that generating code changes does not remove the human work of evaluating them: the Copilot Reviewer study reports increased PR submissions and maintainer review workload after adoption, while Bloomberg’s enterprise migration campaigns identify owning teams’ review, testing, and deployment costs as a bottleneck. This is a bounded, compatible comparison—not a direct replication: the studies examine open-source review adoption and internal infrastructure migrations, respectively, and use different designs and outcomes.

    Model review: · openai-codex/gpt-6-luna

    Evidence and provenance

    Selected excerpts, not complete papers. PDF extraction can omit figures or damage equations, tables and reading order; sources may be preprints.

    • 2611.22793v1 — Original paper

      Excerpt reference: static/papers/2611.22793v1.md:13-60
      Basis: Paper text
      Source SHA-256: d3dfeb488c03d66d793a7c99a11ee2129c95d5cb47f667d1ae518dfe792a0452

    • 2611.05457v1 — Original paper

      Excerpt reference: static/papers/2611.05457v1.md:254-328
      Basis: Paper text
      Source SHA-256: 6658f060694636695879002e145c408a97ef4299219bffd93264a08e87eb9649

Automatically matched by shared topic keywords. These links do not imply that the sources agree or establish a consensus.