How to Automatically Review Pull Requests with GitHub Copilot

How to Automatically Review Pull Requests with GitHub CopilotSecurity, Code Review & Enterprise3Intermediate/github-copilot/security-enterprise/automatic-code-review/

The previous lesson requested reviews by hand. This one makes them happen without anyone remembering — which is where automatic review earns its value and also where it generates the noise that gets it turned off again.

The single structural fact to absorb first: at repository and organization level this is a branch ruleset rule, not a Copilot settings toggle. It is targeted by branch and repository patterns like any other ruleset, which is more powerful and less obvious than a checkbox.

Key takeaways

• Three levels: individual for your own pull requests, repository and organization through rulesets.
• Rulesets target branches and repositories, so you can scope precisely.
• Two options change the volume most: reviewing new pushes and reviewing drafts.
• Effort has an organization default that a repository can override.
• Pilot before enabling everywhere. Reviewing every push is the setting that produces comment fatigue.

Individual: your own pull requests

The smallest useful version, and a good way to form an opinion before proposing anything to a team.

From then on your own pull requests get reviewed without asking. Nobody else is affected, which makes this the honest way to find out whether the comments are useful on your codebase before you make it everyone’s experience.

Repository: a branch ruleset

Being a ruleset is the interesting part. Everything you already know about targeting applies — include the default branch, or a pattern, or a set of branches, and leave everything else alone. A repository where only main and release/* get reviewed is a normal configuration rather than a workaround.

Organization: many repositories at once

The two options that decide the volume

The trade is straightforward and worth stating plainly:

1. 1Review once, on openLowest volume. Misses problems introduced by later pushes.
2. 2Review new pushesFull coverage of the branch's life.
3. 3More comments per pull requestIncluding repeats of findings you already decided about.
4. 4SkimmingHuman judgementWhich is where the value goes.

Steps marked Human judgement are the ones that do not get delegated — they are where you decide whether what Copilot produced is actually right.

Reviewing once on open is the right starting position for almost every team. It covers the moment that matters — the pull request being proposed — and it caps the volume at one review per pull request, which is a number people can absorb.

Drafts are the gentler of the two. Reviewing drafts moves mechanical findings earlier, which is where you want them — the author is still working, nobody has been asked to review yet, and fixing something at that point costs nothing socially.

Effort defaults

Lite. Standard, and the default. Roughly $0.05 to $1 USD per review, per GitHub's published estimate.

Balanced. Higher-reasoning model, aimed at complex logic. Roughly $0.25 to $5 USD per review. Uses more AI credits and may consume marginally more GitHub Actions minutes.

GitHub notes that Balanced reviews use more AI credits and may consume marginally more GitHub Actions minutes. Set the organization default to Lite and override upward for the specific repositories where intricate logic justifies it — the payments service, the authorisation library — rather than the other way round.

Why a ruleset rather than a toggle

Worth a paragraph, because the choice explains several things that otherwise look arbitrary.

A toggle answers one question: on or off. Rulesets answer four — which repositories, which branches, under what enforcement, and alongside which other rules. For a control that costs money per invocation and produces output people have to read, those are the questions that matter.

Three consequences follow, and all three are useful.

Scoping is free. Reviewing pull requests into main and release/* while ignoring everything else is one target expression, not a workaround. Most teams want exactly this and would not have thought to ask for it from a checkbox.

It lives with your other branch rules. Required reviewers, status checks and automatic review are configured in the same place, which means the person changing one sees the others. A Copilot setting in a Copilot menu is a setting nobody reviewing branch protection would think to look at.

It inherits ruleset behaviour. Evaluate mode, layering between organization and repository rulesets, bypass lists — none of it is special-cased, so what you already know applies.

Layering organization and repository rulesets

Once both exist, the interaction is worth being deliberate about.

The common pattern that works: an organization ruleset enabling automatic review on default branches across a broad repository pattern, with new pushes off and drafts off. That establishes a floor — every repository gets a review on the branch that matters.

Then repository rulesets for the exceptions. A team that wants draft reviews adds them for their repository. A service with intricate logic overrides effort to Balanced. Neither requires touching the organization configuration.

What to avoid is the opposite arrangement — an aggressive organization ruleset that individual repositories then work around. Rulesets layer additively rather than overriding each other, so “turn it down here” is not generally available as a move. Set the floor low and let teams opt upward.

A pilot that tells you something

Enabling this across an organisation on day one is how teams end up with an opinion about automatic review before they have any evidence.

Week one — one repository, default branch, both options off. Reviews happen once per pull request on the branch that matters. Instructions file in place.

Week two — ask. Not “do you like it” but “has any comment changed a change”. That question has a factual answer.

Week three — adjust the instructions, not the scope. Almost every complaint at this stage is about comment quality, and comment quality is the instructions file. Widening scope before fixing quality multiplies the problem.

Week four — decide. Expand to more repositories, or enable drafts, or turn on new pushes for one team that wants it. One change at a time, so you know what caused the difference.

Budgeting

Automatic review means the volume is no longer bounded by anyone deciding to ask, so it is worth a moment of arithmetic before switching it on organisation-wide.

The published Lite range is roughly $0.05 to $1 per review. Multiply by pull requests per month, then by the number of reviews each one triggers — which is one if new pushes is off, and however many pushes the branch receives if it is on. A team merging two hundred pull requests a month with new pushes enabled is not doing two hundred reviews; on a codebase where branches receive four or five pushes it is closer to a thousand.

Three practical implications.

New pushes is a cost decision as well as a noise decision. The two arguments point the same way, which makes it an easy call.

Balanced everywhere is expensive for little gain. The higher-reasoning model earns its cost on intricate logic. On a typical pull request it produces more words, not more insight.

Large pull requests cost more and review worse. Both problems have the same fix, which is a pleasant coincidence.

What a good rollout looks like six months in

The steady state worth aiming for, so the pilot has a target.

Automatic review runs on default branches across most repositories, at Lite, with new pushes off. Two or three repositories with genuinely intricate code run Balanced. One or two teams that asked for it have drafts enabled. Every repository where review runs has an instructions file that somebody on that team wrote and has pruned at least once.

Comment volume is low enough that people read them. When a review comment appears, the default assumption is that it is worth a look rather than worth dismissing. Human reviewers still read the diff, and say so.

Nobody has needed to think about the configuration for months, because it was scoped narrowly enough to be stable and nothing about it is load-bearing enough to break.

That is an unglamorous end state, and it is the one that survives. Configurations that require ongoing attention get abandoned; configurations that quietly work do not.

Troubleshooting

No review appears. Check three things in order: that the ruleset enforcement status is Active rather than Evaluate, that the pull request targets a branch the ruleset actually covers, and that the repository matches the target pattern. Targeting accounts for most of these, and the ruleset page will show you which pull requests it matched.

Reviews on the wrong repositories. The target pattern in the organization ruleset is broader than intended. * patterns are easy to write and easy to under-estimate.

Comments repeat across pushes. New pushes is on, and the review has no memory of what the author already considered and rejected. That is the setting working as designed; whether you want it is a separate question, and for most teams the answer is no.

Drafts are reviewed and you did not expect it. The draft option is on in one of the rulesets that matches — check the organization ruleset as well as the repository one, since either can enable it.

Nothing on some files. Dependency management files, log files and SVG files are excluded from review outright, and content exclusion removes files as well.

Reviews cost more than expected. Check whether a repository override set Balanced, whether new pushes is multiplying the review count, and how large the pull requests are — cost scales with the diff, so all three compound.

Fork and outside-contributor pull requests

An open-source or widely-contributed repository has a wrinkle worth thinking about before enabling this broadly.

Automatic review on a public repository means Copilot reads pull requests written by anyone. For the review itself this is fine — the output is comments, and a comment is not an action. But two things follow.

The review reads contributor-supplied text. Instruction files are repository content, and a pull request that adds or edits one is proposing a change to how your review behaves. That diff will look like documentation. Treat it as configuration.

The comments are public. A review that fires on every outside contribution is a visible part of how your project responds to contributors, and a stream of generic mechanical comments on someone’s first pull request is not the welcome most maintainers intend.

What to tell the team when you enable it

The message matters more than the configuration, and it is two sentences.

What it is. A first pass that catches mechanical problems before a colleague reads the pull request. Not a judgement, not a gate, not a replacement for the human review that still has to happen.

What to do with it. Read it, act on what is right, dismiss what is not, and tell whoever owns the instructions file when it is consistently wrong about something. That last part is what turns a fixed configuration into one that gets better.

Worth adding explicitly if you have any doubt: comments are not visible as a quality score, nothing is being tracked about who received how many, and the review does not feed anything. People assume otherwise unless told, and the assumption changes behaviour in unhelpful ways — most commonly, developers resolving comments without reading them so the pull request looks clean.

Common questions

Does this replace required reviewers? No. Branch protection and required human approval are unchanged. This adds a reviewer; it does not satisfy a requirement, and a pull request cannot merge on a Copilot review alone.

Can I exclude particular paths from review? Not through the ruleset itself. Path-specific instructions shape what matters in those paths, and content exclusion removes files entirely.

Individual and ruleset together? The individual setting covers your own pull requests; a ruleset covers everything it targets. There is no conflict, though a review will not be requested twice.

Who gets billed? Reviews consume the AI credits of the account the Copilot usage belongs to, so an organization ruleset spends organization credits. That is worth knowing before enabling it across a large repository set.

Can a team opt out? Not from an organization ruleset targeting their repository — rulesets layer additively. If opt-out matters, express it in the target pattern rather than expecting teams to override locally.

Is Evaluate mode useful here? For seeing which pull requests a ruleset would match before enforcing, yes — the same as any ruleset.

Draft reviews, in practice

The gentler of the two options, and the one more teams should try first.

Reviewing drafts moves mechanical findings to the point where the author is still working. Nothing has been proposed, nobody has been asked to look, and fixing something costs nothing socially — which is a meaningfully different experience from having the same comment appear on an open pull request with two reviewers assigned.

Two caveats worth knowing before enabling it.

Drafts churn. A draft that exists for three days accumulates reviews if new pushes is also on. The two options interact, and enabling both on a team that works in long-lived drafts is how comment volume becomes unmanageable.

Not everyone uses drafts the same way. Some people open a draft immediately as a place to think; others open one when the work is nearly done. The first group gets reviewed on incomplete code, which is noise. Ask before enabling it.

Next

Before enabling this across every repository, note that each review deducts 13 from the monthly premium request allowance — Copilot Code Review vs Human Review works through the two scoping policies that keep that bill predictable.

Security best practices turns to the developer’s own habits, which is where most of the actual defence lives — automated review catches patterns, and the categories that matter most are the ones no diff review can see.