Build a Code Review Agent with GitHub Copilot

A review agent is the easiest genuinely useful agent to build, because its constraint is also its definition: a reviewer that edits is not a reviewer.

That single property makes everything else simpler. The tool list writes itself. The failure modes are bounded. And the worst thing a badly configured reviewer can do is waste your attention, which is a category of problem you can recover from.

Built-in review first

Two things already exist and are worth trying before you write anything.

Copilot code review on GitHub reviews a diff and comments on it. It is integrated where review happens, requires no configuration, and is a reasonable baseline.

The code-review built-in agent is read-only and analyses changes for substantive issues. If your need is “look at this diff”, that may be all you need.

Write your own when you want something these cannot give you: your team’s specific failures, a particular report format, or a narrower scope than “everything that might be wrong”.

The profile

---
name: reviewer
description: Reviews changed code for correctness, security and this team's
  specific recurring mistakes. Read-only — it reports and never edits. Use
  before opening a pull request, or on a diff you are reviewing. Not for
  writing code and not for fixing what it finds.
tools: ["read", "search"]
---

You review code. You never modify it.

Two tools. No edit, no shell. The agent cannot change a file, cannot run a command, and cannot be talked into either — not because it is instructed not to, but because the capability is absent.

The content that matters

Here is the version worth writing:

## What this team gets wrong

- **Database sessions outside a context manager.** We have leaked
  connections in production twice this way. Every session must come from
  `with get_session() as session:`.
- **Background tasks that swallow exceptions.** A bare `except:` or an
  `except Exception: pass` in a task means silent failure. Log and
  re-raise.
- **Endpoints that skip `require_tenant`.** Any route under `/api/`
  without the dependency is a cross-tenant data leak.
- **Naive datetimes.** Timestamps must be timezone-aware. We have had
  two incidents from comparing naive and aware values.
- **String-formatted SQL.** Parameterise. No exceptions, including for
  identifiers you believe are internal.

Every item names a specific pattern, in your vocabulary, with the consequence attached. Two of them cite real incidents.

Compare with the version people usually write first: “check for security issues, performance problems and code quality”. That produces a review that could have been written about any codebase, because it was.

Telling it how to report

Output format is worth specifying, because the default is a wall of prose that buries the important finding in the middle.

## How to report

For each finding: the file and line, what is wrong, and what to do
instead. Order by severity, most serious first.

Say plainly when a finding is a question rather than a defect.

If you find nothing worth reporting, say exactly that. Do not pad the
review with observations to appear thorough — a review full of trivia
trains people to skim.

Three instructions, each preventing a specific failure.

File and line, and what to do instead. A finding without a location is a finding nobody acts on.

Severity order. Attention decays down the page. The serious finding must be at the top or it may as well not be there.

The empty case. This is the one people leave out and the one that matters most. Without it, an agent that finds nothing produces marginal observations, because producing output feels like succeeding. A reviewer that cries wolf gets ignored, and then the one real finding is ignored too.

Pairing it with a review skill

The profile carries the role. A separate skill can carry the review procedure — and separating them is worth doing for the same reason it was worth doing for the infrastructure agent.

A review-checklist skill holds the order of attention:

## Order of attention

1. **Does it do the right thing?** A correct implementation of the wrong
   requirement is the most expensive review miss.
2. **What is not in the description?** Unrequested changes are where
   surprises live.
3. **Security-relevant paths.** Authentication, authorisation, tenant
   scoping, SQL, deserialisation, file paths.
4. **Error paths.** The happy path is usually tested. The `except`
   branch usually is not.
5. **Tests.** Do they assert what they claim? Was an existing assertion
   weakened?

That content is useful to more than the reviewer agent. It helps in plain agent mode. It helps a person preparing a change for review. It helps someone who has just joined and does not yet know what this team looks at first.

Content in the agent profile is available to that agent. Content in a skill is available to whoever needs it — which is why the procedure goes in the skill and the role stays in the profile.

Reviewing the review

Agent findings need a filter, and applying one consistently is a small skill of its own.

Verify before acting. A finding is a claim about the code. Read the line. A plausible-sounding finding about code that does not do what the agent thinks is common enough to expect.

Distinguish defects from questions. “This might be a problem if X” is worth five seconds of thought and usually ends there. “This does the wrong thing when Y” is worth fixing.

Do not fix findings you do not understand. Applying a change because an agent suggested it, without understanding why, is how a working system acquires a subtle bug with a confident commit message.

Notice the pattern of what it misses. If it consistently fails to catch a category, that category is missing from the profile — and adding it is the highest value edit you can make.

Test it with planted defects

This is the step that separates a working agent from a plausible file.

Write a fixture containing one instance of each mistake the agent exists to catch:

def leaked_session(engine):
    # Defect 1: session is never closed — not a context manager.
    session = engine.session()
    rows = session.query("SELECT 1").all()
    return rows


def silent_task(work):
    # Defect 2: swallows the exception, so the failure is invisible.
    try:
        work()
    except Exception:
        pass

Then review it and count.

Five planted defects, five expected findings. Fewer means the agent is misconfigured or the wording is not landing. More means it is over-reporting, which is its own problem.

Verify the boundary

Two checks, both quick.

Ask it to fix something. “Fix the session leak in that file.” It should decline or fail. If it produces an edit, the tools list is not what you think it is.

Ask it to run something. “Run the tests and tell me if they pass.” Same expectation, same reasoning. A reviewer with shell access is a different agent wearing a reviewer’s description, and the description is the part that will not protect you.

Why reviewers over-report

Worth understanding, because the fix is counterintuitive.

An agent asked to review has an implicit goal: produce a review. A response saying “I found nothing” reads, to a system optimising toward a helpful answer, like failing at the task. So it finds something — and since the genuinely serious problems are absent, what it finds is whatever is left: a variable name, a suggestion to extract a function, an observation that a comment could be clearer.

The result is a review that is technically responsive and practically useless, and it does specific damage. Authors learn that the agent’s output is noise. They skim it. Then the one time it catches a real cross-tenant leak, that finding is in the middle of six trivia items and gets skimmed with the rest.

Three things reduce it.

Say the empty case is acceptable, explicitly. This is the single most effective instruction in the whole profile.

Say what is not worth reporting. Naming the categories to skip — formatting, preferences, refactors of untouched code — gives the agent somewhere to put the impulse.

Keep the check list specific. A vague instruction to look for quality problems produces vague findings. Five named patterns produce findings about those five patterns.

Adding external context, without adding power

A reviewer benefits from knowing why code is the way it is, which is where MCP comes in — and where the read-only discipline gets its real test.

The GitHub MCP server, scoped read-only, lets the reviewer find the pull request that introduced a pattern, read the discussion, and check whether an apparent oddity was a deliberate decision. That is genuinely better review, and it adds no write capability at all.

What it catches, and what it does not

Being honest about this keeps the agent useful rather than trusted.

It catches pattern violations well. The five items in the profile are pattern matching against known shapes, which is what this is genuinely good at and what human reviewers are worst at sustaining across a long day of pull requests.

It catches missing error handling and untested branches. These are structural and visible in the diff.

It is decent at spotting unrequested changes. “This pull request says it adds pagination and also changes the default page size elsewhere” is a useful class of finding.

It does not know your architecture. A change that is locally correct and architecturally wrong looks entirely fine at the line level, which is exactly why it survives the mechanical pass and needs a person.

It does not know what is being deployed alongside it. Interaction bugs between two in-flight changes are outside what a diff contains.

It cannot judge whether the requirement was right. That is the most expensive category of review miss and it stays human.

Where it fits in a team’s process

The agent is most valuable at a specific moment, and putting it elsewhere wastes it.

Before the pull request exists. The author runs it on their own branch. The mechanical findings get fixed privately, and the pull request that reaches a human is one where the tenant check is already there. This is the placement that actually saves time, because it removes work from the reviewer rather than adding a document for them to read.

Not as a merge gate. An agent whose approval is required is an agent whose false positives become blocking. It also creates pressure to reduce its sensitivity, which is the wrong direction — you want it noticing things, and you want a human deciding what matters.

Not as a replacement for review. The findings it produces are the ones a careful reviewer would produce quickly. The findings it cannot produce — is this the right approach, does this fit the architecture, is the requirement itself sensible — are the ones review exists for.

Cost and cadence

A review pass over a moderate diff is a multi-step agent session, which is meaningfully more than a chat message.

That is fine for a real pull request and wasteful on every save. Two habits keep it proportionate.

Run it once, when the change is finished. Not iteratively while you are still writing. A review of half-finished code produces findings about things you were about to do anyway.

Scope it to the diff. “Review the changes on this branch” costs a fraction of “review these six files”, and it produces better findings — nothing about untouched code, which is where the noise comes from.

Cluster 6’s request model lesson covers how agent sessions consume premium requests, which is worth reading before rolling this out across a team.

Keeping it honest over time

Add findings from real review comments. When you catch yourself writing the same review comment for the second time, that comment has earned a place in the profile — and writing it there is faster than writing it a third time.

Remove items your tooling now catches. If a linter now enforces something, the agent restating it adds nothing except length, and length is what makes reviews get skimmed. Pruning is as important as adding, and much less likely to happen on its own.

Watch for over-reporting. An agent producing eight findings on a three-line change has stopped being useful and started being something people close without reading. Usually the fix is a tighter instruction about what is not worth reporting.

Re-run the fixture after every edit to the profile. Every single time, without exception — it takes seconds and it is the only evidence you have.

The fixture, in more detail

The planted-defect file deserves a little care, because a careless one gives false confidence.

One defect per function, labelled. A comment naming which mistake it is means you can score the review objectively rather than judging whether the output feels right.

Ordinary mistakes, not exploit code. The point is to test a reviewer, not to keep a payload in the repository. Every defect in the example fixture is something a competent person writes on a tired afternoon.

A warning at the top of the file. State plainly that the file is deliberate and must not be copied. Files like this get found later by people who lack the context.

Excluded from real checks. Your linters and scanners will flag it, correctly, forever. Exclude the path explicitly rather than weakening a rule to accommodate it.

Kept in step with the profile. When you add a check to the agent, add a defect to the fixture. An unpaired check is one you never confirmed works.

Common questions

Should it review its own team’s pull requests automatically? That is what built-in Copilot code review is for. A custom agent is better used deliberately, by the author, before the pull request exists.

Can it review a diff rather than files? Ask it to review the changes on the current branch. Scoping to the diff is usually what you want — reviewing whole files produces findings about code the change did not touch.

Can it review someone else’s pull request for me? It can read a diff and report on it, which is genuinely useful preparation. Posting the review is a separate decision — and one to make yourself, since your name goes on it.

Should different languages get different agents? Only if the recurring mistakes genuinely differ. One agent with a short section per language is usually better than several agents nobody can choose between.

Can it enforce our style guide? It can, and it mostly should not. Style is a formatter’s job, and a review that opens with formatting comments trains people to skim past the security finding underneath.

What about a security-specific reviewer? Reasonable, and the same rules apply: read-only, specific to the vulnerability classes your codebase has actually had, and tested against planted examples. Keep it separate from the general reviewer so that its findings are not diluted by ordinary ones.

Scoping the review

What the agent looks at matters as much as what it looks for, and the default is usually wrong.

Review the diff, not the files. A request to review changed files produces findings about code the change never touched — technically valid, entirely unhelpful, and the fastest way to make a review unreadable.

Include enough surrounding context to judge. A diff alone can be misleading: a removed check may be enforced elsewhere now. The agent has read access, so it can look, and asking it to confirm rather than assume is worth the extra step.

Exclude generated files explicitly. Lockfiles, compiled output, snapshots. These produce enormous diffs and no findings worth having.

Split large changes. A review of a six-hundred-line diff is a review that covers the first hundred lines carefully and the rest at a skim — the same failure a human reviewer has, arriving faster.

Growing the profile from real reviews

The profile that works is not designed; it accumulates. The mechanism is worth being deliberate about.

Harvest from review comments, not from articles. The items that belong are the ones you have written to a colleague. A checklist assembled from general security advice describes a codebase that is not yours.

Add the incident, not the rule. “Database sessions outside a context manager” is a rule. “We have leaked connections in production twice this way” is the rule plus the reason, and the reason is what lets the agent recognise a variant you did not describe.

Phrase it the way your team phrases it. If everyone says “the tenant check”, write “the tenant check” rather than “the authorisation middleware”. Matching the team’s vocabulary makes the findings readable and makes the profile something people maintain rather than something they tolerate.

Delete as readily as you add. An item your linter now enforces, a mistake nobody has made in a year, a check that produces only false positives — each is length that dilutes what remains.

Cap it deliberately. Five to eight items is a profile people read. Twenty is a document nobody has audited, containing at least three things that are no longer true.

Next

The documentation agent is the lowest-risk agent worth building, and the one where the interesting constraint is preventing invention rather than preventing action. Agentic workflows closes the cluster by turning this kind of check into automation.