How does an AI code review by a council of agents work?

What is an AI code review?

An AI code review is an assessment of code by a language model, looking for bugs, security risks and departures from the conventions of a project. It can cover a single change or an entire codebase.

Plenty of AI code review tools do this in pull requests, from Copilot code review to standalone review bots. They catch a lot, and their makers are open about the limits. According to GitHub's documentation⁠↗, a Copilot review does not count toward required approvals by default, Copilot is not guaranteed to spot every problem, and GitHub advises supplementing its feedback with a human review.

One model reading a diff gives you one view of the code. It misses things, and it is not good at weighing which of its own comments matter. A council therefore puts several reviewers on the same code and has their findings checked.

What is a council of agents?

A council is a group of AI agents that assesses the same work from different positions, after which the findings are weighed. The idea has become known as an LLM council.

Andrej Karpathy published llm-council⁠↗, a small web app that shows the pattern. Several models answer the same question. Each model then reviews the others' answers without knowing whose they are, and ranks them. A chairman model combines everything into a single answer. Karpathy himself calls it a weekend hack he will not maintain, but the pattern holds up.

For code review, a council works a little differently. The reviewers do not all answer the same question; each looks at the same code from its own specialism: security, architecture, the connections to external services. Each one acts as a code review agent with a narrow brief. At Studio Northstar, that council runs inside ARCS, our agentic platform. A specialist council reviews your code for security risks, investigates legacy software or checks a delivery ahead of an audit.

Why several reviewers work better than one, and when they don't

Several reviewers with different focuses catch more kinds of problems, and a reviewer that did not write the code does not share the author's blind spots. That only holds if the reviewers genuinely look in different ways and someone weighs what they find.

In Building effective agents⁠↗, Anthropic uses exactly this as an example of parallel work: several prompts review the same code for vulnerabilities and flag it when they find a problem. Research on models as judges shows why the reviewer should not be the maker. In Judging LLM-as-a-Judge⁠↗ (2023), Zheng et al. describe, among other things, biases toward an answer's position, toward longer answers and toward a model's own work.

A council does not always help:

• On small changes. A typo fix or a renamed variable does not need five reviewers. Anthropic advises starting with the simplest solution and adding complexity only when it clearly helps.
• When everyone does the same thing. Running the same instruction five times mostly gives you the same findings five times.
• Without weighing. More reviewers produce more comments. Without a check, the real problems get lost in a long list.

How does a council review work, step by step?

In five steps: scope, split, review independently, check and decide.

1. Set the goal and the scope. What the review must deliver, which code is in scope and which risks weigh most.
2. Split the work. Each reviewer gets its own specialism, such as security, architecture or the connections to external services.
3. Review independently. The reviewers work separately and only read; they change nothing. For each finding they record where it is and what it is based on.
4. Check. An independent reviewer, a different model or a separate context that did not produce the findings, checks them. A fix is followed by a new check: finding, targeted change, recheck.
5. Decide. A person decides what happens with the findings.

Example: reviewing the Plens app

Plens is our app that tells you the best moment to leave if you want to cycle dry. On 24 September, Claude Code ran an independent review of the app with separate read-only agents: one for external connections, one for the web repository and one for code and UX. The first blocking finding was that the entire app lived in a single 317-line file, with no deep links. A link from a notification had nowhere to go.

The review document opens with a fact board: for each environment, what was read, what state it was in and which access or evidence was missing. It also states what the document does not prove, namely that anything works on a real device.

A separate verifier with a fresh context then checked the pull request with the first fixes, before it was merged. It found ten issues, one of them blocking. The code called a function that does not exist in the version of AbortSignal that React Native ships with. On a device, almost every read request failed, while the web and Node tests noticed nothing.

What does a council find, and what does it miss?

A council finds what is visible in the code and its environment: logic that is wrong, security risks, missing error handling, departures from the architecture. What lies outside that context it cannot see without help: behaviour on a real device, or the reason behind a decision nobody wrote down.

A comment is not yet a bug, either. A reviewer can see a problem that is not there, or mistake a deliberate choice for an error. That is why a second party checks the findings before they reach the report, and why every finding comes with the check it is based on. Whoever reads the report can verify a finding for themselves.

For security, reviewers often start from the OWASP Top 10⁠↗, a list of common risks in web applications. The most recent edition is from 2025. A list like that is a framework, not a guarantee that everything has been found.

Reviewing legacy software and preparing for an audit

A council can also examine an existing codebase, not only a new change. We do this for legacy software and for deliveries that have to survive an audit.

With legacy software, you want to know where the risks are before you change anything. The reviewers can read the whole codebase without touching it, and report where each problem is and how serious it is. Code written quickly with AI, for instance through vibe coding, also benefits from a review like this before it goes to production.

In an audit, what counts is that you can show what was checked. That is why you get the findings together with the checks they are based on. A council review is not an audit and does not replace one. It is preparation, so you know what an auditor will find before they arrive.

What do you get back?

You get the findings, each with the check it is based on. We shape the report to the engagement. The Plens review shows what it can contain:

• the severity of each finding: blocking, high, medium or low;
• the screen or file concerned;
• the consequence for the user;
• the proposed fix and its status;
• a fact board showing what was read and which access or evidence was missing.

The fact board also shows you what was not checked. A person then decides what gets fixed and in which order. How that fits into a project is described under build and in our code review service.

Want a council to review your code? Get in touch, and we will start with what the review needs to deliver.