The Review That Found What Review Missed: An Optional Deep Review for Big Changes

One reviewer over a diff approves things that several hostile reviewers with refuters do not. Here is what a multi-agent review found on a 176-commit stack after ordinary reviews had passed it, what it costs, why it is optional, and the loop shape that keeps it from eating your budget.

Last week a stack of 21 pull requests, about 176 commits, went through the ordinary review every Coograph project runs: one read-only reviewer over the diff, every finding backed by a quote from a fresh file read. It passed. Then it went through a different kind of review, and that one found a lock released twice, a test that passed with its fix removed, and a queued job that could overwrite a customer’s files. Each of those had been approved.

This post is about that second review. It ships in Coograph 1.6.0 as /coograph-ultra-review, it is optional on purpose, and it comes with a rule about when to stop, because the first week of using it taught us what happens without one.

What one reviewer misses

An ordinary review agent reads the diff, reads the files, walks a checklist, and reports. It is good at what a careful human is good at: conventions, obvious logic errors, a null that is not handled. It is bad at the same things a careful human is bad at: the interaction between a change and a process that is not on the screen.

The three findings above are all of that kind. The lock released twice was a goroutine that released a claim explicitly and then again in a deferred call, which was fine until a second request took the claim in between. The test that passed without its fix was a test whose setup ran before the code under test re-read the row, so the thing it was supposed to pin never happened. The queued job was a job that checked it still owned a server once, before a twenty-minute wait, and never after; a customer who installed something else during the wait had it overwritten.

None of these show up in a diff. They show up when someone asks one narrow question - what does a redelivery do to this? - and then someone else tries to prove the answer wrong.

What the deep review does instead

/coograph-ultra-review is that, as a saved workflow in Claude Code.

A scout resolves the pull request or the commit list to its files and changed lines, and asks the code graph for the callers and importers of what changed. Then one reviewer per dimension reads the code at the reviewed commit with a single question in mind: state and concurrency; data-loss paths; contracts and compatibility; tests that pass regardless. Each returns findings with a verbatim quote, a file and line, and a concrete failure scenario - inputs and state on one side, the wrong outcome on the other. No quote, no finding.

Then the part that makes it different. Every Critical and Warning goes to refuters, independent agents whose only job is to prove the finding wrong from a fresh read, each with a distinct lens: does the quoted line exist and do what the claim says; does the scenario actually occur or does a guard stop it; does an existing test contradict the reading. A finding survives only if none of them can refute it. Refuted findings are still in the report, with the refuter’s reason, because refuters are wrong sometimes too.

One report, a verdict per commit, and a footer that says what it cost.

Cost is the honest number to lead with: a full run is 1.2 to 1.6 million tokens and about twenty minutes. A 33-commit pull request took 3.1 million and 39 minutes. That is why the agent is told to offer it - when a change is over fifteen files, when it is going to people for approval, when it touches queue jobs, locks, status transitions, migrations or a contract between two services - and never to run it on its own.

The loop that ate a day

Here is what we got wrong first, so you do not have to.

The obvious way to use a tool like this is: run it, fix what it finds, run it again on the fix, repeat until clean. We did that. Four full runs on one fix, about five million tokens, each round finding something real. Round one found the test that did not exercise its fix. Round two found seven problems, all on the edges of the code round one’s fix had added. Round three found a Critical in round two’s fix. It converged, but slowly, and most of what rounds two and three found was in code the original change had not touched.

That is the pattern to watch for: a review that reviews each fix at full breadth will keep finding edges of the previous fix, forever, and every one of them is real enough to fix. The tool is not wrong. The loop is.

So the shipped version has three modes and a stop rule.

Findings outside the changed lines come back as follow-ups, not blockers. The stop rule: fix what is in the diff, record the rest, at most two delta rounds, one full run over the whole PR, then a person. A third round of machine findings on the same lines is not a reason for a fourth; it is a sign the design needs a look, and plan mode is where that look is cheapest.

We tested the plan mode on the design we had already built, written down as it stood before round three. It found round three’s Critical from the plan text alone, before any code, plus two things the code review had missed. It also cost a full run. The value of a plan review is the timing, not the price.

Why it is optional

Because most changes do not need it, and a tool that costs a million tokens per use has to earn each one. The ordinary review stays the default. The agent mentions the deep one - one sentence, with the cost - when a change qualifies, and leaves the decision to you. The retro that measures whether reviews happened counts it as a review; nothing else changes.

If you run it, run it the way the stop rule says, and then hand the pull request to a person. That is still the last step, and after a week with this thing it is the step we trust more, not less: what it hands them is a diff whose lifecycle questions have already been asked and answered in writing.

Also in 1.6.0

Two fixes a user sent in, both worth knowing about if you run the code graph.

An incremental graph update was deleting every call edge an earlier run had resolved: the linker re-resolved node ids as if they were names, found no match, and dropped them. If your graph lost its calls edges after a commit, that was it. Pull, rebuild once with --build, and updates keep them from now on.

And uv run resolves against the machine’s default Python. On a machine whose default is older than 3.10 the MCP server died at start-up with your requirements are unsatisfiable, which the host reported as CONNECTION_CLOSED with the real cause in a log nobody reads. Every uv run Coograph writes now carries -p 3.12; uv downloads the interpreter on demand. The sync script also stopped overwriting a project’s .mcp.json wholesale; it merges the code-graph entry and leaves the rest of the file alone.

Try it

Coograph 1.6.0 is in the Claude plugin marketplace and in the initialiser. In a Coograph project, /coograph-ultra-review (or /coograph:coograph-ultra-review from the plugin) with a PR number or a commit list; the skill proposes the model plan and the cost before anything runs. The workflow file is plain JavaScript in .github/skills/coograph-ultra-review/, so the dimensions are yours to change. Arguments, modes and the stop rule are in the docs.