Peer review without the peer pressure

You add a review step to your coding agent. It takes the diff and runs the same review on a few different models, subagent(kind: "review", model: "gpt-5.6-sol"), subagent(kind: "review", model: "opus-5"), and subagent(kind: "review", model: "grok-4.5"). Being helpful, you give each one everything you have: the PR description, the ticket, the reason the code looks the way it does. They hand back findings, each stamped with a severity and the model that produced it:

[
  { "file": "auth.go", "line": 42, "severity": "critical", "model": "gpt-5.6-sol",
    "note": "token compared with ==, not constant-time" },
  { "file": "auth.go", "line": 42, "severity": "minor", "model": "opus-5",
    "note": "prefer subtle.ConstantTimeCompare here" },
  { "file": "auth.go", "line": 88, "severity": "major", "model": "grok-4.5",
    "note": "error from writeToken is ignored" }
]

The calling agent folds them straight into the review. It reconciles the two auth.go:42 entries, keeps the critical because critical outranks minor, notes that GPT-5.6 Sol was the one that caught it, and writes the whole thing up. That looks like a working pipeline, a careful one even: three models, graded severities, a tidy merged summary.

You just poisoned both ends of that pipeline, in opposite directions.

Two kinds of poison

The reviewers know too much. Tell them why the code is the way it is and they wave the real problems through; tell them to find something and, being goal-oriented, they will invent one. The calling agent knows too much too: it sees which model said what and how sure each claimed to be, and anchors on a verdict it never reached itself. Neither is doing the job you think it is.

Both failures ride in on the same vehicle: leading words. A leading word carries a verdict, not just information, and it sounds precise even when the thing under it is not: critical reads like a measurement, but it is one model's guess wearing a measurement's clothes. Drop one into a pile of evidence and it stops being evidence, it becomes a conclusion the next reader inherits instead of forming their own. The critical in that JSON is a leading word aimed up at the calling agent; "it's a safe refactor, we've done it before" in the PR description is one aimed down at the reviewers. Same trick, both directions.

The fix is the same shape as the problem: isolate context in both directions. Give the reviewers the diff and nothing about why it was written. Give the calling agent the findings with nothing about who produced them or how bad each model claimed they were. Put a layer in the middle to normalize and de-attribute, so no verdict or model name ever reaches the top. It costs more, so save it for the review that matters. The rest of this is how each half works.

The reviewers shouldn't hear your reasons

Tell a reviewer the intent and you have handed them a hypothesis to confirm. People do this: confirmation bias is exactly the tendency to weight evidence toward the belief you walked in with. Blinding fixes it, measurably. In a randomized peer-review trial, reviewers who could see a prestigious author recommended acceptance 87% of the time; reviewers who could not, 68%. Both groups found the same number of planted errors. The blinded ones just stopped excusing them.

Models do the same thing, and there is recent work on all of it. LLM reviewers inflate their ratings for weak work while tracking humans on the strong stuff. They anchor on whatever you show them first, an effect that lives in the shallow layers and does not wash out with better prompting. They get derailed by irrelevant context that a clean problem would never contain. Your carefully written PR description is, from the reviewer's seat, a distraction engineered to make the code look intentional.

So don't send it. A reviewer gets the diff and the contract it is supposed to satisfy. Not the story.

Who found it shouldn't reach the top

Now the other direction. Your reviewers came back with severity: critical and a model name attached. Both are anchors, and the calling agent will lean on them instead of the evidence.

Severity is a classification the reviewer made under all the biases above. Pass it up and the calling agent inherits a verdict it never checked. The model name is worse than it looks: LLM judges recognize and favor their own generations, and the strength of that self-preference tracks how well a model recognizes its own output. Tell a calling agent a finding came from a model in its own family and you have switched on exactly that bias. On top of which, LLM judges carry position, verbosity, and self-enhancement bias, and they are sycophantic: hand one a framing and it tends to agree rather than evaluate.

So strip it. The calling agent should see a finding as location, claim, and evidence. Not critical, not per gpt-5.6-sol. It holds the intent and the codebase; let it do the classifying, from clean evidence.

Put something in the middle

You cannot ask the reviewers to de-attribute their own findings. They do not know what the others said, and you do not want them to. The normalization has to happen somewhere between the reviewers and the calling agent, in its own context. Subagents of subagents.

An intermediate agent collects the raw findings, merges duplicates, drops the severities and the model names, and hands up a single normalized list. None of this is new. Run the same focused review enough times and you are fuzzing: throw many cheap, independent attempts at the code and let breadth catch what any one careful pass misses. Anthropic's Claude Mythos is this taken to its limit, launching the same "find a vulnerability" agent thousands of times across a repo, each run pointed at a different file, with a separate agent to check what comes back. The merge is the other half: Mixture-of-Agents is a layer that consumes many models' outputs and synthesizes one. The aggregation layer is where provenance goes to die, on purpose.

flowchart TB
    orch["Calling agent<br/>wrote the change, holds intent"]
    subgraph reviewers["Reviewers: clean context, blind to each other"]
        r1["Reviewer A"]
        r2["Reviewer B"]
        r3["Reviewer C"]
    end
    agg["Aggregator<br/>dedupe, normalize,<br/>strip severity + provenance"]
    orch -->|"diff + spec only"| reviewers
    reviewers -->|"raw findings"| agg
    agg -->|"normalized, de-attributed"| orch

The aggregator is also where you get to prune, using exactly the labels the calling agent never gets to see. If every model rated a finding minor, drop it. That is safe because it happens below the line: pruning on severity down here cannot anchor a caller that never receives the severity. And you do want to prune, because deduping into one clean set is not the same as making it short. Cut too hard and you bury real bugs; pass everything through and you drown the one agent that is supposed to reason about them.

And keeping it short is not just tidiness. Frontier models are relentlessly goal-oriented and, past a point, forgetful: give one too much to chew on and it will seize on the third item, chase it to the ground, and burn half its context window getting there, right as its grip on that window starts to slip. You have driven your best reasoner into the dumb zone over something minor.

But doesn't a panel of models cancel out bias?

By now the panel might look like the fix itself: run enough different models and the biases wash out. There is a real result behind that. A panel of diverse judges beats a single big judge and shows less bias, because models from different families do not fail in the same places. So more models should help, not hurt.

The catch is the word diverse. A panel only cancels bias if the models are really independent, and that breaks in two ways. If they were trained on similar data, they fail in the same places, so three of them agreeing tells you little. And if you pass the model names and severities up, self-preference and sycophancy bring the bias right back. So you need both: models different enough not to share blind spots, and de-attribution so the calling agent weighs the finding and not who sent it.

This post was researched the same way: a fan-out of subagents that never saw each other's notes, and one pass to put them together.