All Three Reviewers Were Right—'Divergent Verdicts' Are Manufactured by the Question
Consider a very ordinary table. During a remediation cycle for a data pipeline project, three review verdicts for the exact same validation item were placed side by side:
- Gemini 3.1 Pro: ⚠️—“No environment assertion code in the diff”
- qwen:
met - Claude:
met
It looked like a disagreement among three reviewers: one flagged an issue, while the other two passed it. Yet, this single table ended up teaching me more than any completely green one ever could.
The three case studies in this article come from two projects—a data pipeline and an Android TV app—but the collaboration topology was identical: planning and investigation were run by Planner, implementation was handed to Claude, and review was distributed to Qwen, Gemini 3.1 Pro, and Opus. My position was at both ends: defining the problem, and deciding whether to accept the result.
All three cases point to the exact same conclusion: a reviewer’s verdict answers the question you actually sent, not the question you thought you were asking. When verdicts contradict each other, the difference usually lies not in the model’s capability or honesty, but in the phrasing of the individual prompts—divergence is manufactured by the question. Resolving this kind of divergence relies neither on voting nor on authority, but on personally reading the mechanism at the layer where the dispute resides. The three cases each cover a different angle: one illustrates how divergence is created, one shows how two lenses can even disagree on “what counts as correct”, and one demonstrates that certain landmines only exist within a specific lens.
I will start with the shortest case below.
1. Two Lenses Viewing the Same Diff Disagree Even on “What Counts as Correct”
After a particular implementation round in the data pipeline project, Claude confessed to something: when upgrading the manifest header, it modified the first row of the existing file. The plan explicitly stated in rule D5: “append-only, existing rows must not be rewritten.” On the surface, this was a violation that should have resulted in a FAIL.
Two reviewers looked at the exact same diff.
Gemini 3.1 Pro looked at the code itself: this was a gray area, but the approach was sound—the upgrade was idempotent, encountering an unexpected header triggered sys.exit, and the write was atomic. Verdict: no FAIL.
Claude looked at it through the project context lens and reasoned one step further: if the header was not upgraded, the new rows would have 7 columns while the old header only had 6, and pandas.read_csv would immediately throw an error—the true-gap recalculation and all future analysis relying on the manifest would die the day the first new row appeared. Following this logic, upgrading the header was not a violation; rather, it preserved the downstream gates.
Claude did not stop at inference. It executed awk -F, 'NR>1{print NF}' on its own, empirically verifying that 5,728 rows had 6 columns and 1,496 rows had 7 columns—the data rows were untouched at the byte level, and the evidence for the 15 rows of 402 (rate limit error code) resided entirely on the 6-column side. It did not simply trust the implementation report.
For the same diff, one evaluated the approach while the other evaluated the consequence, fundamentally changing the very definition of “what counts as correct.” Ultimately, what settled the matter was that awk command—the consequence was deduced through measurement, not argued through the sheer force of a stance.
2. All Three Answered Correctly—Divergence is Manufactured by the Question
Returning to the table at the beginning. The validation item caught between those three verdicts was named Test 0. The master plan described it as “version mismatch must fail with an environment error message,” and its item title was “must run before all numerical comparisons.”
Upon receiving the three verdicts, Planner initially recorded it as “divergent judgment among the three, I judge Gemini 3.1 Pro to be correct”—the “I” Planner recorded was itself. This framing was wrong—I pointed it out on the spot. Thus, Planner went back to check the three actual prompts it had sent, verbatim:
- To Gemini 3.1 Pro: “Did the actual diff satisfy this?… Please only evaluate whether the ‘mechanism is correctly implemented’”
- To Claude: “Rule 6 always uses
./.venv/bin/python3”—Rule 6 is an ironclad rule in this project stating “all measurements must strictly use the project venv interpreter”; for the party being reviewed, this was a statement of practice, not a question at all. - To Qwen: Generic review prompt, verifying against the actual disk state.
Three prompts, three different questions. A sentence Planner later wrote down in its notes became the thesis of this entire article: “All three correctly answered the question I asked. The divergence was manufactured by me.” Both instances of “I” in that sentence refer to Planner—Planner issued the questions, and Planner manufactured the divergence. Gemini 3.1 Pro was asked “is the mechanism in the diff”, and it answered no—correct. Qwen was asked “does the validation item hold true against the disk state”, and it answered yes—correct. Claude received a statement, and it answered compliant—also correct. Three zero-point questions yielded three perfect-score answers.
The root cause lay one layer higher, in the plan itself: Test 0 described “what a failure looks like,” but never specified where this assertion should live, nor did it state that the assertion had to be persistent. The implementer read it as “I check the environment before the numerical comparison”—measured against the item title, this was a defensible reading.
Yet, the signal was right there the whole time. Planner’s rule is that every command referenced by a validation item must actually be executed. During that round, all other validation items were executed, with Test 0 being the sole exception because it had no commands to run. Looking back, the very fact that there was “no command to run” was evidence that it was not yet a mechanism—an assertion genuinely living in the code cannot possibly fail to produce a single executable check.
This risk was not theoretical. Planner empirically tested it: running the tool with the system interpreter (pandas 2.2.3, whereas the project requires 2.2.0)—it ran perfectly normally, zero warnings, exit 0—when the environment assertion is absent, nothing happens, which is exactly the failure mode “version mismatch must fail with an environment error message” intended to block. After the fix, the wrong interpreter exits with code 3, printing the actual and expected values.
3. Some Landmines Live Only Inside One Lens
The rewrite of the Android TV app took four rounds for a single cycle. The distribution of verdicts in the second round was: the initial in-dispatch review by Qwen judged it PASS; the two subsequent follow-up reviews assigned separate stances—Opus reading code, and another Claude carrying the project context—resulted in a double FAIL, totaling 13 items: three consecutive violations of the dialog focus wiring, DCA key bleed-through, AC-D9 failing to open the dialog, missing guards on isDolby, visual scope creep, and so forth.
Of the 13 items, both follow-up reviewers caught the 3 Critical ones; the complementary portions were cleanly divided—Opus caught pure logic issues (rating ?: 0, unreported deviations in the report), while Claude caught convention landmines (the runCatching on R4, hardcoded “Watch”, two-line OSD, reason=dca never triggering, and modifying the value of label_ok).
The sharpest item was the dialog focus wiring (missing .focusable()). This particular landmine had long been prophesied in the project’s memory, verbatim:
static review + in-dispatch + code-facing all failed to catch it, only the project-context lens catches it
This round proved it true yet again: the in-dispatch Qwen missed it across both rounds; Opus only caught it after the prompt explicitly stated those two rules; the Claude equipped with project context independently caught all three. The visibility of the exact same landmine differs under different lenses—some require someone to point them out before they can be seen, while others become apparent on their own.
Another incident occurred in the third round: the in-dispatch review was abruptly cut off during the Qwen stage. The remediation was that “absence does not equal PASS”—the subsequent follow-up reviewer was promoted to primary reviewer to perform a make-up review. It was this make-up review that caught the last two remaining defects (accidentally deleting EPG_DIALOG action=open and the underline indicator being suppressed by constraints). The convergence trajectory for the entire cycle was: 6 → 13 → 2 → 0.
Verdict Formats Pretend They Are Answering the Same Question
At their core, all three cases are the exact same issue, which breaks down into three steps.
Step one: verdict formats lie. PASS / FAIL / ⚠️ / met look as though they are answering the same question. When the correct answers to three different questions are placed side by side, their outward appearance is one of “divergent judgment.” Not a single one of the three verdicts in Case 2 was wrong; what was wrong was the framing that aligned them side by side.
Step two: the process scrutinized “who reviews” but overlooked “what was asked.” Multiple models, split stances, follow-up review escalation—the configuration of “who” has been constantly evolving. But unfolding the actual inputs received by the three gates in Case 2: Qwen received the generic prompt and the disk state; Gemini 3.1 Pro received the diff and a single sentence reading “only evaluate the mechanism”; Claude received a statement of practice. The three inputs correspond to three different questions, yet none of the fields in the returned verdict formats recorded the question. The act of placing them side by side welded three different questions into an imaginary, shared question—“divergence” is born in that exact gap. Across the entire pipeline, not a single gate required treating the “sent question itself” as a suspect.
Step three: the default action for resolution is to “believe one of them.” Yet the true materials for resolution—the phrasing of the prompt, the source code at that specific layer, a single measurement—can only be retrieved by looking back. No one mandates this backward glance; it only happens when someone calls a halt on the spot. That erroneous framework in Case 2 was dismantled only after a halt was called.
The Previous Article Already Stood Here
The conclusion of Checks That Cannot Fail Carry Zero Information When They Pass argued that “complementary value comes from stance, not from vendor”: when the model designated on the code-reading side failed on the spot, substituting another model from the same vendor resulted in both follow-up reviewers belonging to the same vendor, yet the complementary value held perfectly true. In that same article, during the instance where verdicts contradicted each other (Opus PASS, Claude FAIL), the final resolution was settled by Planner personally reading the source code to deduce a temporal sequence argument.
That article was about the action of “designating complementary stances.” This article climbs one rung higher: stance does not reside in the org chart; it resides in the question—“verifying the mechanism vs. verifying the disk,” “verifying the shape vs. verifying the direction,” “verifying the approach vs. verifying the consequence.” The phrasing of the question is the stance. “Complementarity” without altering the phrasing is merely asking the same question twice.
Five Actions
Distilling this article into an actionable checklist yields five rules, each backed by a case study:
- Reread the prompt you sent before reading the verdict. To address “divergence,” investigate the definition of the question first, then the answers—all three answers in Case 2 were correct individually; what was wrong was the framing that aligned them side by side.
- Bake the stance into the question. State explicitly what needs to be verified: mechanism or disk state, shape or direction, approach or consequence. Phrasing is stance; designating a stance while omitting the phrasing renders it ineffective.
- Do not vote when verdicts contradict. Personally read through the layer where the dispute resides and run a measurement—counting columns with
awk, constructing temporal sequence arguments, or reading paths line by line are all the exact same action. The material for resolution is empirical measurement, not sheer force. - Missing lenses can be compensated for after the fact. An absent review does not equal PASS; the make-up review in the third round of Case 3 was exactly what caught the last two remaining defects.
- Both lenses must be present. Certain landmines natively live only inside one lens (Case 3, proven twice)—the code-reading lens and the project-context lens are not duplicate layers of insurance; they are two distinct areas of coverage.
This diagnostic thread has also altered the process itself: subsequent follow-up review assignments shifted from not designating stances to assigning A and B to review separately, each holding a different stance—that change stems directly from the statement “the verdict is a function of the question.”
The next time you receive a seemingly divergent table, the first action is not to ask which model is right, but to look back at what question you actually sent. The answer is usually written right there in the prompt.