Structured Reflection vs. a Plain Nano Call: Polyreflexion on GPQA Diamond

We tested whether wrapping a small model in a structured reflection system beats asking it directly. The model is gpt-5.4-nano in every condition, the exam is GPQA Diamond with 198 graduate-level multiple-choice questions, and the challenger is polyreflexion, our reflection system. Once both sides see the same material at the final step, every polyreflexion condition beats a plain direct call, with the best configuration reaching 0.460 against a direct baseline of 0.328. An earlier version of this comparison said the opposite because it was quietly unfair, and correcting that is part of the finding.

The question this write-up answers is a narrow, practical one: on a hard multiple-choice exam, does wrapping a small language model in a structured reflection system beat simply asking the model directly? The small model is gpt-5.4-nano. The baseline is a single, plain call to it: the question, its four lettered choices, one answer. The challenger is polyreflexion, running on the very same model. Everything below compares those two.

polyreflexion takes a piece of text — a question, a claim, or a problem — and does not answer it in one breath. It splits the text across three points of view, lets each one speak, splits again, and folds the whole tree back into a single summary. A second layer, the meta layer, then looks at what came out and decides what the next pass should be: go deeper, refine, start over, or stop. The meta layer never writes reasoning of its own. It only chooses. The engine produces; the meta layer selects. That division of labor is the core of the design.

The idea reaches back to Gotthard Günther and what he called a polycontextural logic: the thought that there is no single, flat space of truth, and that a statement carries not only a value from true to false, but also a place — a context in which that value means anything at all. In Günther's spirit, a claim can be read through more than one logic at once: an objective logic that asks is this warranted by the facts?, a conceptual-dialectical logic that asks is this relevant, does it open a new distinction?, and a subjective logic that asks is this coherent within its own frame? A statement then receives not one verdict but a small tuple of them, and the interesting cases are the ones where the logics disagree — where something is narrowly false under one reading yet conceptually necessary to move forward. In that tradition, a contradiction is not a crash. It is information: the system reporting that its current frame is too small.

We have since formalized the full truth tables of this two-layer routing architecture and tested it across five major benchmark suites in Note #08. Here we focus on the single controlled experiment that first proved how sensitive such an architecture is to a clean evaluation setup.

Concretely, the three points of view are objectivity, subjectivity, and conceptuality, and the recursion lays them out as a Sierpinski triangle: each node splitting into three smaller ones, independent branches running in parallel, and the leaves re-integrated upward into one synthesis.

The meta layer is where the polycontextural idea becomes a concrete control loop. After a pass, three judges score the result in parallel, each from its own standpoint: a subjective reading, an objective reading, and a dialectical reading. A deterministic interpreter maps those three verdicts onto four moves in the search:

• three positives → converged, stop;
• two → expand, step wider to test what was left out;
• one → refine, narrow into the single promising perspective while keeping the other two critiques as boundaries;
• zero → reset once, at greater depth.

What the philosophical theory treats as kinds of truth, the code treats as steering signals for a reflection loop. When a verdict is overruled, the losing alternative is not thrown away. It is kept as what we call the semantic surplus — the road not taken, carried forward in case the next cycle needs it.

(One deliberate detail in the code: on the very first cycle, a unanimous positive verdict is read as "expand once" instead of "stop," so the loop performs at least one reflective step before it is allowed to terminate. From the second cycle on, full agreement stops it normally.)

The task is GPQA Diamond, 198 graduate-level multiple-choice questions in biology, physics, and chemistry — the kind of exam where surface bluffing does not work. The answering model is gpt-5.4-nano in every condition. The baseline, labelled direct, is the plain single call. The polyreflexion conditions are the fixed reflection depths (1, 2, 3) and the four meta-steered variants (meta_c1–c4), all running on the same model. A separate judge model scores correctness, and the whole grid was run under two different judges (gpt-4o and gpt-5-mini) to verify that the result does not hinge on the grader.

The corrected comparison is clear: the direct nano baseline answers 0.328 of the 198 questions, and every polyreflexion condition beats it — the fixed reflection depths from 0.394 to 0.424, and the meta-steered variants from 0.414 to 0.460. The strongest configuration, meta_c2, reaches 0.460, and it is the top scorer under both judges. Structured reflection, on this exam and this small model, is worth a clear margin over a single direct pass, and the meta-steered loop adds further gain on top of plain reflection.

That word corrected needs explaining, and the explanation is itself part of the result. An earlier version of this comparison said the opposite — direct beating every reflection and meta condition by a wide margin — and it was wrong because the comparison was quietly unfair. In the final step, the moment the pipeline actually picks a letter, the reflection and meta conditions were handed only the prose reflection summary, and never the question's actual lettered answer choices. The direct condition always saw the choices. When we pulled out three cached summaries and read them by hand, all three reasoned correctly in the abstract, and one explicitly ruled out A, B, and C while never mentioning the text of D, which was the right answer. The reflection engine had done the thinking, and was then asked to choose a letter with the letters hidden. It was blind-guessing the last step.

The fix is one small change: re-attach the lettered choices to that final answer call, leaving the direct condition's path untouched. Because the reflection summaries are cached before that step, the rerun reuses them without regenerating a single reasoning token and only redoes the final selection. Before the fix, the reflection and meta conditions sat between 0.16 and 0.23 while direct sat at 0.354; after it, they clear direct across the board. What had looked like "reflection hurts" was simply "reflection was being made to answer with its eyes closed."

The result also barely depends on the grading judge. Swapping the judge model from gpt-4o to gpt-5-mini shifts no condition by more than three questions:

The entire rerun — both full eight-condition grids under two judges — cost about $0.41 on a local machine calling the standard API.

In a follow-up sweep, we tested what happens when we turn up gpt-5.4-nano's built-in hidden reasoning parameter (reasoning_effort from none through low, medium, and xhigh). At none, external reflection leads direct answering by nearly ten to thirteen points; once hidden internal reasoning is turned on, both direct and reflected accuracy climb sharply and the gap between them on GPQA narrows to within noise.

On a self-contained single-turn science question where all the facts sit in the prompt, internal chain-of-thought and external reflection can substitute for one another. Where they part ways dramatically is on multi-step state tracking and multi-agent perspective tracking — as we show in Note #08, where even a flagship reasoning model spending almost its entire budget on hidden reasoning tokens stalls near ten percent, while a small model wrapped in polycontextural reflection climbs above forty percent on belief tracking and ninety percent on state tracking.

• Code and evaluation scripts (gaialogicag/polyreflexion)
• Theoretical background in German (Dr. Simon Hegelich · Entwurf einer polykontexturalen Logik)
• Note #08: Small Models Think Better When They Reflect Polycontexturally