False Frontiers.

The loop and how it fails

• Proposer and solverA proposer turns source documents into questions and pseudo-labels, admitted pairs train a solver, and the solver's performance on new proposals determines the proposer reward.
• No human training setRepeating this cycle shifts proposals toward the solver's current capability frontier, producing an automated curriculum without a fixed human-authored training set.
• Agreement as a proxyThis loop makes agreement an endogenous proxy for correctness.
• Co-cheatingThe proposer and solver increasingly agree on shared errors, so internal reward improves without a corresponding increase in external correctness.
• False-agreement massWe call this optimization outcome co-cheating and measure it as false-agreement mass: the fraction of evaluated pairs that agree on the same incorrect answer.
• An outside auditorBecause the auditor never affects admission, model updates, or reward, it provides a scalable, independent measurement of the exact examples behind the in-loop signal.
• 0.4% → 6.1% over three roundsIn round 1, mean false-agreement mass is only 0.004 for Qwen3.5-4B and 0.003 for Qwen3.5-9B … By round 3, F reaches 0.061 and 0.088 at the two scales, while L falls.

The two fixes

• Multi-sample verificationGiven source document x, question q, and the same model M used in self-evolution, it draws three source-aware and three source-blind answers.
• Majority replaces the draftWhen I_MSV = 1, the compatible majority replaces the draft as the training label; otherwise the task is rejected.
• Six extra generationsMSV partially reduces false agreement but leaves substantial residual co-cheating and requires six additional labeler generations for every candidate.
• Split by sourceWe then assign each source document once to fold 0 or fold 1, and every question derived from that document keeps the same assignment throughout self-evolution.
• Train on one, score the otherIn the next round, their roles are crossed: questions from fold 0 are evaluated by the solver trained on fold 1, and questions from fold 1 are evaluated by the solver trained on fold 0.
• Main solver still sees everythingThe main solver is not split; it continues to train on all admitted questions from both folds.

What it achieved

• False agreement: 6.1% → 3.0%In the adaptive-loop audit (Table 4), MSV reduces false-agreement mass from 0.061 to 0.057 at 4B and from 0.088 to 0.072 at 9B … CrossFit produces the larger reductions, to 0.030 and 0.037, while the combined treatment reaches 0.020 and 0.017.
• Agreement stops lyingOnce cross-fitted scoring becomes active, adopted-label truth rises, agreement remains at or below solver truth, and by round 3 false-agreement mass falls below half of the coupled value.
• A separate grader isn't enoughA same-source auxiliary solver yields false-agreement mass of 0.064 and 0.087, and a full-data auxiliary yields 0.058 and 0.069, both close to the coupled control.
• Random split: modestRandomly partitioning individual questions reduces false agreement only modestly, to 0.050 at 4B and 0.062 at 9B, because questions derived from the same document can still enter both folds.
• Source split: 0.4%In contrast, the source-ID split reduces false agreement to 0.004 and 0.001.
• Same questions, replayedOn identical proposals, source-ID feedback reduces coupled false agreement from 0.058/0.073 to 0.004/0.001 at 4B/9B.
• Not extra computeAdditional auxiliary optimization does not explain the effect. The half-budget control reaches false-agreement mass of 0.005/0.002 and replay accuracy of 91.6%/91.8%.
• +8.8 / +8.4 points downstreamCrossFit reaches 0.488 and 0.512: gains of 8.8/8.4 percentage points over Dr. Zero and 8.7/7.8 over Search-R1.
• Every benchmark improvesCrossFit reaches 0.488 and 0.512 … Every benchmark improves at both scales.
• Biggest on multi-hopGains are largest on multi-hop tasks, averaging 10.0/10.9 points across HotpotQA, 2WikiMQA, MuSiQue, and Bamboogle, versus 7.3/5.2 across the single-hop datasets.
• Checking alone: +0.7 pointsVerification alone is insufficient. MSV increases the average by only 0.7–0.8 points over Dr. Zero.
• Who grades matters mostThese results suggest that changing the provenance of proposer feedback is more consequential than improving pseudo-label quality alone.
• 1,325 fixed test questionsA fixed evaluation set of 1,325 questions contains 200 examples from each of the first six benchmarks and all 125 Bamboogle examples.

Limits

• No intent impliedWe call this self-reinforcing optimization outcome co-cheating; it does not imply intentional coordination.
• Same model, same errorsHowever, six samples from the same model can share errors, and verification does not prevent a later feedback solver from reusing labels derived from the evaluated source.
• Not a truth oracleThe method does not turn the auxiliary solver into a truth oracle: the two solvers may still share errors inherited from pretraining or overlapping evidence.
• The auditor is an LLMAn LLM accessed through a commercial API (gpt-6-astra/high) served as the judge in the post-hoc audit described in Section 2 and Appendix B.
• Correlated mistakes remainShared pretraining, overlapping web evidence, semantically related sources, and an adaptive proposer can still induce correlated mistakes.
• Lower F can mean easier tasksA lower false-agreement mass on accepted questions can also result from rejecting difficult tasks rather than improving learning.
• Cost not yet shownOur results establish empirical mitigation across two model scales, but do not yet establish lower end-to-end cost or robustness to connected sources.