The Illusion of Thinking.

Standard math and coding benchmarks conflate two problems: they may be contaminated, and their difficulty isn’t cleanly controllable. Instead the authors adopt controllable puzzle environments that let us vary complexity systematically — Tower of Hanoi, Checker Jumping, River Crossing, and Blocks World — where a simulator grades every intermediate move and the core logic stays fixed as size grows. That lets them inspect not just answers but the whole chain-of-thought trace.

These models still fail to develop generalizable problem-solving capabilities for planning tasks, with performance collapsing to near-zero beyond a certain complexity threshold.

The central finding is not gentle degradation. Across diverse puzzles, frontier reasoning models face a complete accuracy collapse beyond certain complexities. These aren’t ordinary LLMs — they’re trained with reinforcement learning to self-reflect — yet past a puzzle-specific threshold their accuracy drops to near-zero and stays there. The hero chart above traces that shape; the widgets below let you drive it.

Matched under equal inference compute, the paper identifies three performance regimes: low-complexity tasks where standard models win, medium-complexity tasks where thinking shows a real advantage, and high-complexity tasks where both collapse.

This is the counter-intuitive scaling limit. Near the collapse point, LRMs begin reducing their reasoning effort as problem complexity increases, despite operating well below generation length limits. It isn’t a budget cap — the 64k token allowance sits largely unspent. Pick a puzzle and watch the two curves diverge:

Accuracy vs. thinking tokens (Fig. 6)

HanoiChecker JumpingBlocks WorldRiver Crossing

AccuracyThinking tokens Collapse (N=8)

Reasoning models initially spend more tokens while accuracy declines gradually — until a critical point where reasoning collapses and effort counterintuitively decreases.

Figure 6 — Accuracy and thinking tokens vs. complexity Original

§4.2.2 · Figure 6

Because the simulator can grade every intermediate move, the authors locate where correct solutions appear along each trace. On simple problems, reasoning models often identify correct solutions early but inefficiently continue exploring incorrect alternatives — the “overthinking” mode. Past the threshold, the failure is total: complex problems (N≥8) exhibit consistently near-zero accuracy.

If the models failed because search is hard, giving them the recipe should help. It doesn’t. And it isn’t only a search problem in the first place — they fail to use explicit algorithms and reason inconsistently across scales and problems. The curves for “find it yourself” and “here’s the algorithm” sit almost on top of each other:

Give the model the algorithm? (Fig. 8)

HanoiChecker Jumping

Default (find it) Algorithm given

Even when the algorithm is provided in the prompt — so the model only needs to execute the prescribed steps — performance does not improve and the collapse occurs at roughly the same point.

Figure 8 — Default vs. algorithm-guided execution Original

§4.4 · Figure 8

If the wall were about raw computation, harder-to-simulate puzzles would fall first. They don’t. The models show very different error-free sequence lengths across puzzles, performing up to 100 correct moves in Tower of Hanoi but only 4 in River Crossing.

The controlled setup is the paper’s strength and its limit. The authors note their puzzle environments represent a narrow slice of reasoning tasks, and that most of our experiments rely on black-box API access to the closed frontier models — so results may not generalize to open-ended reasoning, and no mechanistic look inside is possible. Read the collapse as a sharp, reproducible symptom, not a full theory of why.