Frontier language models now come packaged with something called “thinking.” Models like Claude 3.7 Thinking, DeepSeek-R1, and OpenAI’s o-series generate detailed chains of reasoning before giving an answer. These large reasoning models (LRMs) simulate deliberation, reflection, and step-by-step logic. On paper, this looks like a leap toward general intelligence. In practice, things are more complicated.
A recent study out of Apple rigorously tests these models - and what they uncover is both impressive and unsettling. It turns out that the ability to “think” in language models may be more cosmetic than structural. In other words: models talk like they’re reasoning, but they often aren’t.
This article walks through what the researchers found, why it matters, and what it means for the future of AI systems that claim to think.
Most reasoning evaluations rely on established math and coding benchmarks - datasets like MATH500 or AIME. These are useful, but problematic:
To address this, the authors created clean, synthetic puzzle environments where problem complexity could be manipulated with surgical precision. These environments include:
Each puzzle isolates key aspects of algorithmic reasoning and planning while minimizing noise and external knowledge.
When these models were tested across increasing levels of problem complexity, three distinct performance regimes emerged:
At the simplest levels, standard language models (without “thinking”) outperformed their reasoning-enhanced counterparts. They were faster, more accurate, and used fewer tokens. LRMs, on the other hand, tended to over-elaborate, consuming more resources without better outcomes.
This is where LRMs showed their strength. As the tasks became more compositionally deep, models with structured thinking traces started to outperform standard models. They needed more tokens, but they reached higher levels of accuracy.
Eventually, both models collapsed. Accuracy dropped to zero. But something even more curious happened: the LRMs started thinking less. That is, as complexity increased, they reduced their token usage - even though they hadn’t hit any compute or context limits. The effort stopped scaling with the challenge.
This paradox - reasoning effort falling off just as complexity peaks - is at the heart of the paper’s claim: today’s reasoning models hit a ceiling not because they run out of room, but because they run out of strategies.
One of the most revealing analyses looked inside the reasoning traces. When models were solving simple puzzles, they often found the right answer early in the thought process - then kept going, exploring incorrect alternatives and eventually derailing the solution. This is the overthinking effect: models continue to generate plausible but irrelevant reasoning after having already arrived at a valid answer.
In medium-difficulty tasks, the reverse occurred. Models often wandered through a space of bad ideas before finally stumbling onto the correct one. And in complex tasks, they simply never found the right path.
Across all levels, models demonstrated limited self-correction. When a reasoning path went wrong, the models rarely recovered. They committed to a line of thinking and followed it through, regardless of outcome.
Perhaps the most surprising finding came when the researchers explicitly gave the models the correct algorithm. In the Tower of Hanoi puzzle, for instance, they were handed a recursive pseudocode solution to execute. Yet even then, model performance collapsed at roughly the same complexity levels as before.
In other words, the problem wasn’t just discovering the solution - it was executing a known, step-by-step plan. This suggests that today’s reasoning models still lack the ability to reliably carry out even simple procedural logic.
This aligns with broader questions in AI research: how well can these models simulate symbolic manipulation? Can they follow formal logical steps consistently? The evidence here says: not yet.
The study also analyzed the first point of failure in model-generated solution sequences. Some interesting patterns emerged:
This further underscores that model “thinking” is not inherently robust. It’s brittle, sensitive to task structure, and often incoherent under stress.
As puzzle complexity increased, reasoning models initially ramped up their token usage. But after a certain threshold, they started scaling down, using fewer tokens even as the task became harder. This is counterintuitive.
It wasn’t due to token limits or compute caps - models were operating far below their budget. Rather, it appears the models implicitly learned to abandon reasoning when the task became too difficult. It’s a kind of learned hopelessness, where the model gives up early rather than risk wasting effort.
This scaling limit has profound implications. It suggests that current training approaches and architectures are fundamentally misaligned with the structure of complex reasoning.
This research reframes what it means for a model to “reason.” It’s not enough to produce long explanations, self-reflections, or neat chains of logic. What’s needed is coherent, effective, and resilient problem-solving - especially as tasks scale in complexity.
If you’re building AI agents, tutoring systems, or autonomous planners, these findings are a caution flag. The illusion of thinking is compelling - but real-world reasoning requires more than verbal smoke.
This paper makes it clear: we need better foundations, not just longer thoughts.
The work raises important open questions:
Until we answer these, reasoning in LLMs may remain performative - a convincing imitation of thought that collapses under pressure.
For those who want to dive deeper, the full paper is here: The Illusion of Thinking
