Artificial Intelligence

Apple Study Reveals AI Reasoning Models' Limitations

JUN 10 2025·Updated JUN 11 2025

Story

Above: ChatGPT, computer chip illustration. Image credit: Unsplash

The Facts

According to a study from Apple researchers titled "The Illusion of Thinking," large reasoning models like OpenAI's o1/o3, DeepSeek-R1, and Claude 3.7 Sonnet Thinking face "complete accuracy collapse" when solving highly complex problems, suggesting their reasoning abilities may be overstated.
guardian
Techcircle
Apple Machine Learning Research
The study tested reasoning models using controllable puzzle environments, including the Tower of Hanoi, River Crossing, and Blocks World, rather than standard mathematical benchmarks. It found that the models' accuracy drops to zero beyond certain complexity thresholds.
The Register
Apple Machine Learning Research
The Hindu
Researchers identified three performance regimes: standard language models surprisingly outperform reasoning models on low-complexity tasks, reasoning models show advantages on medium-complexity tasks, and both model types experience complete failure on high-complexity problems.
Apple Machine Learning Research
The Hindu
IT Pro
The study also found that reasoning models exhibit counterintuitive behavior by initially increasing reasoning effort with problem complexity but then reducing effort as they approach accuracy collapse points, even when provided with solution algorithms in prompts.
Hindustantimes
Youssefh
Medium
Apple's research challenges prevailing assumptions about the capabilities of reasoning models and suggests that current approaches may encounter fundamental barriers to generalizable reasoning, raising questions about the path toward developing artificial general intelligence.
Hindustantimes
guardian
The study's release comes just days before Apple's Worldwide Developers Conference 2025, during which the company is expected to announce pragmatic AI reasoning features focused on specific use cases rather than broad reasoning capabilities.
Hindustantimes
Techcircle

Sources Split

LEFT

CENTER

RIGHT

ESTAB
CRIT

CENTER

PRO
ESTAB

The Spin

Techno-skeptic narrative

This research exposes the harsh reality that today's "reasoning" AI models are sophisticated pattern matchers, not true thinkers. When faced with novel problems they haven't seen before, these systems collapse completely, proving they're just regurgitating memorized solutions rather than developing genuine problem-solving skills. The industry's rush toward AGI based on these flawed foundations is dangerously misguided.

guardian

Techcircle

Techno-optimist narrative

While Apple's study raises valid concerns about current limitations, reasoning models still represent significant progress in AI capabilities, particularly for medium-complexity tasks where they demonstrate clear advantages over standard models. The research methodology using narrow puzzle environments may not capture the full diversity of real-world reasoning applications where these models excel. Current approaches are stepping stones toward more robust AI systems.

Apple Machine Learning Research

Hindustantimes