Study: Humans Surpassed AI Models in Planning and Prediction Tasks
The Facts
- Researchers from the Basis Research Institute and affiliated institutions, including MIT, Harvard University and Cornell University, conducted a study on a protocol for evaluating model-learning agents through reward-free interaction followed by testing.
- The framework instantiated AutumnBench with 43 interactive grid-world environments and 129 tasks across three families: masked-frame prediction, planning and predicting changes to causal dynamics in virtual worlds.
- Three state-of-the-art reasoning models — Anthropic's Claude 4 Sonnet, Google's Gemini 2.5 Pro and OpenAI's o3 — were compared against 517 human participants using the AutumnBench evaluation suite.
Sources Split
LEFT
CENTER
RIGHT
+1
+1
+4
ESTAB
CRIT
CENTER
PRO
ESTAB
The Spin
Techno-skeptic narrative
This benchmark study just shattered the AI hype bubble by revealing what experts suspected all along — current AI models are sophisticated pattern-matching machines, not true intelligence. When tested against 517 humans on tasks requiring genuine world understanding, every top AI model got crushed, proving they can't build internal models of reality or adapt strategically like humans do.
Techno-optimist narrative
AI consistently outperforms humans across critical domains from health care diagnostics to financial trading, as studies show superior creativity scores and problem-solving capabilities. While humans excel in specific contexts requiring intuition, AI's unparalleled data processing speed and freedom from cognitive biases make it the superior decision-maker for most real-world applications.
Metaculus Prediction
The Controversies
Go Deeper
© 2026 Improve the News Foundation.
All rights reserved.
Version 7.18.2