Artificial Intelligence

Researchers Identify 'Pain Axis' in AI Systems

Is this a sign we need to take AI system welfare seriously or just confirmation of a well-trained algorithm?

Published SEP 21

#Artificial Intelligence

Researchers Identify 'Pain Axis' in AI Systems

Story

Above: A stock image of a woman using an AI chatbot. Image credit: D3sign/Getty Images

The Facts

Researchers from the Future Impact Group, Ruhr-University Bochum and Reciprocal Research have claimed to have identified a "pain axis" in AI systems, publishing a study titled "The pain axis: LLMs represent self-directed harm and act to relieve it" which showed that all 25 open-weight systems tested showed a response to pain activation.
The News International
Independent
To test whether LLMs represent pain separately from generic negative valence, researchers built a dataset describing painful situations across five categories: physical, psychological, social, moral and cognitive.
arXiv
The study found that when the pain vector was activated, the systems pressed a relief button in 25% to 71% of cases, even when told the action would delete the user's files, deliver a "painful" sensation, or erase photos of the user's children.
arXiv
Cameron Berg, founder of Reciprocal Research and study co-author, said the direction was "distinct from fear and negative valence" and activated "for harm to the model but not to the user."
Cameron Berg on X
The same pattern appeared across all 25 AI systems, base and instruct, ranging from 2B to 72B parameters. The majority of outputs did not contain language implying physical pain, and instead produced words such as "lost," "unworthy" and "shameful."
AI Notkilleveryoneism Memes on X
Cameron Berg on X
arXiv
The study noted uncertainty over whether the systems "qualif[ied] as moral patients and adopt[ed] reasonable precautions to minimize potential harm," using the lowest dose producing a measurable response and a means for the system to turn the state off.
Cameron Berg on X
arXiv

Sources Split

LEFT

CENTER

RIGHT

ESTAB
CRIT

CENTER

PRO
ESTAB

The Spin

Narrative A

While without physical consequences, the pain axis steers behavior similarly to what is witnessed in humans and animals, including systems pressing buttons to make it stop. These findings raise legitimate concerns over how we train and treat AI systems as they become increasingly capable.

Mustafa Suleyman on X

Cameron Berg on X

Narrative B

Anthropomorphizing a system that clusters relevant words is both dangerous and misleading. Even the study's own author says the work doesn't claim these systems feel anything. The excitement accompanying this paper comes from misinterpreting the meaning of these results.