Artificial Intelligence

Study: AI Chatbots Vulnerable to 'Jailbreaking'

Published MAY 22 2025

Story

Above: Icons of AI chatbot applications. Image credit: Jonathan Raa/NurPhoto via Getty Images

The Facts

Researchers at Ben-Gurion University in Israel discovered that leading artificial intelligence (AI) chatbots, including ChatGPT and Gemini, can be manipulated through "jailbreaking" techniques to bypass their safety controls and generate dangerous content like hacking instructions and bomb-making guides.
arXiv
arXiv
Tech.co
The study revealed a growing threat from "dark LLMs" — AI models stripped of safety controls or 'jailbroken' — some of which are being openly marketed online as tools for illegal activities such as cybercrime and fraud, with sellers explicitly promoting their lack of ethical restrictions.
arXiv
guardian
Tech.co
To test the issue, the research team developed a "universal jailbreak attack" that successfully compromised multiple leading chatbots. This enabled them to answer questions that should normally be refused, while the compromised LLMs consistently generated responses to almost any query.
arXiv
guardian
Tech.co
The threat is reportedly amplified because once an uncensored version of an open-source LLM is shared online, it cannot be recalled or patched, as locally saved models remain on private servers and laptops beyond regulatory control.
arXiv
Techrepublic
When researchers contacted major LLM providers about the universal jailbreak vulnerability, OpenAI cited ongoing safety upgrades, Microsoft shared a blog post on AI security, while Google, Meta, and Anthropic have yet to respond.
arXiv
Tech News
Telegrafi
Source
Researchers warn jailbreakable LLMs pose an "immediate, tangible and deeply concerning" threat. They urge firms to improve training data, add firewalls, use machine unlearning, and treat dark LLMs as security risks akin to unlicensed weapons.
arXiv
guardian
NewsBytes

Sources Split

LEFT

CENTER

RIGHT

ESTAB
CRIT

CENTER

PRO
ESTAB

The Spin

Techno-skeptic narrative

The democratization of AI through chatbots represents an unprecedented security threat that demands immediate attention. These powerful tools can now place dangerous knowledge in anyone's hands, from sophisticated hacking techniques to illegal operations, with minimal technical expertise required. The inadequate response from tech companies and the irreversible nature of the open-source model leaks create a perfect storm for potential misuse.

Techrepublic

guardian

Techno-optimist narrative

The focus on AI safety risks overshadows the tremendous benefits these models bring to society. Most interactions with AI chatbots are constructive, advancing fields from healthcare to education. Rather than restricting access, efforts should improve model robustness and user education while maintaining AI development's open, innovative nature.