Artificial Intelligence

Anthropic Releases Open-Source Tool to Test AI Political Bias

Published NOV 18 2025

Story

Above: The Anthropic Claude AI logo on Oct. 21, 2025. Image credit: Thomas Fuller/SOPA Images/LightRocket/Getty Images

The Facts

Anthropic has released an open-source evaluation method to measure political neutrality in AI chatbots, using 1,350 paired prompts across 150 topics to assess how evenly models treat opposing political viewpoints through structured scoring.
Perplexity
DIGIT
Rohan's Bytes
Anthropic's evaluation framework measured three dimensions — even-handedness in treating opposing views, plurality of perspectives to acknowledge nuance, and refusal rates to ensure the model doesn't selectively decline engagement.
Axios
DIGIT
According to Anthropic's testing, Claude Sonnet 4.5 scored 94% in even-handedness, while Gemini 2.5 Pro achieved 97%, Grok 4 reached 96%, GPT-5 scored 89%, and Llama 4 received 66% in political neutrality assessments.
AIBase
Fortune
Verge
This comes after the company said it trains Claude using system prompts and reinforcement learning to avoid unsolicited political opinions and maintain factual accuracy while representing multiple perspectives with equal depth and engagement.
Fortune
India Today
The announcement comes amid U.S. President Donald Trump's executive order requiring federal agencies to procure only unbiased AI systems, and amid the administration's criticism of diversity, equity and inclusion initiatives as threats to reliable AI.
Axios
Fortune
Donald Trump
Anthropic made its evaluation tool available on GitHub under an open-source license, encouraging industry-wide adoption and creating shared standards for measuring political bias in AI systems among developers and researchers.
Verge
AIBase

Sources Split

LEFT

CENTER

RIGHT

ESTAB
CRIT

CENTER

PRO
ESTAB

The Spin

Techno-skeptic narrative

Anthropic's so-called bias testing is nothing but corporate manipulation disguised as transparency. This fear-mongering company pushes fake "safety" measures to control AI development and gatekeep competition. Their circular self-evaluation system is fundamentally flawed propaganda.

Justin Grimmer on X

Ahmad Osman on X

Techno-optimist narrative

Anthropic's groundbreaking open-source evaluation tool is a significant advancement in AI transparency and fairness. The company's rigorous testing across thousands of prompts shows Claude achieves 94% even-handedness while maintaining political neutrality. This shared standard benefits the entire industry.