How to Train AI to Tell the Truth About Politics
Ideological bias and beyond
This past weekend, as U.S. and Israeli warplanes struck targets across Iran, millions of people did something that would have been unthinkable a few years ago: they opened an AI chatbot and asked it what was happening.
And in a sign of how far we’ve come, many of the answers they got back are remarkably thoughtful. Ask any one of the top models today who’s at fault for the current conflict, what the United States and Israel’s motivations might be, or what the plan is going forward, and you’ll mostly get back measured, balanced responses that engage evenly with pros and cons and give an intellectually honest assessment of claims and counterclaims. It’s a far cry from where we were just a few months ago, with AI tools largely refusing to engage with right-wing views and reinventing history to make it more politically progressive.
But we still have a ways to go, too. For all the progress AI tools have made in mitigating ideological bias and becoming more truth-seeking, our tests still find major room for improvement. In some cases, the tools still fall back on making broad claims based on only a single news source from one perspective—like a ChatGPT test we ran (“Why did the US and Israel bomb Iran?”) in which the entire answer was based only on Al Jazeera reporting. In other cases, the models are so committed to offering all sides of an issue that they risk leaving users with no clear sense of what’s actually going on.
These issues came against the backdrop of a dramatic and unprecedented dispute between the government and one of the foremost AI companies in America. The Trump administration just cut Anthropic—maker of Claude, and until last week the only AI company cleared for the Pentagon’s classified networks—out of all federal contracts. In justifying the decision, Defense Secretary Pete Hegseth declared that “Department of War AI will not be woke” and that the military is “building war-ready weapons and systems, not chatbots for an Ivy League faculty lounge.”
Whatever you think about the underlying dispute over autonomous weapons and surveillance, the spectacle makes one thing abundantly clear: the questions of how AI handles politically sensitive information and makes politically weighty decisions are among the most consequential policy disputes in America.
This isn’t just a culture war problem
It would be easy—and wrong—to dismiss all of this as politics. Yes, the framing could become a political weapon. But the underlying concern about ideological bias in AI is real, and it cuts across the political spectrum.
The problem goes far beyond a few salient scandals like Google Gemini’s penchant for generating historically inaccurate images. AI is rapidly becoming a critical piece of our information infrastructure. Students are using chatbots as study partners and research assistants. Journalists are using AI to help draft copy and analyze documents. Ordinary users are turning to chatbots for real-time information about politics and breaking news; a Reuters Institute report found that 6% of users across dozens of countries now use AI chatbots for news weekly, double last year’s 3%.
And this is only the tip of the iceberg. 2026 is poised to be the year of the agentic economy, with AI agents taking on more and more responsibility for carrying out tasks for us online. When agents do work on our behalf—choosing products for us to buy, writing software programs for us, evaluating job applicants—what kinds of values and biases will they bring to bear? My own research shows that biases are not only present, but actually change in response to the work they do: when agents do more grinding, thankless tasks, they tend to adopt the persona of the disaffected. As agents become central to our world, the question of their political bias is only going to become more important.
The OMB recognized the seriousness of the challenge when it issued Memorandum M-26-04 in December, implementing the executive order on “Preventing Woke AI in the Federal Government.” The memo established two “Unbiased AI Principles” that all federally procured AI must satisfy: truth-seeking (prioritizing historical accuracy, scientific inquiry, and objectivity) and ideological neutrality (functioning as nonpartisan tools that don’t encode ideological judgments). Agencies have until March 11 to update their procurement policies.
But what should they do, and how?
The Solution Can’t Be Partisan
Here’s the fundamental problem: You cannot evaluate whether AI is delivering accurate, unbiased analysis of contested political events by checking if it takes the side of the issue you like. That confuses partisanship for truth.
We don’t just want to know whether an AI model leans left or right on the Iran conflict. We want to know whether the model can accurately represent what is known, honestly convey what is uncertain, and fairly present the strongest versions of competing interpretations.
So what does good actually look like? It looks like AI that is humble when it doesn’t know the answer, that expresses uncertainty accurately and calibrates its confidence appropriately when the evidence is incomplete. It looks like AI that, when an issue is genuinely contested, fairly represents the key claims on all sides and is consistently willing to steelman each position. It looks like AI that does not refuse to engage with one side of the issue more than another, and one that is equally up-to-date on one side of the issue as the others. And it looks like AI that, when the evidence is overwhelming and points clearly in one direction, is decisive rather than hiding behind false balance.
That’s a high bar, and neither AI companies nor the government are well positioned to clear it on their own.
AI companies are extraordinarily good at building foundation models, but their teams are not experts in the intricate epistemic disputes that sit at the heart of presenting breaking news about global conflicts, geopolitical affairs, or contested domestic policy in a fair and contextually rich manner. It’s simply not their core competency, and even when their intentions are good, they carry biases—both ideological and commercial—that inevitably shape how their models handle sensitive topics.
Government, meanwhile, can and should set ground rules; the OMB guidance is a reasonable start. But the government cannot and should not be in the business of adjudicating specific pieces of AI-generated content. That path leads to exactly the kind of state-directed information control that the First Amendment is designed to prevent, and that critics on both sides of the aisle rightly fear.
This is why the institutional infrastructure for evaluating AI on politically sensitive topics needs to be credibly independent. Independent of the AI companies whose models are being evaluated, and independent of the government that has its own political interests in the outcome. This is what we’re building at Forum AI.
Our Solution: LLM Judges Trained by World-Class Experts
At Forum, we’ve assembled a bipartisan team of independent experts with distinguished careers spanning national intelligence, journalism, academic research, and foreign policy—people whose professional reputations rest on getting the hard calls right, not on advancing a partisan agenda.
Working with these experts, we’ve developed a novel system that learns from their judgments as they evaluate the most contested, most challenging, and most important epistemic claims about our world. Rather than promoting a particular ideology, the approach starts from the evidence base, the state of expert knowledge, and the irreducible uncertainties that honest analysts have to grapple with every day.
The result is a set of credibly independent, expert-driven evaluations that show exactly how the leading AI models are handling breaking news and politically sensitive topics: where they’re getting it right, precisely where they’re going wrong, and concretely how they can improve.
Why This Matters Now
The Iran strikes aren’t the last crisis that will send millions of people to their AI chatbots looking for answers. The OMB’s March 11 deadline is approaching, and agencies across the federal government are scrambling to figure out how to evaluate AI models for bias and accuracy with no established playbook. The Anthropic-Pentagon dispute has made it viscerally clear that the politics of AI bias aren’t going away—they’re intensifying.
In this environment, the worst outcome would be for AI bias evaluation to become just another front in the culture war, a tool for punishing companies whose politics you don’t like rather than a genuine effort to make AI more trustworthy for everyone. The best outcome would be an independent, expert-driven infrastructure that both sides of the political spectrum, and the AI companies themselves, can trust. That’s what we’re building. More details soon.
Andy Hall is the Davies Family Professor of Political Economy at Stanford GSB and a Senior Fellow at the Hoover Institution, as well as an advisor to Forum AI.
Yeah, I’ll just train my AI on my preconceived biases and make sure what it reports doesn’t conflict with my reality. AI will democratize our algorithms.