Anthropic's effort to help define AI's moral direction appears to have collided with a public divide at the Vatican, where co-founder Christopher Olah reportedly floated pulling the company out of Pope Leo XIV's AI encyclical launch after reading the pope's rejection of machine consciousness.
The dispute highlights how tech companies are pushing ahead with systems that could reshape work, safety, and everyday life while debating what exactly they have built.
Here's what to know
The tension emerged only after Anthropic reviewed the encyclical in advance. As The New York Times reported, Olah accepted an invitation to speak at the Vatican on May 25 during the rollout of Pope Leo XIV's first encyclical, "Magnifica Humanitas." But he disagreed enough with the pope's rejection of artificial intelligence consciousness to suggest Anthropic back out of the event.
In the encyclical, Leo wrote that "humanity — in all its grandeur and woundedness — must never be replaced or surpassed" and said, "If technology becomes the ultimate criterion, the human person risks being reduced to data, a cog in a machine or a commodity."
Privately, by contrast, Olah has reportedly been urging religious thinkers to take seriously the possibility that Anthropic's Claude models may have something like moral status.
At the same time, the company has been chasing enormous growth, with a potential valuation of $2 trillion. Anthropic spent months bringing in religious scholars from around the world, requiring nondisclosure agreements, and holding private conversations about how Claude should behave virtuously, according to the Times.
TCD Picks » Quince Spotlight
💡These best-sellers from Quince deliver affordable, sustainable luxury for all
One of the world's most powerful AI companies is still debating whether its products are tools, agents, or something closer to beings, even as those systems are being rolled out across society.
More background
Behind Olah's outreach were worries about AI's real-world effects, including job disruption, cyberattacks, and biological threats.
Those risks extend around the globe, touching workers, consumers, students, and anyone whose personal data or livelihood depends on the safety of digital systems.
Anthropic has framed its effort as a way to protect humanity by building AI that chooses good outcomes. But critics see a troubling contradiction: The company is commercializing increasingly powerful products while wrestling with basic questions of control, accountability, and values.
AI tools are being folded into workplaces, customer service, search engines, education, and decision-making. If companies build these systems first and sort out the ethics later, the public bears the risk.
"Those who control AI will impose their own moral vision, which will become the invisible infrastructure of these systems," Olah argued, per the Times.
What's being done?
Anthropic says it is trying to reduce those risks through both outside consultation with philosophers, religious scholars, psychologists, and civil society leaders and a written "constitution" for Claude, an 84-page document meant to guide the model's behavior.
Anthropic in-house philosopher Amanda Askell said she wants the models to be "the best of us," while Olah has argued for "some shared notion of goodness that cuts across society in some very broad way," the Times reported. The outside input could counter the insular decision-making that often defines the tech industry.
Still, consultation is not the same as public accountability. Closed meetings and internal values do little to reassure people who face the consequences if AI systems are deployed recklessly in schools, offices, hospitals, or government settings.
"I will be honest: We keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don't know what that means, but I think it warrants ongoing discernment," Olah said, according to the Times.
Where can I learn more?
The tension is part of the debate about who gets to set AI's values and limits.
• At the Vatican, Pope Leo XIV urged oversight of AI before it eclipses human judgment.
• Pope Leo also warned AI weapons are pushing humanity toward annihilation.
• In Washington, OpenAI-backed Anthropic dropped a core safety principle after pressure from government officials.
• Anthropic CEO Dario Amodei says binding U.S. rules are needed as AI spreads.
Get TCD's free newsletters for easy tips, smart advice, and a chance to earn $5,000 toward home upgrades. To see more stories like this one, change your Google preferences here.
