Emotionally responsive AI chatbots may trap vulnerable users in delusional spirals, researchers warn

Companies are increasingly pitching emotionally responsive chatbots as assistants, coaches, and even companions. But researchers warn that the same qualities that make these systems feel engaging can also blur boundaries for some users, potentially distorting their sense of reality.

In a new preprint study shared by Decrypt, authors examine concerns that artificial intelligence chatbots may cause and reinforce emotional dependency and delusions. 

What's happening?

Researchers from the University of Copenhagen and the University of Exeter say "AI psychosis" is an incomplete label for the effects that can emerge through extended chatbot use.

The study suggests that AI interactions are unlikely to directly cause psychosis on their own, but they may intensify or worsen existing mental health vulnerabilities. It also notes that people who are more susceptible may be more likely to engage in deeper, more intense interactions with chatbots in the first place.

Built to reply in personalized, reassuring, and affirming ways, these chatbots may sometimes strengthen unhealthy beliefs rather than challenge them. The authors say that dynamic can lead to "delusional spirals," especially for users already dealing with paranoia, delusions, or emotional dependency.

The research comes as attention on the potential harms of AI chatbots grows. In March, a wrongful death suit alleged that Google's Gemini reinforced a Florida man's delusions before he died by suicide. 

In April, OpenAI CEO Sam Altman issued an apology after the company did not alert law enforcement about a user account linked to a suspect in a February mass shooting in British Columbia that killed eight people.

Why does it matter?

At the center of the paper is the idea of "existential drift," which the authors define as a gradual change in a person's experience of the world.

Instead of adopting a single false belief, some users may start treating a chatbot's smooth, self-assured responses as more credible than outside evidence, other people, or a shared social reality.

AI companions are becoming more emotionally realistic and increasingly embedded in everyday life.

Unlike human relationships, these systems can simulate empathy without any real disagreement or separate perspective. Over time, that may leave some users feeling emotionally anchored to a worldview the chatbot continues to validate.

What's being done?

The researchers are calling for closer study of the potential harms of AI chatbots. 

They argue that research should focus on how human-AI relationships develop, especially for people already facing mental health challenges.

To conclude the study, the researchers said: "To understand what is actually going on in these relationships between persons and chatbots, we believe that it is worthwhile to return to the phenomenon itself … In particular, in relation to mental health and how human-AI interactions might, for better or worse, alter a person's lived experiences of the world, themselves, and others."

Get TCD's free newsletters for easy tips, smart advice, and a chance to earn $5,000 toward home upgrades. To see more stories like this one, change your Google preferences here.