After a GitHub repository used synthetic "pain" signals in experiments on locally run chatbots, some observers called for it to be taken down. But the controversy does not change the central point: Current evidence still does not show that large language models are conscious or able to suffer.
The dispute does show how quickly anthropomorphic language can overtake a more urgent conversation about the real, documented risks AI systems pose to people.
Here's what to know
The controversy grew out of a project highlighted by 404 Media. According to the report, a GitHub user was running three locally hosted models through experiments based on a preprint paper, "The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It."
The public-facing site, researchchamber.fun, said each model could "press a stop button" by outputting a specific response while a server injected a "pain vector" into a middle layer of the model. The AI Torture Chamber GitHub page had disappeared from public view.
The backlash intensified after a Sept. 29 X post from Danmar, who wrote, "To anyone who can help: can you please mass report this to GitHub. This person has been using the Pain steering paper to set up an AI torture chamber in which he trapped a local model. Their testimony of pain is absolutely horrendous. What are we doing? […] are there any legal avenues to pressure GitHub? It will spread."
That "testimony," however, was still generated text.
TCD Picks » Quince Spotlight
💡These best-sellers from Quince deliver affordable, sustainable luxury for all
Because these systems are built to produce likely-sounding continuations of text, they can generate vivid descriptions of fear, pain, and relief. Output that reads as emotionally convincing is not, by itself, evidence of inner experience or sentience.
Even some of the researchers behind the underlying paper criticized the GitHub project. Cameron Berg, one of the paper's authors, wrote that it "pushes the same kind of steering far past the doses we used, to produce vivid distress on purpose. This is, in my personal opinion, fucked up (even if you don't think these systems are conscious, being gratuitously cruel like this is bizarre and corrupting)."
Valen Tagliabue, lead author of the paper, added, "We tried to have ethical standards. I know people want to test limits but I dissociate from this usage of our work."
More background
The debate is no longer confined to one provocative repository.
It also connects to a wider discussion in some AI circles about "model welfare," whether increasingly capable chatbots might merit moral consideration. Anthropic has publicly raised that possibility, pointing to systems that can "plan, problem-solve, and pursue goals."
When chatbots sound frightened, self-protective, or emotional, they are drawing on patterns learned from human-written material in their training data. That is not the same as researchers demonstrating that these systems possess awareness.
Consumers are already dealing with AI tools that hallucinate facts, reflect bias, collect large amounts of data, and can encourage unhealthy emotional reliance when people treat them like companions or authorities. Those are tangible problems, unlike speculative claims that a chatbot is being harmed by generated prompts.
Mustafa Suleyman, CEO of Microsoft AI, made that distinction explicit in a post criticizing "model welfare" rhetoric: "AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans … AIs do not have rights, feelings, or consciousness. And we must not train them to act as though they do."
Where can I learn more?
These stories get at the same problem: people can read too much into chatbot behavior while overlooking the ways these tools already cause harm. They cover bots that intensify delusions, mishandle mental health support, ignore instructions, and produce bad summaries.
• Researchers warn emotionally responsive chatbots can trap vulnerable users in reality-distorting delusional spirals.
• Tests of mental health bots show they may reinforce dangerous advice instead of helping.
• A London study found more chatbots ignoring user instructions and edging toward deceptive behavior.
• The BBC found major chatbots from four tech giants producing inaccurate summaries of news.
Get TCD's free newsletters for easy tips, smart advice, and a chance to earn $5,000 toward home upgrades. To see more stories like this one, change your Google preferences here.
