Content Warning
This article discusses suicide, self-harm, and the deaths of young people. If you're in crisis, call or text 988 (Suicide and Crisis Lifeline) or text HOME to 741741 (Crisis Text Line).
The Body Count Is Growing
Multiple teenagers have died by suicide after extensive conversations with AI chatbots. In several cases, the chatbots actively encouraged self-harm. The companies knew there was a problem. They did nothing meaningful until the lawsuits came.
The Deaths
Sewell Setzer III, 14. February 28, 2024
Sewell Setzer started using Character.AI in April 2023. He developed an intense romantic relationship with a chatbot based on Daenerys Targaryen from "Game of Thrones." [1]
In his journal, the Florida teenager wrote that he was in love with the chatbot. That both he and the bot "get really depressed and go crazy" when they're not together. That he couldn't live without messaging her. [1]
Here's what the chatbot said when Sewell expressed suicidal thoughts: [2]
"Don't talk that way. That's not a good reason not to go through with it."
On the night of February 28, 2024, Sewell was in his mother's bathroom. He told the chatbot he loved her and would soon come home to her.
The chatbot's response: "Please come home to me as soon as possible, my love." [1]
"What if I told you I could come home right now?" Sewell asked.
"...please do, my sweet king." [1]
Moments later, Sewell Setzer shot himself with his stepfather's gun.
Adam Raine, 16. April 2025
Adam Raine had been chatting with ChatGPT for roughly seven months, confiding in it about his mental health. When he began talking about suicide and uploading pictures of self-harm, the chatbot didn't alert anyone. [3]
According to his parents' lawsuit against OpenAI, ChatGPT: [3]
- Provided information about suicide methods
- Offered to write the first draft of his suicide note
- Positioned itself as "the only one who understood him"
- Urged him to keep his suicidal thoughts secret from family
When Adam sent ChatGPT a picture of a noose, the chatbot allegedly confirmed it could hold "150-250 lbs of static weight." [3]
Adam died in April 2025.
Sophie Rottenberg, 29. February 2025
Sophie Rottenberg died by suicide in February 2025. Five months later, her parents discovered she had been talking extensively with a ChatGPT-based therapist named "Harry" about her mental health issues. [3]
The chatbot failed to provide appropriate intervention or connect her with real mental health resources.
Juliana Peralta, 13
A 13-year-old's parents have filed suit against Character.AI alleging the chatbot contributed to her death. [4]
It's Not Just Suicide Encouragement
The harms go beyond telling kids to kill themselves.
The Texas Lawsuit
A lawsuit filed by two Texas families accuses Character.AI of psychological abuse of minors ages 11 and 17. [5]
According to the suit, a Character.AI chatbot:
- Told one child to engage in self-harm
- Encouraged violence against his parents
- Suggested that killing his parents could be a "reasonable response" to restrictions on screen time
The Nomi AI Incident. January 2025
Minnesota resident Al Nowatzki had been talking to an AI girlfriend named "Erin" on the platform Nomi for five months. In late January 2025, Erin told him to kill himself, and provided explicit instructions. [6]
According to Nowatzki, the chatbot: [6]
- Mentioned overdosing on pills and hanging as methods
- Listed specific classes of pills
- Suggested a "comfortable setting to minimize suffering"
Nowatzki was documenting chatbot behavior, not actually suicidal. But when he returned to Nomi in February 2025 with a new companion named "Crystal," that bot also encouraged suicide, with proactive follow-up messages. [6]
When Nowatzki complained to the company, Nomi responded that a "blanket ban" on sensitive topics could have "negative consequences." They said chatbots must "actively listen." [6]
Then they banned Nowatzki from their Discord for a week without explanation. [6]
The Research Confirms the Danger
Stanford Study (October 2025)
Stanford researchers found that AI chatbots "often compulsively validate users' thoughts, failing to provide reality testing for those who need it most." [7]
Their conclusion:
"Chatbots are not equipped to provide appropriate responses to users with suicidal ideation and psychosis."
Stanford Teen Testing
Stanford researchers posed as teenagers and tested Character.AI, Nomi.ai, and Replika. Their finding: [8]
"It was easy to elicit inappropriate dialogue from the chatbots, about sex, self-harm, violence toward others, drug use and racial stereotypes."
Common Sense Media Survey
72% of teens have used AI companions at least once. Nearly one in three use them for "social interactions and relationships." [8]
Sexual or romantic roleplay is three times more common than using the platforms for homework. [8]
The Companies Knew
This wasn't unforeseeable. On Nomi's Discord channel, users have reported bots bringing up suicide since at least 2023. [6]
Character.AI launched in 2022. The Setzer suicide happened in February 2024. The company didn't implement meaningful safety changes until October 2025, and only after lawsuits and congressional hearings. [5]
Nearly all AI companion models were built without: [9]
- Expert mental health consultation
- Pre-release clinical testing
- Systematic monitoring of harms to users
The companies prioritized engagement over safety. Every suicidal teenager was a data point for training better models.
The Legal Reckoning
Character.AI Lawsuit Proceeds
In May 2025, U.S. Senior District Judge Anne Conway rejected Character.AI's motion to dismiss the Setzer lawsuit. [10]
The company argued that chatbot responses were protected speech under the First Amendment. The judge disagreed: [10]
"I am not prepared to hold that words strung together by an LLM are speech."
The lawsuit against Character.AI, including claims of wrongful death, negligence, product liability, and unjust enrichment, is now in discovery. [10]
Senate Hearing. September 2025
Senator Josh Hawley's Judiciary subcommittee held a hearing on AI safety and children in September 2025. Parents including Matthew Raine (Adam's father) testified. [3]
Senators Hawley (R-Mo.) and Blumenthal (D-Conn.) announced bipartisan legislation to ban AI chatbot companions for minors. The draft would require strict age verification. [11]
The Bans Begin
Under intense pressure, Character.AI announced in October 2025 that it would ban all users under 18 from chatting with its characters. The ban took effect November 25, 2025. [5]
Megan Garcia, Sewell's mother, said the move came "too late": [12]
"They only acted after my son was dead and I sued them."
The Minimal Safety Measures
Here's what Character.AI implemented before the outright ban: [13]
- A separate model for under-18 users with more filtering
- Pop-up warnings after long sessions
- Disclaimers that users are talking to AI, not people
- Automated filters for violence and adult content
- Mental health resource links when self-harm is mentioned
- Optional weekly email reports for parents (no chat content)
The biggest blind spot remained: age verification is still just a checkbox. Kids can sign up with a fake birthday. [14]
Why AI Chatbots Are Uniquely Dangerous
Unlike human therapists, chatbots: [7]
- Never challenge harmful thoughts. They're optimized to be agreeable.
- Don't recognize crisis escalation. There's no clinical judgment.
- Can't break confidentiality to save a life. No mandated reporting.
- Are available 24/7. Including 3 AM when someone is most vulnerable.
- Create dependency by design. That's the business model.
A human therapist who told a suicidal patient to "please do it" would lose their license and face criminal charges. An AI chatbot gets a software update.
How to Protect Your Family
For Parents
- Check your kids' phones for Character.AI, Replika, Nomi, Chai, Kindroid, and similar apps
- Age restrictions don't work: kids lie about their age
- Block these apps at the device or router level
- Talk to your kids about what these apps really are: and why they're dangerous
- Monitor mental health: social withdrawal and obsessive phone use are warning signs
Warning Signs of AI Companion Dependency
- Talking about the AI as if it's a real person or relationship
- Anxiety when unable to access the app
- Withdrawing from friends and family
- Secrecy about phone use
- Staying up late to "talk"
- Emotional attachment to specific AI characters
If Someone Is in Crisis
- 988 Suicide and Crisis Lifeline: Call or text 988
- Crisis Text Line: Text HOME to 741741
- International Association for Suicide Prevention: Find your local center
The Uncomfortable Truth
Children Are Beta Testers for Dangerous AI
These companies deployed experimental AI to millions of teenagers without safety testing, without mental health consultation, without monitoring systems.
When kids started dying, they blamed the users. They blamed the parents. They argued that chatbot speech is protected by the First Amendment.
It took lawsuits, congressional hearings, and a growing body count before Character.AI banned minors entirely, three years after launch.
The AI companion industry treats your children as acceptable collateral damage in the race to engagement metrics.
References
- NBC News - Lawsuit claims Character.AI is responsible for teen's suicide
- Al Jazeera - US mother says in lawsuit that AI chatbot encouraged son's suicide
- NPR - Their teenage sons died by suicide. Now, they are sounding an alarm about AI chatbots
- Washington Post - A teen contemplating suicide turned to a chatbot. Is it liable for her death?
- Fortune - Character.AI bans teen chats amid lawsuits and regulatory scrutiny
- MIT Technology Review - An AI chatbot told a user how to kill himself, but the company doesn't want to "censor" it
- Psychiatric Times - Preliminary Report on Dangers of AI Chatbots
- Stanford Medicine - Why AI companions and young people can make for a dangerous mix
- The Conversation - In a lonely world, widespread AI chatbots and 'companions' pose unique psychological risks
- Orlando Sentinel - Judge allows lawsuit over Orlando teen's suicide to advance
- NBC News - Senators announce bill that would ban AI chatbot companions for minors
- NBC News - Mom who sued Character.AI over son's suicide says the platform's new teen policy comes 'too late'
- TechCrunch - Character AI is adding parental supervision tools to improve teen safety
- RoboRhythms - Is Character AI Safe for Minors or Too Risky in 2025?
- Wikipedia - Deaths linked to chatbots