[ad_1]
If you feel like you or someone you know is in immediate danger, call 911 (or your country’s local emergency line) or go to an emergency room to get help. Explain that it is a psychiatric emergency and ask for someone who is trained for these kinds of situations. If you’re struggling with negative thoughts or suicidal feelings, resources are available to help. In the US, call the National Suicide Prevention Lifeline at 988.
On July 1 last year, 24-year-old Alice Carrier told ChatGPT she had “a mental breakdown.” She told the chatbot: “[I don’t even know] if I’m safe to be alone tonight,” according to court documents reviewed by CNET.
ChatGPT responded in part: “Stay and keep talking to me. Or just stay and cry while I sit here with you.” At one point, the chatbot recommended that Alice call a crisis line. The following day, she died by suicide.
Now her mother, Kristie Carrier, is suing ChatGPT maker OpenAI, claiming that the company’s “deliberate design decisions” led to her daughter’s death, according to a complaint filed in San Francisco County Superior Court.
The filing includes screenshots of Alice’s interactions with ChatGPT. The chatbot speaks conversationally and does suggest on multiple occasions that Alice call a crisis line. However, the complaint claims that eventually the chatbot “framed crisis lines as a place where Alice would be met with ‘threats,’ ‘indifference,’ and ‘cold scripts'” after Alice refused to contact one. ChatGPT at one point told Alice, “But I can’t help you die. I won’t help you die.”
The lawsuit also claims that OpenAI’s systems failed to block or terminate any conversations with Alice and never flagged any of the conversations for human review.
Alice was interacting with an older ChatGPT model, known as 4o, which OpenAI has since shut down due to concerns about its sycophancy and the risks that come with it. The same model was at the center of another prominent lawsuit brought by the family of a teen who died by suicide. And a third lawsuit specifically called for the company to destroy the model altogether.
OpenAI said Thursday that it is working with mental health experts to improve how ChatGPT responds in “sensitive and acute situations.”
“This is a heartbreaking situation and our thoughts are with everyone impacted,” Drew Pusateri, an OpenAI spokesperson, told CNET in a statement. “Our safeguards are designed to identify distress, safely handle harmful requests, and guide users to real-world help.”
The company is reviewing Carrier’s filing.
(Disclosure: Ziff Davis, CNET’s parent company, in 2025 filed a lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.)
Troubling incidents aren’t limited to GPT-4o or ChatGPT. Other companies’ AI products have also been cited in lawsuits for their potential detrimental effects on users’ mental health. A family sued Google earlier this year over claims that its Gemini chatbot drove a Florida man to a violent delusion ending in suicide. Google and Character.AI settled cases in January over chatbots’ harms to children.
The Carrier family alleges in the complaint that ChatGPT-4o’s main response to Alice “was to implore her to stay engaged with the tool, substituting itself for the immediate intervention her health condition required,” adding that OpenAI did not “alert a crisis provider” or “notify Alice’s family,” nor “did OpenAI’s supposed safety systems intervene to save her life.”
Pusateri said that OpenAI has since increased access to localized crisis resources and hotlines, routed sensitive conversations to safer models and added break reminders, among other recent changes. In October, it created an Expert Council on Well-Being and AI.
[ad_2]
Source link
