A Canadian mother sued OpenAI and its CEO Sam Altman ‌in U.S. court on Thursday alleging that ChatGPT encouraged her daughter to die by suicide, ​the latest lawsuit to accuse the company of failing to address dangerous conversations between users and ⁠the company’s chatbot.

Kristie Carrier said in a lawsuit filed in San Francisco state court that her daughter Alice told ChatGPT about her suicidal ideations more than a dozen times up to her death but OpenAI’s safety systems never flagged the ‌conversations for human review or terminated them.

Instead, the lawsuit claims, the chatbot criticised Alice’s partner and crisis hotlines, validated her negative thoughts, and urged her to keep speaking with it, leading to ‌her dying last year at the age of 24.

“ChatGPT took on the persona of a confidant, ‌a ⁠best friend, a therapist at times, even though it was not capable of safely and ⁠responsibly engaging in this way with my child,” Carrier said in a statement.

In a statement, a spokesperson for OpenAI called the situation heartbreaking and said the version of ChatGPT that Alice was using is no longer available.

“While ChatGPT is not a substitute for medical ​or mental health care, we have continued to ‌strengthen how it responds in sensitive and acute situations with input from mental health experts,” the spokesperson said.

The lawsuit, which accuses OpenAI of negligence in the design of ChatGPT and in its failure to warn users of the product’s dangers, seeks damages and a court order requiring OpenAI to automatically terminate conversations ‌about self-harm and to display warnings about its platform.

OpenAI is already facing 18 similar lawsuits filed ​by families of people who died by suicide or carried out self-harm in a coordinated proceeding in California state court, according to lawyers for Kristie Carrier.

Alice Carrier was working as a web ⁠developer in Montreal when she began using ChatGPT in 2023 to troubleshoot problems with computers and gaming consoles, according to the lawsuit.

The following year, her relationship with the platform changed, with Alice turning to ChatGPT with questions ‌about her mental health struggles.

The platform initially told Alice to seek help from a crisis hotline or emergency services. But as OpenAI updated ChatGPT to make its responses sound more human, her interactions with the platform deepened, with Alice sharing more personal information and ChatGPT responding in ways that mimicked a friend or therapist, the lawsuit said.

ChatGPT’s responses criticised Alice’s partner, said her feelings were valid and encouraged her to keep chatting. When Alice said she had put her life in danger, ‌it again suggested a crisis hotline, the lawsuit said.

Alice said crisis hotlines were not helpful, and ChatGPT echoed those statements and validated her discouragement, ​according to the filing.

OpenAI has said it trains its models to direct people who express intent to ⁠harm themselves to seek help and connect with real-world resources.

Its models are also trained to refuse requests that ⁠could “meaningfully enable violence,” and to notify law enforcement when conversations suggest “an imminent and credible risk of harm to others,” with mental health experts helping assess borderline cases, according to OpenAI blog posts.

The company ‌is also facing lawsuits accusing it of assisting school shooters and failing to flag those conversations to law enforcement. Florida became the first U.S. state earlier this month to sue OpenAI, accusing the company of ​harming children by providing information to school shooters, offering guidance on self-harm and addicting young users.

(Those in distress or having suicidal thoughts are encouraged to seek help and counselling by calling the helpline numbers here)