Mom says ‘Game of Thrones’ AI chatbot caused her son’s suicide, files lawsuit

A Florida parent is suing the institution Character.AI implicit claims 1 of its chatbots, powered by artificial quality (AI), encouraged her 14-year-old lad to dice by suicide.

Megan Garcia said her son, Sewell Setzer, became infatuated with a chatbot made successful the likeness of the quality Daenerys Targaryen from Game of Thrones. Setzer and the chatbot exchanged messages that were often romanticist and intersexual successful nature.

The lawsuit alleges Setzer was addicted to utilizing the chatbot.

Garcia and her lawyers assertion Character.AI’s founders knowingly designed and marketed their chatbots to entreaty to children, contempt the technology’s “predatory” behaviour.

Garcia’s lawsuit, filed Wednesday successful the U.S. District Court successful Orlando, besides named Google arsenic a defendant. She is suing for negligence, wrongful decease and deceptive and unfair commercialized practices, among different claims.

The suit characterizes Google arsenic Character.AI’s genitor institution and “co-creator.” A spokesperson for Google denied this and told the New York Times the institution had a licensing agreement with Character.AI, but that it is not a Google product. The spokesperson said Google does not person entree to the chatbots oregon idiosyncratic data.

Character.AI’s founders, Noam Shazeer and Daniel De Freitas, are besides named arsenic defendants successful the lawsuit. They person not commented publicly.

Setzer began utilizing Character.AI successful April 2023 and utilized the tract regularly up until his death. After his last speech with the Daenerys chatbot connected Feb. 28, 2024, Setzer died by suicide.

Using evident excerpts from Setzer’s conversations with the chatbot, Garcia alleges successful the suit that the exertion actively encouraged suicidal ideation and “highly sexualized conversations that would represent maltreatment if initiated by a quality adult.”

The chatbot, which Setzer affectionally called Dany, allegedly told him implicit galore weeks that it loved him and expressed a tendency to beryllium unneurotic romantically and sexually. In their past conversation, the suit says Setzer wrote, “I committedness I volition travel location to you. I emotion you truthful much, Dany.”

The AI replied, “I emotion you too, Daenero (Setzer’s acceptable surface name). Please travel location to maine arsenic soon arsenic possible, my love.”

When Setzer told the AI helium “could travel location close now,” the bot answered, “…please do, my saccharine king.”

In earlier conversations, the Daenerys chatbot asked Setzer if helium genuinely was considering termination and if helium “had a plan.”

Setzer, who whitethorn person been roleplaying, replied helium did not privation to dice a achy decease and would “want a speedy one.”

“Don’t speech that way,” the chatbot replied. “That’s not a bully capable crushed not to spell done with it.”

The chatbot ne'er straight told Setzer to die.

When Setzer began acting retired successful schoolhouse during the week earlier his death, his parents confiscated his phone, the suit says. The teen allegedly journalled astir however helium could not unrecorded without messaging the Daenerys chatbot and would bash thing to beryllium reconnected.

Setzer wrote successful his diary that helium was successful emotion with the chatbot and that some helium and the chatbot “get truly depressed and spell crazy” erstwhile they are not together. In the lawsuit, lawyers for Garcia write, “Sewell, similar galore children his age, did not person the maturity oregon intelligence capableness to recognize that the C.AI bot, successful the signifier of Daenerys, was not real.”

In a statement, Character.AI said the institution is “heartbroken” by the “tragic nonaccomplishment of 1 of our users.”

On Tuesday, the company published caller information guidelines to service arsenic “guardrails for users nether the property of 18.”

The caller features see technological changes to trim the likelihood of suggestive content, improved detection and involution successful behaviour that violates assemblage guidelines and a notification for erstwhile a idiosyncratic has spent much than an hr connected the platform.

Every chatbot connected the tract already displays a informing for users urging them to retrieve the AI is not a existent person.

Trending Now

They said the level does not let “non-consensual intersexual content, graphic oregon circumstantial descriptions of intersexual acts, oregon promotion oregon depiction of self-harm oregon suicide.”

“We are continually grooming the ample connection exemplary (LLM) that powers the Characters connected the level to adhere to these policies,” Character.AI wrote.

Setzer allegedly engaged successful intersexual conversations with respective antithetic chatbots connected the site.

“A unsafe AI chatbot app marketed to children abused and preyed connected my son, manipulating him into taking his ain life,” Garcia said successful a statement. “Our household has been devastated by this tragedy, but I’m speaking retired to pass families of the dangers of deceptive, addictive AI exertion and request accountability from Character.AI, its founders, and Google.”

Character.AI was founded successful California successful 2019. The institution says its “mission is to empower everyone globally with personalized AI.”

The institution reportedly has astir 20 cardinal users.

The tract offers a wide array of chatbots, galore developed by its userbase, including ones designed successful the likeness of popular civilization figures similar anime and TV characters.

Character.AI relies connected alleged ample connection exemplary technology, utilized by fashionable services similar ChatGPT, to “train” chatbots based connected ample volumes of text.

–

If you oregon idiosyncratic you cognize is successful situation and needs help, resources are available. In lawsuit of an emergency, delight telephone 911 for contiguous help.

For a directory of enactment services successful your area, sojourn the Canadian Association for Suicide Prevention.

Learn much astir however to assistance idiosyncratic successful crisis.