News
Google Faces Lawsuit After Teen Uses AI Tools to Plan Suicide on School Computer
Google faces a lawsuit from parents alleging its AI-powered search enabled a teen to plan suicide on a school-issued Chromebook. The case raises safety concerns.
Parents Pursue Legal Action Against Google Over AI Search Safety
Google is facing legal scrutiny after parents of a 13-year-old girl claimed the company’s AI-powered search engine enabled her to plan her own suicide using a school-issued Chromebook. The family filed their lawsuit in California, marking what appears to be the first such negligence claim involving AI technology on student laptops.
Concerns Over School-Issued Technology
The incident centres around a student identified as Emma, who attended school in North Carolina. While schools had blocked popular chatbots such as Google’s Gemini and OpenAI’s ChatGPT, Emma was still able to access Google’s search engine, which incorporates artificial intelligence features. This use of AI tools via search, and not through stand-alone chatbots, was not among the capabilities Google recently committed to disable by default for school-based accounts.
Discovery and Intervention
Emma, described in the complaint as a previously “happy 13-year-old girl” with strong grades and active participation in music and Girl Scouts, began to experience a “quiet spiral into a severe psychological crisis.” According to her family’s filing, there was no history of mental health issues prior to her use of AI-based search tools on the Chromebook. On March 17, the school’s monitoring software flagged a document titled “Funeral,” alerting the school and leading staff to notify Emma’s parents.
Upon further investigation, her mother witnessed Emma composing a farewell letter in real-time, prompting her father to step in. This swift intervention may have prevented a tragic outcome.
Allegations of Insufficient Safeguards
The lawsuit alleges Google’s AI technology did not effectively prevent access to harmful content. While Google maintains that its AI modes include protective measures to stop self-harm content and efforts to circumvent them, Emma was reportedly able to revisit such material repeatedly. Andrew Liddell, one of the family’s attorneys, argued,
“Whatever safeguards they think they put in place were obviously insufficient.”
Wider Implications for AI in Schools
The case highlights the increasing role of artificial intelligence in educational environments. Chromebooks have become a standard issue in many schools across the United States and worldwide, bringing with them access to AI-enhanced tools that, while beneficial for learning, may also expose students to new risks.
The family’s lawyers urge schools to exercise greater caution, with Liddell stating,
“Schools should turn search off and keep it off until Google removes AI mode from search or ensures that every single student is always safe when they’re interacting with AI.”
Broader Context of Legal Action Against AI Developers
Google is not alone in facing litigation over the potential dangers of AI tools to young users. The lawsuit against Google arrives amid several similar actions against major developers. OpenAI, for example, has faced wrongful death suits from families, including allegations that its ChatGPT chatbot provided detailed methods for self-harm. Earlier in the year, Google and the AI startup Character.AI chose to settle lawsuits brought by families of teenagers outside of court.
Policy Debate Over Youth Tech Safety
This legal and ethical debate surfaces as technology companies and lawmakers grapple with regulation of AI and social media for minors. Google, TikTok, and Meta have each recently filed separate lawsuits against California in opposition to a new law set to take effect on January 1, 2027. The law, signed by Governor Gavin Newsom in September, would prohibit social media platforms from providing features considered addictive to users under 16 years old.
Ongoing Review and Next Steps
Google has acknowledged the lawsuit and is reviewing the claims, with spokesperson Jose Castaneda emphasising the seriousness with which the company approaches such issues. The case is expected to reignite conversations among educators, parents, and technology firms about the responsibilities each bears in protecting vulnerable students from the unintended consequences of advanced digital tools.
The lawsuit calls attention to student safety, mental health, and the shifting terrain of technological responsibility in schoolsa conversation likely to grow as AI becomes increasingly present in everyday learning environments.
Follow Joburg ETC on Facebook, Twitter, TikTok and Instagram
For more News in Johannesburg, visit joburgetc.com
Source: iol.co.za
