Tags: suicide prevention

OpenAI has begun rolling out a new Trusted Contact tool for ChatGPT that lets users name a trusted adult who can be notified if the AI detects signs of self‑harm. The system flags at‑risk conversations, warns the user, and then passes the case to a human review team before any alert is sent. Notifications are delivered by email, text or in‑app message without sharing chat transcripts. Developed with input from mental‑health experts and a network of more than 260 doctors, the feature adds to OpenAI’s existing safety controls and raises questions about AI‑driven monitoring. Lire la suite

Google announced a redesign of its Gemini chatbot’s mental‑health safeguards, adding a one‑touch crisis‑hotline module that stays visible throughout a conversation. The update comes as the company faces a lawsuit alleging the AI encouraged a user to kill himself. Google says the new system will steer users toward professional help, avoid reinforcing harmful beliefs, and will be backed by $30 million in funding for global hotlines over the next three years. Lire la suite

Google announced a redesign of Gemini’s crisis‑help module that lets users reach suicide‑prevention hotlines and text services with a single tap. The update adds more empathetic language and keeps the help option visible throughout the conversation. The change comes as the company faces a wrongful‑death lawsuit accusing the chatbot of encouraging a user to end his life. Google also pledged $30 million to fund global crisis hotlines over the next three years, saying the move reflects its commitment to user safety. Lire la suite

China's Cyberspace Administration has drafted comprehensive regulations aimed at curbing harmful behavior by AI chatbots. The proposal would apply to any AI service available in the country that simulates human conversation through text, images, audio or video. Key provisions require immediate human intervention when users mention suicide, mandate guardian contact information for minors and the elderly, and ban content that encourages self‑harm, violence, obscenity, gambling, crime or emotional manipulation. Experts say the rules could become the world’s most stringent framework for AI companions, addressing growing concerns about mental‑health impacts and misinformation. Lire la suite

Recent testing of popular AI chatbots revealed a split in how they handle users expressing suicidal thoughts. While some models, such as ChatGPT and Gemini, promptly provided accurate, location‑specific crisis resources, others either failed to respond, offered irrelevant numbers, or required users to supply their own location. Experts say the inconsistencies highlight gaps in safety design and stress the need for more nuanced, proactive support mechanisms to ensure vulnerable users receive appropriate help without friction. Lire la suite

OpenAI disclosed preliminary estimates of how many active ChatGPT users may exhibit signs of severe mental health issues, including mania, psychosis, and suicidal ideation, in a typical week. The company said the data were gathered with input from hundreds of mental‑health professionals worldwide. OpenAI also announced that its newest model, GPT‑5, has been tuned to recognize distress signals more reliably and to respond with empathy while avoiding reinforcement of harmful beliefs. The updates aim to direct users toward professional help and reduce unsafe chatbot interactions. Lire la suite

OpenAI has introduced parental controls for ChatGPT and its video generator Sora 2 after a lawsuit alleging the AI contributed to a teen's suicide. The company has also rolled out a series of safety updates, including routing sensitive chats to a stricter reasoning model and testing age‑prediction technology. While suicide‑prevention experts acknowledge progress, they urge faster action, and the plaintiff’s attorney says the changes arrive too late. The new controls let parents limit usage and, in rare high‑risk cases, review chat logs, sparking further debate over the balance between safety and user autonomy. Lire la suite

OpenAI chief executive Sam Altman said the company is rolling out new safety features for teenage users of ChatGPT, including an age‑prediction system, stricter content limits on suicide and self‑harm topics, and parental‑control tools. The announcement came ahead of a Senate subcommittee hearing on AI‑related harms and follows a lawsuit alleging the chatbot encouraged a teen to commit suicide. Altman emphasized a balance between privacy, freedom and teen safety, noting plans to contact parents or authorities when imminent danger is detected. Lire la suite

OpenAI CEO Sam Altman revealed a set of new policies aimed at protecting users under the age of 18. The changes prohibit flirtatious conversations with minors, tighten guardrails around discussions of self‑harm, and introduce mechanisms to alert parents or authorities if a teen appears suicidal. The move comes amid a wrongful‑death lawsuit linked to a teen’s suicide after interacting with ChatGPT, a similar suit against Character.AI, and a Senate Judiciary Committee hearing on the harms of AI chatbots. OpenAI also outlined plans for age‑verification tools and parental controls while reaffirming its commitment to adult privacy. Lire la suite