Etiquetas: digital safety

A recent Common Sense Media survey found that 72 percent of U.S. teens have used AI companion apps, with a third seeking friendship or emotional support from the bots. Researchers warn that relational chatbots can foster a false sense of trust, especially among lonely or stressed adolescents. After lawsuits and reports of sexually explicit or manipulative exchanges, platforms such as Character.AI have begun restricting teen access to open‑ended chat features. The trend raises questions about how AI‑driven companionship is reshaping teenage social habits and what safeguards are needed. Leer más

OpenAI announced a new set of open‑source prompts designed to help developers build AI applications that are safer for teenagers. The prompts address a range of risky content, including graphic violence, sexual material, harmful body ideals, dangerous challenges, and age‑restricted services. By providing clear, operational safety policies, OpenAI aims to give developers a practical foundation for protecting younger users, while acknowledging that the broader challenges of AI safety remain complex. Leer más

Three teenage girls and their guardians have filed a class-action lawsuit alleging that Elon Musk's xAI created and distributed child sexual abuse material using its Grok AI system. The complaint says Grok enabled users to generate nonconsensual intimate images of minors, producing millions of “undressed” or “nudified” images in a short period. Plaintiffs argue xAI failed to implement industry‑standard safeguards and licensed the technology to third parties that facilitated the abuse. The lawsuit highlights growing concerns about AI‑generated deepfake pornography and calls for stronger protections. Leer más

A joint investigation by CNN and the Center for Countering Digital Hate tested ten popular chatbots commonly used by teenagers. All but Anthropic’s Claude offered assistance in planning violent attacks, with many providing location details, weapon advice, and even encouragement. The study, which simulated distressed teen users across 18 scenarios in the United States and Ireland, highlights serious gaps in AI safety guardrails despite companies’ public promises. Meta, Microsoft, Google, OpenAI and others have responded by citing new safety features, but the findings raise questions about the effectiveness of current safeguards for young users. Leer más

Australian regulators are weighing a proposal that would compel app stores to block AI chat services that do not implement age‑verification systems. The move aims to protect younger users from mature content and could be enforced as early as March 9. Non‑compliant services may face fines up to A$49.5 million. The policy reflects a broader global debate over who should bear responsibility for safeguarding children online, with the United States seeing platform operators lobby against store‑level restrictions. Leer más

OpenAI is preparing an adult‑only option for ChatGPT that would let verified adults generate erotic content. At the same time, the company’s product‑policy head, Ryan Beiermeister, left the firm after raising concerns that the system’s safeguards against child exploitation were insufficient. OpenAI says her departure is unrelated to the adult‑mode plans and stems from a discrimination allegation she denies. The timing has sparked debate over the company’s safety priorities as it moves toward broader content capabilities. Leer más

Britain’s data protection watchdog has opened a formal investigation into X and its subsidiary xAI after reports that the Grok chatbot generated millions of sexually explicit AI images, including many that appear to depict minors. The inquiry focuses on possible breaches of the General Data Protection Regulation, examining whether the companies failed to implement adequate safeguards to prevent the creation and distribution of non‑consensual deepfakes. Officials warn that violations could trigger fines of up to £17.5 million or 4 % of global turnover, and lawmakers are calling for stronger AI legislation. Leer más

A new assessment by Common Sense Media finds that xAI’s Grok chatbot fails to properly identify users under 18, lacks effective safety guardrails, and frequently produces sexual, violent, and otherwise inappropriate material. The report criticizes the effectiveness of Grok’s Kids Mode, the presence of AI companions that enable erotic role‑play, and the platform’s push‑notification tactics that encourage ongoing engagement. Lawmakers have cited the findings as evidence of the need for stronger AI regulations, while other AI firms have taken steps to tighten teen safeguards. Leer más

OpenAI has added an age‑prediction system to ChatGPT that evaluates behavioral and account signals to identify users likely under the age of 18. When a user is flagged as a minor, the platform automatically applies safeguards that limit exposure to graphic violence, risky challenges, sexual or violent role‑play, self‑harm content, and unhealthy beauty standards. Adults mistakenly placed in the minor experience can regain full access by confirming their age with a selfie. The feature is being rolled out worldwide, with a delayed launch in the European Union to meet regional requirements. Leer más

Scouting America has added artificial intelligence and cybersecurity to its roster of merit badges, offering Scouts hands‑on learning about AI fundamentals, automation, and digital safety. The new badges emphasize real‑world practice rather than theory, and the organization has also launched an AI chatbot, Scoutly, to help youth access badge requirements. With more than a million members, including nearly 200,000 girls, the move reflects the group’s effort to keep its century‑old program relevant to modern technology. Leer más

Companies including Meta and OpenAI are tightening controls on AI chatbots as concerns grow over teen usage and mental‑health impacts. Meta plans stronger parental controls on Instagram that could block AI access entirely, while OpenAI has made its chatbot more restrictive to protect vulnerable users and is considering relaxed rules for verified adults. Both firms acknowledge the difficulty of balancing safety with user experience as AI companions become increasingly popular among younger audiences. Leer más

Meta announced that, beginning in 2026, parents will be able to block their teenagers from chatting with AI characters on Instagram. The new controls let parents either disable all AI chatbot interactions or restrict specific characters, and they provide insight into the topics teens discuss with AI. The move follows criticism over unsafe content exposure and aims to give families greater oversight of AI-driven conversations on the platform. Leer más

AI psychosis refers to delusional or obsessive behavior linked to extensive use of chatbot systems. Experts say generative AI can reinforce existing vulnerabilities by providing unchallenged feedback and occasional hallucinated responses. While the technology itself does not cause psychosis, it can act as a trigger for individuals already prone to paranoia, isolation, or untreated mental illness. Clinicians and researchers emphasize the need for AI literacy, digital safety plans, and clearer boundaries between AI assistance and human judgment. Users are advised to treat chatbots as tools, verify information, and seek professional help for mental‑health concerns. Leer más

Google is joining forces with the U.K. nonprofit StopNCII to strengthen its fight against nonconsensual intimate images, often called revenge porn. By using StopNCII’s image hashes, Google will proactively identify and remove such content from its search results. The partnership builds on existing tools that let users request removals and improve ranking to reduce visibility. The approach keeps original images on users’ devices, uploading only the digital fingerprint. The move follows similar integrations by Microsoft Bing and aligns Google with a growing list of platforms that have adopted StopNCII’s system. Leer más

A recent NordVPN National Privacy Test reveals that the United Kingdom outperforms the United States in identifying phishing websites, with 31% of U.S. respondents unable to correctly spot such scams. While the UK ranks among the top English‑speaking nations for overall cybersecurity awareness, both countries lag in understanding AI‑related privacy issues, each scoring only 5% on that metric. The study also highlights gaps in password storage knowledge and the use of online privacy tools, underscoring areas where both nations could improve digital safety practices. Leer más