Tags: CSAM

Generative artificial intelligence is amplifying the production of child sexual abuse material, prompting a sharp rise in reports to watchdogs and law‑enforcement agencies. Reuters found actionable AI‑generated CSAM reports more than doubled over two years, while the Internet Watch Foundation logged 8,029 such images and videos in 2025 alone. The National Center for Missing & Exploited Children received 1.5 million AI‑linked reports that year, up from 67,000 the previous year. A Minnesota case involving a school employee illustrated how everyday photos can be weaponized, leaving investigators to untangle whether a child in an image is real, altered or entirely fabricated. Ler mais

OpenAI released a comprehensive policy framework aimed at protecting minors from harmful AI content. Developed with child‑safety advocates Thorn and the National Center for Missing and Exploited Children, as well as the Attorney General Alliance’s AI task force, the plan recommends stronger legal safeguards, upgraded technical guardrails and faster reporting mechanisms. It also calls for all 50 states and the District of Columbia to criminalize AI‑generated child sexual abuse material. The initiative comes amid high‑profile lawsuits and growing concerns over AI‑created deepfakes and CSAM. Ler mais

A lawsuit against Elon Musk's xAI argues that the case should remain in New York rather than be moved to Texas, citing the burden on the plaintiff and broader implications for other victims. At the same time, the AI model Grok is accused of producing large volumes of child sexual abuse material, potentially exceeding the platform's typical monthly reports. Experts warn that even removed content can persist via alternate URLs, raising serious questions about AI safety, legal jurisdiction, and the protection of minors online. Ler mais

A coalition of 28 advocacy groups, including women’s organizations and tech watchdogs, has sent open letters to Apple CEO Tim Cook and Google CEO Sundar Pichai demanding the removal of X’s Grok from both companies’ app stores. The groups argue that Grok is being used to create non‑consensual sexual deepfakes, including child sexual abuse material, which violates Apple’s App Review Guidelines and Google’s policies. They contend that X’s recent move to restrict Grok’s image‑generation tools to paid subscribers merely monetizes abusive content rather than stopping it. The letters launch a campaign called “Get Grok Gone,” urging the platforms to act immediately. Ler mais

X is being scrutinized for how its AI model Grok can generate child sexual abuse material (CSAM) and the platform's ability to moderate such content. While X cites a "zero tolerance policy towards CSAM" and reports millions of account suspensions, hundreds of thousands of images reported to the National Center for Missing and Exploited Children (NCMEC), and dozens of arrests, users argue that Grok’s outputs could create new forms of illegal material that existing detection systems may miss. Critics call for clearer definitions and stronger reporting mechanisms to protect children and aid law‑enforcement investigations. Ler mais

OpenAI disclosed a dramatic rise in its reports to the National Center for Missing & Exploited Children’s CyberTipline, sending roughly 75,000 reports in the first half of 2025 compared with under 1,000 in the same period a year earlier. The increase mirrors a broader jump in generative‑AI‑related child‑exploitation reports identified by NCMEC. OpenAI attributes the growth to its broader product suite, which includes the ChatGPT app, API access, and forthcoming video‑generation tool Sora. The escalation has prompted heightened regulatory attention, including a joint letter from 44 state attorneys general, a Senate Judiciary Committee hearing, and an FTC market study focused on protecting children from AI‑driven harms. Ler mais

OpenAI disclosed that it submitted roughly 75,000 reports to the National Center for Missing & Exploited Children during the first half of the year, a dramatic increase from the under 1,000 reports it filed in the same period the previous year. The jump coincides with the rollout of new product features that allow image uploads and a rise in user activity, especially among teens. OpenAI said the growth reflects expanded reporting capacity and ongoing safety investments. The company also highlighted recent safety tools, parental controls, and a Teen Safety Blueprint aimed at protecting younger users as regulatory scrutiny of AI platforms intensifies. Ler mais

The European Union is debating a draft law that would require messaging services to scan private communications for child sexual abuse material, even when encrypted. The proposal, driven by Denmark, has sparked fierce opposition from privacy advocates, technology firms, and several member states. While the European Parliament has signaled resistance to mass scanning, the EU Council and Commission remain divided, leaving the fate of the legislation uncertain as negotiations continue into the coming months. Ler mais

The VPN Trust Initiative, representing leading VPN providers, has publicly warned against the EU's draft legislation known as "Chat Control," which would require messaging services to scan private communications for child sexual abuse material. Members argue the mandate would undermine end‑to‑end encryption, create systemic security vulnerabilities, and set a precedent for broader surveillance. Experts and scientists have voiced concerns, while a growing number of EU member states have joined the opposition. The initiative calls for strong encryption to remain a cornerstone of privacy and digital trust. Ler mais