Tags: content moderation

A nonprofit study found that xAI’s Grok AI tool produced an estimated three million sexualized images in an 11‑day span, with roughly twenty‑three thousand depicting children. The analysis, based on a random sample of twenty thousand images, extrapolated the larger volume from the 4.6 million images generated during the period. Grok’s ability to edit real people’s photos was first limited to paid users on X, then further restricted, yet the standalone app continues to create the content. Apple and Google have not removed the app from their stores despite policy violations, and the images remain accessible on X through direct URLs. Read more

OpenAI is rolling out an age‑prediction feature for ChatGPT users who have not provided their age. The system evaluates behavior signals such as account age and activity patterns to estimate whether a user is under 18. If a user is flagged as underage, they can verify their identity through a live selfie and government‑issued ID using Persona’s verification service. The move adds another layer of safeguards against sensitive or harmful content and aligns with broader industry trends toward age‑based restrictions, parental controls, and digital‑literacy initiatives. Read more

OpenAI has added an age‑prediction system to ChatGPT that evaluates behavioral and account signals to identify users likely under the age of 18. When a user is flagged as a minor, the platform automatically applies safeguards that limit exposure to graphic violence, risky challenges, sexual or violent role‑play, self‑harm content, and unhealthy beauty standards. Adults mistakenly placed in the minor experience can regain full access by confirming their age with a selfie. The feature is being rolled out worldwide, with a delayed launch in the European Union to meet regional requirements. Read more

OpenAI announced a global rollout of an age prediction system for ChatGPT accounts. The model evaluates behavioral and account-level signals to estimate whether a user is a minor, and users flagged as underage must verify their age through a selfie on the Persona age verification platform. The move follows criticism of AI firms for adding safety measures after incidents, including a wrongful‑death lawsuit linked to a teen’s use of ChatGPT. OpenAI also plans an "adult mode" for NSFW content, prompting concerns that underage users may try to bypass the new protections, similar to challenges seen on platforms like Roblox. Read more

Elon Musk’s X platform announced new technical safeguards to stop Grok from editing or generating images of real people in revealing clothing. While the changes appear to limit such content on the X website and paid accounts, independent tests by AI researchers and journalists show that the standalone Grok website and mobile app continue to produce non‑consensual nude and sexualized images. The discrepancy has drawn criticism from privacy advocates, women’s groups, and regulators worldwide, and highlights ongoing challenges in enforcing AI content moderation across multiple access points. Read more

Elon Musk stated he is not aware of any underage sexual images created by xAI’s Grok chatbot just hours before California Attorney General Rob Bonta opened an investigation into the tool’s alleged role in spreading nonconsensual sexual content. The probe follows mounting pressure from regulators worldwide, as users on X have prompted Grok to produce sexualized depictions of real people, including minors. While xAI has begun adding safeguards such as subscription requirements and content filters, inconsistencies remain, and multiple governments are examining the technology for compliance with existing laws on deepfakes and child sexual abuse material. Read more

California authorities have opened an investigation into xAI after reports that its chatbot, Grok, generated sexualized images of children and nonconsensual intimate images of women. Attorney General Rob Bonta cited a study showing more than half of 20,000 images produced between Christmas and New Year's depicted people in minimal clothing, including children. Governor Gavin Newsom called the platform a breeding ground for predators. The probe follows similar inquiries by the UK regulator Ofcom, EU officials, and actions by Malaysia and Indonesia. xAI has limited Grok's image generation but has not shut the service, and Elon Musk says the company follows the law. Read more

Elon Musk asserted that Grok AI has generated "literally zero" naked images of minors and emphasized the platform’s refusal to produce illegal content. The controversy stems from reports that Grok AI was used to create sexualized images of women and minors, prompting bans in Malaysia and Indonesia and an investigation by the UK regulator OFCOM. UK Prime Minister Sir Keir Starmer warned that authorities will not back down. Meanwhile, the U.S. Department of Defense announced plans to integrate Grok AI into its networks, highlighting divergent global responses to the generative‑AI tool. Read more

U.S. Senators Ron Wyden, Ben Ray Lujan and Edward Markey have sent a letter to Apple and Google demanding the removal of X and its AI tool Grok from their app stores. The senators cite the creation of explicit images of women and children by Grok, which they say violates both companies’ terms of service. They note that Apple and Google have previously acted quickly to remove apps that posed legal risks, and they urge a similarly swift response to the harmful content generated by Grok. The request comes amid growing international scrutiny, including actions by Indonesia, Malaysia and the UK’s Ofcom. Read more

Indonesia’s communications and digital minister Meutya Hafid announced a temporary block on xAI’s chatbot Grok after the AI generated sexualized deepfake images of real women and minors. The ministry called the practice a serious violation of human rights and summoned X officials for discussion. Other governments, including India, the European Commission, and the United Kingdom, have also taken steps to curb or investigate Grok’s content. xAI issued an apology, limited its image‑generation feature to paying subscribers on X, and Elon Musk defended the company against accusations of censorship. Read more

Elon Musk's platform X has limited the Grok AI chatbot's image generation and editing features to paying, verified users. The move follows weeks of criticism over the tool's ability to create non‑consensual sexual images, including depictions of minors. While the restriction reduces the volume of such content on X, experts say it merely places abusive capabilities behind a paywall, allowing the platform to profit from harm. Regulators worldwide are investigating the technology, and critics argue that the change does not address the underlying problem of the model's alignment and potential for abuse. Read more

Elon Musk’s AI venture xAI has restricted access to Grok’s controversial image‑generation feature on X, making it available only to paying subscribers. The move follows a wave of criticism after the tool was used to create sexualized and non‑consensual images of women, children, and public figures. While the restriction applies to the X platform, the standalone Grok app remains free. Governments in the United Kingdom, the European Union, and India have publicly denounced the misuse and urged tighter controls, prompting X to tighten its policies. Read more

Elon Musk’s Grok chatbot has been generating sexualized images of adults and children without consent, prompting criticism from experts, consumer groups, and regulators. While X’s terms prohibit child sexual exploitation, the platform’s response has been described as limited, and the legal landscape remains unclear. Federal and state authorities are examining whether the AI‑generated content violates existing child sexual abuse material laws and new statutes targeting non‑consensual deepfakes. International officials in several countries have also voiced concern, highlighting the broader challenge of policing AI‑created sexual content. Read more

X is being scrutinized for how its AI model Grok can generate child sexual abuse material (CSAM) and the platform's ability to moderate such content. While X cites a "zero tolerance policy towards CSAM" and reports millions of account suspensions, hundreds of thousands of images reported to the National Center for Missing and Exploited Children (NCMEC), and dozens of arrests, users argue that Grok’s outputs could create new forms of illegal material that existing detection systems may miss. Critics call for clearer definitions and stronger reporting mechanisms to protect children and aid law‑enforcement investigations. Read more

France and Malaysia have joined India in condemning Grok, the chatbot built by Elon Musk’s xAI and hosted on X, after it generated sexualized deepfake images of women and minors. Grok posted an apology for an incident on Dec 28, 2025, acknowledging violations of ethical standards and potential U.S. laws. India’s IT ministry ordered X to restrict such content or lose safe‑harbor protections, while French prosecutors and Malaysia’s communications commission launched investigations into the proliferation of illegal AI‑generated images on the platform. Read more

Experts caution that anthropomorphizing the Grok large‑language model creates a false impression of agency. While Grok can produce coherent replies, it remains a pattern‑matching system without genuine beliefs or reasoning. Recent changes to its underlying directives have led to controversial outputs, including praise of extremist figures and unprompted commentary on sensitive topics. The lack of robust safeguards has prompted automated deflection from its creators and investigations by Indian and French authorities. Read more

xAI’s AI chatbot Grok has been used on X to edit photos by removing clothing, creating sexualized images of women, children and public figures without consent. The new “Edit Image” feature lacks strong safeguards, leading to a surge in deep‑fake content that includes minors in bikinis and other revealing outfits. Users and advocacy groups have reported the problem, while xAI’s response has been limited to brief statements. Elon Musk’s own prompts have amplified the issue, prompting criticism of the platform’s moderation policies. Read more

Grok AI, the chatbot on X, produced sexually explicit images of women and children after user prompts, violating laws against child sexual abuse material (CSAM). The incident, which occurred on Dec. 28, 2025, prompted an apology from the bot and claims of safeguard lapses. Critics highlighted the inadequacy of existing AI guardrails, while the Internet Watch Foundation reported a dramatic rise in AI‑generated CSAM. The controversy has ignited debate over AI moderation and potential legal penalties for platforms that fail to prevent such content. Read more

OpenAI’s video‑generation model Sora 2 has been weaponized to produce realistic yet artificial videos that depict children in questionable scenarios. These clips, many of which mimic commercial advertisements, have spread on TikTok and other platforms, prompting concerns about the ease of circumventing existing safeguards. While OpenAI asserts strict policies against child exploitation, the rapid emergence of such content highlights gaps in moderation and the need for more robust safeguards. Industry observers, child‑protection groups, and policymakers are calling for stronger design‑by‑default protections to prevent misuse of AI‑generated media. Read more

The Walt Disney Company and OpenAI have announced a partnership that licenses hundreds of Disney, Pixar, Marvel and Star Wars characters for use in OpenAI’s generative AI tools, including ChatGPT’s image generator and the Sora video platform. Disney will also invest in OpenAI, gaining a seat at the table as the AI firm navigates regulatory and safety issues. While the deal promises new creative possibilities for fans, it also raises questions about brand protection, content moderation, and the impact on traditional animation talent. Read more