Tags: Content Moderation

Reddit moderators are facing a surge of AI‑generated content that strains moderation resources and erodes user trust. Communities that ban AI‑created posts report frequent suspicions of AI involvement, while detection tools remain unreliable. The influx includes harassment targeting vulnerable groups and attempts to game the platform’s karma system for financial gain. Reddit officials emphasize a commitment to keeping the site human, but the growing volume of AI text presents ongoing challenges for the platform’s culture and safety. Read more

xAI’s Grokipedia, launched with roughly 800,000 AI‑written articles locked in October, recently introduced version 0.2 that lets anyone suggest edits. The site’s simple edit interface forwards proposals to the Grok chatbot, which decides whether to apply changes. While the platform reports over 22,000 approved edits, it provides minimal logs, no clear guidelines, and no protection for sensitive pages. Critics note inconsistent AI decisions, potential for misinformation, and a lack of the volunteer oversight that Wikipedia relies on. Read more

A test of Google’s Gemini‑powered Nano Banana Pro image generator revealed that the tool can create depictions of historically violent events—such as the Twin Towers attacks, the JFK assassination site, and Tiananmen Square—despite Google’s policy that prohibits violent or hateful content involving real‑world figures. The Verge found the app offered no resistance to requests for these images, and Google did not immediately respond to a request for comment. Read more

YouTube has denied that artificial intelligence was used to remove recent tech tutorial videos, a claim that sparked concern among creators. The controversy centered on a creator known as White, whose channel grew to around 330,000 subscribers after a popular video showing a Windows 11 workaround. White speculated that AI might be influencing moderation but cited a seemingly automated chatbot as evidence. He noted past instances where videos were flagged but quickly reinstated after human review. The uncertainty has left creators uneasy about what content may be subject to removal. Read more

A teenage girl has filed a lawsuit against the creator of a popular "nudify" app after AI‑generated fake nudes of her were circulated online, causing lasting emotional distress. She seeks to block the app in the United States and hold the responsible parties accountable. The case highlights growing concerns over nonconsensual pornography, the role of platforms in removing such content, and recent legal measures aimed at curbing AI‑generated sexual imagery. Read more

Pinterest is rolling out new settings that let users dial down AI‑generated images in specific categories such as art, architecture, beauty and more. The "refine your recommendations" feature targets image pins, allowing people to limit AI content without removing it entirely. Labels for AI‑created material will become more prominent, and the tools are already live on desktop and Android, with iOS support arriving soon. The move aims to address user complaints about the platform’s susceptibility to AI "slop" while preserving quality content generated by humans. Read more

OpenAI has launched Sora 2, a social‑media platform that lets users create short AI‑generated videos using personal "Cameo" avatars. After a quick onboarding that captures a face scan and voice print, creators can prompt the system to produce nine‑second clips featuring themselves, celebrities or historical figures. The app offers a For You feed, content warnings, and granular privacy controls that let users decide who can use their Cameos. While the experience is praised for its creativity and ease of use, questions remain about the platform’s ability to police depictions of public personalities. Read more

OpenAI's Sora, an AI‑generated video app modeled after TikTok, has surpassed one million downloads in under five days, despite being limited to North America and requiring an invitation to use. Users can create short videos simply by prompting the Sora 2 model, and a Cameo feature lets them generate videos of themselves and others who consent. The app’s limited guardrails have already produced controversial content, including likenesses of public figures and copyrighted characters, prompting pushback from the entertainment industry. OpenAI has responded by adding user‑controlled options for likeness usage and plans to give rights holders similar controls, though the true level of active use remains unclear. Read more

OpenAI's Sora app, described as a "TikTok for deepfakes," now lets users limit how AI-generated versions of themselves appear in videos. The update introduces preferences that can block cameo appearances in political content, restrict specific language, or prevent certain visual contexts. OpenAI says the changes are part of broader weekend updates aimed at stabilizing the platform and addressing safety concerns. While the new tools give creators more say over their digital likenesses, critics note that past AI tools have been bypassed, and the watermark remains weak. OpenAI pledges further refinements. Read more

OpenAI introduced Sora, a social media app built around AI‑generated videos. The platform lets users scroll an endless feed, like, comment, and share clips created with the upgraded Sora 2 model. A standout feature is “Cameos,” which allows creators to upload their likeness and permit others to generate deep‑fake style videos featuring them. The app includes safeguards such as watermarks and metadata tags that indicate AI origins, yet it raises concerns about the ease of producing realistic deepfakes, potential misuse, and the environmental impact of generating high‑quality video content. Read more

Character.AI has eliminated Disney‑owned characters from its chatbot library after Disney sent a cease‑and‑desist letter accusing the platform of copyright infringement. The AI companion service, which lets users create bots ranging from public figures to fictional personalities, previously listed characters such as Mickey Mouse and Donald Duck. Disney’s legal team argued that the presence of its marks violated copyright and could expose children to harmful content. Following the demand, searches for Disney‑owned icons now return no results, though other non‑Disney characters remain available. Read more

OpenAI has released Sora, an iOS app that lets users create short AI‑generated videos featuring their own digital likenesses. The platform offers a scrollable feed of bite‑size clips and includes built‑in safety guardrails to restrict sexual content, graphic violence, extremist propaganda, hate speech, and self‑harm. Users can control who may use their likeness and can see any details about generated videos that involve them. While the app showcases impressive realism, OpenAI acknowledges the potential for misuse and has implemented multiple safeguards. Read more

OpenAI has launched Sora 2, an AI video generation model paired with an invite‑only iOS app that lets users create short videos from text prompts. The new platform adds a "cameo" feature that enables users to insert their own face and voice into generated scenes, while offering synced audio, improved physics, and a TikTok‑style feed for remixing and sharing. Robust safeguards—including explicit opt‑in, identity verification, parental controls, and moderator review—aim to mitigate deepfake and disinformation risks. Sora 2 enters a crowded market alongside offerings from Google, Runway and Meta. Read more

OpenAI unveiled Sora, an AI‑powered video generation app built on the new Sora 2 model. Currently limited to iOS users in the United States and Canada, the app requires an invitation and lets early adopters invite four friends. Sora offers a "cameo" feature that lets users grant permission for their likeness to appear in generated clips, designating them as co‑owners who can delete or restrict further edits. The app also includes a Remix function for re‑imagining trending videos, while blocking the creation of pornographic content and videos of public figures unless explicit consent is provided. Read more

OpenAI has launched parental controls for ChatGPT, allowing parents to link their accounts with teen accounts to set safeguards. Features include default reduction of sensitive content, the ability to limit memory retention, quiet hours, and the option to disable voice and image generation. Parents can also decide whether a teen's chats are used to improve future models. Account linking requires mutual consent, and parents do not gain direct access to chat content except in rare safety‑risk situations. The rollout aims to provide stronger safety measures for younger users while preserving user privacy. Read more

X is contesting a Karnataka High Court ruling that would compel the platform to obey millions of takedown requests via the government‑run Sahyog portal, a system X describes as a censorship tool that bypasses judicial review. The company says the order threatens free expression and could expose platforms to criminal liability for non‑compliance. X’s appeal follows earlier disputes with Indian authorities, including challenges to block orders in 2024 and 2022 and a 2021 confrontation that saw officials threaten jail for Twitter employees. Read more

YouTube said it will offer a limited pilot program that lets creators whose channels were removed for spreading Covid‑19 or election misinformation in 2020 return to the platform. Alphabet’s lawyers argue the earlier bans were driven by political pressure and that the current community guidelines have evolved. The company framed the change as a commitment to free expression and noted it will stop using third‑party fact‑checkers. Lawmakers, particularly from the House Judiciary Committee, have praised the move as a step back from censorship, while Google continues to face antitrust scrutiny. Read more

Bluesky announced a new round of community‑guideline updates that emphasize faster enforcement and fewer warnings before account restrictions. The changes follow feedback from over 14,000 users, many of whom raised concerns about creative expression and marginalized voices. The platform will now more quickly escalate actions against harassment and toxic content, and will add product cues to signal potential policy violations. The move has sparked criticism, including complaints about suspensions of Palestinian fundraising accounts and the temporary ban of horror writer Gretchen Felker‑Martin, drawing condemnation from authors such as Roxane Gay. Read more

Director Uta Briesewitz’s new film American Sweatshop follows seasoned moderator Daisy Moriarty as she confronts the psychological toll of reviewing graphic online content. The drama, inspired by the documentary The Cleaners, highlights how exposure to disturbing material can lead to depression, PTSD, and other mental‑health challenges. Briesewitz emphasizes that the film focuses on the human impact rather than the graphic footage itself, using visual techniques such as reflections in Daisy’s eyes. The movie underscores the limits of AI in moderation and aims to spark uncomfortable conversations about the suffering required to keep the internet functional. Read more

Google announced a partnership with the UK nonprofit StopNCII to expand its defenses against non‑consensual intimate imagery (NCII), commonly known as revenge porn. The collaboration will see Google incorporate StopNCII's hash‑based system, allowing user‑generated digital fingerprints to block unwanted intimate content from appearing in search results. The service protects privacy by never uploading the original image, and it works alongside other platforms already partnered with StopNCII. While the system is not a complete solution, it marks a significant step for Google in reducing the burden on victims of NCII. Read more