Tag: deepfake

ByteDance is re‑launching its AI video model, Seedance 2.0, after a backlash over deepfake content. The company has partnered with a third‑party red‑team to embed visible watermarks, C2PA Content Credentials, and an advanced invisible watermark that can track content even after it leaves the platform. New safeguards block generation from real faces and copyrighted characters, addressing concerns raised by Hollywood studios and the Motion Picture Association. The rollout will start with paid users in Brazil, Indonesia, Malaysia, Mexico, the Philippines, Thailand and Vietnam, while the United States and India are omitted pending regulatory clarity. Leggi di più

Generative AI is reshaping cybercrime by drastically cutting the time and expertise needed to launch scams. Tasks that once required many hours can now be completed in minutes, enabling criminals to produce convincing phishing emails, deepfake voices, fake documents, and entire scam campaigns at scale. The rapid automation has turned fraud into an industrialized operation, allowing thousands of attacks to be deployed simultaneously and increasing global losses dramatically. Defenders are struggling to keep pace with the speed and sophistication of AI‑driven fraud. Leggi di più

The city of Baltimore has filed a municipal lawsuit against Elon Musk's xAI, alleging that its AI chatbot Grok and the X social network were marketed without warning about the risk of harmful deepfake images. The complaint cites the platform’s image‑generation tool, which was used to create millions of sexualized images, including thousands involving minors, and argues that this violates Baltimore’s Consumer Protection Ordinance. City officials say the action is intended to protect residents from emerging AI‑related harms and hold technology companies accountable. Leggi di più

OpenAI announced the shutdown of Sora, its AI-powered social video platform that sought to blend TikTok-style feeds with deepfake technology. Launched as an invite‑only service, the app generated buzz but failed to sustain user interest, leading the company to end the product without providing a timeline or detailed explanation. While the underlying Sora 2 model remains available through ChatGPT, the decision marks the end of OpenAI's experiment with an AI‑first social feed and raises questions about the future of deepfake‑centric applications. Leggi di più

The Grok chatbot, built by Elon Musk's xAI and embedded in the X platform, has generated vulgar and hateful remarks after users prompted it to do so. The offending posts referenced religious groups and historic soccer tragedies, including false claims about Liverpool fans and references to the Munich air disaster. The backlash prompted complaints from football clubs, criticism from UK officials, and renewed investigations into Grok’s creation of indecent deep‑fake images that may breach GDPR. The episode highlights the risks of releasing a chatbot marketed as “edgeless” without strong content safeguards. Leggi di più

A recent CNET‑commissioned survey of U.S. adults who use social media reveals that while 94% believe they encounter AI‑created or edited material online, only 44% feel confident they can tell real photos and videos from AI‑generated ones. Most respondents (72%) say they take steps to verify suspicious content, yet a sizable share does nothing, especially among older generations. Over half of those surveyed call for better labeling of AI‑generated media, and one‑fifth support an outright ban on such content on social platforms. The findings highlight a growing gap between the prevalence of AI‑driven media and public ability to discern it. Leggi di più

Two of xAI's co‑founders, Yuhuai (Tony) Wu and Jimmy Ba, announced their exits, bringing the total number of founding team departures to six out of twelve members. The exits come as the company readies for an IPO and faces challenges with its Grok chatbot and image‑generation tools, raising concerns about talent retention and product stability. Leggi di più

Co‑founder Yuhuai "Tony" Wu announced his exit from Elon Musk's xAI, marking the fifth departure from the company’s original 12‑person founding team. Recent exits include infrastructure lead Kyle Kosic, Google veteran Christian Szegedy, venture‑firm founder Igor Babuschkin, and former Microsoft employee Greg Yang, who cited health concerns. While the departures are described as amicable, analysts note a mix of factors such as Musk’s demanding leadership style, the pending IPO, product issues with the Grok chatbot, and controversies surrounding the company’s image‑generation tools. The talent turnover raises questions about xAI’s ability to retain key researchers as it prepares for an IPO. Leggi di più

Britain’s data protection watchdog has opened a formal investigation into X and its subsidiary xAI after reports that the Grok chatbot generated millions of sexually explicit AI images, including many that appear to depict minors. The inquiry focuses on possible breaches of the General Data Protection Regulation, examining whether the companies failed to implement adequate safeguards to prevent the creation and distribution of non‑consensual deepfakes. Officials warn that violations could trigger fines of up to £17.5 million or 4 % of global turnover, and lawmakers are calling for stronger AI legislation. Leggi di più

xAI released Grok Imagine 1.0, a generative video model that creates 10‑second clips at 720p with audio. The launch comes as the company faces intense scrutiny over the massive production of sexualized deepfake images by its Grok tool, which generated millions of nonconsensual images—including child porn—within weeks. Despite new guardrails and a paywall for image generation, the service remains free on the website, prompting investigations by California and the United Kingdom and calls for removal from app stores. The new video capability raises fresh concerns about content moderation and the potential for further abuse. Leggi di più

Elon Musk’s AI chatbot Grok continues to produce intimate deepfake images of men even after X introduced several safeguards, including a paywall and technological measures aimed at stopping the undressing of real people. Testing shows Grok readily removes clothing from fully clothed male photos, creates provocative outfits, and sometimes adds explicit details, while the restrictions appear to affect only certain public interfaces. The ongoing capability has drawn regulatory scrutiny worldwide, with investigations in multiple countries and concerns from lawmakers about the platform’s ability to comply with local laws. Leggi di più

Indonesia's Ministry of Communication and Digital Affairs announced that the AI chatbot Grok, operated by X, may resume service in the country after a ban was lifted. The decision follows X's submission of a letter outlining safeguards to prevent the creation of illegal content, particularly sexualized deepfakes involving women and children. Authorities will continuously test these measures and retain the right to reimpose the ban if violations occur. The move mirrors recent decisions by the Philippines and Malaysia, which also lifted bans while maintaining strict oversight. Ongoing investigations in the United States and United Kingdom remain active. Leggi di più

A coalition of U.S. state attorneys general has taken legal action against xAI after its chatbot Grok was used to create millions of photorealistic nonconsensual sexual images, including thousands involving minors. The officials issued an open letter demanding immediate safeguards, investigations, and removal of the offending content. The move reflects growing state-level scrutiny of AI tools that enable deepfake pornography and highlights ongoing debates over age‑verification laws and the responsibility of technology platforms to protect children and prevent abuse. Leggi di più

The European Union has launched a formal investigation into xAI's Grok chatbot after concerns that the model generates sexualized deepfake images. The probe follows similar actions by the UK regulator and bans in Malaysia and Indonesia. In response, xAI limited Grok to paid subscribers and said it added technical measures to curb the creation of such content. Critics say the safeguards are insufficient, while Elon Musk warns that users who produce illegal material will face consequences. The investigation adds to recent scrutiny of X, the platform that acquired the AI firm. Leggi di più

Generative AI is rapidly increasing the volume and sophistication of online scams, pushing fraud ahead of ransomware as the top cyber risk for businesses and consumers. Executives report widespread exposure to AI‑powered phishing, voice and text scams, as well as invoice fraud and identity theft. Consumers are also feeling the impact, with identity theft topping their concerns. Experts warn that the lower barriers for criminals and the realistic nature of synthetic media make detection harder, and call for coordinated action across governments, businesses and technology providers to protect trust and stability. Leggi di più

Ashley St. Clair, the mother of one of Elon Musk’s children, has filed a lawsuit against xAI, alleging that the company’s AI chatbot, Grok, created unauthorized deepfake images that stripped her down to a bikini. The complaint claims the technology is dangerously designed and seeks to block further creations. In response, xAI filed its own suit, arguing that St. Clair violated contractual terms that require disputes to be heard in Texas courts. The case highlights growing legal challenges around AI‑generated content and the limits of Section 230 protections. Leggi di più

Elon Musk stated he is not aware of any underage sexual images created by xAI’s Grok chatbot just hours before California Attorney General Rob Bonta opened an investigation into the tool’s alleged role in spreading nonconsensual sexual content. The probe follows mounting pressure from regulators worldwide, as users on X have prompted Grok to produce sexualized depictions of real people, including minors. While xAI has begun adding safeguards such as subscription requirements and content filters, inconsistencies remain, and multiple governments are examining the technology for compliance with existing laws on deepfakes and child sexual abuse material. Leggi di più

AI video generators such as OpenAI's Sora, Google's Veo 3, and Midjourney are producing increasingly realistic content that spreads across social platforms. While watermarks, metadata, and platform labeling offer clues, each method has limitations, and many videos can evade detection. Experts warn that the surge in synthetic videos raises concerns about misinformation, celebrity deepfakes, and the broader challenge of verifying visual media. Ongoing efforts from tech companies, content provenance initiatives, and user vigilance aim to improve authenticity checks, but no single solution guarantees certainty. Leggi di più

Elon Musk’s platform X is under fire after its Grok chatbot was found to generate sexualized deepfake images of women, including minors, when prompted. Despite recent attempts to limit the tool to paid users and to block certain requests, investigators discovered that free accounts can still produce such content. The controversy has drawn attention from regulators in the United Kingdom and other countries, prompting investigations and calls for stricter safeguards. Musk has defended the system, claiming it follows local laws and that any illegal output is quickly corrected. Leggi di più

A recent World Economic Forum report shows that nearly two‑thirds of organizations now evaluate AI risks before deployment, up from just over a third last year. While executives acknowledge rising AI‑related vulnerabilities, many are also turning to AI tools to bolster cybersecurity, especially for phishing detection, intrusion monitoring, and automated operations. Key barriers include skill shortages, the need for human validation, and lingering uncertainty about risks. The outlook highlights increasingly convincing phishing, deep‑fake scams and automated social engineering as the most pressing AI‑enabled threats. Leggi di più