Anthropic, a leading AI company, has flagged and stopped multiple scientists who were using its AI models to create potential biological weapons, according to a new report. The company's report includes five case studies of Anthropic's models being used to develop biological weapons, the safety measures that brought the misuse to Anthropic's attention, and how it responded.
Detecting possible misuse is complicated, as the company stressed, because research into a new vaccine could look a lot like creating a bioweapon. In all cases, the company says it erred on the side of being overly cautious because of the possible consequences. "You are not seeing someone in a comic book kind of way say, 'Hey, I want to build a biological weapon to kill everybody,'" Jacob Klein, Anthropic's head of threat intelligence, told The New York Times. "It's an incredibly nuanced situation."
In one case study from May, Anthropic says its "biological safety classifier" flagged a request for Claude to author a grant for gain-of-function research on chikungunya virus. The virus has no licensed treatment and can cause debilitating symptoms for weeks or months, the company says. Gain-of-function research explores methods for genetically altering an organism, and in the case of this specific virus, the grant proposed increasing its transmissibility and ability to evade immune response.
Anthropic's report also documents the use of its models for surveillance, cyberattacks, and conventional weapons development. The company said it detected and shut down these operations, which involved its Claude Haiku, Sonnet, and Opus models. None of the cases involved its Fable or Mythos-class models, with the exception of one distillation case. In each instance, the company said it banned the accounts and strengthened its safeguards.
The report spans seven harm areas, including cyber operations, influence operations, surveillance, conventional weapons development, biological misuse, scams, and fraud, and illicit distillation. Anthropic said the misuse involved its models being used to develop software for conventional weapons, including a guided rocket and a multi-stage ballistic missile.
Anthropic's head of threat intelligence, Jacob Klein, told Axios that AI was making state surveillance cheaper and more efficient. "They're effectively automating parts of the job within the intel apparatus," he said. "Authoritarian states are using AI for surveillance, repression, and influence operations today."
The company attributed some of the misuse to state-aligned actors and commercial spyware vendors using Claude to build surveillance systems. In one case, a single subscriber used Claude as the engineering workforce for a platform named Lakana 360, which monitored roughly 25 million SIM cards across all three of the country's mobile operators.
Anthropic said it published the report out of an obligation to disclose the misuse and to give governments and civil society a clearer view of how such threats take shape. The company's findings have implications for the development of AI models and the need for stronger safeguards to prevent misuse.
Cet article a été rédigé avec l'assistance de l'IA.
News Factory APP - actualités agentiques pour booster votre SEO et AEO.