OpenAI says it has sharply reduced attackers' ability to pull the internal reasoning out of its newest models, a tactic the company calls adversarial distillation. In an internal test, the average attack success rate on protected reasoning fell from 54.0% against GPT-6 Astra with its original safeguards to 0.2% against GPT-6.2 Astra with its updated protections.
The company published the findings in a detailed post that also lays out how it finds and disrupts these campaigns, and what it believes other developers should do. The core concern is straightforward. A frontier model's outputs can be used, without permission, to train a different model that has similar abilities but none of the original guardrails. An adversary could, for example, copy a model's cyber capabilities and use the new model to help carry out attacks that the original would have refused to support.
Why distillation is a safety issue
What is adversarial distillation?
Distillation is a common machine-learning technique in which one model learns from another model's data. A student model can learn from a teacher's answers, its reasoning, or its judgments about which answers are better. OpenAI says unauthorized use of that method goes beyond intellectual property concerns. Through adversarial distillation, a second model can pick up advanced capabilities without inheriting the safeguards that govern how the original can use them.
The risk is most acute in dual-use fields such as cybersecurity and biology. A model that helps defenders find software flaws could also help an attacker exploit them. A model that assists biological research could, in the wrong hands, help someone develop a harmful pathogen.
OpenAI's particular focus is on what it calls protected reasoning, the intermediate steps a model generates while working toward an answer. Those traces can contain information that the final answer leaves out, which makes them especially useful as training data.
In an internal study, the company tested how much cyber capability could transfer to a student model. Researchers started each training run from the same pretrained model and fine-tuned it mainly on work from Astra-level teacher models on coding and defensive cybersecurity tasks. One group of students learned from exact reasoning traces, while another learned from reasoning summaries. After fine-tuning, the students trained on exact traces generally performed better. After a light round of reinforcement learning, the exact-trace students still scored higher at the largest inference budget tested, while the summary-trained students did better at lower budgets.
Layered defenses and campaign enforcement
How do OpenAI's defenses stop distillation attacks?
OpenAI says its defenses work at two levels. At the request level, the model's behavior, classifier checks and API controls limit reasoning extraction. The company uses input classifiers to flag attempts to extract protected reasoning and output classifiers to catch leaks in assistant messages and tool-call arguments. Tool-call coverage matters, the company said, because a model can expose protected reasoning through a tool's arguments without including it in the final answer.
At the campaign level, OpenAI looks for patterns across requests and accounts that point to coordinated extraction. When it finds violations, it enforces against the entire network of accounts involved. The company says it has seen multiple unauthorized campaigns on its platform this year, with weekly enforcement actions ranging from thousands to tens of thousands of accounts. The pace of those actions increased significantly over the past summer. OpenAI also shares indicators of compromise with other model providers so they can look into related abuse on their own services.
The company tracks how often distillation abuse occurs and how long it takes to move from detection to enforcement. Its stated goals are to limit the volume and value of data distillers can obtain, raise the cost of running these campaigns and shorten the response time to new attacks.
Why is distillation a national security issue?
OpenAI also frames the issue as a national security matter. A foreign actor that reproduces advanced U.S. model capabilities could deploy them without the protections the original developer built in. The company said it briefed Congress in February on activity suggesting DeepSeek was trying to distill U.S. frontier models. In September, it reported disrupting a campaign to extract its models' protected reasoning and attributed a core part of that activity to individuals associated with Moonshot AI, the developer of Kimi. OpenAI said it plans to share more about recent campaigns soon.
The company said it supports legislation that would give the U.S. government more tools to identify and sanction foreign actors that illicitly extract capabilities from American AI models. It has also told Congress it supports the Chip Security Act, the AI OVERWATCH Act and the MATCH Act. Export controls can limit adversaries' access to the hardware needed to train frontier models, while distillation defenses make it harder to acquire some of the same capabilities from models already trained.
For other developers, OpenAI recommends protecting private reasoning across the full interaction, not just the final message. It suggests keeping logs of successful extraction attacks as regression tests, replaying those attacks whenever defenses are updated, and pairing request-level mitigations with campaign-level investigations. The company also urged developers to coordinate safeguard requirements with deployment partners. OpenAI said it has previously shared findings through the Frontier Model Forum and government information-sharing channels, while keeping details that attackers could exploit out of public view.
This article was written with the assistance of AI.
News Factory APP - agentic news to boost your SEO & AEO.