What is AI model distillation, and why is it so hard to stop?
Scientific American · LC · trust 44/100
AI companies say rivals are ‘distilling’ their models. What does that even mean?
Anthropic and OpenAI accused Chinese AI developers of mining their models’ answers to train cheaper copycats. Here’s how distillation works and why it’s so hard to stop
To make grain-based liquor, you heat a fermented mash in a still until alcohol-rich vapor rises and then cool it back into a stronger liquid. Chemists and whiskey makers know this as distillation. Artificial intelligence developers have borrowed the word to describe boiling down a big AI model into a smaller one—and lately the biggest AI companies say that rivals have been distilling like bootleggers.
In September Anthropic and OpenAI each detailed how they caught and killed campaigns to distill their flagship models—that is, to extract enough information from the models to train new models that mimic the originals at a fraction of the size and cost.
Distillation has become a sore spot for American AI companies as they try to justify high valuations and fight off competition from each other and open-source alternatives. If, say, a Chinese model developer can piggyback on all the money and computing power those companies have invested, the incentive to keep making new models starts to evaporate.
If you're enjoying this article, consider supporting our award-winning journalism by subscribing . By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
And no one expects distillation to stop any time soon—so we asked the researchers who study it to explain the situation.
Just as a still concentrates a large batch of weak liquid into a smaller batch of strong stuff, AI distillation condenses a big model into a smaller one that, ideally, acts like the original. The process starts with the existing large model, called the teacher. Developers prompt the teacher many times over and then feed its responses into the new model, called the student, to train the latter.
In a way, every response contains information about what the teacher learned during training, so that knowledge passes to the student without going through the process of collecting data, finding patterns in those data and getting human feedback. Distillation lets someone reverse engineer an AI model just by using it and watching its responses.
It helps that AI models tend to give away more than just their final answers. “Oftentimes the teacher model doesn’t just give you the label, but it actually gives you more fine-grained information,” says Yevgeniy Vorobeychik , a professor of computer science at Washington University in St. Louis. That could include the odds a model assigns to each possible response or, as in the case OpenAI described, an encrypted record of the step-by-step reasoning its more powerful models work through before they answer.
Plenty of distillation is aboveboard. Smaller models let developers get more for less. In fact, the concept predates large language models (LLMs); by the 2000s, researchers were studying it as a way to increase efficiency for neural networks. “We cared about making it smaller, easier to use, faster to apply,” says Alexandru Niculescu-Mizil , a machine learning researcher at Qube Research & Technologies, who cowrote an early paper on the idea in 2006.
Distilling someone else’s model is another matter. The term first caught mainstream attention in early 2025, when DeepSeek, then a little-known Chinese start-up, released a reasoning model, called R1, that it said it had trained for less than $300,000—a fraction of what American labs had spent. Allegations quickly emerged that DeepSeek had trained R1 in part on responses extracted from OpenAI’s models.
These days the word is generally associated with accusations of model theft, mostly aimed at Chinese developers such as DeepSeek and Moonshot AI, the maker of Kimi. In their September reports, Anthropic and OpenAI both blamed labs based in China for the recent distillation campaigns they caught, calling the activity “illicit” and “unauthorized.” DeepSeek and Moonshot AI have both fended off these accusations in the past. Of course, Anthropic and OpenAI have themselves been accused of training their models on data obtained without authorization from all over the internet.
Still, though the companies also cast distillation as a security risk, their most obvious interest in stopping it is protecting their competitive advantage. “The text that it produces in response to your prompt is being stolen by some rival companies,” says Alexander Panfilov , a Ph.D. student at the Max Planck Institute for Intelligent Systems in Germany. “But on a conceptual level, what you are stealing [are] capabilities.”
In general, distillation is done by collecting thousands and thousands of AI responses to different kinds of queries. Those prompts and responses then become training data for the student, usually through a process called supervised fine-tuning, which treats the teacher’s answers as targets the student should try to emulate. Over many rounds, training nudges the student’s internal settings, called weights, so that its answers more closely match the teacher’s.
The key is getting those data. Anthropic and OpenAI have detailed several creative methods they believe attackers used to do so. According to Anthropic, some labs routed their own customers’ prompts to Claude and then saved Claude’s responses to train their own models. The company also said that distillers try to obscure where they are located. OpenAI, meanwhile, said distillers tried to get at its models’ hidden reasoning by “copying encrypted reasoning from one conversation and asking a model in another conversation to decrypt” it.
Panfilov was co-lead author of a…
Read the original at Scientific American →
Open in TruthVane →