Leading AI companies, including OpenAI, Microsoft, and Meta, are embracing a technique called “distillation” to develop more cost-effective AI models, accelerating competition in the race to build advanced artificial intelligence. The approach, which gained widespread attention after China’s DeepSeek used it to create smaller but powerful AI models, has sparked concerns about Silicon Valley’s ability to maintain its dominance. Wall Street investors reacted by wiping billions off the market value of major US tech firms.
Distillation works by taking a large, complex AI model—referred to as the “teacher” model—and using it to generate data that trains a smaller “student” model. This process allows companies to retain much of the performance of their largest AI systems while significantly reducing costs and computational demands.
“Distillation is quite magical,” said Olivier Godement, head of product for OpenAI’s platform. “It lets us take a very large, smart model and create a much smaller, cheaper, and faster version optimized for specific tasks.”
Training large AI models like OpenAI’s GPT-4, Google’s Gemini, and Meta’s Llama requires vast amounts of computing power, costing hundreds of millions of dollars. Distillation, however, enables businesses and developers to access AI capabilities at a fraction of the cost, allowing them to run AI models efficiently on devices such as smartphones and laptops.
Microsoft, OpenAI’s largest backer, has already leveraged GPT-4 to create its own family of small AI models, known as Phi. However, OpenAI has accused DeepSeek of distilling its proprietary models to train a rival AI system—a move that would violate its terms of service. DeepSeek has not commented on the allegation.
While distillation produces efficient AI models, it also presents trade-offs. “If you make the models smaller, you inevitably reduce their capability,” said Ahmed Awadallah of Microsoft Research. Distilled models excel at specific tasks, such as summarizing emails, but lack the broad functionality of their larger counterparts.
Despite these limitations, many businesses prefer distilled models, which are powerful enough for tasks like customer service chatbots and mobile applications. “Anytime you can reduce costs while maintaining performance, it makes sense,” said David Cox, vice president of AI models at IBM Research.
This shift poses challenges for the business models of major AI firms. Distilled models are less expensive to create and operate, meaning they generate lower revenue for companies like OpenAI. Although OpenAI charges lower fees for distilled models due to their reduced computational demands, the company maintains that large AI models will still be necessary for high-stakes applications where accuracy and reliability are crucial.
However, OpenAI is actively working to prevent its large models from being distilled to train competitors. The company monitors usage patterns and can revoke access if it suspects a user is extracting large amounts of data for this purpose, as it reportedly did with accounts linked to DeepSeek.
Distillation has also fueled debate over open-source AI development. While OpenAI and other firms aim to protect their proprietary models, Meta’s chief AI scientist, Yann LeCun, has embraced distillation as part of the open-source philosophy. “That’s the whole idea of open source—you profit from everyone else’s progress,” LeCun said.
The rapid progress enabled by distillation raises concerns about the sustainability of first-mover advantages in AI. Despite investing billions into developing frontier models, leading AI firms now face rivals that can replicate their breakthroughs in months. “In a world where things are moving so fast, you can spend a lot of money doing it the hard way, only to have the field catch up right behind you,” said IBM’s Cox.
