Audio Edition: How Distillation Makes AI Models Smaller and Cheaper
The Quanta Podcast
Quanta Magazine
4.7 • 638 Ratings
🗓️ 14 May 2026
⏱️ 8 minutes
🧾️ Download transcript
Summary
Fundamental technique lets researchers use a big, expensive “teacher” model to train a “student” model for less.
The story How Distillation Makes AI Models Smaller and Cheaper first appeared on Quanta Magazine.
Transcript
Click on a timestamp to play from that location
| 0:00.0 | Welcome to the Quanta Audio Edition. |
| 0:07.0 | In each of these bi-weekly episodes, we bring you a story direct from the Quanta website about developments in basic science and mathematics. |
| 0:15.0 | I'm Susan Vallett. |
| 0:16.0 | The Chinese AI company DeepSeek released a chatbot last year called R1, which drew |
| 0:23.1 | a huge amount of attention. |
| 0:25.4 | Much of the news coverage implied that the company discovered a new, more efficient way to |
| 0:29.9 | build AI. |
| 0:31.5 | But instead, it used a fundamental technique. |
| 0:34.9 | That's next. |
| 0:47.3 | Quantum Magazine is an editorially independent online publication supported by the Simons Foundation to enhance public understanding of science. |
| 1:01.2 | DeepSeek's R1 chatbot sent a ripple through the industry. Most of the attention focused on the fact that a relatively small and unknown company said |
| 1:06.5 | it had built a chatbot that rivaled the performance of those from the world's most famous |
| 1:11.7 | AI companies, but using a fraction of the computer power and cost. As a result, the stocks of |
| 1:18.4 | many Western tech companies plummeted. Invidia, which sells the chips that run leading |
| 1:24.0 | AI models, lost more stock value in a single day than any company in history. |
| 1:30.7 | Some of that attention involved an element of accusation. |
| 1:34.5 | Sources alleged that DeepSeek had obtained, without permission, knowledge from OpenAI's |
| 1:40.4 | proprietary O1 model by using a technique known as distillation. |
| 1:46.0 | Much of the news coverage framed this possibility as a shock to the AI industry, |
| 1:51.3 | implying that Deepseek had discovered a new, more efficient way to build AI. |
| 1:57.3 | But distillation, also called knowledge distillation, is a widely used tool in AI. |
| 2:03.7 | It's a subject of computer science research going back a decade and a tool that big tech companies use on their own models. |
... |
Transcript will be available on the free plan in 7 days. Upgrade to see the full transcript now.
Disclaimer: The podcast and artwork embedded on this page are from Quanta Magazine, and are the property of its owner and not affiliated with or endorsed by Tapesearch.
Generated transcripts are the property of Quanta Magazine and are distributed freely under the Fair Use doctrine. Transcripts generated by Tapesearch are not guaranteed to be accurate.
Copyright © Tapesearch 2026.

