← Back
technique
Knowledge Distillation
1 creator use or recommend it
▲ 1 active in the last 90 days
1 use
Last discussed Jul 30, 2026
A model compression technique where a smaller 'student' model is trained to replicate the behavior of a larger 'teacher' model, producing efficient models that retain much of the larger model's capability.
Recently discussed in
Jeff Dean: The 1% Rule for Building in AI
Jeff Dean · Jul 30, 2026
What creators say
JD
Jeff Dean
uses
Used for: Producing smaller, faster, cheaper models (Gemini Flash) from larger, more capable models (Gemini Pro) for serving at scale across domains
Praised: Enables highly capable smaller models; Gemini Flash is benchmark-leading for ...
vs: Paper was initially rejected by a conference reviewer who called it 'unlikely...
"We do use distillation in making our Flash models from our larger scale Pro model — that's partly why our Flash models in Gemini are so capable relative to their size and speed. Making cheaper, highly capable models from larger scale models was so..."
More from Jeff Dean →Keep exploring
Topics
About this data
- ✓44,000+ mentions extracted from real podcast transcripts
- ✓Every quote links to the source episode
- ✓Sponsored mentions are always labeled
- ✓Critical takes are shown, never filtered out
Get an email when a new creator recommends it