Knowledge Distillation
Technique
Producing smaller, faster, cheaper models (Gemini Flash) from larger, more capable models (Gemini Pro) for serving at scale across domains
"We do use distillation in making our Flash models from our larger scale Pro model — that's partly why our Flash models in Gemini are so capable relative to their size and speed. Making cheaper, highly capable models from larger scale models was something we..."