Knowledge Distillation
Soft targets and temperature, sequence-level and on-policy distillation, and when a student beats its teacher.
5concepts
64flashcards
36minutes of reading
No beginner concepts in this track. Show all.
Soft targets and temperature, sequence-level and on-policy distillation, and when a student beats its teacher.
No beginner concepts in this track. Show all.