Knowledge Distillation
Soft targets and temperature, sequence-level and on-policy distillation, and when a student beats its teacher.
11concepts
101flashcards
82minutes of reading
No beginner concepts in this track. Show all.
Soft targets and temperature, sequence-level and on-policy distillation, and when a student beats its teacher.
No beginner concepts in this track. Show all.