Distributed Training
Data, tensor and pipeline parallelism, ZeRO/FSDP sharding, mixed precision, and offload.
10concepts
70flashcards
85minutes of reading
No beginner concepts in this track. Show all.
Data, tensor and pipeline parallelism, ZeRO/FSDP sharding, mixed precision, and offload.
No beginner concepts in this track. Show all.