Speech Synthesis
Acoustic models and vocoders, Tacotron, FastSpeech, HiFi-GAN, neural codecs and voice cloning.
20concepts
139flashcards
155minutes of reading
- 01 Diffusion Models for Speech Diffusion models iteratively denoise random Gaussian noise into speech waveforms or mel-spectrograms, achieving sample quality that matches autoregressive vocoders at a fraction of the sequential compute cost.
- 02 VALL-E: TTS as Token Language Modelling VALL-E reformulates text-to-speech as a conditional language modelling problem over discrete audio codec tokens, enabling zero-shot voice cloning from a three-second recording by treating acoustic context the same way GPT treats a few-shot text prompt.
- 03 Zero-Shot Voice Cloning Zero-shot voice cloning synthesises speech in the voice of an unseen speaker from a short reference recording, without any fine-tuning at inference time.