Forward Diffusion and Noise Schedules
The fixed corruption process that makes diffusion training possible, why the closed form in alpha-bar removes the need to simulate it, and how the schedule silently decides which frequencies the model learns.
Training a generative model usually means fighting an intractable likelihood. Diffusion sidesteps the fight with a trick that looks almost like cheating: define a corruption process that destroys data into pure noise, note that each step of it is trivially invertible in expectation, and train a network to undo one step at a time. The forward process is not learned. It is a fixed piece of arithmetic you choose in advance, and that choice does more to determine sample quality than most architecture decisions.
The process, and why you never simulate it
The forward process adds Gaussian noise over \(T\) steps according to a variance schedule \(\beta_1, \dots, \beta_T\):
Written this way it looks like you would have to run \(t\) steps to get a training sample at time \(t\). You do not. Gaussians compose, so with \(\alpha_t = 1 - \beta_t\) and \(\bar{\alpha}_t = \prod_{s \le t} \alpha_s\) the marginal has a closed form:
A training step is therefore: sample a clean image, sample \(t\) uniformly, sample \(\epsilon \sim \mathcal{N}(0, I)\), and form \(x_t = \sqrt{\bar{\alpha}_t}\,x_0 + \sqrt{1-\bar{\alpha}_t}\,\epsilon\) in one line. This is the entire reason diffusion scales. Every step of the reverse chain gets an independent, cheap, exactly-correct training example, and there is no sequential rollout, no discriminator, and no sampling loop inside the training graph.
The \(\sqrt{1-\beta_t}\) scaling on the mean is not cosmetic. It keeps the variance of \(x_t\) at roughly unit scale for data normalised to unit variance, so the network sees inputs of consistent magnitude across all \(t\). Drop it and the signal grows without bound as noise accumulates.
The schedule is a curriculum over frequencies
Only \(\bar{\alpha}_t\) reaches the network, and it acts as a signal-to-noise ratio, \(\text{SNR}(t) = \bar{\alpha}_t / (1 - \bar{\alpha}_t)\). High-SNR steps carry an almost-clean image, so the only information left to recover is fine texture. Low-SNR steps carry near-noise, so the only thing recoverable is coarse layout and colour. The schedule allocates model capacity across spatial frequencies by deciding how many training steps land in each SNR band.
The original DDPM linear schedule ran \(\beta_t\) from \(10^{-4}\) to \(0.02\) over 1,000 steps. On 32x32 images this is reasonable. On 64x64 and above it destroys information too early: by the halfway point \(\bar{\alpha}_t\) has already collapsed, so a large part of the training budget is spent on steps indistinguishable from pure noise. The cosine schedule, which sets \(\bar{\alpha}_t = \cos^2\left(\frac{t/T + s}{1+s}\cdot\frac{\pi}{2}\right)\), holds SNR up much longer and then falls sharply, and it improved sample quality at higher resolution with no change to the network at all (Nichol and Dhariwal, 2021, arXiv:2102.09672).
Resolution changes the right answer, because more pixels means more redundancy: neighbouring pixels are correlated, so a given noise level destroys proportionally less information in a large image than in a small one. Schedules tuned at 64x64 under-noise at 512x512, which is why practitioners shift the schedule toward higher noise as resolution grows rather than reusing a published one.
Discrete steps versus continuous time
Nothing in the derivation requires \(T\) to be finite. Treating \(t\) as continuous on \([0,1]\) turns the forward process into a stochastic differential equation and makes the schedule a continuous function of \(t\), which decouples the schedule from the sampler's step count. Practically this lets you train once and then sample with 20, 50 or 250 steps without retraining, and it is why most modern implementations condition on a continuous noise level, or on \(\log \text{SNR}\), rather than on an integer step index.
When it breaks
Terminal SNR is not actually zero. With a standard linear or cosine schedule, \(\bar{\alpha}_T\) is small but non-zero, so \(x_T\) still contains a faint trace of the image, most visibly its mean brightness. At sampling time you start from pure noise, which has zero mean, so the model operates on an input distribution it never saw. The symptom is an inability to generate very dark or very bright images: everything drifts toward medium grey. Rescaling the schedule so \(\bar{\alpha}_T = 0\) exactly, and adjusting the sampler to match, fixes it.
Uniform sampling of \(t\) wastes gradient budget. The loss is not equally informative at every noise level; the middle SNR band carries most of the useful signal, while very high and very low SNR steps are close to trivial. Importance-sampling \(t\), or weighting the loss by a function of SNR, converges faster and reduces gradient variance. Several published weightings are equivalent to a change of schedule, a good reminder that "schedule" and "loss weighting" are two views of one design choice.
The schedule is baked into the checkpoint. Because the network is conditioned on \(t\) or on the noise level, swapping schedules after training changes the meaning of that conditioning input. Samplers that appear to work with a mismatched schedule are usually degrading quality in ways FID picks up before the eye does.
14 flashcards for this concept
Click a card to reveal the answer.