From Normalising Flows to Flow Matching: How the Change-of-Variables Idea Survived Its Own Architecture

In July 2018, OpenAI's Glow reported 3.35 bits per dimension on CIFAR-10 and generated 256 by 256 faces in about 130 milliseconds on one GTX 1080 Ti (Kingma & Dhariwal, 2018, Glow, arXiv:1807.03039). Three months later Eric Nalisnick and colleagues trained Glow on CIFAR-10 and found it assigned higher likelihood to SVHN house numbers it had never seen (2.39 bits per dimension) than to the CIFAR-10 test set (3.46) (Nalisnick et al., 2019, Do Deep Generative Models Know What They Don't Know?, ICLR, arXiv:1810.09136). Flows had paid an enormous architectural price for a number that did not mean what everyone assumed.

Six years later Stability AI's largest research model was an 8-billion-parameter "rectified flow transformer" (Esser et al., 2024, arXiv:2403.03206), and Meta's 30-billion-parameter Movie Gen was trained with flow matching (Polyak et al., 2024, arXiv:2410.13720). The change-of-variables idea won precisely because it stopped being trained through the change-of-variables formula.

Why this matters: Flow matching is now a default objective for image, video, speech and robot-action generators, usually described as "diffusion with a different loss". Knowing what it inherited from normalising flows (an invertible ODE transport map) and what it discarded (maximum-likelihood training through a Jacobian) explains why it trains cheaply, samples in few steps, rarely reports likelihood, and why its "straight paths" are not straight.

TL;DR

• A discrete flow computes \(\log|\det J|\) per layer per example; for CIFAR-10 a general determinant is about \(2.9 \times 10^{10}\) operations. NICE, RealNVP, Glow, MAF and IAF made that scalar cheap and paid in expressiveness, plateauing at 3.35 (Glow) and 3.08 (Flow++) bits/dim against PixelCNN++ at 2.92.
• FFJORD freed the architecture by trading the determinant for a trace (3.40 bits/dim, under 2% of Glow's parameters) but put ODE solves inside training: six GPUs, about five days.
• Flow matching, rectified flow and stochastic interpolants (2022) regress a velocity onto per-pair targets with no solver or Jacobian, and provably the same gradient as the intractable marginal objective.
• One network, trained two ways on CIFAR-10: flow matching reached 2.99 bits/dim, FID 6.35 and 142 function evaluations; DDPM training reached 3.12, 7.48 and 274.
• Gaussian flow matching and diffusion are one family, and the flow-matching Euler sampler is DDIM. In SD3's study, time weighting decided whether rectified flow beat the baseline at all.
• Straightness holds per pair, not in the learned field. Reflow buys one-step sampling (FID 4.85 after distillation) at a cost in full-simulation quality (FID 2.58 rising to 3.36).
• Likelihood is not settled: Apple's TarFlow, a Transformer MAF, reported 2.99 bits/dim on ImageNet 64x64 against flow matching's 3.31.

At a Glance

flowchart LR
    Z["Noise z ~ N(0, I)"] --> DF["Discrete flow: invertible layers"]
    DF --> LD["Pay log-det per layer"]
    Z --> CF["Continuous flow: ODE dx/dt = v"]
    CF --> TR["Pay trace plus solver per step"]
    Z --> FM["Flow matching: regress v on pairs"]
    FM --> MSE["Pay one MSE, no solver"]
    LD --> X["Data sample x"]
    TR --> X
    MSE --> X

    classDef blue fill:#1e40af,stroke:#3b82f6,stroke-width:1px,color:#fff
    classDef purple fill:#6d28d9,stroke:#a78bfa,stroke-width:1px,color:#fff
    classDef teal fill:#0e7490,stroke:#22d3ee,stroke-width:1px,color:#fff
    classDef amber fill:#b45309,stroke:#fbbf24,stroke-width:1px,color:#fff
    classDef emerald fill:#047857,stroke:#34d399,stroke-width:1px,color:#fff
    class Z blue
    class DF,CF,FM purple
    class LD,TR amber
    class MSE emerald
    class X teal

Three generations, one object: a transport map from noise to data. They differ in what each training step pays to learn it.

The Determinant Era

If \(z = f(x)\) is invertible and differentiable, then

The second term corrects for how much \(f\) stretches volume around \(x\); across stacked layers the log-determinants add. The result is exact maximum likelihood, exact latent inference and one-pass sampling.

The catch is the last term. A general \(D \times D\) determinant costs \(O(D^3)\), and so does its gradient. Every flow from 2014 to 2019 answers one question: which networks have a cheap Jacobian determinant and remain expressive and invertible?

timeline
    title From Jacobian Determinants to Velocity Regression
    2014 : NICE introduces additive coupling with unit determinant
    2015 : Rezende and Mohamed name normalising flows for variational inference
    2016 : RealNVP adds affine coupling and multiscale squeeze
         : IAF makes flows cheap to sample inside a VAE
    2017 : MAF stacks masked autoregressive layers for density estimation
    2018 : Glow adds invertible 1x1 convolutions
         : Neural ODEs and FFJORD swap the determinant for a trace
    2019 : Flow++ reaches 3.08 bits per dim
         : Nalisnick et al. show flows prefer SVHN to CIFAR-10
    2022 : Flow matching, rectified flow and stochastic interpolants go simulation-free
    2024 : Stable Diffusion 3, FLUX.1, Movie Gen and pi0 adopt flow matching
         : TarFlow revives discrete flows with Transformers

NICE split the input in two and let one half shift the other, \(y_2 = x_2 + m(x_1)\), with \(m\) any network (Dinh, Krueger & Bengio, 2014, NICE, arXiv:1410.8516). The Jacobian is triangular with a unit diagonal, so the determinant is 1 and inversion is subtraction. But a volume-preserving map cannot concentrate mass, so a final diagonal scaling did all the compression. Rezende and Mohamed named the family in 2015, chaining invertible maps to enrich variational posteriors (Rezende & Mohamed, 2015, ICML, arXiv:1505.05770). RealNVP made coupling affine, \(y_2 = x_2 \odot \exp(s(x_1)) + t(x_1)\), so the log-determinant became \(\sum_j s_j(x_1)\), and added checkerboard masks and squeezing for multiscale images, reaching 3.49 bits/dim on CIFAR-10 against PixelRNN's 3.00 (Dinh, Sohl-Dickstein & Bengio, 2017, ICLR, arXiv:1605.08803).

[IMAGE: Two-panel diagram of an affine coupling layer. Left: input split into x1 (blue, passed through) and x2 (purple), with x1 feeding a network that outputs s and t, which scale and shift x2. Right: the Jacobian as a 2x2 block matrix, identity top-left, zeros top-right, dense grey bottom-left, diag(exp(s)) bottom-right, diagonal highlighted. Caption: "A coupling layer is expressive where it does not matter for the determinant and constrained exactly where it does."]

Masked Autoregressive Flow shifts and scales each dimension using all earlier dimensions through one masked network (Papamakarios, Pavlakou & Murray, 2017, NeurIPS, arXiv:1705.07057): density in one parallel pass, sampling in \(D\) sequential passes. Inverse Autoregressive Flow reverses this, suiting a VAE posterior that only scores its own samples (Kingma et al., 2016, NeurIPS, arXiv:1606.04934). The determinant constraint reappears as a latency constraint.

Glow replaced fixed channel permutations with a learned 1x1 convolution whose log-determinant, \(h \cdot w \cdot \log|\det W|\), costs \(O(c^3)\); writing \(W = PL(U + \mathrm{diag}(s))\) cuts it to \(O(c)\) (Kingma & Dhariwal, 2018). Every clause is determinant accounting. Flow++ added variational dequantisation, mixture couplings and attention to reach 3.08 bits/dim, still short of PixelCNN++ (Ho et al., 2019, Flow++, ICML, arXiv:1902.00275).

Then the determinant became a trace. For an ODE flow, log-density evolves by the negative trace of the velocity's Jacobian (Chen et al., 2018, Neural Ordinary Differential Equations, NeurIPS, arXiv:1806.07366), and FFJORD estimated that trace unbiasedly in linear time with Hutchinson's estimator (Grathwohl et al., 2019, FFJORD, ICLR, arXiv:1810.01367). The architecture was free; the training loop was not, integrating an ODE forward and an adjoint backward every step (the library's concept on continuous normalising flows covers the derivation).

How the Change of Variables Stopped Being the Loss

A flow without a Jacobian

A flow is a velocity field \(v_t(x)\) on \(t \in [0, 1]\). Integrating \(dx/dt = v_t(x)\) from noise at \(t = 0\) yields data at \(t = 1\), moving densities \(p_t\) that obey the continuity equation

Along a trajectory this is the instantaneous change of variables, \(\frac{d}{dt}\log p_t(x_t) = -\nabla \cdot v_t(x_t)\): the divergence is the infinitesimal log-determinant. That idea survived intact.

Maximum likelihood asks which \(v\) makes \(p_1\) closest to data, and \(p_1\) is only known by integrating. Flow matching asks instead: if we choose a path of densities in advance, which velocity generates it? Then training is regression.

The right target is intractable

Let \(q(x_1)\) be the data distribution. Choose a conditional path \(p_t(x \mid x_1)\) from a Gaussian at \(t = 0\) to a point mass near \(x_1\) at \(t = 1\), generated by a known conditional velocity \(u_t(x \mid x_1)\). The marginal path \(p_t(x) = \int p_t(x \mid x_1)\, q(x_1)\, dx_1\) then ends near \(q\), and Lipman, Chen, Ben-Hamu, Nickel and Le proved it is generated by

(Lipman et al., 2023, Flow Matching for Generative Modeling, ICLR, arXiv:2210.02747). The velocity at a point is the posterior-weighted average of the conditional velocities of every data point that could have put a particle there. Regressing onto it, \(\mathcal{L}_{FM} = \mathbb{E}_{t, x \sim p_t}\|v_\theta(t, x) - u_t(x)\|^2\), is useless as written: \(u_t(x)\) integrates over the whole dataset.

Conditional flow matching: the same gradient for free

Regress on the conditional velocity instead:

Expand both squares. The \(\|v_\theta\|^2\) terms match, because in both losses \(x\) is marginally distributed as \(p_t\). The \(\|u\|^2\) terms do not involve \(\theta\). The cross term of \(\mathcal{L}_{FM}\) is \(\mathbb{E}_x \langle v_\theta(x), u_t(x)\rangle\); substituting the posterior average turns it into \(\mathbb{E}_{x_1, x}\langle v_\theta(x), u_t(x \mid x_1)\rangle\), the cross term of \(\mathcal{L}_{CFM}\). The losses differ by a constant, their gradients are equal, and the minimiser is \(\mathbb{E}[u_t(x \mid x_1) \mid x_t = x]\), the marginal field. It is least squares recovering a conditional mean, and the price is variance: each example supervises with one sampled velocity, not the average.

[IMAGE: Vector field at t = 0.5 for a toy dataset of two Gaussian blobs, top-left and bottom-right. Faint grey arrows show conditional velocities of sampled pairs crossing in the middle; bold teal arrows show the learned marginal field bending around the crossing. Caption: "Each pair supplies a straight-line target. The network learns their average, and averages of straight lines are not straight."]

Choosing the path

For Gaussian conditional paths \(\mathcal{N}(\mu_t(x_1), \sigma_t(x_1)^2 I)\), the generating field is \(u_t(x \mid x_1) = \frac{\sigma_t'}{\sigma_t}(x - \mu_t) + \mu_t'\). A variance-preserving diffusion schedule recovers the probability-flow ODE of score-based diffusion. The linear choice

gives \(x_t = (1 - (1 - \sigma_{\min})t)\,x_0 + t\,x_1\) with \(x_0 \sim \mathcal{N}(0, I)\), and a target constant along each conditional trajectory:

Lipman et al. call it the optimal-transport path: each conditional map is the OT displacement between two Gaussians, and particles move in straight lines at constant speed.

Rectified flow reached the same objective from transport theory, fitting a velocity to \(X_t = tX_1 + (1 - t)X_0\) with target \(X_1 - X_0\) between two arbitrary distributions (Liu, Gong & Liu, 2022, Flow Straight and Fast, arXiv:2209.03003); with \(\sigma_{\min} = 0\) the objectives coincide. Its distinctive move was reflow: generate pairs by running the trained ODE, then retrain on them. Rectification provably never increases convex transport costs, and the best of \(K\) rounds has straightness bounded by \(\mathbb{E}\|X_1 - X_0\|^2 / K\).

Albergo and Vanden-Eijnden, three weeks later, derived a quadratic velocity loss for any interpolant between two densities (Albergo & Vanden-Eijnden, 2023, Building Normalizing Flows with Stochastic Interpolants, ICLR, arXiv:2209.15571). The follow-up adds latent noise, \(x_t = \alpha(t)x_0 + \beta(t)x_1 + \gamma(t)z\), yielding both ODE and SDE samplers, and notes that likelihood control is weaker ("more stringent") for the deterministic ODE than for the stochastic samplers (Albergo et al., 2023, arXiv:2303.08797). Tong et al. paired samples by minibatch optimal transport, reducing crossings and inference steps (Tong et al., 2023, TMLR, arXiv:2302.00482).

Flow matching is diffusion, reparameterised

Write \(x_t = \alpha_t x_1 + \sigma_t x_0\). Networks predicting noise, clean data, score or velocity are affine functions of one another given \(x_t\) and \(t\). Gao, Hoogeboom, Heek, De Bortoli, Murphy and Salimans drew the consequence: Gaussian flow matching and diffusion are one family, and the flow-matching Euler sampler is exactly DDIM (Gao et al., 2024, Diffusion Meets Flow Matching: Two Sides of the Same Coin, ICLR 2025 blogpost track). What differs is the MSE weighting across noise levels and the network's output, which changes optimisation even when the optimum is shared.

That reframes Lipman et al.'s ablation, where one U-Net trained five ways and OT flow matching won every column: a better-conditioned target and weighting, not a new model class. Across 61 formulations for Stable Diffusion 3, rectified flow with uniform timesteps did not beat the older LDM-Linear baseline; only rectified flow with logit-normal timesteps, \(\pi_{ln}(t; m, s) \propto \frac{1}{t(1-t)}\exp\!\big(-\frac{(\mathrm{logit}\,t - m)^2}{2s^2}\big)\), which concentrates training on intermediate noise, did, with \(m = 0, s = 1\) ranking consistently well (Esser et al., 2024). The straight path was necessary, not sufficient.

Seeing It in Motion

Maximum-likelihood training of a continuous flow runs a solver twice per step. Conditional flow matching never runs one during training.

sequenceDiagram
    participant D as Data loader
    participant N as Noise sampler
    participant M as Velocity network
    participant O as Optimiser
    D->>M: data point x1
    N->>M: noise x0 and time t from logit-normal
    Note over M: build xt = (1-t) x0 + t x1
    M->>M: predict v at (t, xt)
    Note over M: target is x1 - x0, no solver, no Jacobian
    M->>O: squared error to target
    O->>M: gradient step
    Note over D,O: sampling later integrates v from t=0 to t=1 in 10 to 50 steps

Three independent choices replaced "which invertible layer".

flowchart TB
    subgraph Choices["Flow matching design choices"]
        P1["Pairing: independent or minibatch OT"]
        P2["Path: OT line, VP diffusion, interpolant"]
        P3["Time weighting: uniform or logit-normal"]
    end
    subgraph Effects["What each choice controls"]
        E1["Crossing paths and target variance"]
        E2["Curvature and solver steps"]
        E3["Which noise levels get capacity"]
    end
    subgraph Risks["Where it goes wrong"]
        R1["Blurry few-step samples"]
        R2["Reflow error accumulation"]
        R3["Good at 50 steps, bad at 5"]
    end
    P1 --> E1 --> R1
    P2 --> E2 --> R2
    P3 --> E3 --> R3

    classDef blue fill:#1e40af,stroke:#3b82f6,stroke-width:1px,color:#fff
    classDef purple fill:#6d28d9,stroke:#a78bfa,stroke-width:1px,color:#fff
    classDef rose fill:#be123c,stroke:#fb7185,stroke-width:1px,color:#fff
    class P1,P2,P3 blue
    class E1,E2,E3 purple
    class R1,R2,R3 rose

Likelihood still comes from the change of variables: integrate the divergence along the sampling ODE, as FFJORD did.

[IMAGE: Trajectories in 2D from five noise samples to a two-moons dataset: VP diffusion paths curving and overshooting (left), 1-rectified flow bending near the midpoint (middle), 2-rectified flow nearly straight (right). Caption: "Straightness comes from the learned coupling, not the loss."]

By the Numbers

Sources: Kingma & Dhariwal, 2018 (RealNVP, Glow); Grathwohl et al., 2019; Ho et al., 2019 (Flow++, PixelCNN++); Kingma et al., 2021 (VDM); Lipman et al., 2023, Table 1, one U-Net per objective; Liu et al., 2022, Table 1a; Zhai et al., 2025. Dequantisation and architectures differ across papers, so compare rows within a paper.

Sources: Esser et al., 2024; FLUX.1 [dev] model card; Polyak et al., 2024; Le et al., 2023; Black et al., 2024.

A Concrete Example

Two calculator-checkable computations, one per era.

Part 1: exact likelihood through an affine coupling layer

Step 1, one dimension. Let \(z = f(x) = 2x + 1\) with a standard normal base. For \(x = 0.25\), \(z = 1.5\), and

This matches \(x \sim \mathcal{N}(-0.5, 0.5^2)\) evaluated directly; the \(+0.693\) corrects for squeezing a unit-width Gaussian to half width.

Step 2, a coupling layer. Take \(x = (0.5, 2.0)\) and a conditioner giving \(s(x_1) = 0.8\,x_1 = 0.4\) and \(t(x_1) = x_1 - 1 = -0.5\). Then \(z_1 = 0.5\) and \(z_2 = 2.0 \times 1.491825 - 0.5 = 2.483649\). The Jacobian is lower triangular with diagonal \((1, e^{0.4})\); the off-diagonal entry, holding the conditioner's derivatives, never enters the determinant, so \(\log|\det J| = 0.4\).

Step 3, likelihood. \(\log \mathcal{N}(0.5) = -1.043939\) and \(\log \mathcal{N}(2.483649) = -0.918939 - 3.084256 = -4.003195\), so

Step 4, inversion. \(x_2 = (2.483649 + 0.5)/1.491825 = 2.000000\), using only a forward pass of the conditioner on \(z_1\).

Step 5, the bill. A dense \(3072 \times 3072\) determinant needs about \(2.9 \times 10^{10}\) operations. The coupling layer paid nothing and left half its inputs untouched, so Glow's CIFAR-10 model stacked 3 levels of 32 steps.

Part 2: a conditional flow matching target for one sample

OT path with \(\sigma_{\min} = 0\) (identical to rectified flow), in two dimensions.

Step 1, sample. Data \(x_1 = (1.0, -2.0)\), noise \(x_0 = (0.3, 0.8)\), time \(t = 0.25\).

Step 2, interpolate. \(x_t = 0.75\,(0.3, 0.8) + 0.25\,(1.0, -2.0) = (0.475, 0.100)\).

Step 3, target. \(u = x_1 - x_0 = (0.7, -2.8)\); with Movie Gen's \(\sigma_{\min} = 10^{-5}\) it is \((0.700003, -2.799992)\).

Step 4, loss. If the network outputs \((0.5, -2.0)\), the loss is \(0.2^2 + 0.8^2 = 0.68\). That is the whole training step.

Step 5, a crossing pair. An equally likely data point \(x_1' = (-1.0, 2.0)\) with noise \(x_0' = (x_t - t\,x_1')/(1-t) = (0.966667, -0.533333)\) passes through the same \(x_t\) at the same \(t\), with target \((-1.966667, 2.533333)\). The loss minimiser there is the average, \(v^* = (-0.633333, -0.133333)\).

Step 6, one Euler step to \(t = 1\). With the conditional target, \(x_t + 0.75\,u = (1.0, -2.0)\), the data point. With the learned field, \(x_t + 0.75\,v^* = (0, 0)\): the mean of the two data points, which is neither. This is the one-step blur, and why Liu et al. observe that a 1-rectified flow simulated with one Euler step generates the mean of the target. More steps let trajectories commit as the posterior sharpens; reflow makes such crossings rare.

[IMAGE: 2D grid for the worked example. Teal data points at (1,-2) and (-1,2), blue noise points at (0.3,0.8) and (0.97,-0.53), dashed straight lines from each noise point to its data point crossing at (0.475, 0.1) in amber. From the crossing, two thin arrows along the conditional targets and one bold rose arrow along their average, ending at the origin. Caption: "Two straight training paths meet; the learned velocity averages them, and one Euler step lands between the data."]

Where It Breaks

Straight conditional paths, curved marginal paths

The learned field is a posterior average, and the average of two straight lines through a point belongs to neither; Gao et al. list "flow matching paths are straight" as a misconception for this reason. Required steps depend on how entangled the pairing is, and neither minibatch OT nor logit-normal weighting yields a one-step model for free.

Reflow trades quality for speed, and errors compound

Reflow trains on the previous model's outputs and inherits its errors. In Liu et al.'s Table 1a, full-simulation CIFAR-10 FID goes from 2.58 (1-rectified) to 3.36 (2-rectified) to 3.96 (3-rectified), while one-step FID improves from 378 to 12.21 to 8.15 undistilled, and from 6.18 to 4.85 then back to 5.21 distilled. The authors advise against many rounds "as it may accumulate estimation error". At scale it is expensive: InstaFlow's one-step Stable Diffusion, FID 23.3 on MS COCO 2017-5k, cost 199 A100 GPU-days (Liu et al., 2024, InstaFlow, ICLR, arXiv:2309.06380).

A schedule tuned for one step count fails at another

In the SD3 study, rf/lognorm(0.50, 0.60) was the best variant at 50 sampling steps and averaged rank 8.5 at 5 steps. Movie Gen replaced linear inference timesteps with a 50-step linear-quadratic schedule emulating 250 linear steps. Changing training weighting or inference discretisation without re-evaluating the other produces regressions that look like a bad checkpoint.

Likelihood: expensive, unoptimised, and a poor proxy anyway

Flow matching does not minimise negative log-likelihood; computing one costs an adaptive ODE solve with a trace estimator, and SD3, FLUX.1 and Movie Gen report none. Little may have been lost. Theis, van den Oord and Bethge showed average log-likelihood, Parzen estimates and sample fidelity are "largely independent" in high dimensions (Theis et al., 2016, ICLR, arXiv:1511.01844). Nalisnick et al.'s flows preferred SVHN to CIFAR-10 by a full bit and gave constant images the highest likelihood of all, 0.589 bits/dim, which they attribute to data variance and model curvature rather than semantics. That is the strongest case for abandoning maximum likelihood, and it is contested below.

Invertible networks were never as invertible as advertised

Behrmann et al. showed common invertible architectures suffer exploding inverses, becoming numerically non-invertible so that the change-of-variables formula fails and sampling breaks (Behrmann et al., 2021, AISTATS, arXiv:2006.09347). ODE flows cannot cross trajectories either, so splitting a Gaussian into distant modes needs steep velocities, which flow matching sees as high-variance targets at intermediate \(t\).

[IMAGE: Grouped bar chart from Liu et al. Table 1a (CIFAR-10 FID, log scale) for 1-, 2- and 3-rectified flow: one Euler step (378, 12.21, 8.15, rose), one step after distillation (6.18, 4.85, 5.21, emerald), full RK45 simulation (2.58, 3.36, 3.96, amber). Caption: "Reflow makes one step usable and full simulation worse."]

Alternative Designs

Glow's exact inverse still wins when data must round-trip through a latent losslessly, and MAF's slow sampling is irrelevant when only densities matter. Because Gaussian flow matching is reparameterised diffusion, that choice reduces to three knobs: parameterisation, weighting and sampler.

How It Is Used in Practice

Image. The SD3 team did not simply swap losses: they paired the tuned objective with an MM-DiT using separate text and image weights, and the 8B model scored 0.74 on GenEval, above DALL-E 3's 0.67 in the same table (Esser et al., 2024). FLUX.1 [dev]'s description as a 12B rectified flow transformer is a vendor claim without a paper.

Video. Movie Gen's reasons were engineering: linear interpolation guarantees zero terminal signal-to-noise ratio, and flow matching was "more robust to the exact choice of noise schedules" (Polyak et al., 2024).

Speech. Voicebox (Le et al., 2023, NeurIPS, arXiv:2306.15687) and the open F5-TTS, trained on 100K hours (Chen et al., 2024, arXiv:2410.06885), both generate speech non-autoregressively with flow matching.

Robotics. Physical Intelligence's pi0 attaches a 300M-parameter flow-matching action expert to a 3B PaliGemma backbone and integrates 10 Euler steps at inference, caching the observation prefix's keys and values (Black et al., 2024, pi0, RSS 2025, arXiv:2410.24164). Few-step sampling is what makes control at up to 50 Hz feasible.

[IMAGE: Four-column panel titled "One objective, four modalities": image latent patches (SD3), spatiotemporal video tokens (Movie Gen), mel-spectrogram frames (Voicebox), and a chunk of robot joint-angle vectors (pi0). Under each, the same equation box "target = x1 - x0" and the sampling step count used. Caption: "Flow matching is modality-agnostic because the regression target never depends on the architecture."]

Operational lessons. Version the inference timestep schedule with the weights, evaluate at the deployed step count, and give reflow or distillation its own quality gate.

Insights Worth Remembering

1. The determinant was a training cost, not a modelling requirement. A continuous flow still obeys the change of variables through its divergence. Flow matching removed the formula from the loss and pays for likelihood only when asked.

2. Every discrete-flow architecture was an accounting trick. Coupling masks, LU-parameterised convolutions and autoregressive masks exist to keep one scalar cheap. Once the scalar left training, none survived into SD3 or Movie Gen.

3. Conditional flow matching is least squares recovering a conditional mean. Tractable and intractable losses share gradients, so the network averages noisy per-pair targets. Target variance, not bias, is the enemy.

4. Flow matching versus diffusion is a false dichotomy. For Gaussian paths they are one family and DDIM is the Euler sampler. SD3 showed the time weighting, not the path alone, decided the result.

5. Straightness belongs to pairs, not to the learned field. One Euler step lands on a conditional mean. Few-step quality is bought with better couplings or distillation, and reflow pays in full-simulation quality.

6. The losing architecture was early, not wrong. TarFlow's 2.99 bits/dim on ImageNet 64 suggests discrete flows were capped by 2018-era conditioners more than by the determinant constraint.

Open Questions

Will exact-likelihood flows return for generation? Measured: TarFlow's ImageNet 64 results and STARFlow's extension to latent space and text-to-image synthesis (Gu et al., 2025, STARFlow, arXiv:2506.06276). Unknown: whether sequential sampling plus their extra denoising step can compete with 10 to 50-step flow matching at matched compute.

Should generative models train on likelihood at all? Kingma and Gao showed common diffusion objectives are weighted integrals of ELBOs over noise levels and, with monotonic weighting, equal the ELBO under Gaussian-noise augmentation (Kingma & Gao, 2023, arXiv:2303.00848). Whether one weighting can serve both bits and perception is unresolved.

How much of the gain is path, and how much weighting? Lipman et al. varied path and loss together; Esser et al. found uniform-time rectified flow lost to an epsilon baseline. No factorial study at scale separates path, parameterisation and weighting, so "straight paths drive the gains" is likely, not shown.

Is there a principled one-step flow? Reflow accumulates error and distillation adds a training run. As of September 2026 no consensus objective is accurate at many steps and usable at one.

Sources and Further Reading

1. Dinh, L., Krueger, D., & Bengio, Y. (2014). "NICE: Non-linear Independent Components Estimation." ICLR 2015 Workshop. arXiv:1410.8516
2. Rezende, D. J., & Mohamed, S. (2015). "Variational Inference with Normalizing Flows." ICML. arXiv:1505.05770
3. Dinh, L., Sohl-Dickstein, J., & Bengio, S. (2017). "Density estimation using Real NVP." ICLR. arXiv:1605.08803
4. Kingma, D. P., et al. (2016). "Improving Variational Inference with Inverse Autoregressive Flow." NeurIPS. arXiv:1606.04934
5. Papamakarios, G., Pavlakou, T., & Murray, I. (2017). "Masked Autoregressive Flow for Density Estimation." NeurIPS. arXiv:1705.07057
6. Kingma, D. P., & Dhariwal, P. (2018). "Glow: Generative Flow with Invertible 1x1 Convolutions." NeurIPS. arXiv:1807.03039
7. Grathwohl, W., et al. (2019). "FFJORD." ICLR. arXiv:1810.01367
8. Ho, J., et al. (2019). "Flow++." ICML. arXiv:1902.00275
9. Nalisnick, E., et al. (2019). "Do Deep Generative Models Know What They Don't Know?" ICLR. arXiv:1810.09136
10. Theis, L., van den Oord, A., & Bethge, M. (2016). "A note on the evaluation of generative models." ICLR. arXiv:1511.01844
11. Lipman, Y., Chen, R. T. Q., Ben-Hamu, H., Nickel, M., & Le, M. (2023). "Flow Matching for Generative Modeling." ICLR. arXiv:2210.02747
12. Liu, X., Gong, C., & Liu, Q. (2022). "Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow." arXiv:2209.03003
13. Albergo, M. S., & Vanden-Eijnden, E. (2023). "Building Normalizing Flows with Stochastic Interpolants." ICLR. arXiv:2209.15571
14. Tong, A., et al. (2023). "Improving and generalizing flow-based generative models with minibatch optimal transport." TMLR. arXiv:2302.00482
15. Esser, P., et al. (2024). "Scaling Rectified Flow Transformers for High-Resolution Image Synthesis." arXiv:2403.03206
16. Gao, R., Hoogeboom, E., et al. (2024). "Diffusion Meets Flow Matching: Two Sides of the Same Coin." ICLR 2025 Blogpost Track. diffusionflow.github.io
17. Zhai, S., et al. (2025). "Normalizing Flows are Capable Generative Models." ICML. arXiv:2412.06329
18. Polyak, A., et al. (2024). "Movie Gen: A Cast of Media Foundation Models." arXiv:2410.13720

Free to read, no ads, no sign-up. If it was useful you can buy me a coffee.