Looped Transformers and Recurrent Depth

Parameters and compute are welded together in a standard transformer: 32 distinct layers means 32 blocks' worth of weights and 32 blocks' worth of FLOPs. Unwelding them is an old idea with a new reason to care. If a problem needs many sequential steps of refinement but not much stored knowledge, you can run one block repeatedly instead of stacking many, and pay in time rather than in memory.

Universal Transformers proposed exactly this in 2018: replace the stack with a single layer looped a fixed number of times, with an Adaptive Computation Time halting mechanism to let each token choose its own loop count (Dehghani et al., 2018, Universal Transformers, arXiv:1807.03819). ALBERT made the parameter-saving version famous a year later, tying all layers to get 18x fewer parameters than BERT-large, 18M against 334M, at close to comparable quality (Lan et al., 2019, ALBERT, arXiv:1909.11942).

Depth is the resource being bought

The sharpest modern statement of why looping helps is that many reasoning problems need depth and not parameters. On addition, \(p\)-hop induction and grade-school maths, a \(k\)-layer transformer looped \(L\) times nearly matches a \(kL\)-layer model and is far better than the \(k\)-layer model it is built from (Saunshi et al., 2025, Reasoning with Latent Thoughts: On the Power of Looped Transformers, ICLR 2025, arXiv:2502.17416). The intuition is that these tasks are iterative algorithms; an iterative algorithm needs the loop, not a different function at every step.

Run the argument forward and the loop count becomes a test-time knob. A 3.5B-parameter recurrent-depth model trained on 800B tokens improves on reasoning benchmarks as you unroll it further, up to a compute load equivalent to a 50B-parameter model, without emitting a single extra token of chain of thought (Geiping et al., 2025, Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach, arXiv:2502.05171). That is a different bargain from the usual one: no specialised reasoning traces to train on, no long context consumed by the reasoning, and an adjustable budget per request.

The on-device version of the same trick

Where memory is the binding constraint, looping is a parameter-count play rather than a reasoning play. MobileLLM found that at sub-billion scale, architecture beats data volume, and combined deep-and-thin shapes, embedding sharing and grouped-query attention with immediate block-wise weight sharing: run a block twice in a row before moving on, so the weights are already in cache the second time (Liu et al., 2024, MobileLLM, ICML 2024, arXiv:2402.14905).

The word immediate is the whole engineering content of that sentence. A phone's inference cost at batch size one is dominated by moving weights from memory into the accelerator, not by arithmetic. Looping a block immediately means the second pass reads weights that are still resident, so the extra depth costs close to nothing in bandwidth. Looping the same block with 15 other blocks in between would evict it and cost a full reload.

When it breaks

Bandwidth-free depth requires locality. The MobileLLM result depends on the repeats being adjacent. Any interleaving that pushes the shared weights out of cache converts a nearly free extra pass into a full memory round trip, which on a bandwidth-bound device is the entire cost of the layer.

Latency scales with the loop count and nobody batched for it. A serving stack sized for a fixed 32-layer forward pass does not know what to do with a request that wants 64 unrolls. Continuous batching assumes a uniform per-token cost across the batch; adaptive depth breaks that assumption, and in practice deployments pin the loop count per request class rather than letting each token halt on its own.

Shared weights are shared failure modes. A tied block must be useful at every position in the computation, so it cannot specialise the way a distinct layer can. That is why looped models tend to trail parameter-matched stacks on knowledge-heavy evaluations while matching or beating them on algorithmic ones. Choose according to which half of your evaluation suite you care about.

Latent reasoning is harder to audit. Chain-of-thought is at least inspectable text, with known limits on how faithful it is (see chain-of-thought faithfulness). Recurrent-depth computation leaves no trace in token space at all. The reasoning happens in the residual stream, so oversight has to reach into activations rather than reading output, which is a strictly harder interpretability problem.