LambdaRank and LambdaMART

nDCG is a step function of the model's scores. Change a score slightly and nothing happens until two documents swap positions, at which point the metric jumps. It has zero gradient almost everywhere and is undefined at the jumps, so gradient descent has nothing to work with. This is the central obstacle in learning to rank, and LambdaRank's answer was to stop trying to solve it.

Skipping the loss

Gradient descent needs a gradient, not a loss. Burges' observation was that you can specify the gradient directly and never write down the function it integrates (Burges, 2010, From RankNet to LambdaRank to LambdaMART: An Overview, Microsoft Research MSR-TR-2010-82).

For a pair of documents \((i, j)\) where \(i\) should outrank \(j\), the RankNet gradient is \(\sigma / (1 + e^{\sigma(s_i - s_j)})\). LambdaRank multiplies it by the metric change that swapping the pair would produce:

Each document's lambda is the sum over all pairs it participates in, and that vector of lambdas is used as the gradient. The interpretation is a force: each document is pushed up or down with a magnitude proportional to how much the metric would benefit.

The consequence is a shift in where the model spends effort. Swapping ranks 1 and 2 changes nDCG substantially, since the discount is \(1/\log_2(2)\) versus \(1/\log_2(3)\); swapping ranks 99 and 100 changes it negligibly. Under a plain pairwise loss those two errors are equal. Under LambdaRank the first receives a gradient perhaps a hundred times larger. The model learns to be right at the top, which is what the metric rewards and what users experience.

Empirical work subsequently showed that these lambdas correspond to the gradient of an actual, if unusual, loss function, so the construction is better founded than the "we just made up a gradient" framing suggests.

LambdaMART

LambdaMART is LambdaRank's gradient inside gradient-boosted regression trees rather than a neural network. Each boosting iteration fits a tree to the lambdas, exactly as ordinary gradient boosting fits a tree to a loss gradient.

The combination was dominant for a decade in commercial search and in every ranking competition of the period, and the reasons are the ones that make GBDT strong on tabular data generally. Ranking features are tabular: BM25 scores per field, click-through rates, freshness, PageRank-like authority, query-document match counts. They are heterogeneous in scale, contain many weakly informative columns, and have irregular relationships with relevance. Trees need no normalisation, ignore useless features, and represent thresholds exactly.

When it breaks

The lambdas depend on the current ranking, which changes every iteration. \(|\Delta\text{nDCG}|\) must be recomputed from the model's own current ordering, so the objective is non-stationary and the training loop is more intricate than plain boosting. It also means the training is sensitive to the ordering early on, when the model is poor.

Truncation at \(k\) creates a blind spot. Optimising \(|\Delta\text{nDCG@10}|\) gives near-zero gradient to documents that are all outside the top 10, so the model receives almost no signal about how to promote a relevant document from rank 300. Where the candidate generator is imperfect, this matters, and it is one reason deep-list quality can be poor in a system that looks excellent at \(k=10\).

Text features are second-class. LambdaMART consumes a feature vector, so the semantic matching has to happen upstream in the features. This is precisely where neural rankers displaced it: a cross-encoder reads the query and document text jointly and needs no hand-built match features. The modern architecture is usually a neural reranker over the top candidates, with a GBDT combining its output alongside the behavioural and quality features.

Position bias is not addressed. If the labels are clicks, LambdaMART faithfully optimises for reproducing the biased click distribution. The propensity correction is a separate mechanism and must be applied to the labels or the loss weights.