Parameter-Efficient Fine-Tuning
LoRA, QLoRA, DoRA, adapters and soft prompts — adapting big models on small budgets.
20concepts
140flashcards
145minutes of reading
- 01 Adapter Modules Adapter modules insert small trainable bottleneck layers into a frozen pretrained transformer, achieving near-full fine-tuning performance while updating fewer than 4% of the model's parameters.
- 02 Catastrophic Forgetting in Fine-Tuning Full fine-tuning on a narrow task silently destroys general capabilities baked in during pre-training, and understanding why this happens is the prerequisite for choosing any mitigation strategy.
- 03 Choosing LoRA Rank and Alpha LoRA rank r controls how much task-specific capacity the adapter has, and alpha controls the scaling of that update; choosing them poorly wastes parameters or destabilises training.
- 04 IA3 and Scaling-Vector Methods IA3 fine-tunes a transformer by learning one scaling vector per targeted activation stream, achieving roughly 0.01% trainable parameters while matching full fine-tuning accuracy on several benchmarks.
- 05 LoRA Training Pitfalls LoRA's low-rank approximation introduces subtle failure modes around rank selection, learning rate asymmetry, and target-module coverage that can silently degrade fine-tuned model quality.
- 06 LoRA vs Full Fine-Tuning LoRA constrains weight updates to low-rank matrices, cutting trainable parameters by orders of magnitude while matching full fine-tuning quality on most tasks, but that constraint is also the source of its failure modes.
- 07 LoRA: Low-Rank Adaptation LoRA freezes a pretrained model's weights and inserts trainable low-rank matrix pairs into each target layer, cutting trainable parameters by up to 10,000x with no added inference latency.
- 08 Merging LoRA into Base Weights After LoRA training, the low-rank adapter matrices can be folded directly into the frozen base weights, eliminating inference overhead and adapter management complexity while producing a standard dense model.
- 09 Prefix Tuning Prefix tuning freezes all pretrained model weights and instead optimises a small set of continuous, task-specific vectors prepended to every layer's key-value cache, achieving within a few points of full fine-tuning while training roughly 0.1% of the original parameters.
- 10 Prompt Tuning with Soft Prompts Soft-prompt tuning prepends a small set of learnable continuous vectors to the input of a frozen language model, achieving full-fine-tuning parity at billion-parameter scale while touching less than 0.1% of model weights.
- 11 QLoRA: 4-bit Base with LoRA QLoRA reduces the GPU memory needed to fine-tune a 65-billion-parameter model from hundreds of gigabytes to a single 48 GB card by storing the frozen base model in 4-bit precision and training only small LoRA adapters at full 16-bit precision.
- 12 The Intrinsic-Dimension Hypothesis Pre-trained language models can be fine-tuned in a surprisingly low-dimensional subspace of their parameter space, and measuring that dimension explains why parameter-efficient methods work at all.
- 13 The Memory Maths of Fine-Tuning Fine-tuning a 7B-parameter model with full gradients and Adam state consumes roughly 112 GB of GPU memory; parameter-efficient methods cut that by an order of magnitude by training only a tiny fraction of weights.
- 14 Which Layers to Adapt Choosing which transformer weight matrices to inject LoRA or adapter modules into determines both the parameter budget and downstream task quality, and the right choice is not self-evident.
- 15 Why Parameter-Efficient Fine-Tuning Parameter-efficient fine-tuning methods adapt large pretrained models to new tasks by training only a small fraction of parameters, making customisation practical without the compute and storage costs of full fine-tuning.