TIES and DARE Merging

Merging the weights of two fine-tuned language models with a simple average reliably degrades performance on both tasks. The culprit is not averaging itself; it is interference between the delta parameters each model has accumulated. TIES and DARE are two algorithmic responses to that interference, and together they underpin most of the serious model-merging work happening in 2023-2025.

The Task Vector Picture

To reason about merging, you need the concept of a task vector (introduced in Ilharco et al., ICLR 2023). Given a pre-trained model with weights \(\theta_0\) and a fine-tuned model with weights \(\theta_{ft}\), the task vector is simply:

A merged model is then assembled as:

where \(\lambda\) is a scaling coefficient and the sum runs over \(n\) fine-tuned models you want to combine. The arithmetic is seductive in its simplicity. The problem is that \(\tau_i\) and \(\tau_j\) will often disagree on the same weight: one pushes it positive, the other pushes it negative. Summing them cancels signal from both models. That is parameter interference.

TIES: Trim, Elect Sign, Merge

TIES-Merging (Yadav et al., NeurIPS 2023) attacks interference with three sequential steps applied to each weight position independently.

Step 1 - Trim. Most fine-tuned weights move very little from \(\theta_0\). Those tiny perturbations are noise, not signal, and including them adds interference without adding task knowledge. TIES trims by zeroing out all delta values whose absolute magnitude falls below a top-\(k\) threshold: only the \(k\%\) largest-magnitude deltas in each task vector are kept. Typical values of \(k\) are 20 to 50 percent.

Step 2 - Elect sign. After trimming, each surviving weight position \(j\) has a set of non-zero delta values across the \(n\) models. Some are positive, some negative. TIES resolves this by electing a consensus sign for each position: whichever sign has greater total absolute mass wins.

Step 3 - Disjoint merge. Only the models whose delta at position \(j\) agrees with the elected sign \(\hat{s}_j\) contribute to the final value. The rest are masked out. The survivors are averaged:

where \(\mathcal{A}_j\) is the subset of models with sign-aligned deltas at position \(j\).

The intuition: a weight that strongly wants to go up across several tasks should go up. A weight pulled in opposite directions by different tasks is a conflict; TIES lets the majority coalition win rather than letting them cancel.

# Pseudo-trace for 3 models, single weight position j
delta = [-0.8, +0.3, +0.6]          # raw deltas after trim
elected_sign = sign(sum) = sign(0.1) = +1
aligned_models = {model_2, model_3}  # model_1 is negative, excluded
merged_delta = mean([+0.3, +0.6]) = +0.45

DARE: Drop And Rescale

DARE (Yu et al., 2023) starts from a different empirical observation: the deltas produced by supervised fine-tuning are extremely sparse in practice. Specifically, for instruction-tuned 7B-class models the delta magnitudes are typically under 0.002 in absolute value, and the vast majority do not encode any task-specific semantics at all. DARE takes advantage of this by randomly pruning deltas at a high dropout rate \(p\) (often 0.9 to 0.99) and then rescaling the survivors to keep the expected weight sum unchanged:

This is structurally identical to dropout applied to delta parameters rather than activations. After sparsifying, the thinned-out task vectors can be summed with far less interference because fewer positions overlap. DARE then typically hands off to a standard merge (either simple addition or TIES) on the pruned deltas.

In practice, DARE is often used as a preprocessing step before TIES. The combination, sometimes called DARE-TIES, gives the best empirical results: DARE reduces the density of conflicting deltas, and TIES resolves the sign ambiguity in whatever overlapping deltas remain. The HuggingFace mergekit library exposes both dare_ties and dare_linear as first-class merge strategies.

Why This Works At All

Both methods rest on the same underlying fact: supervised fine-tuning moves a relatively small number of weights strongly and leaves the majority nearly unchanged. The strong movers encode task-specific inductive biases; the weak movers are numerical noise. TIES uses the top-k filter to separate them. DARE uses random masking to do the same thing in expectation.

The rescaling step in DARE is non-trivial. Without it, dropping 90% of deltas reduces the effective weight magnitude, shifting the merged model's outputs in a way that degrades perplexity. The \(1/(1-p)\) correction restores the original expected magnitude, analogous to inverted dropout in training. Empirically, DARE can prune 90-99% of deltas with negligible performance loss on the individual model, and models merged from sparsified deltas often outperform models merged from the full deltas.

The NeurIPS 2023 TIES paper reported improvements over simple task arithmetic across vision and NLP benchmarks when merging between 2 and 8 models simultaneously. The DARE paper showed a merged 7B model reaching top-tier performance on the Open LLM Leaderboard as of late 2023.

When It Falls Down

Heterogeneous architectures. Both methods require that all models share the exact same architecture and a common pre-trained checkpoint \(\theta_0\). Merging a Llama-3-8B fine-tune with a Mistral-7B fine-tune makes no sense at the weight level; the token embeddings alone are sized differently. This is a hard constraint, not a nuance.

Divergent fine-tuning regimes. If one model was RLHF-tuned with a large KL penalty and another was SFT-only on a narrow domain, the delta distributions can be incommensurable. TIES sign election will still run, but the elected signs may not represent any coherent task combination.

Catastrophic forgetting of alignment. Merging a safety-tuned model with an instruction-tuned model that received no safety training can dilute the refusal behaviour of the former. DARE's random pruning is symmetric and does not preferentially preserve safety-relevant weights. This is an active research concern.

High-\(p\) DARE on small models. The sparsity argument is empirically weaker for smaller models (below roughly 3B parameters). Delta magnitudes are less redundant, so aggressively pruning at \(p = 0.9\) can cause noticeable regression. Tuning \(p\) per model size matters.

No gradient signal for \(\lambda\). The scaling coefficient \(\lambda\) in task arithmetic is typically set by grid search or human intuition. There is no efficient closed-form solution; too large and you overfit to the merged tasks, too small and you preserve \(\theta_0\) at the expense of fine-tuned capabilities.

Layer sensitivity is ignored. Both TIES and DARE apply uniform treatment across all layers. In practice, attention heads in early layers are far more sensitive to parameter perturbation than mid-network MLP layers. Layer-wise scaling, as used in SLERP or DARE variants, can close some of this gap.

Further Reading

• Yadav, P. et al. (2023). "TIES-Merging: Resolving Interference when Merging Models." NeurIPS 2023. https://arxiv.org/abs/2306.01708
• Yu, L. et al. (2023). "Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch." https://arxiv.org/abs/2311.03099
• Ilharco, G. et al. (2023). "Editing Models with Task Arithmetic." ICLR 2023. https://arxiv.org/abs/2212.04089
• Labonne, M. (2024). "Merge Large Language Models with mergekit." Hugging Face blog. https://huggingface.co/blog/mlabonne/merge-models