At inference time, a merged model costs:

One call per request — The constituents stop existing — you have one set of weights. Ensembling is the one with a higher inference cost.

Sequential finetuning on task A then task B mainly risks:

Catastrophic forgetting — Training on B overwrites what the model learned about A (Kirkpatrick et al., 2016).

A task vector is obtained by:

Subtracting the base model from a finetuned model — Also called delta parameters — a direction in weight space capturing what the finetuning taught.

Task arithmetic (Ilharco et al., 2022) allows you to remove a capability by:

Subtracting its task vector — Used for removing invasive capabilities such as facial recognition, and for pushing back against pre-training biases.

Linear combination of task vectors is most effective when the models:

Were finetuned on top of the same base model — Then the vectors are measured in the same coordinate system, so adding them is meaningful rather than merely possible.

Before summing models whose parameter values sit in different ranges, you should:

Rescale them — Otherwise the larger-scaled model simply dominates the result.

Model soups (Wortsman et al., 2022) showed that averaging the whole weights of multiple finetuned models:

Improves accuracy without increasing inference time — Though in practice it is more common to combine specific components such as adapters.

SLERP interpolates between two vectors by:

Walking the shortest path along a sphere's surface — Which preserves magnitude, where a straight-line average of two divergent vectors produces something shorter than either.

A SLERP interpolation factor below 0.5 produces a result:

Closer to the first vector — And 0.5 is exactly halfway between them.

AI-ENGINEERING

advanced

Model Merging

AI Engineer path · lesson 100 of 124

Combine models by mixing their parameters rather than their outputs. Build a task vector, use task arithmetic to add a capability or subtract one, interpolate along a sphere instead of a straight line, and see why TIES keeps only the top 20%.

The idea

Blending paint rather than hanging two paintings side by side. Ensembling keeps both canvases and asks a judge which to look at; merging mixes the pigments into one tin. The tin is cheaper to carry — and mixing badly gives you brown.

How to use

On the task-arithmetic tab, toggle subtraction to see the same machinery used to remove a capability rather than add one.

Two things merging buys: performance — a model answering the first 60% of questions plus one answering the last 60% might together answer 80% — and a reduced memory footprint, since two task models become one.

Three ways to get one model good at two tasks

Finetune on A and on B independently — in parallel — and combine the results.

Each model learns its task with the base model's full attention, and neither overwrites the other. Better per-task learning with less risk of catastrophic forgetting.

Merging is not ensembling

Merging mixes parameters

The constituent models stop existing; you get one set of weights. One inference call per request.

Ensembling combines outputs

Each constituent model stays intact, and their answers are combined by majority vote or a trainable module. Higher inference cost — multiple calls per request.

Many models at the top of Hugging Face's Open LLM Leaderboard are merged models — which is partly a statement about what merging achieves and partly a statement about what a leaderboard measures.

On-device deployment

Merge several task models into one that fits limited device memory. Needed when data cannot leave the device for privacy reasons, or when the internet is limited or unreliable — and it reduces inference cost either way.

Federated learning

Deploy model X to many devices, let each copy learn from local data, then merge the copies back into a new base model (McMahan et al., 2016). Merging is the step that makes the whole pattern work.

Key insight

Without further finetuning, model merging can be done without GPUs. That single property is why it became an indie-developer technique rather than a lab one — the expensive part of making a new model was the training run, and merging skips it.

What does this merge method need?

Sort

Sort each method by whether it needs a common base model.

Tap a piece, then tap where it goes — or drag it. Tap the tray to take one back.

Needs a shared base model

Drop items here

Works without one

Drop items here

5 pieces left to place

Test your knowledge

20 questions

Question 1 of 20

Score: 0

Merging differs from ensembling in that merging combines:

The short version

In more depth

Model Merging

Finetuning alters one model; merging combines several, by mixing parameters rather than outputs.

Merging vs ensembling

	Merging	Ensembling
What is combined	Parameters	Only outputs
Constituent models	Cease to exist	Stay intact
Inference cost	One call	Multiple calls per request

Without further finetuning, merging can be done without GPUs — which is why it became an indie-developer technique. Many models at the top of Hugging Face's Open LLM Leaderboard are merges.

Multi-task finetuning, three ways

1.Simultaneous on a combined dataset — harder to learn multiple skills at once; needs more data and training.

2.Sequential (A then B) — risks catastrophic forgetting (Kirkpatrick et al., 2016).

3.Merging — finetune separately in parallel, then combine. Better per-task learning, less forgetting.

Approach 1 — Summing

Linear combination: Merge(A,B) = (W_A·A + W_B·B) / (W_A + W_B). Studied as early as Perrone (1993). Model soups (Wortsman et al., 2022): averaging whole weights improves accuracy without increasing inference time. Rescale first if parameter ranges differ.

Task vectors: finetuned − base = a direction capturing the task. Task arithmetic (Ilharco et al., 2022) — add to combine capabilities, subtract to remove them (e.g. facial recognition, pre-training biases). Most effective on models sharing a base.

SLERP: walk the sphere's surface rather than cutting through it. Factor < 0.5 is closer to the first vector; 0.5 is halfway. Defined for only two vectors — apply sequentially for more.

Pruning first: TIES-Merging (Yadav et al., 2023) showed keeping the top 20% of task-vector parameters gives comparable performance to keeping 100%. DARE (Yu et al., 2023) drops and rescales. The more models merged, the more important pruning is. Note: this improves performance, not memory — only the task vectors become sparse.

Approach 2 — Layer stacking (passthrough, frankenmerging)

Typically requires further finetuning.

Goliath-120B — two finetuned Llama 2-70Bs (Xwin, Euryale), 72 of 80 layers from each.

Sparse upcycling (Komatsuzaki et al., 2022) — copy layers, add a router, train. Can produce MoE models that outperform MoE trained from scratch.

Mixture-of-Agents — Together AI mixed six weaker open models to comparable performance to GPT-4o on some benchmarks.

SOLAR 10.7B — from a 7B, 32-layer model: copy, sum 16 layers and stack the rest ⇒ 32 × 2 − 16 = 48 layers, then train further.