Before combining AI adapters, check what they actually change

Small updates, potentially large interference

LoRA, short for low-rank adaptation, represents changes to a language model’s numerical weights using relatively small sets of additional parameters. These adapters can specialize a model without updating all its original weights. Once trained, several adapters can be combined into one model without another round of training.

The difficulty is preserving existing abilities while adding new ones. A model adapted for several purposes might become worse at solving arithmetic problems or writing code. Researchers often describe such deterioration as catastrophic forgetting.

In an arXiv preprint, Lenny Aharon, Neta Glazer and Lior Aharon ask whether some apparent forgetting instead reflects unsuitable ingredients. Their experiments distinguish adapter compatibility, individual adapter performance and the effects of combining adapters.

Two checks before merging

The researchers first checked whether each adapter’s configuration named the intended base-model version. Similar names were not enough: an update trained for Llama-3 was rejected for experiments using Llama-3.1.

They also reconstructed the actual weight change produced by each adapter and measured its overall size, called its norm. This matters because an adapter can contain substantial files yet make almost no change to the model.

One candidate code adapter had a change norm of roughly 0.02 and behaved almost like the unchanged model. Another targeted the wrong Llama version and produced substantial damage when included. These are different problems: the first contributes little, while the second can make a merge look much worse.

In a later comparison using the same evaluation setup, an original, unaudited four-adapter pool scored 49.1% on the GSM8K arithmetic benchmark, versus 61.5% for the audited pool. That approximately 12-percentage-point difference illustrates how adapter selection can alter conclusions about merging. It is separate from the paper’s headline results, which used an earlier evaluation-harness version.

Simple scaling preserved arithmetic, not everything

The main experiment combined four audited adapters for Llama-3.1-8B-Instruct, covering mathematics, coding, general language tasks and medicine. The researchers evaluated each adapter separately as well as in combination.

They kept training examples used for probing and parameter selection separate from the final GSM8K test set. This separation reduces the risk of selecting a method because it happens to perform well on the very questions used to report its success.

Simply adding all four weight changes reduced GSM8K accuracy from 70.7% to 60.3%. On HumanEval, a programming benchmark, the share of problems solved by a single generated answer fell from 62.8% to 44.5%.

Task Arithmetic, which here multiplied each adapter’s contribution by 0.5 before adding it, performed considerably better: 70.5% on GSM8K and 56.1% on HumanEval. Another method, TIES, scored 71.2% and 54.3%, respectively.

The arithmetic scores were within one reported standard error of the unchanged model. That is not proof of equivalence, and the coding results still showed losses. Preserving one benchmark should not be mistaken for preserving every capability.

There is another important qualification: none of the individual Llama adapters produced a convincing GSM8K improvement beyond measurement noise. Remaining near the original model therefore demonstrates retention, not successful accumulation of useful new arithmetic skills.

Could subtracting an adapter help?

An exploratory search allowed adapter contributions to be negative as well as positive. On a 200-example training probe, the selected mathematics coefficient was −0.5: the merge subtracted half that adapter’s weight change rather than adding it.

The resulting configuration reached 67.2% on the full GSM8K test—better than naive merging, but worse than the unchanged model and the tested Task Arithmetic and TIES settings. It also sacrificed the mathematics adapter’s observed contribution on MATH-500, a separate benchmark.

This is a search-space observation, not an established superior merging method. A negative contribution may help a protected task while undermining the specialization the adapter was supposed to supply.

Compatibility is only the beginning

A smaller replication used Mistral-7B-Instruct-v0.3. Two of four candidate adapters failed the base-version check. With the remaining pair, Task Arithmetic again stayed close to the original GSM8K score, while naive merging reduced it. Coding replication was not reported because of evaluation-format problems.

The evidence remains limited to small adapter collections and two model families for merging experiments. Configuration metadata can also be wrong, and passing compatibility and norm checks does not establish usefulness or safety.

The practical lesson is narrower than a solution to forgetting: check that updates belong to the intended model, establish what each does alone, and keep tuning separate from final testing before judging how well a merger works.