Catastrophic Forgetting

Catastrophic forgetting is the loss of skills a model already had after it is fine-tuned on a narrow task, and in LLMs it is the cost of fine-tuning that the task metric does not show. You set out to make it better at one thing and it gets worse at others: a model tuned hard on contract drafting starts fumbling arithmetic, casual conversation or formatting instructions it handled before. McCloskey and Cohen described the effect in neural networks in 1989 and called it catastrophic interference. It is still a common way for a fine-tuning project to go wrong, because the damage does not show on the metric everyone is watching.

Why one task overwrites everything else#

A model's general abilities are not stored in a separate compartment. Everything it can do, from following instructions to recalling facts, lives in one shared set of weights. Fine-tuning moves those weights in whatever direction lowers the loss on your dataset, and your dataset is all the optimizer can see. If all of the training signal is "classify this support ticket", nothing in the gradient rewards keeping conversation or world knowledge intact, so the weights that encoded them get reused.

Picture a 7B instruction model fine-tuned for three epochs on 8,000 labeled tickets. Task accuracy climbs. The model also starts answering almost any input with a category label, and stops honoring "reply in the user's language". Luo et al. measured forgetting across models from 1B to 7B parameters and found it in all of them, with forgetting growing worse as size increased in that range, which they suggest may come from the larger models' stronger starting performance. Kalajdzievski, fine-tuning Llama 2 7B chat with LoRA, found that forgetting grows as a shifted power law in the number of parameters fine-tuned and the number of update steps, and that it rises in a straight line as fine-tuning loss falls. In his experiments it could not be avoided by stopping early or by changing how many parameters were trained. Fitting the new task and forgetting old ones come from the same weight movement.

Mix general data back in#

The most reliable defense is replay: mix general-purpose data into the task dataset so the old abilities stay in the gradient. There is no published standard ratio, and the papers that study replay tune it per task, so pick a starting share, measure both the task and the general suite, and adjust. For the ticket example that means 8,000 tickets plus a set of general instruction-following examples, from an open mix or, better, from your own past production prompts that worked. Yang et al. found a further step useful: have the base model rewrite the training responses in its own words before fine-tuning. Their self-distillation method reduced forgetting on Llama 2 chat models while matching or beating ordinary fine-tuning on the task, because the model is taught the task in the way it already writes rather than in another dataset's style.

Move the weights less#

Forgetting scales with how far the weights move, so the second lever is to move them less. The learning rate sets the size of each weight update, and published full fine-tunes of instruction models use rates around 1e-5 to 2e-5: Alpaca trained its 7B model at 2e-5, and Tulu 2 used 2e-5 except 1e-5 at 70B. A rate an order of magnitude higher fits the task faster and erodes general ability faster. Keep epochs low for the same reason, since every extra pass over the training set adds update steps, and forgetting grows with update steps while task gains shrink. Save a checkpoint every few hundred steps, score each one on the task and on general checks, and ship the earliest one that clears the task bar. The last checkpoint is rarely the best trade.

Freeze the base: LoRA and adapters#

Parameter-efficient methods change the problem. With LoRA/QLoRA the base weights stay frozen and training touches only small low-rank adapter matrices; the LoRA paper cut trainable parameters by 10,000 times against full fine-tuning of GPT-3 175B. Biderman et al. compared the two directly in 2024 and titled the result plainly: LoRA learns less and forgets less. In their code and math experiments LoRA reached lower peak task performance than full fine-tuning, but kept out-of-domain ability better than full fine-tuning regularized with weight decay or dropout, and they found full fine-tuning learns weight changes with a rank 10 to 100 times higher than typical LoRA settings. Adapters also make rollback simple: you can unload one at once, serve many on a single base model, and revert without redeploying weights. LoRA reduces forgetting without removing it, as Kalajdzievski's LoRA-only experiments show, and a high-rank adapter trained at a high learning rate behaves much like a full fine-tune. Start with LoRA when the job is a format, a structure or a register the base model nearly has already, because a small low-rank change is enough for that and the base stays intact. Move to a full fine-tune only when the LoRA run plateaus below the task bar and the general suite shows you can afford the extra movement.

Measure the general suite at every checkpoint#

Forgetting does not show on the training task. Task accuracy rises, the demo looks good, and the regression appears weeks later as odd production behavior nobody connects to the fine-tune. Hold out a test set in two halves. One is a task split the model never trained on, to catch ordinary overfitting. The other is a general suite: a few hundred items covering instruction following, formatting and JSON output, multi-turn chat, safety refusals and some broad knowledge questions, ideally drawn from production prompts that worked before the fine-tune. Run both halves at every checkpoint and plot the two curves together.

Averages hide part of the damage. Harmon et al. showed in 2025 that a task average mixes items that flip from correct to wrong with items that flip the other way, so a flat average can hide large changes. Compare per-item results on the prompts you care about most, and write the ship gate with both numbers, for example task accuracy at least 90 percent and no general check down by more than 2 points. See evals for how to build the harness.

The less a fine-tune has to learn, the less it can overwrite, so narrow its job before tuning anything. On our trademark drafting system the fine-tune was given a narrow job. It handles structure, legal reasoning patterns and formal register; citations come from retrieval over a whitelisted index and are checked by validators, because fine-tuning improves how a citation is phrased, not whether the authority exists. Production logging compares the base and fine-tuned models on the same inputs across six scored axes, so a regression from a model change shows up as a score movement rather than as a complaint weeks later. If the gap you want to close is missing facts rather than missing behavior, do not fine-tune at all: put the facts in retrieval, where updating them cannot cost the model anything it already knew.