Fine-Tuning (SFT)

Fine-tuning means taking an already-trained model and training it a little further on examples you have hand-picked, so its weights shift toward the behavior you want. The most common flavor is supervised fine-tuning (SFT): you assemble pairs of inputs and the exact outputs you wish the model had produced, then run a few more passes of gradient descent so those outputs become the model's default. Nothing about the architecture changes; you are nudging the same weights with new evidence. Done well, it turns a capable generalist into a specialist that no longer needs to be talked into the right behavior on every call.

How supervised fine-tuning works#

An SFT job takes a dataset of conversations, each ending in the response you want the model to produce, and trains the model to make that response more likely. On hosted platforms you upload a JSONL file of chat-formatted examples and the provider runs the job; OpenAI, Google, and Mistral all offer this, and OpenAI's platform now exposes SFT alongside preference-based methods like DPO and reinforcement fine-tuning for cases where you have rankings or a programmatic grader instead of gold answers. For open-weight models such as Llama, Qwen, or Mistral, you run the same loop yourself, almost always through a parameter-efficient adapter rather than touching every weight. Either way the mental model is identical: the model already knows how to write; you are showing it, by example, exactly which of the things it could write is the one you want.

The adaptation ladder#

Think of model adaptation as a ladder where each rung costs more and commits you harder. The cheapest rung is a good prompt: minutes to write, instantly reversible, zero infrastructure. Next is few-shot prompting, a form of in-context learning where you paste a handful of worked examples into the context; it often fixes formatting and style at the price of extra tokens on every call. The third rung is RAG, which injects fresh facts at query time; it needs real infrastructure, but the knowledge stays editable. Only when those run out of road do you reach for fine-tuning, because it is the most expensive and least reversible rung: you now own a dataset, a training pipeline, an evaluation suite, and a custom model artifact that must be versioned, hosted, and retrained when the base model or the task changes. The operating rule is simple: climb in order and stop at the first rung that works.

When it wins, and when it is the wrong tool#

Fine-tuning wins when the problem is about behavior rather than knowledge. The classic cases: a rigid output format the model keeps drifting from, a house style or tone that a prompt cannot fully pin down, a narrow task like classification or extraction where a small specialized model can match a frontier model, and latency or cost pressure, where you bake a 2,000-token instruction-heavy prompt into the weights so you stop paying for it on every call. A worked example: a team routing support tickets across 40 categories fine-tuned an open 8B model on a few thousand labeled tickets and matched the accuracy of a frontier model that cost roughly twenty times more per request, at a fraction of the latency.

The flip side is just as important. Fine-tuning is for form, not facts. It will not reliably teach the model new knowledge, and it cannot keep up with facts that change weekly; the weights are frozen at training time and go stale the moment your docs change. If your failures come from missing or outdated information, the answer is RAG. If they come from wrong format, tone, or behavior, fine-tuning is on the table.

Data quality over quantity#

The dataset is the single biggest lever, and quality dominates volume. The model faithfully imitates whatever you show it, including your inconsistent labels, sloppy formatting, and contradictions, so a few hundred clean, consistent, on-distribution examples routinely beat tens of thousands of noisy ones. Realistic sizing as of 2026: hosted APIs accept as few as 10 examples, but meaningful improvement typically starts around 50 to 100 well-crafted demonstrations. Classification and extraction tasks usually land in the 200 to 500 range, style and content generation in the 500 to 2,000 range, and complex domain behavior in the 1,000 to 5,000 range. Below roughly 50 examples you are doing few-shot prompting with extra steps. The real work is curation: deduplicate, fix label disagreements, make formatting uniform, and hold out a genuine test set the model never sees, with evals defined before you train so you can prove the fine-tune actually improved anything.

LoRA, forgetting, and distillation#

Full fine-tuning updates every weight, which is expensive and produces a whole new copy of the model. The default cheap path is LoRA: freeze the base model and train small low-rank adapter matrices, typically under 1 percent of the parameters, that get added to the model's behavior at inference. Quality is close to a full fine-tune for most behavioral tasks, and adapters are small files you can swap per customer or per task. QLoRA pushes cost down further by quantizing the frozen base to 4 bits; the original paper fine-tuned a 65B model on a single 48GB GPU.

The standing risk is catastrophic forgetting: train hard on a narrow task and the model can lose general skills it used to have, like instruction following or refusal behavior. Light training (low learning rate, one to three epochs), mixing in some general data, and LoRA's frozen base all limit the damage, but you should always re-run broad evals after tuning, not just task evals.

Finally, distillation is the cost play that ties this together: use a large teacher model to generate high-quality outputs for your task, then fine-tune a small student on them. The idea dates to Hinton's 2015 paper and is now productized; OpenAI's platform, for example, lets you store frontier-model completions and distill them directly into a cheaper model. For a narrow, well-defined task, a distilled student often delivers near-teacher quality at a tenth or less of the serving cost.

Practical takeaway#

Climb the ladder in order: prompt, few-shot, RAG, and only then fine-tune, stopping at the first rung that works. Reach for fine-tuning to lock in format, style, narrow-task accuracy, or cost and latency wins, never to add fresh facts. Spend your budget on a small pristine dataset and honest held-out evals rather than scale, default to LoRA or QLoRA over full fine-tuning, watch for forgetting, and when the goal is cheap serving, distill a big teacher into a small student instead of tuning the big model itself.