Few-Shot Prompting & In-Context Learning

Few-shot prompting is putting a few worked examples of a task in the prompt so the model follows their pattern. In-context learning is the name for the model's ability to pick up a task that way, with no training at all. You show it a few input and output pairs, it infers the pattern, and it applies the pattern to your real input. The weights never change. Whatever the model "learned" exists only inside that one call and is gone when the call ends. The GPT-3 paper, "Language Models are Few-Shot Learners" (Brown et al., 2020), made this its headline result: one frozen model did many tasks competitively just by being shown what each task looked like, and the larger the model, the more it got out of the examples.

Zero-shot, one-shot and few-shot prompting#

The number of examples sets the name. Zero-shot is an instruction only: "Classify this support ticket as billing, technical, or account." One-shot adds a single worked example. Few-shot adds several. The GPT-3 paper used between 10 and 100, as many as fit in its 2,048-token window, and accuracy rose with the count on most of its tasks, with each added example worth less than the one before. A production prompt usually carries far fewer, because every example costs tokens on every call.

Why do examples beat a written rule?#

A written rule describes the task in the abstract, and an example shows it. For a precise output format, an unusual label set, or a judgement call at the boundary, two examples say more than two paragraphs, because prose leaves edge cases open and an example settles them.

Take a model asked, with no examples, to extract {"name", "amount"} from invoice text. You get JSON most of the time, but the key names drift, the currency symbol sometimes lands inside the number, and now and then a helpful sentence wraps the output. Add two examples, one ordinary and one where no amount is present and the answer is "amount": null, and the output settles on your schema, including the "nothing found" case you never managed to describe in words. Pair the examples with structured output enforcement to close the last gap.

What do the examples cost?#

Every example sits in the context window on every call. Ten examples at 150 tokens each add 1,500 tokens to each request. As of October 2026, Anthropic lists Claude Opus 5.5 at $4 per million input tokens, so those examples cost $0.006 a call, or $6,000 per million calls, before anything else in the prompt.

Prompt caching changes that. If the examples form a stable prefix at the top of the prompt, later calls read them from the cache. On Opus 5.5 a cache read costs 5% of the base input price, and on most other Claude models it costs 10%, so the $6,000 falls to about $300 plus the occasional cache write. Anthropic's documentation also says time to first token generally improves for long cached prompts, without giving a figure. To keep the prefix cacheable, put the examples before the variable user input and never shuffle or rotate them per request, because a changed prefix is a cache miss and the full price comes back. Check the size too. A prefix shorter than the model's minimum is processed without caching and without an error, and the minimum is 512 tokens on Opus 5.5 but 4,096 on Haiku 4.5, so a short example block can look cached in the design and cost full price in the bill.

How should the examples be chosen and ordered?#

Cover the space rather than repeating the easy case. Five near-identical happy-path examples teach less than three that span the task: one typical case, one boundary case, and one showing what to do when the input is malformed or the answer is "none". Whatever the examples leave out stays undefined, and the model fills it in.

Keep the format consistent across examples. On our trademark drafting platform, which is a fine-tune rather than a prompt, Bigcircle first unified years of attorney opinions into one five-section template, because a model trained on a dozen individual writing styles averages them. A few-shot set copied from documents written by different people carries the same risk at a smaller scale, so rewrite the examples into the one format you want back.

Balance the labels and watch the order. Zhao et al. (2021) found three biases in few-shot prompts: the model leans toward the label that appears most often, toward the label of the last examples, and toward tokens common in its training data. Reordering the same four examples moved the 2.7 billion parameter GPT-3 model on a sentiment task from 54.3% accuracy, near chance, to 93.4%. Lu et al. (2022) found that order alone could separate near state-of-the-art accuracy from random guessing. So keep label counts even, interleave them so the last few examples do not all say "approve", and rerun your evals after any change to the set or its order.

When should the examples move into weights?#

Long context windows turned few-shot into many-shot. Agarwal et al. (2024) showed that hundreds or thousands of examples keep improving results on many tasks and can override biases from pretraining. On their machine translation tasks, for Bemba and Kurdish, many-shot prompting came close to supervised fine-tuning. With a 1 million token window and a cached prefix, a prompt holding several hundred labelled examples is a real design, and it stays editable in a way trained weights are not.

Fine-tuning earns its cost when the example set no longer belongs in the prompt. Move when the per-call token cost of the examples exceeds the cost of a training run spread over your traffic, when a long prompt breaks your latency budget even when cached, or when a smaller tuned model would serve the task for less. Until then the examples win on change: a new set ships in minutes, rolls back at once, and needs no training infrastructure. A task whose wanted behaviour still changes every week is a reason to stay in the prompt even when the token bill argues otherwise.