Few-Shot Prompting and In-Context Learning

Learning without training

Every model you deploy arrives with its behavior frozen. Its weights, the billions of numbers that encode what it knows and how it responds, were set during training and do not move when you use it. So the practical question of adaptation is: how do you get a fixed model to do your task, in your format, without the expense of changing those weights? Few-shot prompting is the most direct answer. You put a few worked examples of the task directly into the prompt, and the model does the rest.

The mechanism behind this is called in-context learning, and it is a genuinely surprising property of large models. Shown a handful of input/output pairs inside a single prompt, the model infers the pattern they share and continues it for a new input, all at inference time, in a single forward pass. Nothing is stored. Nothing is trained. The "learning" happens entirely within the context window and evaporates the moment the request ends. Ask the same model the same question tomorrow without the examples, and it will have no memory of the pattern you taught it today.

That is the whole appeal. Because few-shot touches no weights, there is no training run, no labeled dataset to assemble, no pipeline to stand up, and no model artifact to host and version. You edit a prompt, you get new behavior, and you can iterate in seconds. On the adaptation ladder, few-shot sits on the second rung, just above plain instructions in a system prompt and well below anything that involves retraining. It is the cheapest form of real task adaptation available, which is exactly why it should be the first thing you try and, just as often, the thing you never need to move past.

The discipline that follows from this is worth stating up front, because the rest of the article keeps returning to it. Few-shot is cheap enough that it is tempting to reach past it for something heavier. Most of the time you should not. Understand precisely what examples can and cannot steer, invest your effort in choosing them well, and you will get remarkably far without ever leaving the prompt.

Zero, one, few

The "shot" in few-shot counts the examples you place in the prompt, and it names a spectrum rather than a single technique. Zero-shot gives the model no examples at all: you describe the task in words and trust its training to cover the rest. One-shot supplies a single demonstration, which is often enough to pin down an ambiguous output format. Few-shot supplies several, letting the model triangulate the pattern from more than one instance and see how the task handles variation.

More examples generally help, but only up to a point, and the curve flattens faster than people expect. The first example or two usually carries most of the gain, because it resolves the ambiguity that plain instructions leave open. Beyond a handful, returns diminish: additional examples add tokens and cost without teaching the model much it has not already grasped, and past some threshold they can actively crowd the context window, pushing out the actual query or the retrieved material that the answer depends on. The right number is task-specific and worth testing rather than assuming; treat any figures here as illustrative.

The practical reading of this table is that the interesting decision is rarely "how many examples" in the abstract. It is where you sit on the curve for your task. Start at zero-shot, because if the model already handles the task from a plain description, examples are pure cost. Add examples one at a time and measure, stopping when the improvement stalls. When you find yourself wanting dozens of examples on every call to get consistent behavior, that is not a few-shot problem to solve with more shots; it is a signal that you have reached the top of this rung and should look at the ones above it.

What few-shot actually steers

The most consequential misunderstanding about few-shot is what the examples actually teach. Watch a model snap into producing clean, correctly shaped output after two demonstrations and it is easy to conclude you have taught it something new. You have, but the something is narrower than it looks. Examples steer how the model performs a task, not what it knows. They shape output format, structure, style, and the framing of the task itself. What they do not reliably do is install new facts.

This distinction has teeth in practice. Show a model three examples of classifying support tickets into your taxonomy, and it will learn the taxonomy's shape and start producing the right labels in the right format, because that is a task and a format. Show it three examples of your product's return policy, and it will learn to talk like a policy expert while remaining just as likely to invent the details of a policy it was never trained on, because those are facts. The examples taught the register, not the record. A model given examples of a fact it does not hold tends to produce fluent, confident, wrong answers, which is worse than an obvious failure because it looks right.

Getting this boundary right is what separates few-shot that works from few-shot that quietly misleads. When a task fails despite good examples, the first diagnostic is to ask which side of the line it sits on. A formatting or framing failure means better or better-chosen examples. A factual failure means you are asking examples to do a job they cannot do, and no amount of prompt tuning will fix it. Reach past few-shot only once you are sure the problem is knowledge, not behavior.

Example selection is everything

If few-shot teaches by demonstration, then the examples you pick are the entire curriculum, and a model is a fast, uncritical student. It will faithfully imitate whatever you show it, including the parts you did not intend to teach. This is the single most underrated lever in the technique: the gap between mediocre and excellent few-shot is almost never the number of examples, it is their quality. Three good examples beat ten careless ones, and one wrong example can undo the rest.

Three properties do most of the work. Examples should be representative of the real inputs the model will face, not the tidy cases you thought of first. They should be diverse, spanning the variation and the hard edges of the task, because a model shown only easy cases learns only the easy case. And they must be correct, because an error in an example is not noise the model averages out; it is a lesson it learns. Bad examples teach bad behavior, and the most dangerous version is subtle: if every example happens to share an incidental trait, the model will pick that up as if it were the rule, quietly encoding a bias nobody wrote down.

  BAD example set              GOOD example set
  -----------------            ------------------
  all short inputs             mix of short and long inputs
  all one category             every category represented
  all positive sentiment       positive, negative, ambiguous
  one silent typo in a label   every label verified correct
  same phrasing each time      varied real-world phrasing
        |                             |
        v                             v
  model overfits the           model learns the actual
  incidental pattern           task and its edges

The operational takeaway is to treat example sets as a curated asset, not a quick sample. Pull candidates from real inputs, check every label by hand, and deliberately include the awkward cases you would rather ignore. When behavior drifts in production, audit the examples before you touch anything heavier, because a skew in the set is the most common and most fixable cause. Examples are small enough to feel disposable and powerful enough to define the whole task; respect the mismatch.

Dynamic few-shot

Everything so far has assumed a static example set: you choose a handful of demonstrations once and ship them with every request. That works well when the task is narrow. It strains when the task is broad, because the examples that best illustrate one kind of query are often useless for another, and a fixed set that tries to cover everything ends up covering nothing well. The response is dynamic few-shot: instead of one frozen set, you keep a large pool of examples and select the most relevant ones for each incoming query at request time.

The machinery is borrowed almost directly from retrieval-augmented generation. You maintain a store of candidate examples, embed them, and at query time retrieve the ones most similar to the current input, then assemble those into the prompt on the fly. It is the RAG pattern pointed at demonstrations rather than documents: retrieve what is relevant, put it in context, generate. The model still learns in context and its weights still never move; you have only made the "few" in few-shot adaptive to the request in front of it.

  incoming query
        |
        v
  [ embed query ] --> [ example store ]   (many examples)
        |                    |
        |            retrieve top-k most similar
        v                    |
  [ assemble prompt: k tailored examples + query ]
        |
        v
  [ model ] --> in-context answer, tuned to this query

This is real machinery, and it is not always worth it. A retrieval step adds an embedding store, a similarity search, and the latency and operational surface that come with them. For a narrow task where one good static set covers the space, that overhead buys nothing. Dynamic selection earns its keep when the input distribution is wide and heterogeneous, when a well-chosen example genuinely lifts accuracy on that specific query, and when you already run retrieval infrastructure the example store can ride alongside. Reach for it when a static set visibly fails to cover the range of inputs, not before.

The costs and limits

Few-shot is cheap to adopt, which is not the same as free to run. The examples live in the prompt, and the prompt is sent on every single call. That means you pay for those example tokens on every request, forever, in both money and latency. A generous example block that looked trivial in a notebook becomes a standing tax at production volume, and it competes for the same finite context window as the user's query and any retrieved material. Every token spent on demonstrations is a token unavailable to the actual work, so more examples is never a purely additive choice.

There are subtler limits too. Example order can matter: the sequence in which demonstrations appear sometimes shifts the output, and a model can lean on the most recent example more heavily than the earlier ones, so an arrangement that looks neutral may not be. More fundamentally, few-shot does not scale like a trained solution. It re-teaches the task from scratch on every call, which is fine for moderate volume and variable tasks but wasteful and inconsistent when you need the same precise behavior across millions of requests. The instruction that lives only in the prompt is never fully internalized; the model is always reading the pattern, never remembering it.

None of this argues against few-shot. It argues for using it with its cost model in view. The technique is at its best when the token overhead is small relative to the value it adds and the task benefits from the flexibility of examples you can edit in seconds. It is at its worst as a permanent crutch for high-volume consistency, a job the next rung does better. Knowing which situation you are in is most of the skill.

The architect view

Few-shot occupies a specific and valuable place in the architect's toolkit: it is the second rung of the adaptation ladder, cheap enough to try first and powerful enough that you often stop there. The default posture is to reach for it before anything heavier. It needs no training run, no dataset, and no model to host; you edit a prompt and measure. When a plain instruction is not enough to steer format or task framing, a few well-chosen examples usually are, and you have spent minutes rather than a project.

The three failure modes are worth naming because they are so common. The first is asking few-shot to teach facts, when it only teaches behavior; that is a retrieval or fine-tuning problem wearing a prompt-shaped disguise. The second is treating examples as a quick sample rather than a curated asset, and inheriting whatever bias the careless set encodes. The third is staying on this rung too long, shipping ever-larger example blocks to force consistency that the weights should be carrying instead. Each has a clean tell, and each points somewhere specific.

So the progression is disciplined rather than fashionable. Start with examples, static and few. When one fixed set cannot cover a wide input distribution, add dynamic selection and let retrieval tailor the demonstrations per query, accepting the extra machinery only where the range of inputs justifies it. And when volume and the need for consistent, internalized behavior make the per-call token cost the dominant expense, climb to fine-tuning, having first earned the right by exhausting the cheaper rung. Few-shot is the fastest way to steer a model and, used with clear eyes about what it steers and what it costs, frequently the last technique you need to reach for.