- Adaptation techniques form a ladder from cheap and fast to expensive and heavy: prompting, then few-shot, then RAG, then fine-tuning, then the rarely-justified rungs above. The discipline is to climb only as high as the problem actually requires.
- Most teams reach for fine-tuning far too early, usually to fix a knowledge gap that belongs in retrieval or a formatting gap that a system prompt and a few examples would have closed for free.
- Start at the bottom, climb only when the rung below genuinely cannot meet the need, and combine rungs freely: RAG plus a light fine-tune is a common, cheaper answer than either taken to its extreme.
Climb only as high as you must
There is a reliable pattern in enterprise AI projects that stall: the team decided how to build before it decided what the problem was. Someone heard "fine-tuning" in a vendor pitch, and a formatting inconsistency that a two-line system prompt would have fixed became a six-week data-labeling program. The antidote is to see the options as they actually relate to each other, which is not a menu of equals but a ladder. Each rung buys more control over the model's behavior, and each rung costs more to build, more to run, and more to unwind.
The rungs run from cheapest and fastest to most expensive and heavy: prompt engineering, then few-shot examples, then retrieval, then fine-tuning, and above that the rarely-justified work of continued pre-training or building a model from scratch. The single most useful discipline in the whole exercise is this: climb only as high as the problem requires, and prove the rung below cannot do the job before you step up. Higher is not better. Higher is more capable and more expensive, and you pay the expense whether or not the capability was needed.
cost / effort / lock-in ^
|
build a model ..........| almost never (enterprises)
continued pre-training | rare
------------------------|--------------------------
fine-tuning ............| consistent behavior at scale
RAG ....................| knowledge the model lacks
few-shot ...............| steer format and task
prompt engineering .....| cheapest, often enough
+------------------------->
climb only when forced
This piece walks each rung: what it does, what it costs, and the tell-tale sign that you have climbed one rung too far. It complements RAG vs Alternatives, which picks the right tool per need; here the emphasis is the escalation itself, and the case for trying cheap first.
Rung 1: prompting
The bottom rung is prompt engineering: shaping behavior with the instructions you send, most durably through a system prompt that defines the model's role, its constraints, its output contract, and its refusals. It changes nothing about the model. There is no training run, no dataset, no artifact to version separately, and no lock-in. You edit text, you observe the result, you iterate in minutes. For a large share of enterprise tasks, this is not the starting point on the way to something heavier; it is the finish line.
The reason prompting reaches further than teams expect is that frontier models are already extraordinarily capable and instruction-following. A great deal of what looks like a capability gap is really an instructions gap: the model was never told the format, the tone, the edge-case handling, or the standard it was being held to. A precise system prompt that names the four required fields, states the house style, and specifies what to do when the input is ambiguous will often close a gap that a team was ready to spend a quarter fine-tuning to fix.
Prompting has real limits. Instructions can be verbose, they consume context budget, and behavior at the edges can drift in ways that a single prompt struggles to fully pin down. Those limits are exactly what the next rungs address. But you cannot know which limit you have actually hit until you have pushed this rung to its honest ceiling, and most projects abandon it long before that point.
Rung 2: few-shot
The second rung adds examples to the prompt. Zero-shot gives the model an instruction and no demonstrations; one-shot gives it a single example; few-shot gives it several. The model learns the task from those examples at inference time, without any change to its weights, a behavior usually called in-context learning. It is still cheap and still fast: no training, no new artifact, just a better-constructed prompt. What you are buying over plain instructions is demonstration. Some things are far easier to show than to describe, and a handful of correct input-output pairs can pin down a format or a task framing that a paragraph of prose leaves ambiguous.
Few-shot is strongest at steering form: output structure, style, tone, the shape of a classification, the exact JSON your downstream system parses. It is weak as a way to teach genuinely new facts. Examples demonstrate how to behave; they do not reliably install knowledge the model lacks, and trying to teach facts by piling on examples is a common way to climb this rung in the wrong direction. If the gap is knowledge, the next rung, not more examples, is the answer.
Two practical cautions. First, example quality is the whole game: representative, diverse, correct examples help, while sloppy or unrepresentative ones actively teach bad behavior, and even ordering can shift results. Second, examples are not free at runtime. They consume tokens on every call, spending both money and context budget, and few-shot does not scale the way fine-tuning does for consistent, high-volume behavior. A useful middle pattern is dynamic few-shot, which retrieves the most relevant examples per query, effectively a retrieval technique applied to the prompt. Details live in the Few-Shot deep-dive.
Rung 3: RAG
The first two rungs shape behavior. The third rung supplies knowledge. Retrieval-augmented generation fetches relevant material at query time, from a document store, a database, or an index, and places it into the prompt so the model can answer from current, proprietary information it was never trained on. This is the correct rung whenever the problem is a knowledge problem: the model does not know your policies, your product catalog, your customer's account state, or anything that changed after its training cutoff. No amount of prompting or few-shot fixes that, because the information simply is not in the model.
RAG is a genuine step up in cost and effort, and the jump is mostly infrastructure. You now own an ingestion pipeline, a chunking strategy, embeddings, a vector or hybrid index, a retriever, and the evaluation to keep all of it honest. That is real engineering, and it is why RAG sits above prompting rather than beside it. What you get in return is decisive for enterprise use: the knowledge stays external and current. Update a document and the next answer reflects it, with no training run. You can scope retrieval to a user's entitlements, and you can cite sources, which matters enormously for auditability and trust.
RAG also composes. It is routinely combined with a strong system prompt and few-shot formatting, and later with a light fine-tune for manner, so treating the rungs as mutually exclusive is a mistake. The full comparison of when retrieval beats the alternatives is in RAG vs Alternatives, and the moving parts are explorable in the RAG Explorer.
Rung 4: fine-tuning
Fine-tuning changes the model's weights, training it on a curated dataset of examples so a behavior becomes native rather than instructed. This is the rung that earns its cost when you need consistent behavior at scale: a narrow, stable, high-volume task where a system prompt plus examples is either too long, too inconsistent, or too token-hungry to run economically. Done well, a small fine-tuned model can match a prompted frontier model on that one task at a fraction of the per-call cost, while emitting the exact format every time. That is a real and valuable outcome.
It is also the rung with real teeth. You take on a dataset that must be clean, deduplicated, provenance-checked, and split for evaluation, and dataset quality, not compute, decides whether the tune succeeds. You take on a training and serving pipeline, a versioned artifact tied to a specific base model, and lock-in: when the base model you tuned is superseded, your investment is pinned to yesterday's model until you redo the work. None of that is disqualifying. It is simply expensive, and it is expense you should incur deliberately.
The honest posture toward this rung is patience. Fine-tuning is the right answer for a real set of problems, and reaching it after the rungs below have genuinely topped out is sound architecture. Reaching it first, as a reflex or because it sounds serious, is how quarters get spent training a model to do what a prompt already did.
Beyond the ladder
Above fine-tuning the ladder keeps going, but the air gets thin fast. The next rung is continued pre-training: taking a base model and running further self-supervised training on a large corpus of your own, to shift its foundational sense of a domain rather than a single task. Above that is building a model from scratch, pre-training a new foundation model on your own data and compute. Both are legitimate techniques. Neither is a normal enterprise move, and it is worth being blunt about why.
The economics are unforgiving. Continued pre-training wants a very large, high-quality domain corpus and a serious training budget, and it still yields a model you must serve, evaluate, and re-do when the frontier advances, which it does relentlessly. Building from scratch multiplies every one of those demands and adds the cost of matching general capabilities that frontier labs spend enormous sums to achieve. For the overwhelming majority of enterprises, the money and talent that a from-scratch model would consume produce a better outcome when spent on retrieval quality, evaluation, and the operational scaffolding around a strong bought-in model.
These rungs make sense in narrow situations: a genuinely distinct domain language that off-the-shelf models handle poorly, a regulatory or sovereignty requirement that forbids external models, or a core-differentiating capability where the model itself is the product. Those cases exist, and they are rare. The practical guidance is simple. This is where the ladder tops out, almost no one needs to climb here, and if you believe you do, the burden of proof is on demonstrating that every rung below genuinely could not meet the need, not on the enthusiasm of the idea.
The architect view
The rule is one sentence: start at the bottom, and climb only when the rung below genuinely cannot meet the need. Framed that way, adaptation stops being a technology choice and becomes a diagnosis. You are not asking "should we fine-tune?" You are asking what kind of gap you have, form or knowledge or scaled consistency, and letting the answer name the rung. The mistake the ladder guards against is the reflex to jump, choosing a heavy solution before the problem is framed or the data checked, which is the same failure the adaptation ladder primer flags in the wider architect workflow.
| The need looks like | Right rung | Because |
|---|---|---|
| Wrong tone, role, or output contract | Prompting | An instructions gap, not a capability gap |
| Format or task hard to describe but easy to show | Few-shot | Demonstration steers form without training |
| Model lacks current or proprietary knowledge | RAG | Keeps data external, current, and citable |
| Narrow, stable, high-volume behavior at cost | Fine-tuning | Bakes the behavior in, cheaper per call |
| Distinct domain language or model-as-product | Beyond | Rarely justified; prove the rest failed |
The other half of the discipline is that the rungs combine. The strongest enterprise systems are rarely one rung; they are a good system prompt, a little few-shot for format, RAG for knowledge, and sometimes a light fine-tune for manner, each doing the job it is cheapest at. RAG plus a small fine-tune, in particular, is a common answer that beats pushing either technique to an extreme: retrieval carries the facts, the tune carries the voice and the reflexes.
So the architect's contribution is restraint that reads as judgment. Anyone can specify the heaviest option. The value is in reaching the lowest rung that clears the bar, proving it with real evaluation, and climbing only under demonstrated pressure. When fine-tuning is finally the answer, you will know exactly why, and the case for it will fit in a sentence. The mechanics of that rung wait in Fine-Tuning in Practice.