The Adaptation Ladder: Prompt, RAG, Fine-Tune
The adaptation ladder is the ordered sequence of interventions for closing the gap between a base model and your task: prompt engineering, then retrieval-augmented generation, then fine-tuning, with each rung adding cost and operational weight. The routing question is diagnostic. Knowledge gaps (the model does not know your products, policies, or codebase) are RAG problems, because facts change and retraining on every update is untenable; behavior gaps (wrong format, wrong tone, ignored domain conventions, poor tool selection) are fine-tuning problems, because no volume of retrieved context reliably reshapes how a model acts. Most teams err in one of two directions: they stop too early, shipping a brittle 4,000-token mega-prompt where a tuned model would behave consistently, or they overreach, fine-tuning to inject facts that go stale within a quarter. The architect's discipline is to build an evaluation set first and climb only when the current rung measurably plateaus.
Fine-Tuning Methods: LoRA, PEFT, Full FT
Fine-tuning updates a model's weights on your own examples, and the methods differ mainly in how many weights move and what that costs. LoRA (low-rank adaptation) freezes the base model and trains small rank-decomposition matrices injected into the attention layers, typically touching well under 1 percent of parameters; that cuts GPU memory enough to tune a 70B model on a small cluster and lets you hot-swap adapters per tenant at serving time. QLoRA quantizes the frozen base to 4-bit and shrinks the footprint further, while the broader PEFT family (prefix tuning, IA3) trades some quality for even smaller deltas. Full fine-tuning moves every parameter and is justified mainly when the target domain sits far from the pretraining distribution (legal, genomics, low-resource languages) or when you own the base model outright. The governing risk is catastrophic forgetting: aggressive tuning on narrow data erodes general capability and safety behavior, so regression evals on held-out general benchmarks matter as much as the task metric.
Distillation: Big-Model Quality at Small-Model Cost
Distillation trains a small, cheap student model to reproduce the behavior of a large teacher, capturing most of the quality at a fraction of the serving cost. The enterprise-practical form is sequence-level distillation on teacher outputs: run a frontier model over tens of thousands of representative inputs, capture its outputs and often its reasoning traces, filter hard for correctness, then fine-tune a 7B-to-14B open-weight model on the result. The classical variant instead matches the teacher's softened token probability distributions (soft targets) rather than sampled outputs, which transfers more signal per example but requires logit-level access most hosted APIs do not expose. The economics are the point: a distilled 8B model can serve a narrow task at roughly an order of magnitude lower cost per token, with latency low enough for interactive products. The catches an architect must own: the student inherits the teacher's errors wholesale, degrades faster off-distribution, and many frontier providers' terms restrict training competing models on their outputs, so read the license before building the pipeline.
Synthetic Data Generation
Synthetic data generation uses a strong model to manufacture training or evaluation examples when real labeled data is scarce, sensitive, or poorly distributed. A production pipeline is generation plus filtering, never generation alone: seed with real examples or schemas, generate variations at elevated temperature, then filter with programmatic checks (schema validation, execution tests for code) and an LLM judge, typically discarding 30 to 70 percent of candidates. The technique earns its keep on edge cases: fraud patterns, rare error states, adversarial phrasings that appear once a month in production yet must be handled correctly. Two risks dominate. Contamination: if synthetic examples leak into your evaluation set, you are grading the generator, not the system. And drift toward the generator's priors: models trained on their own kind of output lose lexical and structural diversity across iterations, so cap the synthetic fraction and keep a real-data anchor in every training mix.
Data Readiness for AI
Data readiness is the strongest predictor of whether an enterprise AI program ships: quality, lineage, access, and entitlements for everything the models will consume. In practice, most blockers labeled as AI problems decompose into data problems: documents with no owner and no freshness signal, entitlement rules that live in SharePoint ACLs and nowhere machine-readable, PII scattered through free-text fields, and three conflicting definitions of customer across systems. RAG raises this bar rather than lowering it, because retrieval must respect per-user permissions at query time; an index that flattens document ACLs is a breach waiting for a demo. The readiness work is concrete: a documented source of truth per domain, freshness SLAs on ingestion pipelines, lineage that traces a bad answer back to a bad document, and entitlement propagation into the vector store. Architects who budget 60 to 70 percent of program effort here are usually about right; those who budget 20 percent discover the real number later.
Choosing & Adapting Embedding Models
Embedding models convert text into the vectors that retrieval runs on, and picking one from a public leaderboard is the most common quiet failure in enterprise RAG. General-purpose embedders are trained on web text, and your corpus is full of vocabulary they have never learned to separate: internal product codes, legal terms of art, clinical abbreviations that collide with everyday words. Benchmark on your own corpus before committing: build a few hundred query-document relevance pairs from real search logs or expert judgment and measure recall at k, because MTEB rank predicts in-domain retrieval quality only weakly. When the gap is real, fine-tune the embedder with contrastive learning on mined (query, positive, hard negative) triplets; a few thousand pairs often lift domain recall by double digits. Dimensionality is a cost dial: 3072 dimensions versus 768 is roughly a 4x difference in index memory and query latency, and Matryoshka-style truncation buys much of that back. Budget for the hidden migration cost: changing embedders means re-embedding the entire corpus.