- Designing an AI system is a risk-first, iterative sequence, not a model choice: use case, data, pattern, platform, operations, then iterate. Jumping straight to the model is the classic mistake, because it commits to a solution before the problem is framed or the data is checked.
- Frame the use case and its non-functionals (accuracy, latency, cost, governance) first, then gate on data readiness, then pick the pattern on the adaptation ladder, climbing only as high as the need genuinely requires.
- Design on the reference architecture rather than a bespoke stack, and bake in evals, guardrails, observability, cost control, and governance from day one. That operational design is what separates a demo from a system that survives production.
From blank page to design
Put an architect in front of a new AI request and watch where the instinct pulls. The pressure in the room is to name a model. The stakeholder has read about the latest release, the board wants a headline, and the fastest way to look decisive is to answer the question everyone is asking: which model are we using. That is the classic mistake, and it is expensive precisely because it feels like progress. A model is an implementation detail. Choosing it first means committing to a solution before you have framed the problem or checked whether your data can support any solution at all.
The work of an architect is not to pick a component but to design a system that survives production, and that is a risk-first, iterative discipline rather than a shopping decision. Risk-first means you attack the things most likely to kill the project before you invest in the things that are cheap to change. The model is cheap to change; a swap is a config line. Whether the use case is worth building, whether the data exists in a usable state, whether the pattern fits the need: those are the expensive commitments, and they belong at the front of the sequence, not the end.
USE CASE -> what problem, for whom, at what bar
|
v
DATA -> is it accessible, clean, governed?
|
v
PATTERN -> prompt / RAG / workflow / agent / tune
|
v
PLATFORM -> model, gateway, integration, non-func
|
v
OPERATIONS -> evals, guardrails, observability, cost
|
+---------< iterate back as gaps appear >-------+
The order is not decoration. Each stage constrains the one after it: the use case sets the accuracy and latency bar, the data determines which patterns are even viable, the pattern shapes the platform, and the platform sets what operations must watch. Work it in order and every decision rests on a settled foundation. Jump ahead and you design on assumptions the later stages quietly violate. And it iterates, because a data gap can rule out the pattern you assumed, or an eval can expose a weak premise, and the honest response is to loop back rather than paper over it. The value the architect adds is holding this order under the pressure to short-circuit it.
Start with the use case
Every sound design starts with a question that has nothing to do with AI: what problem are we solving, for whom, and how will we know it worked. A use case is not a feature request. It names the user, the task they are trying to complete, the decision or output they need, and the bar that output has to clear to be useful rather than merely impressive. Skip this and everything downstream is built to hit a target nobody defined. It is also where you decide whether the problem deserves a system at all, a triage covered in use-case prioritization.
The part teams routinely underweight is the non-functional requirements, which is where most AI systems actually live or die. Functionally, a demo that summarizes a document works. The real question is the envelope around it: how accurate does the answer have to be, what latency will the user tolerate, what does each call cost at the volume you expect, and what governance and audit obligations attach to the domain. A legal-review assistant and a brainstorming aid can look identical in a demo and demand entirely different systems once you name those numbers. The non-functionals, not the feature, decide the architecture.
- Accuracy. What error rate is acceptable, and what does a wrong answer cost the user and the business.
- Latency. Is this an interactive experience measured in a second, or a batch job measured in minutes.
- Cost. What are the per-call and per-user economics at real volume, not at demo scale.
- Governance. What must be logged, explained, restricted, or reviewed for the domain and its regulation.
Assess the data
With the problem framed, the next gate is the one most likely to stop the project quietly: the data. Nearly every pattern an architect might choose is a way of putting data in front of a model, and if the data is not there in a usable state, the pattern is a plan with no foundation. This is why data assessment comes before pattern selection, not after. The choice you make later is downstream of what the data can actually support, and reversing that order means designing on hope.
Three questions decide readiness, and a use case can fail any one of them independently. Is the data accessible: can you reach it past the silos, approvals, and awkward formats that separate a tidy demo export from a live source system. Is it clean enough: accurate, complete, consistent, and fresh, given that a model reproduces flaws faithfully and states wrong values with the same fluency as right ones. Is it governed: do you know its lineage and ownership, and do the usage rights and permissions actually allow this use, enforced at query time rather than asserted in a policy nobody can point the pipeline at. Clear all three for the specific use case and you have earned the right to choose a pattern.
The discipline is to score this honestly and per use case before committing, because the same corpus can be ready for internal search and nowhere near ready for a customer-facing assistant with stricter accuracy and permission needs. The full treatment, with the failure modes and a scoring template, is the data readiness deep-dive; the point here is only its position in the sequence. Data readiness is a prerequisite, assessed early, not a parallel workstream you hope will catch up. Discover the gap now and it is a work plan. Discover it after you have chosen and built the pattern and it is a rebuild.
Choose the pattern
Only with the problem framed and the data assessed does the pattern become a real decision rather than a guess. The options form a familiar set: a well-built prompt or system prompt, few-shot examples that steer format and task, retrieval when the model lacks current or private knowledge, a fixed workflow when the steps are known, an agent when the path is genuinely open-ended, and fine-tuning when you need consistent behavior at high volume. The discipline is the adaptation ladder: start at the cheapest rung that could work and climb only when the problem forces you to, because each rung up adds cost, effort, and lock-in.
Two failure directions are common. Teams reach for fine-tuning first, drawn by its heaviness, when a prompt plus retrieval would have solved the problem at a fraction of the cost and none of the data-labeling burden; fine-tuning is the rung most reached for too early. In the opposite direction, teams build an autonomous agent where a deterministic workflow would do, buying open-endedness they do not need and paying for it in unpredictability, harder testing, and more failure modes. The rule in both cases is the same: the least power that meets the need. Rungs also combine, so retrieval plus light fine-tuning is a legitimate answer when one alone falls short.
| The need | Reach for | Why, and what it beats |
|---|---|---|
| Steer format, tone, or task framing | Prompt + few-shot | Cheapest rung; no weights change, steers at inference |
| Answer from current or private knowledge | RAG (retrieval) | Keeps data external and fresh, versus baking it into a model |
| Known, fixed sequence of steps | Deterministic workflow | Predictable and testable, versus an open-ended agent |
| Open-ended path, dynamic tool use | Tool-using agent | Only when a workflow genuinely cannot express the task |
| Consistent behavior at high volume | Fine-tuning | Last resort; real cost, data work, and lock-in |
The mapping is a starting heuristic, not a verdict; the right pattern depends on the need and the data behind it, and the honest comparison of retrieval against its alternatives is the subject of RAG vs alternatives. What the architect refuses to do is start at the top of the ladder because it is fashionable. Cheap first, climb only on evidence, and combine rungs when the evidence says one is not enough.
Design on the platform
With the pattern chosen, the design becomes concrete, and the first principle is to build on a reference architecture rather than assemble a bespoke stack for every project. A reference architecture gives you the layers already worked out: model access, the gateway, the retrieval and data layer, orchestration, integration into systems of record, and the cross-cutting concerns of security and observability. Designing on it means each new use case fills in known slots instead of reinventing the plumbing and its failure modes one project at a time.
Model selection lands here, late and deliberately, as one decision among several rather than the headline. You choose against the non-functionals from the use case: the cheapest, fastest model that clears the accuracy bar, not the largest on the leaderboard. A gateway keeps that choice reversible, giving you one interface, routing, and fallback so a model swap is a config change rather than a rewrite. Integration is where value is realized or lost: an AI system that cannot reach the systems of record and act on their data is a clever demo, and the integration patterns are as much of the design as the model.
requirements + chosen pattern
|
v
+-----------------------------+
| REFERENCE ARCHITECTURE |
| model <- gateway |
| retrieval / data layer |
| orchestration |
| integration to systems |
| security / observability |
+-----------------------------+
|
v
non-functionals met? --no--> revise model / layer
| yes
v
design ready to build
The through-line is that the platform absorbs the hard, shared problems so the design does not solve them per use case. Bespoke stacks feel faster on the first project and cost you on every one after, as each carries its own gateway, its own logging, its own integration quirks, and its own on-call burden. Design on the platform and the non-functionals have a home; design around it and they become surprises in production.
Design for operations from day one
The gap between a demo and a production system is almost entirely operational. A demo has to work once, for a friendly user, on a chosen input. A production system has to keep working for adversarial users on inputs you never imagined, prove that it is still working, and do so inside a cost budget. That is not a phase you add after launch. It is a set of design decisions that have to be present in the first architecture, because retrofitting them tends to mean rebuilding.
Five operational concerns belong in the design, not the backlog. Evals give you a repeatable measure of quality, so a change is a tested hypothesis rather than a guess and you can tell whether today's system is better or worse than yesterday's. Guardrails constrain inputs and outputs against the failure modes the use case invites, from prompt injection to unsafe actions. Observability makes behavior visible: what was retrieved, what the model did, where latency and errors accumulate. Cost control keeps per-call and per-user economics inside the envelope the use case set. And governance ties it together with the logging, access, and audit the domain requires.
The right amount of this scales with autonomy and stakes. A deterministic, low-risk internal tool needs light scaffolding; an agent acting on live systems for external users needs heavy evals, tight guardrails, and deep observability, because more autonomy buys more capability at the price of predictability. The architect sizes the operational design to the pattern and the risk, and treats it as part of the system rather than a maintenance concern to inherit later.
The architect view
Strip the workflow to its spine and it is a short, repeatable sequence that an architect runs for every system, holding its order against the pressure to skip ahead. Frame the use case and its non-functionals. Assess the data. Choose the pattern on the adaptation ladder. Design on the platform. Design for operations from day one. Then iterate, because real designs loop back when a later stage exposes a flaw in an earlier assumption.
- Use case. The problem, the users, the non-functionals; the bar the output must clear.
- Data. Accessible, clean, governed, assessed per use case, before anything is chosen.
- Pattern. The least powerful rung of the ladder that meets the need.
- Platform. Model, gateway, integration, non-functionals, on the reference architecture.
- Operations. Evals, guardrails, observability, cost, governance, built in from day one.
What every rule in this workflow guards against is the same reflex: jumping to the model, or to any solution, before framing the problem and checking the data. That reflex feels productive and is the most reliable way to build something that demos well and cannot ship. The sequence exists precisely because the expensive commitments, the use case, the data, and the pattern, come first and change hardest, while the model, the thing everyone wants to argue about, is the cheapest to swap.
Done well, this is where strategy becomes a system that survives production. The use-case framing connects a design to a business outcome someone actually cares about; the platform and operational discipline make it something that keeps working after the launch demo is over. That connection between enterprise intent and engineered reality is the substance of AI-first enterprise architecture, and the architect's workflow is how it gets built one system at a time: risk-first, in sequence, and honest about the data before the model.