- The AI tooling ecosystem is large, fragmented and fast-moving, so a map of durable categories, not a list of vendors, is the way to reason about it. Categories outlast the logos that fill them.
- The main building blocks are model providers, gateways and routers, vector databases, orchestration frameworks, observability, evaluation, guardrails, adaptation platforms, and data and retrieval pipelines. They map cleanly onto the reference architecture layers.
- Build versus buy is decided per category, not once: buy the commodity layers, build the differentiating ones. Categories overlap and consolidate over time, so over-buying point tools risks sprawl. Evaluate on fit, lock-in, integration and maturity.
A fragmented, fast-moving map
Open any market map of AI tooling and you are looking at something already stale. There are hundreds of products, dozens of them launched since the last time you looked, and a fair number that have been acquired, pivoted or quietly abandoned since the diagram was drawn. Any attempt to hold the landscape in your head as a list of vendors is a losing game. The list is too long, it changes weekly, and the names carry no structure that helps you decide anything.
The durable move is to stop tracking vendors and start tracking categories. A category is a job that needs doing in the stack: routing requests to models, storing embeddings, tracing what an agent did, catching unsafe output. Those jobs are stable. The products that do them churn constantly, but the need to route, to store, to trace and to guard does not go away, and it does not get renamed every quarter. A CTO who reasons in categories can absorb a new entrant in seconds, because the only question is which existing job it does, and whether it does that job better than what is already in the slot.
This is why a map beats a list. A list ages the moment it is printed; a map of categories tells you where any new tool fits and what it would displace. The categories in this article are deliberately generic, and no specific vendor is named or ranked, because the point is the shape of the space, not this month's leader in any box.
The building-block categories
The ecosystem resolves into a manageable number of categories once you group tools by the job they do rather than the company that sells them. The table below names the main building blocks of an enterprise AI stack and what each one is for. The boundaries are approximate and, as the next sections argue, they blur at the edges, but the set is stable enough to plan around.
| Category | What it does |
|---|---|
| Model providers | Foundation model APIs and open-weight models: the reasoning and generation engine itself, hosted or self-served. |
| Gateways & routers | A single seam in front of many models: authentication, routing, fallback, rate limiting, caching and cost control. |
| Vector databases | Storage and similarity search over embeddings, the retrieval substrate that grounds answers in your data. |
| Orchestration frameworks | The libraries that compose prompts, tools, memory and multi-step agent or workflow logic into an application. |
| Observability & tracing | Capturing prompts, responses, latency, token cost and traces so you can see what the system actually did. |
| Evaluation tools | Scoring output quality against test sets, rubrics or judges, offline in CI and online in production. |
| Guardrails & safety | Filtering unsafe input and output: policy enforcement, PII handling, jailbreak and injection defense. |
| Adaptation platforms | Fine-tuning, distillation and related tooling to specialize a base model to your task or domain. |
| Data & retrieval pipelines | Ingesting, chunking, embedding and syncing source content into a form the retrieval layer can use. |
Two things are worth noticing about this set. First, it is short: nine categories cover the overwhelming majority of what any AI platform assembles, which is a far more tractable object than a market map of hundreds of products. Second, the categories are not equally mature or equally commodity, which is exactly why the sourcing decision differs from one to the next rather than being settled once for the whole stack.
Where the categories fit
Categories become useful the moment you place them in the stack, because position tells you what depends on what and where each seam belongs. The reference architecture arranges an AI system into layers, from the models at the bottom to the application logic at the top, with cross-cutting concerns running alongside. Every category from the previous section lands somewhere on that picture, and none of them is homeless.
APPLICATION domain logic, agents, workflows
------------------------------------------------
ORCHESTRATION orchestration frameworks
------------------------------------------------
RETRIEVAL data & retrieval pipelines
vector databases
------------------------------------------------
ACCESS gateways & routers
------------------------------------------------
MODEL model providers
adaptation platforms
================================================
CROSS-CUTTING (span every layer above)
observability | evaluation | guardrails
The vertical layers form a dependency chain. Model providers and adaptation platforms sit at the base. A gateway is the access seam directly above them, the point through which everything else reaches a model. Retrieval, both the pipelines that prepare data and the vector store that serves it, sits above the access layer, and orchestration composes retrieval, tools and models into the application at the top.
The three cross-cutting categories deliberately do not live in a single layer. Observability, evaluation and guardrails apply at every level: you trace, score and guard the model call, the retrieval step and the full agent trajectory alike. Drawing them as spanning concerns rather than boxes is what stops a team from bolting evaluation onto only one layer and calling the system observed.
Build or buy, per category
Once the categories are placed in the stack, the sourcing decision stops being one big question and becomes nine smaller ones. The mistake is to decide build versus buy for the platform as a whole. The discipline, covered at length in build vs buy vs orchestrate, is to make the call per category, because the economics differ sharply from one box to the next. Some categories are deep commodity markets where a specialist vendor will always outrun your in-house version; others are where your actual advantage lives.
The rule of thumb is short and it holds: buy the commodity layers, build the differentiating ones. A vector database, a gateway, an observability backend and a model API are infrastructure that a dozen credible vendors sell as a commodity, and rebuilding them from first principles spends your best engineers on something that adds cost without a moat. What differentiates you sits higher up and further out: your proprietary data and the pipelines that turn it into retrieval, your domain logic and workflows, and the orchestration and evaluation tuned to your specific tasks. Those a vendor cannot ship, because they are specific to you.
- Lean buy: model providers, gateways and routers, vector databases, observability backends. Deep commodity markets; a specialist runs them better and cheaper than you will.
- Lean build: domain logic, the orchestration of your specific tasks, and the data and retrieval pipelines over your proprietary content. This is where the advantage is.
- It depends: evaluation, guardrails and adaptation. Buy the harness and the base capability, but the test sets, policies and tuning that make them yours are built.
The decision is not permanent either. What you buy today because the capability is not yet core, you may bring in-house once it is; what you built early because nothing existed, you may retire to a vendor once the market matures. Re-scoring the stack periodically is part of the method, not an admission that the first pass was wrong.
Overlap and consolidation
The clean category boundaries of the table are a planning fiction, useful but leaky. In practice the categories overlap, and they consolidate over time as vendors expand outward from their original box to capture adjacent ones. A gateway that started as a pure routing seam adds evaluation and cost analytics. An observability tool that began by capturing traces adds guardrails and online scoring. An orchestration framework grows its own tracing and its own eval harness. The map is real, but the territory keeps redrawing its internal borders.
This matters because it changes how you should buy. If you treat every category as a mandatory separate purchase, you end up with a stack of eight or nine point tools that overlap heavily, cost a fortune in aggregate, and each demand their own integration, their own dashboard and their own renewal. That is tool sprawl, and it is one of the more expensive failure modes in enterprise AI, because the cost is spread across many small invoices that no single owner ever adds up.
The countervailing risk is real: a consolidated suite can lock you in more deeply than any single point tool, because leaving means unwinding several capabilities at once. The judgment is not "always consolidate" but "know which overlaps you are paying for twice." When a tool you already own quietly grows into an adjacent category, that is often a reason to retire a separate purchase, not to keep both running out of habit.
Evaluating within a category
Having chosen to buy in a given category, you still have to pick, and this is where the churn punishes anyone chasing hype. The launch that dominated the timeline last quarter is a poor guide, because the benchmark it won will be irrelevant by the time you are in production and the tool may not survive its own funding round. The durable move is to evaluate on criteria that outlast the news cycle rather than on which product is loudest this month.
Four criteria survive the churn, and they apply inside every category:
- Fit: does it do the specific job you need, at your scale and latency, without forcing you to reshape your problem to suit the tool? A close fit beats a longer feature list.
- Lock-in: what does exit cost? Measure it in the concrete: how much code, data migration and re-testing would leaving actually take? A tool you can leave in a sprint is a different risk than one whose departure needs a reorganization.
- Integration: how cleanly does it sit next to what you already run? A tool that speaks your existing formats and slots behind your gateway is worth more than a marginally better one that fights your stack.
- Maturity: is it stable enough to depend on? Reliability, security posture, documentation, support and the odds it still exists in two years matter more than a headline benchmark.
Notice that raw capability is not on the list as a standalone item; it is folded into fit. That is deliberate. In a fast-moving market, the most capable tool today is routinely overtaken, so buying purely on peak capability optimizes for a lead that evaporates. Buying on fit, exit cost, integration and maturity optimizes for a decision that still looks sound after the next three product launches, which is the only kind of decision worth making in a landscape this volatile.
The architect view
The landscape will keep moving, and that is the one fact you can build on. New categories will appear, existing ones will merge, and this year's leader in any box will not be next year's. None of that is a problem if you have stopped anchoring your understanding to vendors and started anchoring it to categories and capabilities. The map survives the churn; the list never could.
So the architect's stance is a small set of habits, applied in order. Think in categories and capabilities, not vendors, so that any new entrant is just a candidate for a known slot. Map your needs onto the stack, using the reference architecture so you know what depends on what and where each seam belongs. Decide build versus buy per layer, buying the commodity and building where you differentiate, and revisit that call as you and the market mature. And expect the landscape to keep shifting, so favor tools you can leave, seams you control, and a stack small enough to stay coherent.
Done this way, the fragmentation stops being intimidating. A market of hundreds of products collapses into nine or so durable jobs, each with a sourcing decision you have reasoned through and can defend. If you want to see where a specific tool lands in the full picture, the Architecture Designer in the Lab places categories into a working stack, and the reference architecture deep-dive is the map those categories sit on. The vendors will keep changing. The way you reason about them should not.