- A shared enterprise AI platform serves many teams and tenants on common infrastructure, which is efficient only if strong identity and tenant isolation keep each tenant's data, prompts, indexes and models separate from every other's.
- The signature failure is cross-tenant retrieval: on a shared vector store, RAG must be scoped per tenant through namespaces, metadata pre-filters or separate indexes, or a query returns another tenant's documents and the model grounds on them.
- Put identity, access and quota at the gateway, treat agents as first-class principals with their own scoped identity, and prove tenant boundaries with canaries and standing tests rather than assuming them.
The shared platform problem
A mature enterprise AI platform is a shared asset. One set of model endpoints, one retrieval layer, one gateway and one pool of accelerators serve dozens of teams, applications and business units at once. The economics are hard to argue with: consolidating inference and infrastructure behind a common platform is far cheaper than letting every team stand up its own stack, and it is the only way governance, cost control and observability get applied consistently instead of reinvented badly in each corner of the estate.
That sharing is efficient right up to the moment it is not. The instant two tenants sit on the same infrastructure, every boundary between them becomes something the platform has to enforce rather than something physics enforces for you. A prompt, a retrieved document, a cached response, a log line or a fine-tuned adapter that belongs to tenant A must never surface for tenant B, and nothing about running on shared compute guarantees that. The guarantee has to be designed, built and proven.
This is why identity and multi-tenancy are not a late-stage hardening task but a foundational property of the platform. Identity answers who is calling; isolation answers what they are allowed to touch. Get either wrong and you have built an efficient machine for moving one tenant's confidential data onto another tenant's screen. The failure is rarely dramatic. It is a missing filter, a default namespace, a query that forgot its tenant scope, and it stays invisible until the wrong person reads something they were never entitled to see.
Identity: who is calling
Every request to the platform is made by some principal, and the first job of the platform edge is to establish, with confidence, which one. In a classic application that meant a human user behind a login. On an AI platform the population of principals is broader and still growing, and treating them all uniformly is where a lot of designs quietly go wrong.
- Humans authenticate through the enterprise identity provider using SSO, OIDC or SAML and MFA; their group and role membership drives what data and features they may reach.
- Services are the applications and pipelines calling the platform on a user's behalf or on their own schedule; they carry workload identities and short-lived credentials, never a shared static key pasted into config.
- Agents are the newer and harder case: autonomous or semi-autonomous processes that call tools, retrieve data and act in loops. They are non-human principals in their own right and need their own identity, not the borrowed authority of whoever started them.
Authentication proves the principal; authorization decides what it may do, and both belong at the platform edge rather than scattered through application code. The gateway is the natural place to validate the token, resolve the tenant and role, and reject anything that fails before a single model or index is touched. Enforcing it once, on the path every call already takes, beats trusting every team to reimplement it correctly.
Agent identity is the piece the industry is still working out. An agent acting for a user should carry a scoped, delegated authority that names both the agent and the human it serves, so its access can be constrained to less than the user's own and every action traces back to a real accountable identity. Wiring an agent to a broad service account because it is convenient is how a helpful assistant becomes a confused deputy holding the keys to every tenant. Treat agents as first-class principals from the start; the guardrails and sandboxing deep-dive covers the containment side.
Tenant isolation
Isolation is the discipline of keeping each tenant's data, prompts, indexes, adapters and logs separate from every other tenant's, so that no request, however malformed or malicious, can read across the line. The cardinal sin of a multi-tenant platform is one tenant accessing another's data, and every isolation decision is ultimately about making that outcome structurally impossible rather than merely unlikely.
Isolation is not one switch but a spectrum, and different layers of the stack often sit at different points on it. At the soft end, a shared store partitions tenants by a namespace or a tenant_id on every row and every query: cheap and dense, but only as strong as the filter that is never allowed to be omitted. In the middle, per-tenant schemas or indexes give each tenant its own logical container inside shared infrastructure. At the hard end, separate stores, separate keys and separate compute give a physical separation that a forgotten filter cannot defeat, at higher cost and lower density.
SOFTER / DENSER / CHEAPER ---> HARDER / SAFER / COSTLIER +----------------+ +----------------+ +----------------+ | shared store | | per-tenant | | dedicated | | tenant_id on | | index or | | store + keys | | every row | | schema | | + compute | +----------------+ +----------------+ +----------------+ one bad filter logical walls, physical walls, leaks everything shared substrate nothing to share
The right choice is rarely uniform across the platform. A reasonable pattern is to isolate softly where the blast radius is small and the data is low-sensitivity, and reserve hard separation, dedicated stores and keys, for regulated or high-value tenants whose data must never share a substrate with anyone else. What matters is that the choice is deliberate per data class, encoded in the platform rather than left to each team, and defended by defaults that fail closed: a query that arrives with no tenant scope should return nothing, never everything.
The RAG isolation hazard
Retrieval-augmented generation concentrates the multi-tenant risk into one especially sharp form. A vector store holds embeddings of many tenants' documents, a user's question is embedded and matched against that store by similarity, and the top results are pasted into the prompt as grounding. Similarity search does not know about tenants. If retrieval is not explicitly scoped, the nearest neighbors to a question can be someone else's documents, and the model will faithfully answer using data the user was never entitled to see.
This is the failure that catches capable teams off guard, because nothing looks broken. The pipeline runs, the answer is fluent and well grounded, and the citations even look plausible. The defect is silent: the grounding was drawn from the wrong tenant, and you find out when a user notices a competitor's figures in their answer, or when an auditor asks to see why they did not. Scoping retrieval per tenant is therefore not an optimization but a correctness requirement.
There are three durable ways to scope it, and they compose. Namespaces partition the vector store so a query only ever searches its own tenant's segment. Metadata filters attach a tenant_id to every vector and constrain the query to matching records, which must be applied as a hard pre-filter, not a post-hoc trim of results. And separate indexes give each tenant its own store outright, the strongest option for sensitive tenants. Whichever you choose, the tenant scope must be derived from the authenticated principal and injected by the platform, never passed as a parameter the caller can set or forget.
Access, quota and cost
Beyond keeping tenants apart, the platform has to decide what each one is allowed to reach, how much of it, and who pays. These are three distinct controls, model and feature access, rate limits and quotas, and cost attribution, and they share a natural home: the model gateway, the single point every request already traverses. Placing them there means they are enforced once, uniformly, on the same choke point that carries identity, rather than reimplemented per application.
Access control decides which models and features a tenant may use at all. Not every tenant should reach every model; a frontier reasoning model, an external provider, or a feature still in preview may be restricted to specific tenants by policy. Rate limits and quotas protect the shared substrate from any one tenant, whether through a runaway agent loop or a genuine spike, monopolizing capacity or running up an unbounded bill. Cost attribution closes the loop by tagging every call with its tenant so spend lands where it belongs instead of arriving as one undifferentiated invoice.
| Control | What it enforces | Where it lives |
|---|---|---|
| Model & feature access | Which models and features a tenant may reach | Gateway policy, keyed on tenant identity |
| Rate limits | Requests per second or minute, per tenant and principal | Gateway, at the edge before inference |
| Quotas | Token, request or spend ceilings per period | Gateway, with usage tracked per tenant |
| Cost attribution | Every call tagged to a tenant and use case | Gateway telemetry and billing pipeline |
One caution on quotas for reasoning-class models: their internal thinking tokens bill as output, so a ceiling defined on input size alone will understate real spend and let a tenant quietly exceed budget. Numbers here are illustrative, but the direction is reliable, so define quotas on total tokens and watch the escalated path. The model gateway deep-dive covers the enforcement machinery in full.
Audit and the security review
A multi-tenant platform lives or dies at the security review, and that review will not accept assurances. It asks for evidence, and the evidence is the audit trail. Every consequential action, who called, as which tenant, which model and data they reached, what came back, should produce a per-tenant, tamper-evident log that can be reconstructed after the fact. Without it you cannot answer the two questions a reviewer, regulator or customer will eventually ask: where does AI touch this decision, and can you prove no tenant saw another's data.
Per-tenant audit is also what turns your isolation design from a claim into a demonstrable property. A data-boundary guarantee is worth only what you can show, so the reviewer will want to see the isolation mechanism, the fail-closed defaults, the retrieval scoping and the tests that exercise them, not a diagram asserting they exist. The strongest thing you can bring to that meeting is a suite that actively attempts cross-tenant access and proves it fails.
Expect a security review to demand at least the following before go-live:
- Isolation evidence: a clear map of where each tenant's data, indexes, models and logs live, and the control that keeps others out.
- Retrieval scoping proof: tests showing no query can retrieve another tenant's documents, canaries included.
- Per-tenant audit trails: immutable logs of access and actions, retained to policy and free of unnecessary PII.
- Access and quota records: who may reach which models, and evidence the limits are enforced rather than aspirational.
These obligations tie directly into the wider governance program. The boundary guarantees a review scrutinizes are the same ones model risk management and US privacy regimes care about; hedge the specifics, since requirements as of mid-2026 vary by state and sector, but treat the audit trail as the connective tissue between the platform and model risk and US AI governance.
The architect view
The through-line of everything above is that isolation is an architectural property, not a feature you add later. Retrofitting tenant boundaries onto a platform that assumed a single tenant is one of the more painful projects in enterprise engineering, because the assumption is baked into schemas, indexes, cache keys and log formats in a hundred small places. Designing for many tenants from the first commit, even while the first tenant is the only one live, costs little and saves you that excavation.
Put identity and quota at the gateway. The single point every request traverses is the one place where authenticating the principal, resolving the tenant, enforcing access and metering spend can be done once and trusted for everyone, including the team that has not read the policy yet and the agent nobody remembers deploying. Controls scattered through application code are controls that some application will omit under deadline.
Treat agents as first-class principals rather than an afterthought bolted onto human identity. As more of your traffic comes from autonomous processes acting in loops, an agent without its own scoped, delegated identity is an accountability gap and a confused deputy waiting to happen. Give it an identity, constrain it to less than its user's authority, and log what it does under its own name.
Above all, prove your boundaries rather than assume them. A tenant boundary you have not tested is a hypothesis, and the way you usually learn it was false is a customer or a regulator finding the leak first. Seed canaries, run cross-tenant access as a standing test, and make the proof part of every release. You can sketch how identity, gateway and isolation fit a full stack in the Architecture Designer, and pressure-test the policy side in the Governance Sandbox.