Governing AI-Generated Code

Governance at a new volume

Every code governance practice most enterprises run today rests on a quiet assumption: that a human being read the code before it merged. Pull request review, pairing, the second set of eyes on a risky change, the tacit knowledge that a senior engineer would notice something off. The whole apparatus is calibrated to the throughput of people reading diffs at human speed. Coding agents break that calibration, not by doing anything the tools forbid, but by producing sound-looking code far faster than anyone can read it.

The shift is one of volume and velocity at once. A single engineer directing an agent can open more changes in a day than a team used to review in a week, and the diffs are larger, touching more files, spread across more services. The reviewer who used to read every line now faces a queue that cannot be read every line. The honest outcomes are two: either review becomes a rubber stamp that inspects nothing, or it becomes the bottleneck the agents were supposed to remove. Neither is governance. Both are just the old process failing quietly under a load it was never sized for.

The reframe this deep-dive argues for is to stop treating review as the place governance happens and start treating the platform as that place. If a person can no longer read every line, then the properties you care about, that the code is attributable, scanned, license-clean, and covered by tests, have to be guaranteed by machinery that runs on every change regardless of who or what wrote it. This is the same move enterprises already made for the AI-native SDLC and for AI CI/CD: the pipeline, not a person's attention, becomes the thing that guarantees safety. Applied to generated code, that means making provenance, scanning, and test-based review something the platform does by default rather than something a reviewer is asked to remember.

What is different about generated code

Generated code is not simply human code written faster. It fails in characteristic ways that human code does not, and those failure modes are the reason it needs its own governance rather than a bigger version of the review you already run. A skeptical architect should be able to name them, because each maps to a specific control later in this article.

Five risks are specific enough to plan around. Plausible-but-wrong code is logic that reads cleanly, compiles, and even passes the happy path, but is subtly incorrect on an edge the model did not reason through. Insecure patterns appear because the model reproduces the average of its training data, and the average of public code includes a great deal of injection-prone, unvalidated, and outdated security practice. Hallucinated or malicious dependencies are imports of packages that do not exist, or worse, that an attacker has registered under exactly the plausible name a model tends to invent. License contamination is the reproduction of a licensed snippet, verbatim or near, with no attribution and no record of where it came from. Outdated practices are baked in because training data has a horizon: a model happily writes the idiom that was correct three years ago and is deprecated or unsafe now.

Provenance and attribution

You cannot govern what you cannot identify. Before any control in this article can run, the platform has to know which code was AI-generated, produced by which tool, and, where feasible, from what prompt or session. Provenance is not paperwork; it is the index that makes review, audit, and accountability possible. Without it, generated code and hand-written code are an undifferentiated mass, and every downstream policy has to be applied blindly to everything or not at all.

Provenance answers questions that arrive later and unannounced. When a vulnerability class turns up in a particular tool's output, which merged changes need re-examining? When a licensing dispute lands, can you show where a suspect snippet originated? When an incident review asks who is accountable for a defect, is the answer a person, a tool configuration, or a prompt nobody kept? These are not hypothetical. They are the ordinary questions of running software at scale, and they become unanswerable the moment authorship goes unrecorded.

Capture provenance as close to generation as possible and carry it through the lifecycle. The practical signals are worth naming:

• Origin flag: whether a change, or ideally a hunk, was agent-generated, agent-assisted, or hand-written, recorded at commit time rather than reconstructed later.
• Tool and version: which coding agent or model produced it, so a later finding about one tool has a queryable blast radius.
• Prompt or session reference: a link to the instruction and context that generated the code, subject to retention and privacy limits, so a decision can be reconstructed.
• Human accountability: the engineer who directed and accepted the change, because provenance names a responsible person, never a tool, as the accountable party.

Treat these as metadata that travels with the code, in commit trailers, review records, and the artifacts the pipeline emits, not as a report someone assembles by hand. Provenance you have to reconstruct after the fact is provenance you do not have. This connects directly to how coding agents are configured in the first place: an agent that cannot stamp its own output is an agent that has externalized a governance cost onto everyone downstream.

Security and dependency scanning

Generated code should clear the same security bar as human code, and given its failure modes, arguably a higher one. The controls are not new. Static analysis, dependency scanning, and secret detection are mature practice. What changes is that they can no longer be advisory checks a diligent team runs before merge; they become automated gates the platform enforces on every change, because the volume has removed the human who used to be the backstop. Automated gates, not manual hope.

Two categories deserve extra attention for generated code specifically. The first is the insecure pattern the model reproduces from its training average, which is exactly what static application security testing is built to catch, so run SAST on generated diffs as a blocking gate rather than a dashboard nobody reads. The second is the dependency, where generated code introduces a distinctive risk that human code rarely does: the package that was hallucinated into existence. A model invents a plausible import name; an attacker, anticipating the pattern, has already registered that name with a malicious payload. The defense is to verify that every dependency an agent adds actually exists, is the package you think it is, and comes from a trusted source, before it ever resolves in a build.

The point of putting these in the pipeline rather than the review checklist is that they scale with the code and do not tire. A gate wired into AI CI/CD runs identically on the thousandth diff of the day as on the first, which is precisely the property a reviewer under queue pressure cannot offer. Calibrate the thresholds to your own risk tolerance, but make the enforcement automatic and the exceptions recorded.

Licensing and IP

Licensing is the risk most easily deferred and most expensive to discover late. A coding agent trained on public repositories can reproduce a licensed snippet closely enough to inherit its obligations, with no attribution and no note of where it came from. The exposure is twofold: code entering your codebase may carry a license incompatible with how you ship, and the ownership status of what the agent produced may itself be unclear. Neither problem announces itself. Both surface at the worst time, during due diligence, an audit, or a dispute, when the record you needed was never kept.

Manage it with policy and tooling together, because either alone leaves a gap. Policy without tooling is a document no one can enforce at volume; tooling without policy is a scanner that flags findings no one has decided what to do with. The pairing is what makes the control real.

Provenance, again, is the foundation the licensing controls stand on. A snippet-matching scan tells you that code resembles a licensed source; the provenance record tells you which tool and prompt produced it, which is what lets you decide whether the match is a coincidence or a contamination, and what to do about tooling that keeps producing them. Keep the policy specific about which tools and models are approved for what kinds of work, because approving a tool is, in effect, accepting the IP posture of whatever it generates. Treat legal review as a stakeholder in the platform's design, not a court of appeal after the code has shipped.

Review that scales

If a human can no longer read every line, then reading every line has to stop being the definition of review. The alternative is not to review less; it is to review differently, moving human attention to where it is scarce and valuable and letting the platform cover the volume. Three shifts do most of the work: review the specification rather than only the implementation, treat tests as the executable contract, and reserve line-level human reading for the diffs that actually carry risk.

The spec is where intent lives, and reviewing intent scales in a way that reviewing output does not. If the change is driven by a clear specification, a reviewer can judge whether the specification is right, which is a smaller and more durable artifact than the code that implements it, and lean on spec-driven development to keep implementation faithful to it. Tests are the other half. A test the code must pass is a contract stated in executable form: it encodes what correct behavior is and fails loudly when the generated implementation drifts from it. Tests catch the plausible-but-wrong code that reads fine and behaves badly, precisely the failure a skimming reviewer misses.

  agent produces change
          |
          v
  +-------------------+   missing/weak
  | PROVENANCE STAMP  |------------------+
  +-------------------+                  |
          | tagged                       v
          v                        +-----------+
  +-------------------+  finding    |  BLOCK /  |
  | AUTOMATED GATES   |------------>|  ROUTE TO |
  | SAST . deps .     |             |  HUMAN    |
  | secrets . license |             +-----------+
  +-------------------+                  ^
          | clean                        |
          v                              |
  +-------------------+  fail            |
  | TESTS AS CONTRACT |------------------+
  +-------------------+                  |
          | pass                         |
          v                              |
  +-------------------+  high-risk diff  |
  | RISK-BASED REVIEW |------------------+
  +-------------------+
          | low-risk, all green
          v
        MERGE
            

Risk-based routing is what makes the queue tractable. A change that touches authentication, payment flow, data handling, or an external boundary earns full human reading; a well-tested change to an internal, low-blast-radius surface, already clean through every gate, does not need a person to re-read what the machinery has already checked. This is the shape the AI-native SDLC converges on: humans spend their judgment where judgment is decisive, and the platform absorbs the volume everywhere else.

The architect view

The failure mode to design against is the organization that answers the volume of generated code with an appeal to discipline: reviewers told to be more careful, teams reminded to check licenses, a policy document that assumes someone still reads every line. It holds until the day it does not, and then a plausible-but-wrong change, or a hallucinated dependency, or an uncredited snippet reaches production with no record of where it came from. Heroic manual review does not scale to agent output, and a governance model that depends on it is already failing quietly.

The architect's job is to make the governed path the default path, so that generated code is governed by the platform rather than by anyone remembering to govern it. That means provenance stamped at generation and carried through the lifecycle; SAST, dependency, secret, and license scanning wired as automated gates on every change; licensing policy backed by tooling and a clear list of approved tools; and review reshaped around specs, tests as the executable contract, and risk-based routing of the diffs that warrant a human. Each of these is a paved road, not a guideline: something a team gets for free by shipping on the platform, and something they would have to work to opt out of.