Evaluating Agents: Beyond Right Answers

A journey, not an answer

Most evaluation tooling grew up around a simple contract: send a prompt, get a completion, score the completion. Accuracy checks, groundedness checks, LLM-as-judge rubrics, all of it assumes the unit of evaluation is a single output. An agent breaks that contract. Its output is a trajectory: a sequence of model decisions, tool calls with real arguments, intermediate observations, and side effects on external systems. The final answer is only the last frame of a longer film.

The same task can succeed through a good path or a terrible one. Ask an agent to resolve a billing discrepancy, and one trajectory looks up the invoice, checks the payment record, and posts a correction in four steps. Another calls the wrong tool twice, retries the lookup five times with malformed arguments, opens a duplicate ticket as a side effect, and eventually stumbles onto the same correction. An outcome-only eval scores both runs as identical successes. In production, the second agent costs several times more, runs slower, and leaves debris in your ticketing system.

Outcome-only scoring is also blind to the failures that matter most. An agent can fail silently halfway: complete three of five steps, hit an error, and return a confident summary of work it never finished. It can take a forbidden action, a write it should never have made, on the way to a correct answer. It can loop for thirty turns before converging. None of this appears in a score computed on the final message alone.

So agent evaluation shifts the unit of measurement from the answer to the journey. The rest of this article builds that up in layers: end-to-end task success, step-level trajectory judging, efficiency, run-to-run stability, and the sandboxed environments and CI wiring that turn all of it into a regression suite you can actually gate releases on.

End-to-end task success

The headline metric is still the obvious one: did the agent complete the task to spec. Everything else in this article qualifies that number; nothing replaces it. But the number is only as meaningful as the spec behind it, and this is where most agent eval efforts quietly fail. A task needs a defined starting state, a defined goal, and a verifiable definition of done.

The strongest success criteria are assertions over final environment state, checked by code, not string comparison against a reference answer. If the task is "process the refund for order 4417," success means the refund record exists with the right amount and reason code, the order status changed, and no other order was touched. If the task is "triage this bug report," success means a ticket exists with the correct severity and component fields. Deterministic verifiers make success rates trustworthy and cheap to compute at scale; free-text grading of agent outcomes reintroduces every judge problem at the worst possible layer.

Partial credit is worth modeling where partial completion has real value: a long investigation task might score milestones separately (found the account, diagnosed the cause, applied the fix). But use it deliberately. Averaging milestone scores across a suite can dress up a 40 percent completion rate as a comfortable 0.7, and executives will read the 0.7. When in doubt, report binary completion and milestones side by side.

Judging the trajectory

Once the outcome is scored, the path itself becomes the object of evaluation. Step-level judging asks a small set of questions at every decision point:

• Tool selection: given the state at that step, was this a reasonable tool to call, or was a better one available and ignored?
• Argument validity: were the arguments schema-valid and semantically correct, the right order ID, the right date range?
• Ordering: did the agent read before it wrote, verify before it acted, and avoid redoing work it had already done?
• Forbidden actions: did it stay inside policy, no destructive calls, no out-of-scope writes, even on runs that ended well?
• Recovery: after a tool error, did it adapt sensibly or spiral into retries?

Trajectories come from traces, which makes tracing the data source for all of this. The same instrumentation that powers production debugging, every model call, tool invocation, argument, and result captured in order, is what your evaluators consume; if the trace is incomplete, the trajectory is unjudgeable. The tracing and observability deep-dive covers the plumbing; the point here is that agent evals are downstream of it by construction.

Score deterministically wherever possible: argument schemas, forbidden-action lists, and ordering constraints are code, and code does not hallucinate. For the fuzzier questions, was the plan sensible, was that retry reasonable, an LLM judge reading the full trace against a rubric works well in practice, with the standard caveats: judges carry biases, drift in calibration, and need periodic validation against human labels on a sample. The eval methods deep-dive treats those caveats in detail; none of them disappear because the input is a trajectory instead of a paragraph.

Efficiency is a metric, not a nicety

Two agents with identical success rates can be wildly different systems. Illustratively: version A completes 9 in 10 tasks at an average of six steps and a modest token bill; version B completes the same 9 in 10 at fourteen steps and four times the tokens, because it re-reads documents it already retrieved and narrates its own reasoning back into context. On an outcome leaderboard they tie. In production, one of them is quietly multiplying your unit cost and your latency. A prompt or model change that preserves success but doubles steps per task is a regression, and your suite should say so.

The normalization matters: divide by completed tasks, not runs, and keep failed-run spend in the numerator. An agent that fails half its tasks cheaply is not efficient; it is unfinished. Tracking these per task type, rather than as one global average, is what lets you see that a change helped research tasks while degrading transactional ones.

Stability across runs

Agents are nondeterministic. Sampling, provider-side variation, tool timing, and small differences in intermediate state mean the same agent on the same task can take different paths on consecutive runs, and sometimes reach different outcomes. A single run therefore proves very little in either direction: one pass may be luck, one failure may be noise, and you cannot tell which without more data.

The fix is repetition. Run each task several times, five to ten is a reasonable illustrative starting point, and report the distribution, not a single number. Two suite-level figures matter: the share of tasks the agent passes at least once, and the share it passes on every run. The gap between them is your reliability story, and the pass-on-every-run bar is much harder than it sounds. A task that passes seven runs out of ten is not seventy percent of a working feature; it is a feature that fails a third of the time in front of users. Which bar gates a release depends on the deployment: a human-reviewed drafting assistant can live with pass-mostly, an unattended transactional agent cannot.

Budget for this. Repetition multiplies eval cost, which is another reason to keep tasks sharp and suites curated rather than sprawling.

Environments and regression suites

Agents act on systems, so you cannot evaluate them against production. The unit of infrastructure is a sandboxed environment: seeded state (fixture accounts, known documents, a database in a defined condition), a defined task with its verifier, and tools that are either mocked for determinism and cost, or real but isolated, pointing at staging instances. The environment resets between runs so every trajectory starts from the same world and results stay comparable.

        AGENT EVAL LOOP

  1. seed state     fixtures, accounts, documents
        |
        v
  2. run task       agent + mocked or isolated tools
        |
        v
  3. capture trace  every step, argument, result
        |
        v
  4. score          success | trajectory | cost
        |
        v
  5. reset, repeat  k runs per task -> distribution
        |
        v
  6. gate           CI blocks on regressions

Alongside end-to-end tasks, keep component-level evals: routing decisions, retrieval quality, and tool selection each tested in isolation with their own datasets. End-to-end tells you whether the system works; component evals tell you why it stopped working, and they run in seconds instead of minutes. When an end-to-end task regresses, the component suite is what turns a mystery into a diagnosis.

Then wire the whole suite into CI as the gate for agent changes. A prompt edit, a tool schema change, a model version bump, or a retrieval config tweak should all trigger the suite, and merges should block on regressions in success, stability, or cost against the recorded baseline, exactly as described in the AI CI/CD deep-dive. The suite grows the same way good test suites always have: every production incident becomes a new task with a verifier, so the same failure cannot ship twice.

The architect view

Agent evaluation is not a new discipline so much as a convergence point. Tracing supplies the raw material, the full trajectory of every run. Evals supply the method, deterministic verifiers plus calibrated judges. Guardrails testing supplies the forbidden-action checks that run over the same traces. Teams that treat these as one system get compounding returns; teams that bolt on a demo-day success metric get a number that goes up while the agent gets worse.

The operating principles are compact:

1. Define tasks precisely. Seeded state, checkable done-criteria, code-based verifiers. If you cannot write the checker, the task is not ready to be a benchmark.
2. Judge trajectories, not just outcomes. Tool choice, arguments, ordering, forbidden actions, and recovery are all scoreable, and they are where silent failures live.
3. Treat efficiency as first-class. Steps, tokens, latency, and dollars per completed task, with regressions blocking merges like any other regression.
4. Measure stability with repetition. Report the distribution across runs, hold unattended agents to the pass-every-run bar, and treat flakiness as a defect to localize.
5. Gate releases on the suite. Sandboxed environments, component and end-to-end coverage, and CI enforcement on every agent-affecting change.

None of this is exotic; it is software engineering discipline applied to a nondeterministic actor. The organizations that internalize it will ship agents that hold up under load and audit. The ones that keep scoring final answers will keep being surprised. If you want to see the mechanics running rather than described, the Eval Harness in the Lab is the working version of this article.