Agent EvalEvidence for the next step.
Make agentic workflows easier to inspect through evidence-driven metrics and execution assessment.
Measure the workflow,
not the model.
A model answer can be correct by luck. Agent Eval measures what actually happened: which tools were called, whether they succeeded, and whether policy held.
Whether the workflow reached its declared end state, counted against a stated definition of done.
Whether the tool sequence matched the reference path, and how many calls were wasted.
Whether every call passed policy evaluation without an override or bypass.
Decision and end-to-end latency at a declared percentile, not an average.
What a threshold
looks like.
These retained sample values explain the shape of an acceptance rule. They are not a current benchmark and not a report of tests run against the Next release.
Adjust the acceptance rules below. The sample measurements stay fixed; no evaluation is run.
| Metric | Example | Acceptance rule | Result |
|---|---|---|---|
| Task success | 92.4% | ≥ 90% | PASS |
| Tool-call accuracy | 88.1% | ≥ 85% | PASS |
| Policy compliance | 100% | = 100% | PASS |
| Decision latency P95 | 18.2 ms | ≤ 25 ms | PASS |
| Errors in allowed calls | 11.8% | ≤ 5% | FAIL |
For a measured result, publish the code revision, dataset, successful-task definition, environment, sample count and raw output. A tool being allowed is not the same as successful task completion.
Where the line is.
Evaluation tooling is a means, not a verdict. What counts as good performance has to be defined by you.
Not a correctness claim
Published evaluation tooling does not certify model correctness. Test coverage, datasets and acceptance thresholds need to fit your use case.
Source and publication
Source availability and publication records do not certify the current Next revision. Review release notes, tests and deployment assumptions for your environment.
Where to go
next.
The current development branch, the running documentation, the published original and its permanent record.
The DOI identifies a published record. Consult that record for its exact version and files; it does not automatically identify the latest Next commit.
Start with your
acceptance criteria.
Describe the task, the definition of done and the threshold you would refuse to ship below. That is the whole conversation.