You score it against a labeled test set on every change. Most AI systems instead are measured once, on launch day, by the people who built them. Then a prompt gets edited, a model gets deprecated, a supplier redesigns a form, and nobody finds out that accuracy moved.
A golden set from real traffic, a gate in CI that blocks the bad merge, and a taxonomy of how the agent fails.
A traditional bug announces itself. An AI system that has quietly dropped from 94% to 81% field accuracy does not — it keeps returning confident answers, and the wrong ones look exactly like the right ones. In work that feeds a client-facing or regulated process, that difference shows up months later, when someone asks how a number was reached.
An eval harness, as Rottawhite builds it, is a labeled test set drawn from your own documents plus a scoring run that gates every prompt, model, and code change. It is remarkable how many vendors ship without one.
Built from your own real documents, deliberately weighted toward the hard cases: the scanned copy, the one-off format, the supplier whose template changed last year.
Accuracy per field, not one number for the document. A system that nails the customer name and fumbles the amount is not 90% right in any way that matters.
Prompt edits, model swaps, and code changes are scored before they ship. If accuracy drops below the agreed threshold, the change does not go out.
Broken out by document type and source, so a weak format surfaces instead of being averaged away by the easy cases.
An illustrative scoring run for a document-intake system. The shape is what matters: per-field, per-segment, with a threshold that blocks the release.
| Field | Clean PDF | Scanned | Threshold | |
|---|---|---|---|---|
| Customer name | 99.1% | 96.4% | 95% | pass |
| Document date | 98.7% | 94.2% | 95% | fail |
| Line-item total | 97.3% | 95.8% | 95% | pass |
| Reference number | 96.9% | 93.1% | 95% | fail |
Two fields fail on scanned documents. That is not a reason to abandon the system — it is a reason to route scanned document dates and reference numbers to a human until the number comes up. Without the harness you would not know which two fields, or that scans were the problem at all.
Smaller than people expect. One to two hundred labeled examples, chosen to cover the document types and edge cases you actually receive, catches most regressions. A thousand random samples that are all the clean case is worth less than eighty deliberately awkward ones.
We build the first version during the pilot or audit and your team corrects it, because the person who does the work is the one who knows what the right answer is. After that it grows from the exceptions the system routes to humans.
The replacement is scored against the same test set before it goes anywhere near production. A model upgrade becomes a measured decision with a number attached rather than a forced migration you hope went fine. That is the whole of our Model Migration engagement.
It is the same instinct applied to a non-deterministic system. The difference is that the pass condition is statistical, accuracy above a threshold on a population, rather than a single assertion being true.
Per-field accuracy, split into false positives and false negatives, broken out by document type and source. A single blended number hides exactly the failure modes that matter; a field that quietly invents values is worse than one that abstains.
Any change, whether a prompt edit, a model swap, or code, is scored against the test set first. If a field drops below its agreed threshold, the change does not ship. The gate is mechanical, not a judgment call made under deadline pressure.
As numbers, per field, agreed before the build starts, using the pilot or audit measurement as the baseline. "Works well" is not a criterion; "document date at or above 95% on scanned inputs" is.
Evals answer is it right. The audit trail answers how did it arrive at that. Regulated work needs both, which is why they ship together rather than as options.
Both are part of every production build, and the test set is yours to keep from the end of the Workflow Pilot onward. For an agent your team already runs, the Agent Reliability Audit installs the same harness in your repo, and Model Migration uses it to switch models without regressions.
Next step
Then ask how they measured it, and when they last re-ran it. We would rather you asked us that question than took our word for anything.