Requirements documents describe what a system should do. Evaluation suites prove what it does. In agentic work the second replaces the first, and teams that internalise this early ship faster and argue less.
The method is blunt. Before anyone builds anything, pull several hundred real cases from the operation's history: claims that were triaged, contracts that were reviewed, tickets that were resolved. Have the best people in the operation grade them the way they would grade a colleague's work. That graded set is the specification. The agent is built to pass it.
What the exam changes
First, it ends the abstraction. "Handle claims accurately" is a wish; ninety-eight percent agreement with senior adjusters on the golden set is a target. Second, it surfaces disagreement early: when two experts grade the same case differently, you have found a policy question the organisation needed to answer anyway, and it is far cheaper to answer it before the agent embodies the confusion.
A requirements document is an opinion about the future. An evaluation suite is a fact about the present.
The suite outlives the launch
The same cases that specified the system become its regression suite. Every model upgrade, prompt change and integration fix reruns the exam. The score is the deploy gate. This is what makes change safe: the system can evolve weekly because the definition of correct is stable and executable.
When we say we refuse to ship without evals, this is what we mean. Not a benchmark from a paper. Your cases, graded by your people, rerun forever. That document is worth more than any deck we could leave behind.