How to set acceptance criteria for an operational system

Teams often evaluate a new system by reviewing several good outputs or asking whether users like the interface. Those observations are useful, but they do not establish whether the operation can recognize incomplete work, recover from a failed dependency, preserve required records, or place a decision with the right owner.

Operational acceptance criteria translate intended behavior into observable conditions. They cover successful work, edge cases, authority, recovery, support, and evidence. The criteria should be agreed before broad use so the standard is not rewritten around whatever the system happens to do.

Start with one recognizable unit: a request, case, document set, order, reconciliation, or decision packet. Describe what must be true when that unit finishes. Include required state changes and records, not merely the quality of a generated response.

Separate mandatory conditions from desirable qualities. A complete system-of-record update may be mandatory. A preferred writing style may be adjustable after launch. This distinction prevents subjective refinements from obscuring a failure that leaves the operation incomplete.

Build the evaluation set from the range of work the system is expected to receive. Include ordinary examples, incomplete inputs, ambiguous cases, duplicates, conflicting sources, unusual volume, and dependencies that respond slowly or not at all. Sensitive production material should be handled according to its approved boundary; representative does not mean copying uncontrolled data into a test environment.

For each case, state the expected outcome or expected handling. An acceptable result may be a correct completion, a request for missing information, a review queue with the necessary context, or a controlled stop. The system does not need to resolve every exception automatically, but it must not hide them.

Acceptance should verify what the system is allowed to read, propose, change, and communicate. Attempt actions outside that boundary. Confirm that approvals cannot be bypassed and that a reviewer can identify the source, proposed action, and consequence before deciding.

Test correction as well as approval. If an operator changes a classification, amount, destination, or drafted response, determine whether the correction is recorded and whether downstream work uses the corrected value. A review screen that cannot safely change the work is an observation point, not a control.

Disable or simulate the loss of a material dependency. Determine where the work stops, what state is recorded, whether a retry can duplicate an external action, and how an operator learns that intervention is required. Recovery should return the item to a known state instead of asking the operator to infer how far it progressed.

Include degraded operation in the criteria. The acceptable response to an unavailable model may be deterministic handling, a manual queue, delayed processing, or a temporary stop. What matters is that the behavior is deliberate, visible, and proportionate to the authority the system holds.

Identify who reviews the acceptance record and who can authorize a broader operating boundary. The implementer can assemble evidence, but the responsible operational owner should confirm that the cases, controls, records, and recovery behavior represent the work.

Acceptance is not permanent certification. Record the model, rules, interfaces, sources, configuration, and operating assumptions evaluated. A material change to one of those elements may require selected cases or the full set to run again.

Keep one concise record that connects the agreed standard to what was actually observed.

• The bounded unit of work and required end state are named.

• Mandatory conditions are separated from preferences and later refinements.

• Representative ordinary, incomplete, ambiguous, and conflicting inputs are included.

• Expected handling is defined for every evaluation case.

• Permitted actions, approvals, corrections, and prohibited actions are exercised.

• Dependency failure, retry behavior, duplicate prevention, and recovery are observed.

• Required records and operator visibility are checked.

• The evaluated versions, sources, configuration, and known limits are recorded.

• An accountable owner makes and records the authority decision.

A short list of meaningful criteria is more useful than a large scorecard no one can interpret. Each condition should help an owner decide whether the system may take responsibility, must remain supervised, or needs another change.

When criteria are written before implementation, they improve the design itself. The team has to decide how the operation should respond to uncertainty, failure, correction, and change before those conditions arrive under production pressure.

Continue with the complete resource library or read Grayhackle’s implementation approach.