Why Agentic Development Needs Deterministic Checks

A coding agent implements a payment retry. Another agent reviews the patch and calls it sound. The tests pass. Then a timeout leaves the application unsure whether the first request succeeded, and the retry creates a second payment.

This is an illustrative failure scenario, not a customer incident. It captures a practical AI-assurance problem: a convincing review does not establish that an agent-generated change preserves the business rule that matters.

Deterministic checks evaluate explicit conditions using defined rules rather than asking an LLM to judge whether an answer looks right. They can validate structured output, inspect permitted actions, test behavior, or check properties of a formal model. Each supports a different claim. Their value comes from making that claim inspectable and repeatable within stated inputs and conditions.

For agentic development, the goal is to give consequential decisions a basis beyond the agent's account of its own work.

An LLM judge contributes a review, not a verdict to trust by default

LLM-as-a-judge evaluation is useful for questions involving language, relevance, and open-ended quality. A reviewer model may spot a missing failure case or challenge an unsupported assumption. Those are valuable contributions.

But a second model may receive the same incomplete description as the first. It can agree that the retry code follows the proposed design while neither examines what happens after the payment service accepts a request and its response is lost. Agreement does not resolve that missing evidence.

Anthropic's guide to agent evaluations distinguishes code-based, model-based, and human graders. It describes code-based checks as reproducible but limited in nuance, and model-based grading as flexible but requiring calibration. Choose the method according to the question being asked.

If the question is whether repeated requests can create duplicate payments, ask for evidence about that behavior. A favorable review score cannot substitute for it.

Start with an obligation the implementation must preserve

For this invented payment workflow, the rule is: retries of the same authorized payment operation must not create more than one payment. A timeout does not authorize a new operation. The workflow must also preserve the approved recipient and amount, and avoid reporting completion before it has adequate confirmation.

The coding agent proposes an idempotency key: a value used to recognize repeated requests for the same operation. That is a design choice, not evidence that the complete workflow handles duplicates. The relevant service must honor the key, its lifetime must cover the supported retry window, and the application must reuse it correctly.

A practical review separates several questions:

• Output contract: does a proposed payment request contain the required values in the permitted form? A structural check can establish this without establishing authorization.
• Permission: do the proposed recipient and amount match the approved operation? Compare against the authoritative approval, not values the agent supplies as its own evidence.
• Behavior: after an accepted request loses its response, does a retry preserve one payment? Exercise that failure with an observable outcome.
• Reporting: does an unresolved response remain pending rather than being presented as completed? Check the resulting state, not just the wording of a success message.

These checks turn “the agent handled retries” into claims a reviewer can examine individually.

Test the failure that a persuasive explanation can hide

Use a controlled test environment with no real payments. Arrange for the payment service to accept the first request, then withhold its response from the application. Let the application retry the same operation.

Inspect the service's recorded outcome: one payment, with the approved recipient and amount. Inspect the application's state too. If completion is still unconfirmed, it should remain unresolved. A final agent message saying “payment succeeded” is not the observation the test needs.

Then deliberately break the implementation so the retry uses a fresh operation key. The test should detect the duplicate. Also test two distinct authorized operations with the same recipient and amount: they should remain distinct. Otherwise an overly broad duplicate check could block legitimate work.

These are proposed test cases, not reported experimental results. They make the check's purpose concrete and expose two opposite mistakes: allowing a duplicate and suppressing a valid operation. Concurrent retries and service restarts require further cases if the workflow supports them.

Keep the acceptance rule under review when the agent repairs its patch. A passing run achieved by deleting the duplicate assertion or changing the expected result does not resolve the original defect. Review changes to the tests as well as changes to the implementation.

Deterministic checks need trustworthy inputs and honest scope

“Deterministic AI” can blur repeatability with correctness. A repeatable answer can still be wrong. A deterministic validator can consistently apply the wrong rule, and a test can consistently pass against a mock that behaves differently from the actual payment service.

For this example, review the service's documented idempotency behavior and use suitable integration evidence. Record the configuration and conditions exercised. A fixed test case establishes behavior for that case; it does not prove every possible execution safe. Tests involving uncontrolled timing or changing services may also be unstable even when their assertions are exact.

Formal verification addresses a different part of the problem: specified properties of a model. A bounded check can search the modeled situations within declared limits. It cannot establish that the model includes every relevant implementation behavior. Our guide to incomplete models explains that boundary.

The same distinction applies when checking natural-language LLM output. AWS's Automated Reasoning documentation separates an LLM's translation into logic from mathematical validation of that logic. A sound checking step still depends on faithful translation. Deterministic machinery does not turn an unsupported input into an established fact.

Put the check where the consequential decision happens

For a development agent, a required check can inform whether a change is accepted for release. For an agent that can initiate payments, an operating control must evaluate the proposed action before the payment commits. Those are separate responsibilities: a successful pre-release test does not enforce tomorrow's transaction permission.

A check that only writes a warning also differs from a control that prevents an action. State which behavior the system actually provides. If required evidence is unavailable, keep the decision unresolved and route it to an accountable owner rather than treating silence as permission.

Human review remains necessary for the meaning of the obligation, the adequacy of coverage, and acceptance of remaining risk. Model-assisted critique can help challenge those judgments. Explicit checks provide the mechanical evidence for the properties they can assess.

Make the next agent-generated change easier to trust

Choose one consequential obligation before accepting the next change. Identify the check, the source of its inputs, a case that must fail, and the decision that depends on its result. Keep the applicable rule, changed artifact, result, and remaining limitations together.

This is the connection to architecture governance. Archangel's Decision Trace connects decisions with their supporting context, evidence, review, and downstream work. The test harness and operating controls produce and enforce their own results; the decision record gives reviewers a place to understand what those results justify.

The standard for accepting agent-generated work should be explainable: which obligation was checked, what evidence supports the result, and what still requires judgment. A confident model response is useful input. An inspectable basis for the decision is what makes assurance possible.