Turning Requirements into Checkable Obligations

A coding agent can turn a brief into a design, implementation, and tests. But if it misinterprets the requirement at the start, those outputs can agree with each other while missing the original obligation. A passing test suite would then support the wrong claim.

Consider the requirement: “Every transfer of protected reporting data must be authorized and encrypted.” What would justify accepting an agent’s implementation of it?

At Keller, we study how AI-assisted engineering decisions can become easier to examine, challenge, and trust. This series shares practical methods teams can apply in their own reviews.

This is the first article in a three-part series on practical AI assurance: making claims explicit, examining their support, and revisiting them when that support changes. The reporting system below is invented. We use it to explain an engineering method, not to report an experiment or describe a shipped product workflow.

Decompose the obligation without changing the rule

Constraint decomposition turns a requirement into conditions that can be checked. Its purpose is to preserve the requirement's meaning while making clear what must be established.

Consider an internal reporting service that sends protected records to a reporting store. The invented company rule requires authorization for both endpoints and encryption throughout each transfer. It permits no exceptions.

Three details are essential. Every means the review covers all applicable transfers. Both endpoints means sender authorization cannot stand in for recipient authorization. And means authorization and encryption are required together.

A useful question becomes: is there an applicable transfer with an unauthorized endpoint or an unencrypted channel? Finding one gives the team a specific problem to investigate. Examining one compliant transfer cannot establish that all the others comply.

The same discipline applies to more complicated requirements:

• Applicability: define which entities and activities the requirement covers. Unknown data classification leaves applicability unresolved.
• Quantifiers: preserve “every,” “at least one,” and “exactly one.” They ask different questions.
• Alternatives: if a rule permits A or both B and C, retain those complete alternatives. B alone is insufficient.
• Exceptions and precedence: carry over only exceptions the source permits, including their limits and authority. A deadline does not create a waiver.
• Time: distinguish authorization before a transfer from authorization obtained afterward. A later correction does not erase an earlier violation.

Read the translated conditions back against the original requirement. This review checks whether the logic asks the right question before trusting its answer.

When an agent turns the brief into its own acceptance criteria

Here, agentic development means software work in which an AI agent can choose steps, use tools, and revise its approach as it implements a task. Anthropic's guidance on agents describes this model-directed tool use and emphasizes feedback from the environment and human review of broader requirements. Our assurance question is whether the resulting engineering decision has adequate support.

Suppose the agent implements the reporting transfer and writes a test confirming that the channel is encrypted. It then describes the feature as meeting the requirement. The test may be correct, yet neither endpoint's authorization has been established. If the agent's tests follow its own shortened interpretation, agreement between code, tests, and summary can conceal the omission.

Keep the original obligation available during review. Ask which checks address each required condition and where their inputs come from. A test using an invented “authorized recipient” fixture can exercise program behavior; it cannot establish permission for the actual recipient.

This does not make agent-written tests unhelpful. It separates what they demonstrate from what still needs independent evidence. The useful handoff is a mapping from the requirement to checks, evidence, and unresolved questions—not a confident completion message.

Keep the rule separate from its evidence

The rule defines what must hold. Evidence establishes whether it holds for a particular transfer.

For our reporting flow, reviewers need the data classification, the sender's permission, the recipient's permission, and support for the channel's encryption. They also need a reason to believe the transfer inventory covers the system being assessed.

A label such as “authorized” cannot supply that support by itself. It needs an applicable permission and evidence that the permission covered this transfer at the relevant time. Likewise, a diagram showing an encrypted connection is a design claim; confirming deployed behavior requires suitable evidence from the actual environment.

This boundary matters when AI helps translate requirements. AWS distinguishes natural-language translation from logical validation in its Automated Reasoning concepts. A correct logical result still depends on a faithful translation.

Give missing evidence an honest outcome

Suppose the reporting inventory lists three transfers. Two have supported authorization and encryption. The third has no current recipient-authorization evidence.

That third transfer is unresolved. Missing evidence does not establish that it is unauthorized, but it also cannot support a pass. If reviewers later establish that the recipient lacked permission, the finding becomes a concrete violation.

Now suppose all three have adequate evidence, but nobody has checked whether diagnostic exports belong in the inventory. The three transfers may satisfy the rule individually. The wider system claim remains unsupported because its coverage is unresolved.

For this teaching example, distinguish four outcomes: a supported pass within a stated scope; a known violation; an unresolved result because necessary evidence is missing; and not applicable where evidence establishes that the rule does not apply. A known violation remains a violation even if other facts are missing. An empty inventory deserves a coverage check, not an automatic clean bill of health.

Try three cases before relying on the translation

Take one requirement and walk through an acceptable case, a concrete violation, and a case with missing evidence. Add a new relevant transfer and ask whether it receives the same scrutiny. For a temporal requirement, include an action performed too late.

These examples can expose mistranslation and false passes. They do not establish that the test set covers every possible failure. Record what was examined and which judgments still need an owner.

Before accepting an agent-generated change, pick one requirement and trace it through the implementation, checks, and supporting evidence. Identify what remains unresolved and who can resolve it. That gives the reviewer a basis for trusting the work beyond the agent’s own account of completion.

Next: how a correct checker can validate an incomplete model.