When Evidence Changes, Reconsider the Decision

A development agent may keep working after a design review: adding a dependency, changing a configuration, or introducing an export while resolving an error. Completing the assigned task does not establish that every resulting change remains within the earlier approval.

An approval records a decision made with particular information. When that information changes, the approval date alone cannot explain whether the decision still holds.

This final article in our practical AI-assurance series follows the reporting example from requirements decomposition and model-completeness review. It shows how a changed fact can affect an assurance claim. The scenario is illustrative; it is not a measured experiment or an account of an automated product feature.

Start with what supported approval

Suppose reviewers approved the reporting design after inspecting the primary transfer and its diagnostic flow. The evidence supported authorization for each endpoint and encryption for each channel. The team also examined the inventory's coverage and recorded its limits.

Approval covered a particular reporting workflow and a specified diagnostic configuration. The decision depended on those conditions remaining relevant. It did not grant permanent approval to every future export from the service.

An inspectable decision explains the requirement, the chosen design, the supporting evidence, remaining assumptions, and the person responsible for review. This connects an approval to its rationale rather than leaving the next team to infer it from a status label.

Argument-based requirements research provides an established foundation for reasoning with explicit assumptions and challenges. Jureta and colleagues examine this approach in regulatory compliance. Here we use an invented company rule to make the engineering question concrete.

Identify the claim a change could affect

Now the service adds a diagnostic export to a new archive. The earlier evidence covered the old destination. It cannot establish authorization for the new recipient.

The requirement has not changed. Its application has: there is another transfer to assess. Reviewers need to establish what the export contains, who receives it, whether the relevant permissions apply, and how the channel is protected.

This is why the word “every” mattered in the first article. A newly introduced transfer creates another instance of the obligation. Copying the old approval onto the new flow would skip that work.

Other changes can affect different parts of the argument. A permission revocation undermines authorization support. A modified encryption configuration calls for different evidence. A revised company rule requires checking whether the earlier translation still preserves its meaning. A cosmetic description change may have no effect on the claim at all.

Watch what the agent changed while completing the task

In the reporting example, suppose an agent adds the new diagnostic export to investigate a failure. The export helps with debugging, but it introduces another recipient for protected records. The task can become easier to complete while the original assurance claim becomes less well supported.

Review the consequential changes, including dependencies and configuration, rather than relying only on the agent's final summary or the success of the immediate task. Ask whether a new recipient, permission, or data path affects an obligation already reviewed. Changes that preserve those conditions need not trigger the same response as changes that alter them.

Also distinguish permission to use a tool from authorization for the information sent through it. An agent's ability to call an export tool does not establish that a particular recipient may receive protected records. Tool access and release controls must enforce their own boundaries; a written assurance argument does not enforce them.

This applies to the development process as well as the software it produces. If an agent uploads diagnostic material while debugging, that action needs its own appropriate authorization. If it merely proposes an export in code, reviewers must assess the proposed behavior before accepting the change. Keep those two claims separate.

Distinguish contradiction from missing support

If evidence establishes that the new recipient is unauthorized, there is a concrete violation to address. If the permission has not yet been checked, support is incomplete. These findings need different explanations even if both prevent approval of the proposed export.

An old observation can also remain historically correct while becoming insufficient for a current claim. Evidence that a channel was encrypted before a configuration change does not establish its behavior afterward. Preserve the earlier record and identify the new evidence required.

A review date provides a useful checkpoint, but the significance of change depends on the claim. Choose the review interval and event triggers according to the system's consequences and operating needs. There is no universal number of days after which every piece of evidence becomes invalid.

Review the affected decision, then record the result

For the new diagnostic export, pause its approval under the organization's change process while the owner gathers the missing information. Whether an already operating activity must stop is a separate operational decision governed by the applicable requirements and response procedures.

Reviewers can then accept the proposal with supported conditions, revise the design, reject it, or keep it unresolved pending evidence. If the change affects a modeled property, update the model and the claims extracted from the revised design, then check them against the applicable obligations again. A repeated pass on the old model does not assess the new transfer.

Keep the new decision connected to the old one. Explain what changed, which part of the rationale was affected, what evidence resolved the question, and what remains open. The existing architecture review guide helps choose the appropriate review; an exception exit plan addresses any explicitly permitted temporary departure.

Test the approach before calling it effective

Evaluate this approach on a small set of changes your reviewers understand. Include a change that should reopen a decision, one that should not, and one where evidence is missing. Ask whether the process helps reviewers reach supported decisions without sending every change through unnecessary review. Any improvement needs to be demonstrated in that evaluation.

For one agentic development task, identify the changes that could outgrow the current approval, the evidence that would reveal them, and the person responsible for resolving the question. The aim is to keep trust connected to what the agent changed and what the evidence supports—not to treat a completed task as a permanent assurance result.