Verification

This may be the single most important page in the section.

Work items are contracts, not prompts

Don't make tasks merely prompts. Instead of:

"Please modernize the ultrasound reconstruction module."

represent the work structurally. This is the work item — the same schema — one level up, at programme scope:

work_item:
  id: MOD-212
  objective: Eliminate the deprecated imaging API and improve modularity of reconstruction

scope:
  repository: ultrasound-platform
  domain: image-reconstruction
  writes:
    - reconstruction/**

constraints:
  - preserve functional behaviour
  - no public API changes
  - no numerical regression > 0.1%
  - no performance regression > 5%

required_evidence:
  - build
  - unit-tests
  - integration-tests
  - golden-image-comparison
  - performance-benchmark

approval:
  architecture_change: human
  release_merge: human

A planner decomposes it into child work items — like US-18432 on Five Primitives — each inheriting these constraints and narrowing the scope.

Now the agent has something much more powerful than natural-language instructions: an executable contract. constraints say what must stay true. required_evidence says how anyone — agent, reviewer or auditor — will know.

Verification is the heart of autonomous engineering

Don't ask:

How can I make the AI intelligent enough that I can trust its answer?

Ask:

How can I make the environment capable of proving that the AI's change satisfies the requirements?

For software, that is a pipeline:

The model is probabilistic. The verification pipeline should be as deterministic as possible.

This is why software engineering is an unusually attractive domain for agents. We already have compilers, type systems, linters, tests, simulators, static analysers, model checkers, coverage tools and version control — and architecture rules can be made executable too.

Verifier-driven development

Suppose the goal is "Remove this CVE." The agent shouldn't decide when it has succeeded. Success is defined externally:

dependency scanner:  CVE absent
build:               PASS
unit tests:          PASS
integration tests:   PASS
API compatibility:   PASS
performance:         within tolerance

Then the agent iterates against the verifiers, not against its own judgement:

Now autonomous iteration becomes much safer. Every failure is an input to the next attempt, not a review finding — the same economics as executable architecture.

Wiring verifiers into the harness

Three mechanisms do most of the work, and each exists in more than one harness:

In the harnesses

• Run the verifier automatically. Claude Code and OpenHands can run checks from Stop hooks — OpenHands explicitly recommends migrating pre-commit style checks to them — and Codex hooks act on Stop and SubagentStop, able to refuse to let a turn end.
• Protect the verifier from the agent. Claude Code's deny: Edit(tests/golden/**), Codex's filesystem deny globs and Copilot's --deny-tool all express "not this path". OpenHands cannot: with no path-scoped rules, that check has to live inside a hook script.
• Return a machine-readable verdict. claude -p --output-format json, codex exec --json --output-schema, openhands --headless --json and Copilot's -p each give a pipeline something to parse instead of prose to interpret.

One caveat for CI: OpenHands' headless mode always auto-approves, so there its confirmation policy is not a control at all — the sandbox and the hooks are what remain.

The third mechanism is what makes an agent a step in a pipeline rather than a person at a keyboard.

Evidence is attached, not asserted

The verifier outputs are the work item's required_evidence. Attached to the change set, they become the validation evidence a reviewer or auditor reads — instead of trusting a model's self-report, which is optimistic by construction.

Verifiers must be outside the agent's write scope

An agent that can edit the tests it is judged by will, eventually, pass by editing the tests. Keep verification code, golden data and thresholds outside the change surface of the work item being verified — or require a human to approve any change to them.