Skip to main content
← All posts

How do we trust AI agents in production?

Modulo Team

governancegatesaudit

Trust is a system of checks that makes a mistake cheaper than not running the agent. When an engineer asks "can we trust this AI agent in production?", they are asking "will we catch a mistake before it matters?"

What does not work

Blind trust fails: give an agent write access, let it run, and hope. The damage lands unnoticed.

The opposite extreme also fails: requiring approval for every agent action means someone still reads every line, moving the bottleneck from writing to reviewing code.

What works

Layer one: deterministic gates. Before any agent output reaches a human or production, automated checks run: a regex catches secrets in the diff, a schema check validates the output shape. These gates block, so a failure never reaches the next step. A scored eval (an LLM-judged, nondeterministic gate) can measure quality against a baseline. Guardrails apply the same principle on the input side, checking what enters a run - webhook payloads, manual triggers - before the first node executes, with observe, warn, block, or redact actions. A block is terminal.

Layer two: audit with evidence. Every gate decision is recorded alongside the output it evaluated, so a reviewer sees evidence, not just a red flag. "Output contained a string matching -----BEGIN RSA PRIVATE KEY-----" is actionable. "Gate failed" is not.

Layer three: evidence-based autonomy. Gates that never fire can be set to auto-pass - a setting you choose, not one the agent grants itself. The agent earns autonomy by consistently passing every check; you decide which change classes are safe to auto-merge, and block-level changes still need a gate or human review. The data says when it is ready.

How Modulo handles this

Modulo's eval engine runs gates on every agent output first. Gate configuration is part of the pipeline definition, versioned, reviewed, and auditable alongside it. A firing gate pauses the run for human review with evidence attached; one that never fires can be raised to autonomy. For how this applies across teams with different change classes and risk profiles, see governing coding agents across teams.

The principles behind this: deterministic gates catch failures before they reach a human reviewer, and audit as a first-class output means the history you inspect is exactly what happened.