Every gate type in Modulo and when to use each

Gates are the mechanism that catches agent mistakes before they reach a human reviewer or production. Every agent output in Modulo passes through at least one gate before it moves to the next stage. The gate either passes, and the output proceeds, or it fails, and the run pauses for human review or spawns a correction run.

Modulo has four output-side gate types, plus input-side guardrails that check what enters a run. Each is suited to a different kind of check.

Regex gate

A regex gate checks the agent's output against a regular expression pattern. It is the simplest and fastest gate type.

What it catches: Secrets and credentials in output (API keys, tokens, private keys), format violations (missing headers, incorrect date formats), banned patterns (TODO markers, debugging code).

Configuration: A regex pattern and a description of what the pattern checks for. The gate passes if the pattern does not match (for secrets) or does match (for required elements).

When to use it: As a first line of defense on every agent output. Regex gates are cheap to run and catch the most common and most dangerous failure modes. Every pipeline should have at least one regex gate checking for secrets.

JSON Schema gate

A JSON Schema gate validates the agent's output against a JSON Schema definition. It checks structure, types, required fields, and value constraints.

What it catches: Missing required fields, wrong data types, values outside allowed ranges, structural violations (array when object expected).

Configuration: A JSON Schema document (draft-07). The gate passes if the output validates against the schema.

When to use it: Whenever the agent is expected to produce structured output. If the agent generates a pull request description, a JSON Schema gate can validate that it has the required sections. If the agent generates a configuration file, the gate can validate it against the schema. Schema gates are the most common gate type in Modulo pipelines.

Custom function gate

A custom function gate runs a programmatic check against the agent's output. It can implement any validation logic that can be expressed in code.

What it catches: Business logic violations (a discount code applied to an ineligible product), cross-field validation (start date must be before end date), integration-specific checks (the generated code compiles against the project's type definitions).

Configuration: A function that receives the agent's output and returns pass or fail with an optional evidence message.

When to use it: When the validation logic is too complex for a regex or schema check. Custom function gates are the most flexible but also the most expensive to write and maintain. Use them only when the simpler gate types cannot express the check.

Scored eval gate

A scored eval gate uses an LLM to judge the quality of the agent's output against a rubric. It returns a score and a pass/fail decision based on a threshold.

What it catches: Quality issues that cannot be expressed as rules: the code is correct but poorly structured, the documentation is accurate but unclear, the design is valid but not idiomatic.

Configuration: A rubric describing what good output looks like, a scoring scale, and a pass threshold. The gate passes if the score meets or exceeds the threshold.

When to use it: For subjective quality checks where a rule-based gate would miss the problem. Scored eval gates are the most expensive gate type (they require an LLM call per check) and should be used sparingly, on the outputs where quality matters most.

Guardrails (input-side)

Guardrails are the input-side analogue of gates: deterministic checks, regex or JSON Schema only, evaluated at run creation before the run's input payload is persisted.

What it catches: Secrets, credentials, and disallowed content in webhook payloads and manual triggers, before they reach any node.

Configuration: An eval definition with eval_type="guardrail". Actions are observe, warn, block, or redact. A block is terminal - the run never dispatches and has no HITL gate. Redact masks matched values before persistence.

When to use it: Protecting the ingestion edge. When an incoming payload may carry structured credentials or banned content, a guardrail stops it at the boundary, before anything has executed. Remediation is the guardrail-override endpoint, which re-runs the guardrail pass on the operator-supplied input (re-block safe) rather than resuming on the blocked payload.

Block vs warn

Every gate can be configured as block or warn. A block gate stops the run and requires human review before it can proceed. A warn gate logs the failure but allows the run to continue.

When to block: Secrets detection, schema validation failures, any check where a failure means the output should never reach production without human review.

When to warn: Quality checks, style violations, any check where a failure is informative but not blocking. Warn gates are useful for building confidence in a new agent or gate configuration - you can see what the gate would catch without blocking the pipeline.

Putting it together

A typical Modulo pipeline uses multiple gate types in sequence:

1. A regex gate catches secrets in the raw output.
2. A JSON Schema gate validates the output structure.
3. A custom function gate checks business logic.
4. A scored eval gate assesses overall quality.

Each gate catches what the previous one might miss. Together, they provide layered defense that catches failures at the right level of abstraction.

The principle behind this: deterministic gates catch failures before they reach a human reviewer, and the run history shows which gate caught what.