Skip to main content
Modulo

Feedback System

The Feedback System captures human-in-the-loop (HITL) rejections as structured FeedbackRecords, routes them through configurable handlers, and enables AI-powered correction runs.

Core Concepts

  • FeedbackRecord: Created when a HITL reviewer rejects a pipeline output. Stores rejection reason, rejected output, producing node/agent, and routing info.
  • Feedback Handler: Determines how a rejection is triaged – manual review (human), automatic AI correction (ai_correction), or AI correction with human review (ai_correction_with_human_review).
  • Correction Run: A new LangGraph run spawned from a feedback record, pre-configured with the original run’s context and the rejection details.
  • Feedback Inbox: UI surface for reviewing, annotating, and resolving feedback records across all pipelines.

API Endpoints

See PRD §8.20 for the full endpoint specification.

Routing Strategy

A pipeline's default_feedback_handler cascades to all HITL gates unless overridden by a gate-level feedback_handler. When both reject_target and feedback_handler are set on the same gate, the feedback_handler takes precedence.

Eval Gap Detection

When a rejection is recorded, the system can retroactively run the pipeline's eval suite against the rejected output. If no existing eval catches the failure, the record is tagged as an eval_gap – indicating an opportunity to expand the eval suite.