Feedback System
A rejection carries information a bare failure does not. When a reviewer rejects a pipeline output at a human-in-the-loop (HITL) gate, the Feedback System records why, decides what should happen next, and can hand the run to a correction pass that keeps the original context.
Core concepts
- FeedbackRecord: Created when a HITL reviewer rejects a pipeline output. Stores rejection reason, rejected output, producing node/agent, and routing info.
- Feedback Handler: Determines how a rejection is triaged: manual review (
human), automatic AI correction (ai_correction), or AI correction with human review (ai_correction_with_human_review). - Correction Run: A new LangGraph run spawned from a feedback record, pre-configured with the original run’s context and the rejection details.
- Feedback Inbox: UI surface for reviewing, annotating, and resolving feedback records across all pipelines.
API endpoints
| Endpoint | Method | Description |
|---|---|---|
/api/v1/feedback |
GET | List feedback records (paginated) |
/api/v1/feedback/{id} |
GET | Get a single feedback record |
/api/v1/feedback/{id}/resolve |
POST | Mark a feedback record as resolved |
/api/v1/feedback/{id}/annotate |
POST | Add annotation or notes to a feedback record |
Routing strategy
A pipeline-level default_feedback_handler applies to all HITL gates unless a gate overrides it with its own feedback_handler. When both reject_target and feedback_handler are set on the same gate, feedback_handler takes precedence.
Eval gap detection
When a rejection is recorded, the system can retroactively run the pipeline’s eval suite against the rejected output. If no existing eval catches the failure, the record is tagged as an eval_gap, indicating an opportunity to expand the eval suite.
Related
- Pipelines: the directed-graph execution model
- Human-in-the-Loop: review gates and governance
- Evals: quality checks on agent output