Feature Spotlight: How model backends let you mix AI providers in one pipeline

Each agent in a Modulo pipeline binds to a model backend - a provider configuration that routes the LLM call to the right provider. Each backend wraps a provider (Anthropic, OpenAI, Ollama, or any OpenAI-compatible API) with its own credentials, model selection, and default parameters. Agents bind to a backend at configuration time, and Modulo routes the request to the right provider at runtime.

In short

• What it does: model backends let you configure LLM providers once and bind agents to them, so each agent in a pipeline can use a different model.
• When you would use it: when different tasks need different models - code review with Claude, customer text with GPT-4, classification with a local model.
• When you would not: when every agent in your pipeline uses the same model and provider.
• Health checks and rotation catch provider outages before they crash your pipeline.

The problem it solves

Say you run a pipeline that classifies incoming support tickets, drafts responses, and posts them to your ticketing system. The classification agent needs a fast, cheap model. The drafting agent needs a high-quality model that understands tone. The posting agent needs almost no model at all - it just formats and sends.

Without model backends, you either force all three agents through the same provider (paying premium prices for classification) or maintain three separate pipeline stacks (one per provider). Both options waste money or time. Model backends let you pick the right provider for each agent, bind it once, and let Modulo handle the routing.

How it works

Provider configuration

Each model backend is configured with a provider, model ID, and credentials. Modulo supports:

• Anthropic - Claude models (Opus, Sonnet, Haiku)
• OpenAI - GPT-4, GPT-4o, GPT-3.5-turbo
• Ollama - local models (Llama, Mistral, Phi, any Ollama-compatible model)
• Custom - any OpenAI-compatible API endpoint

Credentials are encrypted at rest and never enter agent state, logs, or checkpoints. The setup flow generates a one-time URL where you enter the API key directly - the key never passes through the LLM or the Modulo backend.

Agent binding

Each agent in a pipeline binds to a model backend at configuration time. The binding is a simple reference:

{
  "name": "classify-ticket",
  "type": "llm",
  "model_backend": "fast-classifier",
  "prompt": "Classify this ticket by urgency..."
}

The model_backend field references a configured backend by name. You can change the backend without changing the agent code - swap from Claude to GPT-4 by updating one field.

Health checks and rotation

Modulo runs periodic health checks against each model backend. If a provider is down or rate-limited, the health check fails and the backend is marked unhealthy. A run bound to an unhealthy backend is routed to a fallback (if configured) or pauses with a clear error.

Rotation means you can configure multiple backends for the same provider, and requests distribute across them automatically. This is useful for high-throughput pipelines where a single API key might hit rate limits.

Cost tracking

Each model backend tracks token usage and cost per run. The analytics dashboard shows which providers are consuming the most budget, so you can optimize model selection over time. If a classification agent is using GPT-4 when GPT-3.5-turbo would suffice, the cost data makes that visible.

Journey example: before and after

Before

Imagine a team (led by an engineer named Jordan) that runs every agent through Claude Opus. It works, but the monthly bill is $4,000 - most of it from classification and summarization agents that do not need the most capable model. Jordan knows a cheaper model would work for those tasks, but switching means changing code in 8 different pipeline configs.

After

Jordan configures two backends: Claude Opus for code review and drafting, and Claude Haiku for classification and summarization. Each agent binds to the appropriate backend. The monthly bill drops to $1,200 without any change in output quality - the classification accuracy is identical, and the drafting quality is indistinguishable for the team's use case.

When to use it vs. when not to

Use model backends when different tasks in your pipeline need different models. The backend gives you provider-level control without changing agent code, and the health checks catch provider outages before they cascade.

Do not create a separate backend for every model variant. If you have Claude Opus and Claude Sonnet, one backend per provider is enough - bind agents to the specific model at the agent level. Similarly, do not use model backends as a secrets manager. The backend stores credentials, but the primary purpose is provider routing, not credential lifecycle.

Where it fits in a pipeline

Model backends sit at the agent-configuration layer. Each agent binds to a backend, and Modulo routes the request at runtime. For the tool-access layer that connectors provide, see Feature Spotlight: How connectors let you swap tools without changing pipelines. For the governance layer, see Feature Spotlight: How HITL gates stop bad agent output before it ships. For the broader context, see the principles of Modulo.

Go deeper

Read the Model Backends docs for the full backend configuration, or the related docs on agents and secrets. Try the hosted demo at demo.modulo.run, or use the contact page to talk to us about configuring your model backends.