> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dagraph.app/llms.txt
> Use this file to discover all available pages before exploring further.
# Configure multi-provider fallback chains in dagraph
> Keep workflows running through provider outages and rate limits by routing each node through an ordered fallback chain of alternative models.
Production AI workflows fail for reasons outside your control: a provider hits a capacity event, your account reaches its rate limit, or a network timeout drops a request. Rather than failing the whole DAG when one call fails, dagraph lets you attach a `fallback_chain` to any agent node — an ordered list of alternative models to try before giving up. This guide shows you how to set up fallback chains, which errors trigger them, and how to combine fallbacks with rate limiting to run reliable multi-provider pipelines.
## Model prefix routing
dagraph uses the prefix of a model string to determine which provider to call. No separate provider configuration is needed — the prefix is the routing key:
| Prefix | Provider |
| ------------ | ---------------------- |
| `anthropic/` | Anthropic Messages API |
| `openai/` | OpenAI API |
| `gemini/` | Google Gemini API |
| `ollama/` | Local Ollama instance |
A bare model name like `claude-haiku-4-5-20251001` (no prefix) uses the executor backend you specify with `--backend`. Add a prefix to explicitly route to that provider regardless of backend.
## Adding a fallback chain
Attach `fallback_chain` to any `agent` node. The scheduler tries the primary `model` first, then works down the list in order, stopping at the first successful response:
```yaml theme={null}
# multi_provider_fallback.yaml
name: multi_provider_fallback
inputs:
topic:
type: string
nodes:
- id: research
type: agent
model: anthropic/claude-sonnet-4-6 # primary: try this first
fallback_chain:
- openai/gpt-4o # fallback 1
- gemini/gemini-2.0-flash # fallback 2
- ollama/llama3.2 # last resort: local, free
prompt: |
Research the topic: {{ topic }}.
Return 3 key findings as bullet points.
- id: critique
type: agent
model: openai/gpt-4o
fallback_chain:
- anthropic/claude-sonnet-4-6
depends_on: [research]
prompt: |
Critique this research:
{{ research }}
What's missing? What's overstated?
```
Run it against the API backend so multi-provider routing is active:
```bash theme={null}
agentgraph run multi_provider_fallback.yaml \
--input topic="battery storage technology" \
--backend api
```
Each provider needs its own API key set as an environment variable. For Anthropic: `ANTHROPIC_API_KEY`. For OpenAI: `OPENAI_API_KEY`. For Gemini: `GEMINI_API_KEY`. Ollama requires a running local instance but no API key.
## Which errors trigger the fallback chain
The scheduler walks the fallback chain when it receives a **retriable** error from the current provider:
* **HTTP 429** — rate limit exceeded
* **HTTP 5xx** — provider-side server error
* **Network errors** — connection refused, DNS failure, dropped connection
* **Timeouts** — request exceeded the node's `timeout_seconds`
These errors are retried because a different provider might succeed where the first failed.
## Which errors bypass the chain
The scheduler does **not** try fallbacks for errors that a different provider cannot fix:
* **HTTP 401 / 403** — authentication or authorization failure. A bad API key is still bad at the next provider. Fix your credentials instead.
* **HTTP 400 / 422** — bad request or validation error. The problem is in your prompt or parameters, not the provider.
## Cost and traces
You are only charged for the **successful** attempt. If the primary model fails and the first fallback succeeds, you pay only for the fallback call — the failed attempt's tokens are not billed (they were never completed).
Every attempt, successful or not, appears in the run trace at `runs//trace.jsonl`. Inspect a run to see which provider actually answered:
```bash theme={null}
agentgraph inspect
```
The trace entry for a node that used a fallback includes both the failed attempt (with the error) and the successful attempt (with the model that answered).
## Rate limiting with `--rpm`
When using the `--backend api` option, you can hit Anthropic's (or another provider's) per-minute request limits even without a provider outage. Use `--rpm` to throttle the request rate before you trigger 429s:
```bash theme={null}
agentgraph run multi_provider_fallback.yaml \
--input topic="climate policy" \
--backend api \
--rpm 30 \
--max-concurrent 5
```
`--rpm` applies to the executor as a whole — across all nodes and all providers. It is most useful in combination with large fan-out DAGs where many nodes fire simultaneously.
A good default starting point for the Anthropic API free tier is `--rpm 20 --max-concurrent 3`. Bump these up as you upgrade your plan or add provider fallbacks that spread the load.
## Fallback chains inside `evaluator_loop`
Generator and evaluator roles inside an `evaluator_loop` node also support `fallback_chain` via the `AgentRole` spec:
```yaml theme={null}
- id: haiku
type: evaluator_loop
max_iterations: 3
generator:
model: anthropic/claude-haiku-4-5-20251001
fallback_chain:
- openai/gpt-4o-mini
prompt: "Write a haiku about {{ topic }}. ..."
evaluator:
model: anthropic/claude-sonnet-4-6
fallback_chain:
- openai/gpt-4o
prompt: "Evaluate: {{ candidate }} ..."
```
Each role's chain is independent — the generator and evaluator can fail over to different providers.
Combine fallback chains with parallel execution for resilient fan-out workflows.
Add fallback chains to generator and evaluator roles for reliable iterative refinement.