---
title: "Fallbacks"
subtitle: "When a call fails, the gateway tries the next attempt: another key or provider for the model, then the next model of your route."
---

Fallback happens at three levels:

- **Keys and providers.** A model name such as `gpt-5.5` resolves to every offering of it on every key that can pay for it, your own keys first. If one attempt fails, the next is tried. You get this without setting anything.
- **Models.** A [route](/docs/ai-gateway/routing/routes-and-aliases) you configure in the platform lists models to try in order. A fallback route is a **Fallback** node: the next entry runs only when the one before fails.
- **Respan's keys.** With [managed fallback](#managed-fallback), Respan's keys take over when your own provider key is rate limited.

Create a fallback route once:

```bash
curl -X POST https://api.respan.ai/api/routes/ \
  -H "Authorization: Bearer $RESPAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "smart",
    "mode": "fallback",
    "entries": [
      {"name": "gpt-5.5"},
      {"name": "claude-sonnet-4-5-20250929"},
      {"name": "gemini-2.5-pro"}
    ]
  }'
```

Then send its name as `model`:

<Tabs>
<Tab title="cURL">
```bash
curl https://gateway.respan.ai/v1/chat/completions \
  -H "Authorization: Bearer $RESPAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "smart", "messages": [{"role": "user", "content": "Hello!"}]}'
```
</Tab>
<Tab title="Python">
```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://gateway.respan.ai/v1", api_key=os.environ["RESPAN_API_KEY"])

response = client.chat.completions.create(
    model="smart",
    messages=[{"role": "user", "content": "Hello!"}],
)
```
</Tab>
<Tab title="TypeScript">
```typescript
import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://gateway.respan.ai/v1", apiKey: process.env.RESPAN_API_KEY });

const response = await client.chat.completions.create({
  model: "smart",
  messages: [{ role: "user", content: "Hello!" }],
});
```
</Tab>
</Tabs>

A request for `smart` tries every attempt of `gpt-5.5`, then every attempt of `claude-sonnet-4-5-20250929`, then every attempt of `gemini-2.5-pro`. Your code doesn't change when you change the route.

## Settings

| Setting | Where | Default | Description |
|---|---|---|---|
| [Fallback route](#fallback-routes) | `/api/routes/` with `mode: "fallback"` | none | Models to try, in order, under one name you send as `model`. |
| [`x-respan-managed-fallback-enabled`](#managed-fallback) | request header | off | Lets Respan's keys serve after your own key answers `429`. |
| [Provider offering](/docs/ai-gateway/routing/provider-routing#pin-a-provider) | `model` or a route entry, such as `azure/gpt-5.5` | none | Keeps a request, or one route entry, on one provider. |

## How failover works

1. **Checks.** The gateway authenticates the request and applies your [limits](/docs/ai-gateway/spend/budgets-and-limits) and credit check. A request refused here never reaches a provider and doesn't fall back.
2. **Plan.** `model` resolves to an ordered list of attempts, each one model offering on one key. A route contributes its entries in order; each model contributes its offerings, on your own keys first. See [How attempts are built](/docs/ai-gateway/routing/provider-routing#how-attempts-are-built).
3. **First attempt.** The gateway sends it. If the provider answers with a success status, that answer is returned, streamed or not.
4. **Next attempt.** A rate limit, server error, timeout, or dropped connection moves to the next attempt: another of your keys, another provider of the model, then the next entry of the route.
5. **Respan's keys, if opted in.** Attempts on Respan's keys that stand in for your own key run only if you sent [`x-respan-managed-fallback-enabled: true`](#managed-fallback) and your key's attempt answered `429`.
6. **Result.** The first success is returned. If every attempt fails, you get the last error.

Two errors end the request at once, because no other attempt can fix them: input larger than the model's context window, and `401` from one of your own provider keys. Once an answer has started streaming, it isn't switched to another attempt. Each provider call has a fixed upstream timeout of 600 seconds.

## Attempts for one model

Even without a route, one model name gives the gateway several attempts. For `gpt-5.5` the order is:

1. Your own provider keys ([BYOK](/docs/ai-gateway/spend/credits-vs-byok#byok)) saved in the platform that serve the model, the model's maker first (OpenAI before a reseller such as Azure OpenAI). Several keys for one provider are [load balanced](/docs/ai-gateway/routing/load-balancing#balance-across-your-own-keys) by weight.
2. Keys your workspace shares with the project.
3. Respan's keys, on your credits, offering by offering: the model's default provider first, then the rest from cheapest.

When any of your own keys can serve the model, step 3 is left out unless you opt in to [managed fallback](#managed-fallback). See [Provider order](/docs/ai-gateway/routing/provider-routing#provider-order).

## Fallback routes

A fallback route is a **Fallback** node: its entries are tried in order, and each entry expands into its own attempts as above. An entry can be a model name, a provider offering such as `azure/gpt-5.5`, or a [Respan alias](/docs/ai-gateway/routing/routes-and-aliases#respan-aliases).

```json
{
  "name": "smart",
  "mode": "fallback",
  "entries": [{"name": "azure/gpt-5.5"}, {"name": "gpt-5.5"}, {"name": "claude-sonnet-4-5-20250929"}]
}
```

- Weights are ignored in a fallback route.
- To pick by weight instead, use a split route, a **LoadBalance** node. See [Load balancing](/docs/ai-gateway/routing/load-balancing#nest-load-balancing-and-fallback).
- Change the chain with one `PATCH` to `/api/routes/{name}/`. No deploy is needed.

Creating, listing, and deleting routes is covered in [Routes and aliases](/docs/ai-gateway/routing/routes-and-aliases#manage-routes).

## Managed fallback

When one of your own provider keys can serve a model, the gateway uses only your keys for it. To let Respan's keys take over when your key is rate limited, send this header:

```bash
curl https://gateway.respan.ai/v1/chat/completions \
  -H "Authorization: Bearer $RESPAN_API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-respan-managed-fallback-enabled: true" \
  -d '{"model": "gpt-5.5", "messages": [{"role": "user", "content": "Hello!"}]}'
```

- Respan's keys are tried only after your key's attempt answers `429`. Any other error from your key doesn't move the request to Respan's keys.
- Calls on Respan's keys use your credits, and only for offerings credits can pay for.
- Without the header, a request on your keys never moves to Respan's keys.

## What you're billed

You pay for the attempt that answered, not the model you asked for: that offering, on the key that served it. A call on Respan's key is charged to your credits from the provider's reported usage. A call on your own key is billed by your provider, and is still logged and counted against your limits.

To see which attempt answered, open the request's `X-Respan-Log-Id` in **Gateway** › **Logs**. The log names the model, provider, and deployment that served it, and earlier failed attempts are rows in the same trace.

## Next steps

- [Routes and aliases](/docs/ai-gateway/routing/routes-and-aliases): create and manage routes.
- [Load balancing](/docs/ai-gateway/routing/load-balancing): split traffic by weight, and nest it with fallbacks.
- [Provider routing](/docs/ai-gateway/routing/provider-routing): how attempts are ordered.