> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://respan.ai/docs/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://respan.ai/docs/_mcp/server.

# Retries and fallback

When a call fails, Respan retries it on the same model after a short delay. If it still fails, Respan tries your fallback models one at a time and returns the first response that succeeds. Retries are on by default, with up to 3 attempts starting 0.2 seconds apart.

## Set them for your organization

#### Open the settings

Go to **Gateway** › **Routing config** and open the **Retries and fallback** tab.

#### Set retries

With **Retry the same request** on, pick how many times to try (**1 time** to **3 times**) and the first delay (**0.2 seconds** to **2 seconds**), then click **Save**.

![The Retries and fallback tab of Routing config, with Retry the same request turned on, set to retry 3 times starting after 0.5 seconds](/docs/_fern-img/706c0bbd7c3a5fab05b97bd124996838c5d03c8049d0ba0d2ac55269d972cace.webp)![The Retries and fallback tab of Routing config, with Retry the same request turned on, set to retry 3 times starting after 0.5 seconds](/docs/_fern-img/c40ea896ecc9f50415b47b92239c1d83eab1618ebb690e552aa9327ef4962188.webp)

#### Build a fallback chain

Turn on **Retry with a different model**, click **New model**, and pick a model. To add a model that isn't listed, type its name and click **Custom model: …**. Use the arrows to reorder the chain and the red **×** to remove a model, then click **Save**.

![The Fallback section with Retry with a different model turned on, and a chain that retries with claude-sonnet-4-5-20250929, then gemini-2.5-pro](/docs/_fern-img/f1b13f83c6ee92a72b98e732ab2c172c47bddbdf5db7023c8c04285c20a9500f.webp)![The Fallback section with Retry with a different model turned on, and a chain that retries with claude-sonnet-4-5-20250929, then gemini-2.5-pro](/docs/_fern-img/7ab9b90a868a1ead8918892d6f48e6292f334567e1fd98a524d1d63efc83b380.webp)

The retry count includes the first call, so **3 times** means at most three calls to each model. On `/api/chat/completions`, each wait doubles the one before, plus some jitter, up to 60 seconds. On `/api/responses`, every wait is the first delay. To call each model only once, turn off **Retry the same request** and confirm with **Disable**.

Only organization admins can change the fallback chain. It applies to every request that doesn't send its own `fallback_models`.

## Set them per request

`fallback_models` replaces your organization's chain for that request:

#### OpenAI (Python)

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.respan.ai/api/",
    api_key=os.environ["RESPAN_API_KEY"],
)

response = client.chat.completions.create(
    model="gpt-5.5",
    messages=[{"role": "user", "content": "Hello!"}],
    extra_body={"fallback_models": ["claude-sonnet-4-5-20250929", "gemini-2.5-pro"]},
)
```

#### OpenAI (TypeScript)

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.respan.ai/api/",
  apiKey: process.env.RESPAN_API_KEY,
});

const response = await client.chat.completions.create({
  model: "gpt-5.5",
  messages: [{ role: "user", content: "Hello!" }],
  // @ts-expect-error Respan parameter
  fallback_models: ["claude-sonnet-4-5-20250929", "gemini-2.5-pro"],
});
```

#### cURL

```bash
curl https://api.respan.ai/api/chat/completions \
  -H "Authorization: Bearer $RESPAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.5",
    "messages": [{"role": "user", "content": "Hello!"}],
    "fallback_models": ["claude-sonnet-4-5-20250929", "gemini-2.5-pro"]
  }'
```

On `/api/chat/completions`, your organization's retry settings always apply. On `POST /api/responses`, you can override them per request with `retry_params`, where `num_retries` is the total number of attempts, including the first, and `retry_after` is the wait in seconds before each retry. Send all three fields: one you leave out takes its default (`retry_enabled` `true`, `num_retries` `3`, `retry_after` `0.2`), not your organization's setting.

```json
{
  "model": "gpt-5.5",
  "input": "Summarize this incident.",
  "respan_params": {
    "retry_params": { "retry_enabled": true, "num_retries": 2, "retry_after": 0.5 }
  }
}
```

## What happens when a call fails

1. **Checks.** Respan validates the request, your API key, and your [limits](/docs/documentation/features/gateway/limits). A request rejected here never reaches a model, so it isn't retried and doesn't fall back.
2. **Retries.** Respan calls the model up to the retry count, waiting between attempts as described above.
3. **Fallback.** When the attempts run out, Respan moves to the next fallback model and retries that one the same way. It skips fallback models that can't take the request, such as ones with too small a context window or without image input.
4. **Result.** Respan returns the first success. If every model fails, it returns the last error.

Some errors end the request at once, with no retry and no fallback, because repeating the call can't fix them:

* Input larger than the model's context window.
* An invalid provider key of your own (`401`).
* A model that timed out while generating.

Other provider errors, such as rate limits, server errors, and dropped connections, are retried.

Each fallback model uses your provider key or Respan credits, chosen the same way as for the first model. Requests to `span-router` don't use your organization's chain. In Logs, a request that needed a retry or a fallback is marked as a warning, and its log shows which model served it.

## Troubleshooting

* **The request failed without trying a fallback.** A check rejected it, or it hit one of the errors above. The log's error message says which.
* **Requests are slow while a provider is struggling.** Each fallback model gets its own attempts, so the worst case grows with both. Lower the retry count or shorten the fallback chain.
* **You want to spread traffic rather than recover from failures.** Use [load balancing](/docs/documentation/features/gateway/load-balancing).