Retries
Retry a failed call on the same model before falling back.
When a call fails with an error that can pass, the gateway retries it on the same provider and key. When the tries run out, the request moves to the next provider, key, or fallback model. Retries are on by default.
Set retries
Open the Routing tab of the Gateway page . Under Retries, turn the switch on and pick how many times to try each model and the first wait. The setting applies to every request in the organization.
| Setting | Default | Detail |
|---|---|---|
| Switch | On | Off tries each model once. Fallbacks still run. |
| Try each model up to | 3 times | Tries per model, the first one included. 1 to 3. |
| With waits from | 0.2 seconds | The first wait, 0.2 to 2 seconds. Each wait doubles the one before, up to 60 seconds. |
Retries apply to chat requests. Images, audio, embeddings, and other requests are sent once per attempt.
What is retried
| Failure | Retried |
|---|---|
408, 425, 429, 500, 502, 503, 504 | Yes |
| Connection failed before any response | Yes |
Other 4xx, such as 400 or 404 | No, moves to the next attempt |
Input larger than the context window, 401 from your own key | No, ends the request |
| Refused by your guardrails or limits | No, ends the request |
Each model in a fallback gets its own tries, so the worst-case time grows with both. Lower the count for latency-sensitive traffic. A provider call times out after 600 seconds, and a stream that has started isn’t retried.
Logs
Every try is a row in the request’s trace. The Retries and fallback chart on the Gateway page’s Overview counts requests that were retried or sent to a fallback model.