---
title: "Routes and fallback"
subtitle: "Give a model name a list of providers, and choose how they are tried: in order, with retries, or raced."
---

A route is a model name that clients ask for, and the candidates it goes to. Without routes, a model goes to the provider whose model list has it (see [Providers and models](/docs/oss-gateway/providers#how-a-model-name-picks-a-provider)).

```toml
[routes.smart]
to = ["anthropic/claude-haiku-5-5", "openai/gpt-6-luna"]
```

A client that asks for `"model": "smart"` gets Claude Haiku, and GPT if that call fails. The client's code doesn't change when you change the route.

## Candidates

`to` is one candidate or a list. Each candidate is:

| Form | Example | Sends |
| --- | --- | --- |
| `provider/model` | `"openai/gpt-6-luna"` | that model to that provider |
| `provider/{model}` | `"anthropic/{model}"` | the name the client asked for, to that provider |
| `provider` | `"local"` | the name the client asked for, to that provider |

`provider` is a name from your `[providers.*]` tables.

## Fallback

Candidates are tried in order. If one fails, the next one gets the request.

```toml
[routes.smart]
to      = ["anthropic/claude-haiku-5-5", "openai/gpt-6-luna"]
tries   = 2          # per candidate, retries included
backoff = "250ms"    # first wait before a retry, doubled after each
```

| Key | Default | Meaning |
| --- | --- | --- |
| `fallback` | `true` | Try the next candidate when one fails. `false` uses the first candidate alone. |
| `tries` | `1` | Attempts per candidate, retries included. 1 to 10. |
| `backoff` | `"250ms"` | The first wait before a retry, doubled after each. |
| `max_backoff` | `"10s"` | The longest wait. A provider's `Retry-After` longer than this means no retry. |
| `until` | `"head"` | How far an answer is read before it counts as a success: `head` (a success status) or `end` (the whole answer, held until it ends cleanly). Nothing is retried after that point. |

A candidate is retried only for transient failures: the provider couldn't be reached, or it answered 408, 425, 429, 500, 502, 503 or 504. Any failure moves on to the next candidate. A candidate that failed its last 3 calls within 30 seconds is tried last.

## Hedge

`hedge` races the candidates instead of waiting for one to fail. Each candidate starts this long after the one before, if no answer has come yet. The first success wins.

```toml
[routes.fast]
to    = ["openai/gpt-6-luna", "anthropic/claude-haiku-5-5"]
hedge = "200ms"
```

Here GPT gets the request first. If it hasn't answered after 200 ms, Claude gets it too, and the client gets whichever answers first. Use it to cut tail latency; it can cost two calls for one request. `hedge` can't be combined with `fallback = false`, and `tries` and `backoff` don't apply to a hedged route.

## Pick

`pick` orders the candidates before they are tried:

```toml
[routes.cheap]
to    = ["openai/gpt-6-luna", "anthropic/claude-haiku-5-5"]
pick  = "cheapest"
needs = ["tools"]
```

| `pick` | Order |
| --- | --- |
| `first` | As listed. The default. |
| `cheapest` | By each model's price: input plus three times output. |
| `fastest` | By the gateway's own record of each candidate's time to a response. |
| `weighted` | At random, by each candidate's `weight`. |
| `random` | At random. |

`needs` drops candidates whose model lacks a capability, such as `tools` or `vision`, as its provider's model list says. A candidate the list says nothing about is kept.

Weights go on the candidates:

```toml
[routes.split]
to   = [{ target = "openai/gpt-6-luna", weight = 3 }, { target = "anthropic/claude-haiku-5-5", weight = 1 }]
pick = "weighted"
```

## Model aliases and globs

A route's name is any model name, so a route is also an alias. `smart`, `fast` and `cheap` above are names your clients use, and you can change what they point at in one place.

A route name with `*` is a glob that matches many model names. With `{model}`, the name the client asked for is passed through:

```toml
[routes."claude-*"]
to = "anthropic/{model}"

[routes."llama-*"]
to = "local"
```

An exact route name is matched first, then `provider/model` names, then globs in the order they appear in the file.

## Limit a route

`when` limits a route to some requests. Other requests skip it and are routed as if it didn't exist.

```toml
[routes.internal]
to   = "openai/gpt-6-luna"
when = { clients = ["ops"] }       # client keys, by name
```

`clients` takes [client key](/docs/oss-gateway/client-keys) names. `kinds` takes request kinds such as `chat` or `embedding`.

## Explain

To see where a model would go, without sending anything:

```bash
curl "http://localhost:8080/ui/api/routes/explain?model=cheap"
```

```json
{
  "by": { "type": "rule", "name": "cheap" },
  "pick": "cheapest",
  "mode": "fallback",
  "to": [
    { "provider": "openai", "model": "gpt-6-luna", "price": { "input": 0.25, "output": 2 }, "seen": null },
    { "provider": "anthropic", "model": "claude-haiku-5-5", "price": { "input": 1, "output": 5 }, "seen": null }
  ],
  "passed": []
}
```

`by.type` is `rule` (a route), `named` (`provider/model`), `listed` (a provider's model list) or `default`. `mode` is `single`, `fallback` or `hedge`. `passed` lists the candidates dropped and why, such as `lacks tools`. Add `&kind=embedding` for another kind of request, or `&client=NAME` for a route limited by `when`. A model nothing serves returns `{ "by": null }`.

This is part of the admin API: with client keys set, it needs an admin key. The web UI shows the same thing.

<Note>
A request the [tool loop](/docs/oss-gateway/tools#tool-loop) runs goes to the route's first candidate only, with no fallback, retries or hedge.
</Note>