---
title: "Providers and models"
subtitle: "Add model providers with a key, point the gateway at any compatible endpoint, and see how a model name picks a provider."
---

A provider is where the gateway sends model calls. It is either a built-in vendor plus a key, or any endpoint that speaks a supported format.

```toml
[providers.openai]
vendor = "openai"
key = "env:OPENAI_API_KEY"

[providers.anthropic]
vendor = "anthropic"
key = "env:ANTHROPIC_API_KEY"
```

The table name (`openai`, `anthropic`) is the provider's name. You use it in routes and in `provider/model` names. Write keys as `env:NAME` to keep them out of the file; a literal string also works.

## Built-in vendors

A vendor already knows its base URL, how it takes a key, and which formats it speaks.

| `vendor` | Key variable | Formats |
| --- | --- | --- |
| `openai` | `OPENAI_API_KEY` | `openai-chat`, `openai-responses`, `openai-realtime`, `responses-ws`, `openai-images`, `openai-transcriptions`, `openai-speech`, `openai-embeddings` |
| `anthropic` | `ANTHROPIC_API_KEY` | `anthropic-messages` |
| `gemini` | `GEMINI_API_KEY` or `GOOGLE_API_KEY` | `gemini-generate`, `openai-chat`, `gemini-live`, `gemini-images`, `gemini-veo`, `openai-embeddings` |
| `respan` | `RESPAN_API_KEY` | `openai-chat`, `openai-responses`, `anthropic-messages`, `gemini-generate`, `decisions` |
| `typesafe` | `TYPESAFE_API_KEY` | `decisions` |
| `groq` | `GROQ_API_KEY` | `openai-chat`, `openai-transcriptions`, `openai-speech` |
| `deepseek` | `DEEPSEEK_API_KEY` | `openai-chat` |
| `openrouter` | `OPENROUTER_API_KEY` | `openai-chat` |
| `xai` | `XAI_API_KEY` | `openai-chat`, `openai-images` |
| `mistral` | `MISTRAL_API_KEY` | `openai-chat`, `openai-embeddings`, `openai-transcriptions`, `openai-speech`, `mistral-realtime` |
| `together` | `TOGETHER_API_KEY` | `openai-chat`, `openai-images`, `openai-embeddings` |
| `fireworks` | `FIREWORKS_API_KEY` | `openai-chat`, `openai-embeddings` |
| `cerebras` | `CEREBRAS_API_KEY` | `openai-chat` |
| `perplexity` | `PERPLEXITY_API_KEY` | `openai-responses` |
| `azure` | `AZURE_OPENAI_API_KEY` | `openai-chat` |
| `cloudflare` | `CLOUDFLARE_API_TOKEN` | `openai-chat`, `openai-embeddings` |
| `ollama` | none | `openai-chat`, `openai-embeddings` |
| `vllm` | none | `openai-chat` |

OpenAI, Anthropic, Gemini, Respan and TypeSafe are verified end to end against the live API. The others are maintained but not yet verified that way. `azure` and `vllm` need a `base_url`. `cloudflare` needs `CLOUDFLARE_ACCOUNT_ID` or a `base_url`.

## No config file

With no `gateway.toml`, every vendor whose key is set in the environment becomes a provider, named after the vendor:

```bash
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
respan-gateway
```

Ollama is added if `OLLAMA_HOST` is set. `RESPAN_<VENDOR>_BASE_URL` moves a vendor's base URL, for example `RESPAN_OPENAI_BASE_URL=https://my-proxy.example/v1`. Both only apply without a config file. With one, only the file's `base_url` counts. `respan-gateway init` writes a config file from what it finds in the environment.

## Base URLs

`base_url` replaces a vendor's URL. It is the URL the vendor's own SDK takes: OpenAI-compatible ones include the version (`https://api.openai.com/v1`), Anthropic's does not (`https://api.anthropic.com`).

```toml
[providers.openai]
vendor = "openai"
key = "env:OPENAI_API_KEY"
base_url = "https://my-proxy.example/v1"
```

A vendor whose formats sit at different URLs, such as Gemini, moves them with a table of `formats`, one base URL per format:

```toml
[providers.gemini]
vendor = "gemini"
key = "env:GEMINI_API_KEY"
formats = { gemini-generate = "https://gemini.internal/v1beta", openai-chat = "https://gemini.internal/v1beta/openai" }
```

Only the formats in the table are served. A single `base_url` on such a vendor is refused, since it can't say where each format is.

## OpenAI-compatible endpoints

Anything that speaks a supported format is a provider. Give it `formats` and a `base_url` instead of a `vendor`:

```toml
[providers.local]
formats = ["openai-chat"]
base_url = "http://localhost:8000/v1"
key = "env:LOCAL_KEY"          # optional
```

`formats` is one format name or a list. The chat formats are `openai-chat`, `openai-responses`, `anthropic-messages` and `gemini-generate`; see [API formats](/docs/oss-gateway/api-formats) for the rest. `key` is sent the way the format expects. To send it in a different header, use `auth` instead:

```toml
[providers.internal]
formats = ["openai-chat"]
base_url = "https://llm.internal/v1"
auth = { type = "header", name = "x-internal-key", key = "env:INTERNAL_KEY" }
# or: auth = { type = "bearer", key = "env:INTERNAL_KEY" }
```

A provider with no `key`, no `auth` and no vendor key passes on the client's own credentials.

## Local models

<Tabs>
<Tab title="Ollama">
```toml
[providers.ollama]
vendor = "ollama"              # http://localhost:11434/v1 by default
proxy = false                  # skip [network]'s proxy, if you set one
```

To reach Ollama on another machine, set `base_url = "http://gpu-box:11434/v1"`. Without a config file, set `OLLAMA_HOST=gpu-box:11434`.
</Tab>
<Tab title="vLLM">
```toml
[providers.vllm]
vendor = "vllm"
base_url = "http://localhost:8000/v1"
# key = "env:VLLM_API_KEY"     # only if the server was started with one
```

`vllm` has no default URL, so `base_url` is required. It handles vLLM's reasoning fields; a plain `formats = ["openai-chat"]` provider also works.
</Tab>
</Tabs>

## How a model name picks a provider

The `model` in each request decides where it goes. The gateway tries, in order:

1. A [route](/docs/oss-gateway/routes) with exactly that name, such as `smart`.
2. `provider/model`: the provider named before the first `/` gets the rest as the model. `anthropic/claude-haiku-5-5` goes to the provider named `anthropic`; `groq/openai/gpt-oss-20b` sends `openai/gpt-oss-20b` to `groq`.
3. The first glob route that matches, such as `[routes."claude-*"]`.
4. The provider whose model list has it. The gateway reads every provider's model list at start and every 10 minutes, so `claude-haiku-5-5` goes to Anthropic and `gpt-6-luna` to OpenAI with no routes at all.
5. Otherwise, the default provider for that kind of request: the first provider in the file that serves it.

The provider can speak a different format from the client. An Anthropic SDK asking for a GPT model is translated to OpenAI's format and back.

To set the default yourself, or to turn off model lists:

```toml
[plugins.routing]
default = "openai"
listings = false     # route by routes and provider/model names only
```

`GET /ui/api/routes/explain?model=M` shows where a model would go and why. See [Routes and fallback](/docs/oss-gateway/routes#explain).

## List models

`GET /v1/models` merges every provider's model list into one OpenAI-shaped list. Each model is named the way a client should ask for it: as is, if routing sends that name to its provider, otherwise as `provider/model`.

```bash
curl http://localhost:8080/v1/models
respan-gateway models             # the same list from the command line
```

Each entry has `id`, `owned_by` (the provider) and `kind` (`chat`, `embedding`, `image`, `transcription`, `speech`, `video` or `realtime`). `GET /v1beta/models` returns the same list in Gemini's shape, and `/v1/models` with an `anthropic-version` header returns Anthropic's.

## Respan as a provider

A Respan API key reaches the models in Respan's gateway catalog with one key:

```toml
[providers.respan]
vendor = "respan"
key = "env:RESPAN_API_KEY"
```

Respan speaks OpenAI Chat, OpenAI Responses, Anthropic Messages and Gemini's `generateContent`. The gateway sends each model in its maker's own format (Claude models as Anthropic Messages, Gemini models as `generateContent`) and anything else as Chat Completions. Its model list is Respan's public catalog.

To trace requests in Respan rather than route through it, see the `respan` plugin in [Plugins](/docs/oss-gateway/plugins).