--- title: "Config file" subtitle: "Reference for gateway.toml: where it is read from, every section, environment variables and reloading." --- The gateway runs without a config file: every vendor whose key is in the environment becomes a provider. A config file, `gateway.toml`, adds client keys, providers, routes, prices, tool servers and plugins. `respan-gateway init` writes a commented starter. ## Where the config comes from The gateway uses the first of these that exists: 1. The file named by `--config` (or `-c`), or by the `RESPAN_CONFIG` environment variable. 2. `gateway.toml` in the current directory. 3. The environment: a provider for each vendor whose key is set (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, ...), and a tool server for each tool vendor whose key is set (such as `LINKUP_API_KEY`). If there is no key at all, the gateway exits and says how to start. Before reading anything, the gateway loads the nearest `.env` file in the current directory or one of its parents. A variable already set in the environment wins over `.env`. ## Secrets: `env:NAME` Write a secret as `env:NAME` to read it from the environment (or `.env`) instead of keeping it in the file: ```toml [providers.openai] vendor = "openai" key = "env:OPENAI_API_KEY" ``` This works for secret values: a provider's `key` and `auth` key, `client_keys`, a tool server's `key`, `headers`, `env` and `auth` values, proxy URLs, and the Respan plugin's `key`. It is not general substitution: other strings, such as `base_url`, are read as written. If the variable is unset or empty, the config is refused with an error that names it. ## Values - **Durations** are a number of seconds, or a string such as `"500ms"`, `"30s"`, `"2m"` or `"1h"`. - **Sizes** are a number of bytes, or a string such as `"32MiB"`, `"512 KB"` or `"1GiB"`. - **Unknown keys are refused.** A typo is an error with its line and column, not a silently ignored setting. Errors don't echo the value, in case it is a secret. ## Top-level settings These go at the top of the file, before any `[table]`. | Key | What it does | Default | | --- | --- | --- | | `listen` | Where to listen: `"host:port"`, `":port"` or a port (on `127.0.0.1`). Beyond loopback, `client_keys` are required. | `"127.0.0.1:8080"` | | `client_keys` | Keys clients must send. See [Client keys](/docs/oss-gateway/client-keys). | None: loopback only | | `log` | How each finished request is printed to stderr: `text`, `json` or `off`. | `text` | | `ui` | Whether the web UI and admin API are served under `/ui/`. | `true` | | `formats` | The client formats served, by name. See [API formats](/docs/oss-gateway/api-formats#which-formats-are-served). | Chosen from the providers | | `body_limit` | The largest request body, and the most of a provider's answer held at once. Past it, a request fails `413` or an answer `502`. | `32MiB` | | `ws_message_limit` | The largest WebSocket message in a session. Past it, the session closes with code 1009. | `16MiB` | | `shutdown_grace` | How long open responses and end hooks get after a stop signal. | `30s` | ## Sections | Section | What it holds | Details | | --- | --- | --- | | `[providers.]` | A provider: a `vendor` and its `key`, or any compatible endpoint with `formats` and `base_url`. Order matters: the first provider of a kind is its default. | [Providers and models](/docs/oss-gateway/providers) | | `[models."/"]` | A model card laid over the provider's list: `pricing`, `display_name`, `context_window`, `capabilities`, ... | [Providers and models](/docs/oss-gateway/providers) | | `[routes.]` | A model name or glob clients ask for, and where it goes: `to`, `pick`, `needs`, `when`, `fallback`, `tries`, `backoff`, `max_backoff`, `hedge`, `until`. | [Routes and fallback](/docs/oss-gateway/routes) | | `[tools.]` | A tool server: an MCP server at a `url` or run as a `command`, or a tool `vendor`. Served at `/mcp/` and through `/mcp`. | [Tools and MCP](/docs/oss-gateway/tools) | | `[plugins.routing]` | The `default` provider, and whether to read model `listings`. | [Plugins](/docs/oss-gateway/plugins#routing) | | `[plugins.tool_loop]` | Run tool servers' tools inside chat requests. Off unless set. | [Plugins](/docs/oss-gateway/plugins#tool-loop) | | `[plugins.respan]` | Send each request to Respan as a trace. Off unless set. | [Plugins](/docs/oss-gateway/plugins#respan) | | `[plugins.responses]` | Conversation state for `previous_response_id`. On unless `enabled = false`. | [Plugins](/docs/oss-gateway/plugins#responses) | | `[plugins.request_log]` | `history`: how many finished requests the web UI keeps. | [Plugins](/docs/oss-gateway/plugins#request-log) | | `[cors]` | Browser origins allowed to call the gateway. | [below](#cors) | | `[network]` | The outbound proxy and extra trusted certificates. | [below](#network) | ### cors ```toml [cors] origins = ["https://app.example.com"] ``` `origins` are scheme, host and port only, or `"*"` for any origin, which needs `client_keys`. Preflight requests are answered without a key; other requests from those pages still need one. Without `[cors]`, pages on other origins can't call the gateway. ### network Every outbound connection, to providers, tool servers, OAuth endpoints and Respan, goes through one proxy or none. By default that is the environment's `HTTPS_PROXY`, `HTTP_PROXY` and `ALL_PROXY`, minus `NO_PROXY`. ```toml [network] proxy = "env:CORP_PROXY" # http://user:password@proxy.corp:3128, or false no_proxy = ["localhost", ".corp.internal", "10.0.0.0/8"] ca = "corp-root.pem" # trusted besides the system's ``` | Key | What it does | | --- | --- | | `proxy` | A proxy URL (`http://` or `https://`), literal or `env:NAME`, or `false` for none even if the environment names one. A provider or tool server can set its own `proxy`, which replaces this one. | | `no_proxy` | Hosts reached directly: a host, a domain from its leading `.`, an IP address or CIDR range, or `*`. | | `ca` | A PEM file of certificate authorities, relative to the config file, such as a TLS-inspecting proxy's. | ## A complete example ```toml listen = "0.0.0.0:8080" log = "json" client_keys = [ { name = "ops", key = "env:OPS_KEY", admin = true }, # can use the admin API { name = "app", key = "env:APP_KEY" }, { name = "bot", key = "env:BOT_KEY", tools = ["search__*"] }, ] [cors] origins = ["https://app.example.com"] [providers.openai] vendor = "openai" key = "env:OPENAI_API_KEY" [providers.anthropic] vendor = "anthropic" key = "env:ANTHROPIC_API_KEY" [providers.local] # any OpenAI-compatible endpoint formats = ["openai-chat"] base_url = "http://gpu-box:8000/v1" proxy = false [models."local/qwen3-8b"] pricing = { input = 0.05, output = 0.2 } [routes.smart] # clients ask for "model": "smart" to = ["anthropic/claude-haiku-5-5", "openai/gpt-6-luna"] tries = 2 backoff = "250ms" [routes.fast] to = ["openai/gpt-6-luna", "anthropic/claude-haiku-5-5"] hedge = "200ms" [routes."llama-*"] to = "local" [tools.search] vendor = "linkup" key = "env:LINKUP_API_KEY" [tools.files] command = "npx" args = ["-y", "@modelcontextprotocol/server-filesystem", "/data"] [plugins.tool_loop] servers = ["search"] allow = ["search__*"] max_steps = 8 [plugins.responses] keep = "1h" [plugins.request_log] history = 1000 [plugins.respan] key = "env:RESPAN_API_KEY" content = false ``` Check it before you start, and see what it serves: ```bash respan-gateway check respan-gateway check --online # also call each provider with its key ``` The repository's [`gateway.example.toml`](https://github.com/respanai/respan-gateway/blob/main/gateway.example.toml) is another commented example. ## Environment variables Each can also be set in `.env`. A flag wins over its variable. `RESPAN_LISTEN` and `RESPAN_LOG` win over the config file's `listen` and `log`. | Variable | What it does | | --- | --- | | `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, `GROQ_API_KEY`, ... | Provider keys. With no config file, each one makes its vendor a provider. A `vendor` in the config with no `key` reads its variable too. | | `RESPAN_CONFIG` | The config file, as `--config`. | | `RESPAN_LISTEN` | Where to listen, as `--listen`: `127.0.0.1:8080`, `localhost:8080` or `8080`. | | `PORT` | A port to listen on, on every address (`0.0.0.0`), for hosts that assign one. Used only if neither `--listen`, `RESPAN_LISTEN` nor the config's `listen` is set. | | `RESPAN_CLIENT_KEYS` | Client keys, comma separated, added to the config's `client_keys`. | | `RESPAN_LOG` | `text`, `json` or `off`, as `--log`. | | `RESPAN_COLOR` | `auto`, `always` or `never`, as `--color`. | | `NO_COLOR` | Turns off automatic color when set and not empty. | | `RESPAN__BASE_URL` | Moves a vendor's base URL when there is no config file, such as `RESPAN_OPENAI_BASE_URL`. With a config file, only its `base_url` counts. | | `OLLAMA_HOST` | Where Ollama is. With no config file, setting it makes Ollama a provider. | | `HTTPS_PROXY`, `HTTP_PROXY`, `ALL_PROXY`, `NO_PROXY` | The outbound proxy, unless `[network]` sets one. | ## Reloading The gateway reloads its config without a restart in three ways: - Start it with `--watch` to reload when the file changes. It checks every 2 seconds. - Send the process `SIGHUP` (Unix only). - `POST /ui/api/reload`, or **Reload** on the web UI's Config page. With client keys set, this needs a key with admin access. ```bash kill -HUP "$(pgrep respan-gateway)" curl -X POST http://localhost:8080/ui/api/reload -H "Authorization: Bearer $OPS_KEY" ``` Requests already in flight finish on the old config. An invalid config is refused, and the gateway keeps serving the old one. A changed `listen` address needs a restart. Overrides from flags and the environment, such as `--listen` and `RESPAN_CLIENT_KEYS`, apply again on every reload. Tools turned off in the web UI stay off across a reload, but not across a restart.