---
title: "Plugins"
subtitle: "The plugins built into the gateway, what each one does, and how to configure it."
---

Everything the gateway does beyond translating formats is a plugin: routing, the tool loop, pricing, tracing. The built-in plugins use the same public Rust API you can use to [write your own](/docs/oss-gateway/write-a-plugin).

Some plugins are always on. The others are off until you add their section to `gateway.toml`. Each is configured under `[plugins.<name>]`, and the gateway runs them in a fixed order, whatever order the file lists them in.

| Plugin | Crate | On by default | Config |
| --- | --- | --- | --- |
| [Routing](#routing) | `respan-gateway-plugin-routing` | Always on | `[routes.<model>]`, `[plugins.routing]` |
| [Tool loop](#tool-loop) | `respan-gateway-plugin-tool-loop` | No | `[plugins.tool_loop]` |
| [MCP router](#mcp-router) | `respan-gateway-plugin-mcp-router` | Always on | `[tools.<name>]` |
| [Pricing](#pricing) | `respan-gateway-plugin-pricing` | Always on | `[models."<provider>/<model>"]` |
| [Respan tracing](#respan) | `respan-gateway-plugin-respan` | No | `[plugins.respan]` |
| [Conversation state](#responses) | `respan-gateway-plugin-responses` | Yes | `[plugins.responses]` |
| [Client credentials](#credentials) | `respan-gateway-plugin-credentials` | Always on | none |
| [Relay](#relay) | `respan-gateway-plugin-relay` | Only in `proxy` | none |
| [Request log](#request-log) | part of the binary | Always on | `[plugins.request_log]` |
| [Video jobs](#video-jobs) | `respan-gateway-media` | Always on | none |

A misspelled key, or a section the gateway doesn't know, is refused when the config loads. `respan-gateway check` shows it before you start.

## Routing

Picks the provider for each request from its model name, then sends it to the route's candidates: in turn with retries and fallback, or raced with `hedge`. Routes are top-level `[routes.<model>]` tables; [Routes and fallback](/docs/oss-gateway/routes) covers them.

`[plugins.routing]` holds two settings:

```toml
[plugins.routing]
default = "openai"     # where a request goes when nothing else routes it
listings = true        # read each provider's model list at start and every 10 minutes
```

| Key | What it does | Default |
| --- | --- | --- |
| `default` | The provider for requests no route, `provider/model` name or model list claims. | The first provider that serves the request's kind |
| `listings` | Whether to read each provider's model list. With `false`, a model goes to a provider only by a route, a `provider/model` name or the default, and routes can't pick by price or capability. | `true` |

Routing can't be turned off.

## Tool loop

Offers the tools of your tool servers to the model and runs the calls itself, then returns one ordinary reply. It works for every chat format and provider, and in realtime sessions. A client can skip it for one request with the header `x-respan-tool-loop: off`.

It is off until you add the section:

```toml
[tools.search]
vendor = "linkup"
key = "env:LINKUP_API_KEY"

[plugins.tool_loop]
servers = ["search"]
allow = ["search__*"]
max_steps = 8
```

| Key | What it does | Default |
| --- | --- | --- |
| `servers` | The `[tools.<name>]` servers whose tools are offered. A server with `auth = "forward"` can't be used. | Every server `/mcp` fronts |
| `allow` | Tools to offer, as `server__tool` globs. | All |
| `deny` | Tools never offered. Wins over `allow`. | None |
| `models` | Model name prefixes the loop applies to, such as `["gpt-", "claude-"]`. | Every chat model |
| `max_steps` | Model calls per request, 1 to 64. The last step forbids tools. | `8` |
| `timeout` | Limit for one request's whole loop. Past it, the request fails with `504 tool_loop_timeout`. | `5m` |
| `tool_timeout` | Limit for one tool call. | `60s` |
| `tool_list_ttl` | How long a server's tool list is cached. | `5m` |
| `parallel` | Tool calls run at once in one step, 1 to 64. | `4` |
| `max_result` | Bytes of one tool's answer the model sees. | `64KiB` |
| `max_results` | Bytes of all tool answers the model sees in one request. | `256KiB` |

A request the tool loop runs goes to its route's first candidate only, with no fallback, retry or hedge.

<Warning>
A tool's description and output are untrusted input to the model, and every client key can make the model call every tool you offer. Offer only tools from servers you trust, and put destructive ones in `deny`.
</Warning>

To turn it off, remove `[plugins.tool_loop]`.

## MCP router

Serves your tool servers to MCP clients. `/mcp/{name}` reaches one server as it is. `/mcp` fronts every server as one, with tools and prompts named `server__name` and resources under `mcp://server/...`. Servers with `auth = "forward"` are left out of `/mcp`.

It has no section of its own. It serves whatever `[tools.<name>]` tables the config has; with none, there is no `/mcp`. See [Tools and MCP](/docs/oss-gateway/tools).

## Pricing

Prices every call to a provider as its usage is reported, at the price of the model that call sent. Retries, fallbacks, a race's loser and each tool-loop step are separate calls, so a request's cost is the sum of its calls. The request log and the web UI show it, and `pick = "cheapest"` routes use the same prices.

Prices come from each provider's model list. Declare or override one with a model card, in US dollars per million tokens:

```toml
[models."openai/my-finetune"]
pricing = { input = 1.0, output = 2.0, cached_input = 0.5 }
```

`pricing` also takes `cache_write` and `per_second` (for audio). Pricing is always on; it only measures and never blocks a request.

## Respan tracing

Sends each request to [Respan](https://respan.ai) as a trace over OTLP. A request is a root span, and each call it made is a child span: a `chat` span for a model call, with model, usage, messages and reply, and a `tool` span for each tool the tool loop ran. Spans are sent in batches off the request path. If Respan is slow or down, at most 4096 traces wait and the rest are dropped, so requests never slow down.

It is off until you add the section:

```toml
[plugins.respan]
key = "env:RESPAN_API_KEY"
content = false              # metadata only: no messages or replies
```

| Key | What it does | Default |
| --- | --- | --- |
| `key` | Your Respan API key, literal or `env:NAME`. | Required |
| `content` | Whether messages, replies and tool arguments are sent. | `true` |
| `service` | The service name on the traces. | `respan-gateway` |
| `url` | The OTLP trace endpoint. | `https://api.respan.ai/api/v2/traces` |

Only your clients' requests are traced, not the gateway's own calls to tool servers. To turn it off, remove `[plugins.respan]`.

## Conversation state

Keeps conversation state that a client or provider can't keep:

- A Responses API client sent to a provider that keeps no state (Anthropic, Gemini, Chat Completions). Its `previous_response_id` keeps working: the gateway gives each response an id and sends the whole conversation on the next turn. A request with `store: false` is not kept.
- A Chat Completions client whose call goes to OpenAI's Responses API. The reasoning behind a turn's tool calls is kept and put back on the next turn, so a reasoning model doesn't lose it.

It is on by default:

```toml
[plugins.responses]
keep = "1h"          # how long after it was last read
# enabled = false    # turn it off
```

| Key | What it does | Default |
| --- | --- | --- |
| `enabled` | Whether state is kept. | `true` |
| `keep` | How long a turn is kept after it was last read. | `1h` |

State lives in one gateway's memory, up to about 256 MiB. It doesn't survive a restart and isn't shared between gateways, so a client whose requests reach several gateways must reach the one that kept its conversation.

## Client credentials

Sends a client's own `Authorization`, `x-api-key` or `x-goog-api-key` to providers that have no key of their own, and to tool servers with `auth = "forward"`. It only does this when the client speaks the provider's own format: a key never crosses a translation.

It has no section. A provider gets the client's credentials when it has no key of its own: no `key` or `auth`, and either no `vendor` or a vendor that takes no key, such as `ollama` or `vllm`. A provider with a key never does. Client keys from `client_keys` are removed before this, so a provider never sees them.

## Relay

Used only by [`respan-gateway proxy`](/docs/oss-gateway/cli#proxy). It passes the vendor's response headers and error bodies back to the client as they came, so a tool such as Claude Code sees the vendor's own answers.

## Request log

Records every finished request for the web UI, the admin API and the log lines on stderr: client key, model, provider, status, tokens, cost and latency. It is always on, and holds requests in memory.

```toml
[plugins.request_log]
history = 1000       # finished requests kept for the web UI
```

| Key | What it does | Default |
| --- | --- | --- |
| `history` | How many finished requests are kept in memory. Must be above 0. | `1000` |

The top-level `log` setting (`text`, `json` or `off`) controls the stderr lines. See [Config file](/docs/oss-gateway/config#server).

## Video jobs

Sends a video job's follow-ups (status, content, delete) to the provider that started the job, by the job's id. It is always on and has no settings.

## Retired settings

These older keys are refused with a message that says what replaced them:

| Old key | Use instead |
| --- | --- |
| `[plugins.fallback]` | A route's `to`, `fallback` and `tries` |
| `[plugins.hedge]` | A route's `hedge` |
| `[plugins.routing] models` | Top-level `[routes.<model>]` |
| `[mcp.<name>]` | `[tools.<name>]` |
| `prices` | `[models."<provider>/<model>"]` with `pricing` |