---
title: "Routing & passthrough"
subtitle: "Pick an endpoint, and see how Respan chooses the model, provider, and key for each request."
---
Respan has two kinds of endpoints. The unified endpoint takes OpenAI-format requests for any model, so you switch providers by changing `model`. Passthrough endpoints take a provider's own format, so its SDK or CLI works unchanged.
| Endpoint | Base URL | Use it when |
|---|---|---|
| Unified | `https://api.respan.ai/api/` | You want one SDK for every provider, with fallbacks, retries, and load balancing. |
| Passthrough | `https://api.respan.ai/api/{provider}/...` | You already use a provider's SDK or CLI, such as Claude Code, or need a field only its native API has. |
Every request through either one is logged, and you can mix them in one project.
## Unified endpoint
| Path | Format |
|---|---|
| `POST /api/chat/completions` | OpenAI Chat Completions |
| `POST /api/responses` | OpenAI Responses. See [Responses API](#responses-api). |
| `POST /api/embeddings` | OpenAI Embeddings |
To switch provider, change the model slug. Nothing else in your code changes:
```python
import os
from openai import OpenAI
client = OpenAI(base_url="https://api.respan.ai/api/", api_key=os.environ["RESPAN_API_KEY"])
messages = [{"role": "user", "content": "Hello!"}]
client.chat.completions.create(model="gpt-5.5", messages=messages)
client.chat.completions.create(model="claude-sonnet-4-5-20250929", messages=messages)
client.chat.completions.create(model="gemini-2.5-pro", messages=messages)
```
### How a model is chosen
For each request, Respan:
1. **Picks the model.** It uses `model`. If the request has `load_balance_group` or `load_balance_models`, Respan picks one of those models by weight instead. A group wins over `load_balance_models`. Set `model` to `span-router` to let [span-router](/docs/documentation/features/gateway/span-router/concept) pick the model for each task.
2. **Picks the provider.** The model slug decides it: `gpt-5.5` goes to OpenAI and `azure/gpt-4o` goes to Azure OpenAI. To send a slug to a different provider, [pin a provider](/docs/documentation/features/gateway/models-catalog#pin-a-provider) with the `X-Respan-Route-Provider` header.
3. **Picks the key.** If you added a key for that provider under **Gateway** › **API keys** › **Bring your own key**, Respan uses it, split across your keys by [load balance weight](/docs/documentation/features/gateway/load-balancing). Otherwise the call runs on Respan credits.
4. **Handles failures.** It retries the same model, then tries your fallback models in order. See [Retries and fallback](/docs/documentation/features/gateway/retries).
The model that served the request is on its log.
## Provider passthrough
A passthrough endpoint relays the provider's native request and response. In every case, the `Authorization` header or API key carries your Respan API key, not the provider's.
```python
import os
import anthropic
client = anthropic.Anthropic(
base_url="https://api.respan.ai/api/anthropic/",
api_key=os.environ["RESPAN_API_KEY"],
)
message = client.messages.create(
model="claude-sonnet-4-5-20250929",
max_tokens=1024,
messages=[{"role": "user", "content": "Hello!"}],
)
```
For Claude Code, set these instead. See [Claude Code](/docs/integrations/gateway/claude-code).
```bash
export ANTHROPIC_BASE_URL="https://api.respan.ai/api/anthropic/"
export ANTHROPIC_AUTH_TOKEN="$RESPAN_API_KEY"
```
```python
import os
from google import genai
client = genai.Client(
api_key=os.environ["RESPAN_API_KEY"],
http_options={"base_url": "https://api.respan.ai/api/google/gemini"},
)
```
For the Gemini CLI, set `GEMINI_API_KEY` to your Respan API key and `GOOGLE_GEMINI_BASE_URL` to `https://api.respan.ai/api/google/gemini`. For Vertex AI, use `https://api.respan.ai/api/google/vertexai/`.
```python
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.respan.ai/api/openai/v1", # or /api/xai/v1, /api/groq/v1, ...
api_key=os.environ["RESPAN_API_KEY"],
)
client.chat.completions.create(model="gpt-5.5", messages=[{"role": "user", "content": "Hello!"}])
```
To keep a passthrough's request format but send the call to another provider, for example Anthropic-format requests to Vertex AI or Bedrock, [pin a provider](/docs/documentation/features/gateway/models-catalog#pin-a-provider).
### Chat completions passthrough
`POST /api/{provider}/v1/chat/completions` relays an OpenAI-shaped provider's chat completions payload and returns that provider's response byte for byte. Respan authenticates the request, applies your [limits](/docs/documentation/features/gateway/limits), and logs it. It doesn't translate the request or recompute usage.
`{provider}` is the provider ID, such as `openai`, `azure_openai`, `xai`, `groq`, `mistral`, `deepseek`, `togetherai`, `fireworks`, or `perplexity`. A provider without an OpenAI-shaped API returns `404`. Anthropic and Google have their own endpoints, shown above, and so does [OpenRouter](#openrouter).
A provider with no fixed public endpoint, such as Azure OpenAI, needs its endpoint URL saved with its key under **Bring your own key**. Without it, the request returns `400`.
| | Behavior |
|---|---|
| Request | Forwarded unchanged, including unknown and provider-only fields. Respan removes `respan_params` and merges a literal `extra_body` object into the body. |
| Response and usage | The provider's own bytes and usage numbers. Streams are relayed frame by frame. |
| Errors | The provider's status and body. Respan's own: `400` no known endpoint for the provider, `401` no usable key for the provider (code `activation_required` when you have no credits and no provider key for the model), `402` the request's customer is over their budget, `404` provider not available here, `429` your Respan key's rate limit or an API key limit's block rule. |
| Limits | No size, context, or truncation limit below the provider's own. Requests time out after 600 seconds. |
| Headers sent upstream | `accept`, `accept-language`, `content-type`, `user-agent`, and headers that start with `x-` plus the provider ID, with `_` written as `-`, such as `x-azure-openai-`. Nothing else. |
| Headers returned | `X-Respan-Log-Id`, `X-Respan-Upstream-Request-Id`, and the provider's `x-ratelimit-*` and `retry-after`. |
### OpenRouter
`POST /api/openrouter/v1/chat/completions` relays your request to OpenRouter and returns OpenRouter's own body, status, and headers. It's always bring-your-own-key: `Authorization` carries your Respan API key and `x-openrouter-api-key` carries your OpenRouter key. A missing OpenRouter key returns `400`.
```python
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.respan.ai/api/openrouter/v1",
api_key=os.environ["RESPAN_API_KEY"],
default_headers={"x-openrouter-api-key": os.environ["OPENROUTER_API_KEY"]},
)
response = client.chat.completions.create(
model="anthropic/claude-sonnet-4-5",
messages=[{"role": "user", "content": "Hello!"}],
extra_body={"provider": {"order": ["anthropic"]}, "reasoning": {"effort": "high"}},
)
```
Use OpenRouter's model slug. An `openrouter/` prefix is also accepted and removed. The `http-referer`, `x-title`, and `x-openrouter-*` headers are forwarded. See [Create an OpenRouter chat completion](/docs/apis/gateway/create-openrouter-chat-completion) for the full reference.
The unified endpoint also reaches OpenRouter, through `openrouter/`-prefixed models. See [OpenRouter](/docs/integrations/gateway/openrouter). Pick one:
| | Unified endpoint | OpenRouter passthrough |
|---|---|---|
| OpenRouter request fields | Only object-shaped `reasoning` and `provider` | All of them, including fields OpenRouter adds later |
| Response | Respan's OpenAI-format response | OpenRouter's own, including `provider`, `native_finish_reason`, and `reasoning_details` |
| Upstream `402`, `404`, `429` | Returned as a gateway error | Returned with the same status |
| Key | Your OpenRouter key, or Respan credits | Your OpenRouter key only |
## Responses API
`POST https://api.respan.ai/api/responses` takes OpenAI Responses-format requests. The `X-Respan-Route-Provider` header picks the provider:
| Route | Header | Request needs | Per-request key in `respan_params.credential_override` |
|---|---|---|---|
| OpenAI | None, or `openai` | `model` and `input` | Keyed by the exact model: `{"gpt-5.5": {"api_key": "..."}}` |
| Azure OpenAI | `azure` | `model` as `azure/`, and `input` | Keyed by the exact model, with `api_key`, `api_base`, and `api_version` |
| Perplexity Agent API | `perplexity` | `input`, plus `model` or `models` | Keyed by an empty string: `{"": {"api_key": "..."}}` |
Only the header turns on the Perplexity route. `route_provider_override: perplexity` in the body, in `respan_params`, or in a saved prompt is ignored. The `perplexity` value works only on this endpoint.
The per-request key is optional. Without it, Respan uses the provider key you added under **Bring your own key**, or Respan credits where the route supports them.
```bash
curl https://api.respan.ai/api/responses \
-H "Authorization: Bearer $RESPAN_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "gpt-5.5", "input": "Explain why small language models are useful."}'
```
```bash
curl https://api.respan.ai/api/responses \
-H "Authorization: Bearer $RESPAN_API_KEY" \
-H "Content-Type: application/json" \
-H "X-Respan-Route-Provider: azure" \
-d '{
"model": "azure/YOUR_DEPLOYMENT",
"input": "Summarize the benefits of retrieval-augmented generation.",
"respan_params": {
"credential_override": {
"azure/YOUR_DEPLOYMENT": {
"api_key": "YOUR_AZURE_OPENAI_KEY",
"api_base": "https://YOUR_RESOURCE.openai.azure.com",
"api_version": "2025-04-01-preview"
}
}
}
}'
```
Respan removes the `azure/` prefix before calling Azure. Use an API version your deployment supports.
```bash
curl https://api.respan.ai/api/responses \
-H "Authorization: Bearer $RESPAN_API_KEY" \
-H "Content-Type: application/json" \
-H "X-Respan-Route-Provider: perplexity" \
-d '{
"model": "perplexity/sonar",
"input": "Summarize the latest developments in small language models.",
"max_steps": 1,
"respan_params": {"credential_override": {"": {"api_key": "YOUR_PERPLEXITY_API_KEY"}}}
}'
```
Full examples in Python and TypeScript are in the [OpenAI](/docs/integrations/gateway/model-providers/openai), [Azure OpenAI](/docs/integrations/gateway/model-providers/azure), and [Perplexity](/docs/integrations/gateway/model-providers/perplexity) provider guides. For the full schema, see [Create response](/docs/apis/gateway/create-response).
### Perplexity parameters
These fields work only with the Perplexity header. Other routes drop them.
| Field | Type | Purpose |
|---|---|---|
| `model` | string | A provider-prefixed model, such as `perplexity/sonar` or `openai/gpt-5.1`, passed through unchanged. See [Perplexity's Agent API models](https://docs.perplexity.ai/docs/agent-api/models). |
| `models` | string[] | Models for Perplexity to try in order. |
| `preset` | string | Perplexity's settings preset, such as `medium`. Send it with `model` or `models`: a preset alone fails Respan's model validation. |
| `max_steps` | integer | Maximum agent steps. |
| `tools` | array | Tools for the agent, such as `{"type": "web_search", "filters": {"search_domain_filter": ["respan.ai"]}}`. |
| `skills` | array | Perplexity Agent API skills. |
| `language_preference` | string | Preferred response language. |
| `response_format` | object | Structured output settings. |
Set `stream: true` for server-sent events. The stream ends with `response.completed`. The response's `model` is the model Perplexity actually used.
### Logging and billing
Every successful response has an `X-Respan-Log-Id` header. OpenAI and Azure requests are logged with the model that served them and its usage and cost. Perplexity requests are logged with `provider_id: perplexity`, the model Perplexity returned, and Perplexity's reported cost, which is also what Respan deducts from credits. If Perplexity reports no valid cost, Respan estimates it from catalog token prices.
## Troubleshooting
| Symptom | Fix |
|---|---|
| A model goes to the wrong provider, or isn't available | Copy its ID from the [Model catalog](/docs/documentation/features/gateway/models-catalog#find-a-model), or [pin a provider](/docs/documentation/features/gateway/models-catalog#pin-a-provider). |
| `401` asks for a provider key | Add the key under **Bring your own key**, or send `credential_override` keyed exactly as in the [Responses API](#responses-api) table. |
| Passthrough returns `400` "No endpoint is known for provider" | Save the provider's endpoint URL with its key under **Bring your own key**. |
| OpenRouter passthrough returns `400` | Send your OpenRouter key in `x-openrouter-api-key`. `Authorization` carries your Respan key. |
| OpenAI or Azure Responses request fails before reaching the provider | Send both `model` and `input`. |
| Azure can't find the deployment | Check that the name after `azure/` matches the deployment and that `api_base` points to its resource. |
| Request goes to OpenAI instead of Perplexity | Add the `X-Respan-Route-Provider: perplexity` header. A body field doesn't work. |
| Perplexity returns `404` "Requested model: is not available" | Send `model` or `models`. A preset alone isn't enough. |
| Perplexity-only fields are dropped | Send them to `POST /api/responses` with the Perplexity header, not to Chat Completions. |