Routing & passthrough

Pick an endpoint, and see how Respan chooses the model, provider, and key for each request.

Respan has two kinds of endpoints. The unified endpoint takes OpenAI-format requests for any model, so you switch providers by changing model. Passthrough endpoints take a provider’s own format, so its SDK or CLI works unchanged.

Every request through either one is logged, and you can mix them in one project.

Unified endpoint

To switch provider, change the model slug. Nothing else in your code changes:

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.respan.ai/api/", api_key=os.environ["RESPAN_API_KEY"])
messages = [{"role": "user", "content": "Hello!"}]

client.chat.completions.create(model="gpt-5.5", messages=messages)
client.chat.completions.create(model="claude-sonnet-4-5-20250929", messages=messages)
client.chat.completions.create(model="gemini-2.5-pro", messages=messages)

How a model is chosen

For each request, Respan:

1. Picks the model. It uses model. If the request has load_balance_group or load_balance_models, Respan picks one of those models by weight instead. A group wins over load_balance_models. Set model to span-router to let span-router pick the model for each task.
2. Picks the provider. The model slug decides it: gpt-5.5 goes to OpenAI and azure/gpt-4o goes to Azure OpenAI. To send a slug to a different provider, pin a provider with the X-Respan-Route-Provider header.
3. Picks the key. If you added a key for that provider under Gateway › API keys › Bring your own key, Respan uses it, split across your keys by load balance weight. Otherwise the call runs on Respan credits.
4. Handles failures. It retries the same model, then tries your fallback models in order. See Retries and fallback.

The model that served the request is on its log.

Provider passthrough

A passthrough endpoint relays the provider’s native request and response. In every case, the Authorization header or API key carries your Respan API key, not the provider’s.

import os
import anthropic

client = anthropic.Anthropic(
    base_url="https://api.respan.ai/api/anthropic/",
    api_key=os.environ["RESPAN_API_KEY"],
)
message = client.messages.create(
    model="claude-sonnet-4-5-20250929",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}],
)

export ANTHROPIC_BASE_URL="https://api.respan.ai/api/anthropic/"
export ANTHROPIC_AUTH_TOKEN="$RESPAN_API_KEY"

import os
from google import genai

client = genai.Client(
    api_key=os.environ["RESPAN_API_KEY"],
    http_options={"base_url": "https://api.respan.ai/api/google/gemini"},
)

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.respan.ai/api/openai/v1",  # or /api/xai/v1, /api/groq/v1, ...
    api_key=os.environ["RESPAN_API_KEY"],
)
client.chat.completions.create(model="gpt-5.5", messages=[{"role": "user", "content": "Hello!"}])

To keep a passthrough’s request format but send the call to another provider, for example Anthropic-format requests to Vertex AI or Bedrock, pin a provider.

Chat completions passthrough

POST /api/{provider}/v1/chat/completions relays an OpenAI-shaped provider’s chat completions payload and returns that provider’s response byte for byte. Respan authenticates the request, applies your limits, and logs it. It doesn’t translate the request or recompute usage.

{provider} is the provider ID, such as openai, azure_openai, xai, groq, mistral, deepseek, togetherai, fireworks, or perplexity. A provider without an OpenAI-shaped API returns 404. Anthropic and Google have their own endpoints, shown above, and so does OpenRouter.

A provider with no fixed public endpoint, such as Azure OpenAI, needs its endpoint URL saved with its key under Bring your own key. Without it, the request returns 400.

OpenRouter

POST /api/openrouter/v1/chat/completions relays your request to OpenRouter and returns OpenRouter’s own body, status, and headers. It’s always bring-your-own-key: Authorization carries your Respan API key and x-openrouter-api-key carries your OpenRouter key. A missing OpenRouter key returns 400.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.respan.ai/api/openrouter/v1",
    api_key=os.environ["RESPAN_API_KEY"],
    default_headers={"x-openrouter-api-key": os.environ["OPENROUTER_API_KEY"]},
)
response = client.chat.completions.create(
    model="anthropic/claude-sonnet-4-5",
    messages=[{"role": "user", "content": "Hello!"}],
    extra_body={"provider": {"order": ["anthropic"]}, "reasoning": {"effort": "high"}},
)

Use OpenRouter’s model slug. An openrouter/ prefix is also accepted and removed. The http-referer, x-title, and x-openrouter-* headers are forwarded. See Create an OpenRouter chat completion for the full reference.

The unified endpoint also reaches OpenRouter, through openrouter/-prefixed models. See OpenRouter. Pick one:

Responses API

POST https://api.respan.ai/api/responses takes OpenAI Responses-format requests. The X-Respan-Route-Provider header picks the provider:

The per-request key is optional. Without it, Respan uses the provider key you added under Bring your own key, or Respan credits where the route supports them.

curl https://api.respan.ai/api/responses \
  -H "Authorization: Bearer $RESPAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-5.5", "input": "Explain why small language models are useful."}'

curl https://api.respan.ai/api/responses \
  -H "Authorization: Bearer $RESPAN_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Respan-Route-Provider: azure" \
  -d '{
    "model": "azure/YOUR_DEPLOYMENT",
    "input": "Summarize the benefits of retrieval-augmented generation.",
    "respan_params": {
      "credential_override": {
        "azure/YOUR_DEPLOYMENT": {
          "api_key": "YOUR_AZURE_OPENAI_KEY",
          "api_base": "https://YOUR_RESOURCE.openai.azure.com",
          "api_version": "2025-04-01-preview"
        }
      }
    }
  }'

curl https://api.respan.ai/api/responses \
  -H "Authorization: Bearer $RESPAN_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Respan-Route-Provider: perplexity" \
  -d '{
    "model": "perplexity/sonar",
    "input": "Summarize the latest developments in small language models.",
    "max_steps": 1,
    "respan_params": {"credential_override": {"": {"api_key": "YOUR_PERPLEXITY_API_KEY"}}}
  }'

Full examples in Python and TypeScript are in the OpenAI, Azure OpenAI, and Perplexity provider guides. For the full schema, see Create response.

Perplexity parameters

These fields work only with the Perplexity header. Other routes drop them.

Set stream: true for server-sent events. The stream ends with response.completed. The response’s model is the model Perplexity actually used.

Logging and billing

Every successful response has an X-Respan-Log-Id header. OpenAI and Azure requests are logged with the model that served them and its usage and cost. Perplexity requests are logged with provider_id: perplexity, the model Perplexity returned, and Perplexity’s reported cost, which is also what Respan deducts from credits. If Perplexity reports no valid cost, Respan estimates it from catalog token prices.

Troubleshooting