GatewayLLM SDKs

Respan API (gateway)

The Respan API is the gateway itself. Every request is automatically traced — no SDK or instrumentation needed.

Endpoints

API surfaceBase URLRequest pathAuth
OpenAI-compatible Chat Completionshttps://api.respan.ai/api/chat/completionsAuthorization: Bearer $RESPAN_API_KEY
Anthropic Messageshttps://api.respan.ai/api/anthropic/v1/messagesAuthorization: Bearer $RESPAN_API_KEY or SDK api_key
Google Gemini SDK proxyhttps://api.respan.ai/api/google/gemini/v1beta/models/{model}:generateContentx-goog-api-key: $RESPAN_API_KEY
OpenRouter passthroughhttps://api.respan.ai/api/openrouter/v1/chat/completionsAuthorization: Bearer $RESPAN_API_KEY plus x-openrouter-api-key

The one-key OpenAI-compatible route verified for these setup examples is Chat Completions. Live verification with a Respan gateway key returned a provider-credential error on /api/responses; use /chat/completions unless Responses API access is configured for your key.

Environment switching: Respan doesn’t support an env parameter in API calls. To switch between environments (test/production), use different API keys — one for your test environment and another for production. Manage keys in API Keys settings.

Setup

1

Install packages

Install requests if you want to run the Python example below.

pip install requests
2

Set environment variables

export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
3

Make a request

import os
import requests
response = requests.post(
    "https://api.respan.ai/api/chat/completions",
    headers={
        "Content-Type": "application/json",
        "Authorization": f"Bearer {os.environ['RESPAN_API_KEY']}",
    },
    json={
        "model": "gpt-5.5",
        "messages": [{"role": "user", "content": "Say 'Hello World'"}],
    },
)
print(response.json())
4

Verify

Open the Logs page to see your gateway requests captured as traces.

Switch models

Change the model parameter to use any supported provider through the same endpoint.

response = requests.post(
    "https://api.respan.ai/api/chat/completions",
    headers=headers,
    json={"model": "gpt-5.5", "messages": messages},
)
response = requests.post(..., json={"model": "claude-sonnet-4-5-20250929", "messages": messages})
response = requests.post(..., json={"model": "gemini/gemini-3.5-flash", "messages": messages})

Browse the full model list to see all available models.

OpenAI-compatible parameters

All standard OpenAI chat completion parameters are supported.

ParameterTypeDescription
messagesarrayList of messages in OpenAI format (role + content).
modelstringModel to use (e.g. gpt-5.5, claude-sonnet-4-5-20250929).
streambooleanStream back partial progress token by token.
temperaturenumberControls randomness (0-2).
max_tokensnumberMaximum tokens to generate.
top_pnumberNucleus sampling threshold.
toolsarrayList of tools/functions the model may call.
tool_choicestring|objectControls tool selection.
response_formatobjectForce JSON output.

Respan parameters

Pass Respan-specific parameters in the request body alongside OpenAI parameters. When using the OpenAI SDK, pass them via extra_body.

Observability

ParameterTypeDescription
customer_identifierstringTag to identify the user. See customer identifier.
metadataobjectCustom key-value pairs for filtering and search.
custom_identifierstringExtra indexed tag (shows as “Custom ID” in spans).
disable_logbooleanWhen true, only metrics are recorded.
request_breakdownbooleanReturns a summarization of the response (tokens, cost, latency).

Reliability

ParameterTypeDescription
fallback_modelsarrayBackup models ranked by priority.
load_balance_groupobjectBalance requests across models.
retry_paramsobjectConfigure retries (retry_enabled, num_retries, retry_after).

Caching

ParameterTypeDescription
cache_enabledbooleanEnable response caching.
cache_ttlnumberCache time-to-live in seconds (default: 30 days).
cache_optionsobjectSet cache_by_customer: true to scope cache per customer.

Credentials

ParameterTypeDescription
customer_credentialsobjectPass your customer’s provider API keys.
credential_overrideobjectOne-off credential overrides for specific models.
model_name_mapobjectMap default model names to custom Azure deployment names.

Prompt management

ParameterTypeDescription
promptobjectUse a Respan-managed prompt template.
{
  "model": "gpt-5.5",
  "messages": [],
  "prompt": {
    "prompt_id": "your-prompt-id",
    "variables": {"user_name": "Sarah"}
  }
}

Response format

{
  "id": "chatcmpl-e1b9665b-c354-41c5-bbe5-178bd0b69773",
  "object": "chat.completion",
  "created": 1761546960,
  "model": "claude-sonnet-4-5-20250929",
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "message": {
        "role": "assistant",
        "content": "I'm doing well, thank you for asking!"
      }
    }
  ],
  "usage": {
    "completion_tokens": 20,
    "prompt_tokens": 2619,
    "total_tokens": 2639
  }
}