GatewayLLM SDKs

Respan API (gateway)

The Respan API is the gateway itself. Every request is automatically traced — no SDK or instrumentation needed.

Endpoints

API surfaceBase URLRequest pathAuth
OpenAI-compatible Chat Completionshttps://api.respan.ai/api/chat/completionsAuthorization: Bearer $RESPAN_API_KEY
Anthropic Messageshttps://api.respan.ai/api/anthropic/v1/messagesAuthorization: Bearer $RESPAN_API_KEY or SDK api_key
Google Gemini SDK proxyhttps://api.respan.ai/api/google/gemini/v1beta/models/{model}:generateContentx-goog-api-key: $RESPAN_API_KEY

The one-key OpenAI-compatible route verified for these setup examples is Chat Completions. Live verification with a Respan gateway key returned a provider-credential error on /api/responses; use /chat/completions unless Responses API access is configured for your key.

Environment switching: Respan doesn’t support an env parameter in API calls. To switch between environments (test/production), use different API keys — one for your test environment and another for production. Manage keys in API Keys settings.

Setup

1

Install packages

Install requests if you want to run the Python example below.

$pip install requests
2

Set environment variables

$export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
3

Make a request

1import os
2import requests
3
4response = requests.post(
5    "https://api.respan.ai/api/chat/completions",
6    headers={
7        "Content-Type": "application/json",
8        "Authorization": f"Bearer {os.environ['RESPAN_API_KEY']}",
9    },
10    json={
11        "model": "gpt-5.5",
12        "messages": [{"role": "user", "content": "Say 'Hello World'"}],
13    },
14)
15print(response.json())
4

Verify

Open the Logs page to see your gateway requests captured as traces.

Switch models

Change the model parameter to use any supported provider through the same endpoint.

1response = requests.post(
2    "https://api.respan.ai/api/chat/completions",
3    headers=headers,
4    json={"model": "gpt-5.5", "messages": messages},
5)
6response = requests.post(..., json={"model": "claude-sonnet-4-5-20250929", "messages": messages})
7response = requests.post(..., json={"model": "gemini/gemini-3.5-flash", "messages": messages})

Browse the full model list to see all available models.

OpenAI-compatible parameters

All standard OpenAI chat completion parameters are supported.

ParameterTypeDescription
messagesarrayList of messages in OpenAI format (role + content).
modelstringModel to use (e.g. gpt-5.5, claude-sonnet-4-5-20250929).
streambooleanStream back partial progress token by token.
temperaturenumberControls randomness (0-2).
max_tokensnumberMaximum tokens to generate.
top_pnumberNucleus sampling threshold.
toolsarrayList of tools/functions the model may call.
tool_choicestring|objectControls tool selection.
response_formatobjectForce JSON output.

Respan parameters

Pass Respan-specific parameters in the request body alongside OpenAI parameters. When using the OpenAI SDK, pass them via extra_body.

Observability

ParameterTypeDescription
customer_identifierstringTag to identify the user. See customer identifier.
metadataobjectCustom key-value pairs for filtering and search.
custom_identifierstringExtra indexed tag (shows as “Custom ID” in spans).
disable_logbooleanWhen true, only metrics are recorded.
request_breakdownbooleanReturns a summarization of the response (tokens, cost, latency).

Reliability

ParameterTypeDescription
fallback_modelsarrayBackup models ranked by priority.
load_balance_groupobjectBalance requests across models.
retry_paramsobjectConfigure retries (retry_enabled, num_retries, retry_after).

Caching

ParameterTypeDescription
cache_enabledbooleanEnable response caching.
cache_ttlnumberCache time-to-live in seconds (default: 30 days).
cache_optionsobjectSet cache_by_customer: true to scope cache per customer.

Credentials

ParameterTypeDescription
customer_credentialsobjectPass your customer’s provider API keys.
credential_overrideobjectOne-off credential overrides for specific models.
model_name_mapobjectMap default model names to custom Azure deployment names.

Prompt management

ParameterTypeDescription
promptobjectUse a Respan-managed prompt template.
1{
2  "model": "gpt-5.5",
3  "messages": [],
4  "prompt": {
5    "prompt_id": "your-prompt-id",
6    "variables": {"user_name": "Sarah"}
7  }
8}

Response format

1{
2  "id": "chatcmpl-e1b9665b-c354-41c5-bbe5-178bd0b69773",
3  "object": "chat.completion",
4  "created": 1761546960,
5  "model": "claude-sonnet-4-5-20250929",
6  "choices": [
7    {
8      "index": 0,
9      "finish_reason": "stop",
10      "message": {
11        "role": "assistant",
12        "content": "I'm doing well, thank you for asking!"
13      }
14    }
15  ],
16  "usage": {
17    "completion_tokens": 20,
18    "prompt_tokens": 2619,
19    "total_tokens": 2639
20  }
21}