Respan API | Respan Docs

Set up Respan

Sign up — Create an account at platform.respan.ai
Create an API key — Generate one on the API keys page
Add credits or a provider key — Add credits on the Credits page or connect your own provider key on the Integrations page

Use AI

Add the Docs MCP to your AI coding tool to get help building with Respan. No API key needed.

1 {
2   "mcpServers": {
3     "respan-docs": {
4       "url": "https://mcp.respan.ai/mcp/docs"
5     }
6   }
7 }

Overview

The Respan gateway provides an OpenAI-compatible API endpoint that gives you access to 250+ models from all major providers through a single API key and base URL.

Endpoint	Base URL
OpenAI-compatible	`https://api.respan.ai/api/`
Anthropic proxy	`https://api.respan.ai/api/anthropic/`
Google Gemini proxy	`https://api.respan.ai/api/google/gemini`

Environment Switching: Respan doesn’t support an env parameter in API calls. To switch between environments (test/production), use different API keys — one for your test environment and another for production. Manage keys in API Keys settings.

Quickstart

Step 1: Set environment variables

$ export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

Step 2: Make a request

1 import requests
2 
3 def demo_call(input,
4               model="gpt-4o-mini",
5               token="YOUR_RESPAN_API_KEY"
6               ):
7     headers = {
8         'Content-Type': 'application/json',
9         'Authorization': f'Bearer {token}',
10     }
11 
12     data = {
13         'model': model,
14         'messages': [{'role': 'user', 'content': input}],
15     }
16 
17     response = requests.post(
18         'https://api.respan.ai/api/chat/completions',
19         headers=headers,
20         json=data,
21     )
22     return response
23 
24 messages = "Say 'Hello World'"
25 print(demo_call(messages).json())

Step 3: Verify

Open the Logs page to see your gateway requests.

Switch models

Change the model parameter to use any supported provider through the same endpoint:

1 # OpenAI
2 model = "gpt-4o"
3 # Anthropic
4 # model = "claude-sonnet-4-5-20250929"
5 # Google
6 # model = "gemini-1.5-pro"
7 # DeepSeek
8 # model = "deepseek-chat"
9 
10 response = requests.post(
11     'https://api.respan.ai/api/chat/completions',
12     headers=headers,
13     json={'model': model, 'messages': [{'role': 'user', 'content': 'Hello'}]},
14 )

Browse the full model list to see all available models.

OpenAI-compatible parameters

All standard OpenAI chat completion parameters are supported:

Parameter	Type	Description
`messages`	`array`	List of messages in OpenAI format (`role` + `content`).
`model`	`string`	Model to use (e.g. `gpt-4o-mini`, `claude-sonnet-4-5-20250929`).
`stream`	`boolean`	Stream back partial progress token by token.
`temperature`	`number`	Controls randomness (0-2).
`max_tokens`	`number`	Maximum tokens to generate.
`top_p`	`number`	Nucleus sampling threshold.
`frequency_penalty`	`number`	Penalize tokens by existing frequency.
`presence_penalty`	`number`	Penalize tokens by whether they appear in text so far.
`stop`	`array`	Stop sequences.
`tools`	`array`	List of tools/functions the model may call.
`tool_choice`	`string\|object`	Controls tool selection (`none`, `auto`, or specific tool).
`response_format`	`object`	Force JSON output (`json_object`, `json_schema`, or `text`).
`n`	`number`	Number of completions to generate.
`logprobs`	`boolean`	Return log probabilities of output tokens.

Respan parameters

Pass Respan-specific parameters in the request body alongside OpenAI parameters. When using the OpenAI SDK, pass them via extra_body.

Observability

Parameter	Type	Description
`customer_identifier`	`string`	Tag to identify the user. See customer identifier.
`metadata`	`object`	Custom key-value pairs for filtering and search. See custom properties.
`custom_identifier`	`string`	Extra indexed tag (shows as “Custom ID” in spans).
`disable_log`	`boolean`	When `true`, only metrics are recorded — input/output messages are omitted.
`request_breakdown`	`boolean`	Returns a summarization of the response (tokens, cost, latency).

Example

1 {
2   "model": "gpt-4o-mini",
3   "messages": [{"role": "user", "content": "Hello"}],
4   "customer_identifier": "user_123",
5   "metadata": {"session_id": "abc123", "team": "ml"},
6   "custom_identifier": "feature-x"
7 }

Reliability

Parameter	Type	Description
`fallback_models`	`array`	Backup models ranked by priority. See fallback models.
`load_balance_group`	`object`	Balance requests across models. See load balancing.
`retry_params`	`object`	Configure retries (`retry_enabled`, `num_retries`, `retry_after`). See retries.

Example

1 {
2   "model": "gpt-4o-mini",
3   "messages": [{"role": "user", "content": "Hello"}],
4   "fallback_models": ["gemini-1.5-pro", "claude-sonnet-4-5-20250929"],
5   "retry_params": {
6     "retry_enabled": true,
7     "num_retries": 3,
8     "retry_after": 1
9   }
10 }

Caching

Parameter	Type	Description
`cache_enabled`	`boolean`	Enable response caching. See caches.
`cache_ttl`	`number`	Cache time-to-live in seconds (default: 30 days).
`cache_options`	`object`	Set `cache_by_customer: true` to scope cache per customer.

Credentials

Parameter	Type	Description
`customer_credentials`	`object`	Pass your customer’s provider API keys. See provider keys.
`credential_override`	`object`	One-off credential overrides for specific models (e.g. Azure deployments).
`model_name_map`	`object`	Map default model names to custom Azure deployment names.

Credential override example

1 {
2   "model": "azure/gpt-4o",
3   "credential_override": {
4     "azure/gpt-4o": {
5       "api_key": "your-azure-key",
6       "api_base": "your-azure-base-url",
7       "api_version": "2024-02-01"
8     }
9   }
10 }

Prompt management

Parameter	Type	Description
`prompt`	`object`	Use a Respan-managed prompt template. See prompt management.

Prompt example

1 {
2   "model": "gpt-4o-mini",
3   "messages": [],
4   "prompt": {
5     "prompt_id": "your-prompt-id",
6     "variables": {
7       "user_name": "Sarah"
8     }
9   }
10 }

Response format

1 {
2   "id": "chatcmpl-e1b9665b-c354-41c5-bbe5-178bd0b69773",
3   "object": "chat.completion",
4   "created": 1761546960,
5   "model": "claude-sonnet-4-5-20250929",
6   "choices": [
7     {
8       "index": 0,
9       "finish_reason": "stop",
10       "message": {
11         "role": "assistant",
12         "content": "I'm doing well, thank you for asking! How can I help you today?"
13       }
14     }
15   ],
16   "usage": {
17     "completion_tokens": 20,
18     "prompt_tokens": 2619,
19     "total_tokens": 2639,
20     "completion_tokens_details": {
21       "accepted_prediction_tokens": 0,
22       "audio_tokens": 0,
23       "reasoning_tokens": 0,
24       "rejected_prediction_tokens": 0
25     },
26     "prompt_tokens_details": {
27       "audio_tokens": 0,
28       "cached_tokens": 2601,
29       "cache_creation_tokens": 0
30     },
31     "cache_creation_input_tokens": 0,
32     "cache_read_input_tokens": 2601
33   }
34 }

Next Steps

Full API reference

Complete parameter documentation

Gateway setup

Choose models and configure the gateway

Advanced configuration

Load balancing, retries, fallbacks, and caching

Prompt management

Manage and version your prompts