OpenAI SDK (gateway)

Route OpenAI SDK calls through the Respan gateway to use 250+ models from different providers. Only your RESPAN_API_KEY is needed — no separate provider keys required.

Setup

Install packages

$ pip install respan-ai respan-instrumentation-openai openai

Set environment variables

$ export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

No OPENAI_API_KEY needed — the Respan gateway handles provider authentication.

Point the OpenAI client to the Respan gateway

1 import os
2 from openai import OpenAI
3 
4 client = OpenAI(
5     api_key=os.getenv("RESPAN_API_KEY"),
6     base_url="https://api.respan.ai/api",
7 )
8 
9 response = client.chat.completions.create(
10     model="gpt-4.1-nano",
11     messages=[{"role": "user", "content": "Say hello in three languages."}],
12 )
13 print(response.choices[0].message.content)

Switch models

Change the model parameter to use 250+ models from different providers through the same gateway.

1 response = client.chat.completions.create(model="gpt-4o", messages=messages)
2 response = client.chat.completions.create(model="claude-sonnet-4-5-20250929", messages=messages)
3 response = client.chat.completions.create(model="gemini-2.5-flash", messages=messages)

See the full model list.

Prompt management

Use Respan prompt management to serve prompt templates from the platform. Use schema_version: 2 for all new integrations.

Chat Completions

Prompt messages are the base layer (system/context). Body messages are appended as runtime user turns.

1 PROMPT_ID = "YOUR_PROMPT_ID"
2 
3 response = client.chat.completions.create(
4     model="gpt-4.1-nano",
5     messages=[
6         {"role": "user", "content": "Add dark mode support to the dashboard"},
7     ],
8     extra_body={
9         "prompt": {
10             "prompt_id": PROMPT_ID,
11             "schema_version": 2,
12             "variables": {
13                 "feature_request": "Add dark mode support",
14             },
15         }
16     },
17 )
18 print(response.choices[0].message.content)

Responses API

For the Responses API, pass prompt config under respan_params. The prompt template becomes instructions and body input is preserved.

1 PROMPT_ID = "YOUR_PROMPT_ID"
2 
3 response = client.responses.create(
4     model="gpt-4.1-nano",
5     input=[
6         {"role": "user", "content": "Add dark mode support to the dashboard"},
7     ],
8     extra_body={
9         "respan_params": {
10             "prompt": {
11                 "prompt_id": PROMPT_ID,
12                 "schema_version": 2,
13                 "variables": {
14                     "feature_request": "Add dark mode support",
15                 },
16             }
17         }
18     },
19 )
20 print(response.output_text)

Respan parameters

Pass additional Respan parameters via extra_body for gateway features.

1 response = client.chat.completions.create(
2     model="gpt-4.1-nano",
3     messages=[{"role": "user", "content": "Hello"}],
4     extra_body={
5         "customer_identifier": "user_123",
6         "fallback_models": ["gpt-3.5-turbo"],
7         "metadata": {"session_id": "abc123"},
8         "thread_identifier": "conversation_456",
9     },
10 )

See Respan params & metadata for the full list.