Portkey (gateway) | Respan Docs

Use the Respan gateway when you want the Portkey-style OpenAI-compatible routing flow with Respan request logs, routing, fallbacks, prompt management, and metadata. For tracing calls that still use the Portkey Python SDK, see Portkey tracing setup.

Setup

Install packages

pip install openai python-dotenv

Set environment variables

export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

No PORTKEY_API_KEY is required for gateway calls through Respan.

Point an OpenAI-compatible client to the Respan gateway

import os
from dotenv import load_dotenv
from openai import OpenAI
load_dotenv()
client = OpenAI(
    api_key=os.environ["RESPAN_API_KEY"],
    base_url=os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api"),
)
response = client.chat.completions.create(
    model=os.getenv("RESPAN_MODEL", "gpt-5.5"),
    messages=[{"role": "user", "content": "Say hello in three languages."}],
)
print(response.choices[0].message.content)

Switch models

Change the model parameter to use 1000+ models from different providers through the same gateway.

response = client.chat.completions.create(model="gpt-5.5", messages=messages)
response = client.chat.completions.create(model="claude-sonnet-4-5-20250929", messages=messages)
response = client.chat.completions.create(model="gemini/gemini-3.5-flash", messages=messages)

See the full model list.

Respan parameters

Pass additional Respan parameters via extra_body for gateway features.

response = client.chat.completions.create(
    model="gpt-5.5",
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={
        "customer_identifier": "user_123",
        "fallback_models": ["gpt-5-mini"],
        "metadata": {"session_id": "abc123"},
        "thread_identifier": "conversation_456",
    },
)

See Respan params & metadata for the full list.