GatewayLLM SDKs

AWS Bedrock (gateway)

Route Bedrock calls through the Respan gateway to use 1000+ models from different providers. Only your RESPAN_API_KEY is needed. No separate AWS credentials are required in application code.

Setup

1

Install packages

pip install openai
2

Set environment variables

export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

No AWS credentials are needed in your application runtime. The Respan gateway handles provider authentication.

3

Point an OpenAI-compatible client to the Respan gateway

import os
from openai import OpenAI
client = OpenAI(
    api_key=os.environ["RESPAN_API_KEY"],
    base_url="https://api.respan.ai/api",
)
response = client.chat.completions.create(
    model="bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0",
    messages=[{"role": "user", "content": "Say hello in three languages."}],
)
print(response.choices[0].message.content)

Switch models

Change the model parameter to use 1000+ models from different providers through the same gateway.

response = client.chat.completions.create(model="bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", messages=messages)
response = client.chat.completions.create(model="openai/gpt-6-sol", messages=messages)
response = client.chat.completions.create(model="gemini/gemini-3.8-flash", messages=messages)

See the full model list.