Skip to navigation
IntegrationIntegrationsLLM SDKs

SageMaker (gateway)

Route SageMaker calls through the Respan gateway after the SageMaker endpoint/model is available in your Respan model list.

Setup

Live verification with the tested gateway key returned model unavailable for the previous SageMaker example. Use a SageMaker endpoint/model ID from your Respan model list.

1

Install packages

pip install openai
2

Set environment variables

export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
3

Point an OpenAI-compatible client to the Respan gateway

import os
from openai import OpenAI
client = OpenAI(
    api_key=os.environ["RESPAN_API_KEY"],
    base_url="https://api.respan.ai/api",
)
response = client.chat.completions.create(
    model="YOUR_SAGEMAKER_MODEL_ID",
    messages=[{"role": "user", "content": "Say hello in three languages."}],
)
print(response.choices[0].message.content)

Switch models

Change the model parameter to use 1000+ models from different providers through the same gateway.

response = client.chat.completions.create(model="YOUR_SAGEMAKER_MODEL_ID", messages=messages)
response = client.chat.completions.create(model="openai/gpt-6-sol", messages=messages)

See the full model list.