Skip to navigation
IntegrationIntegrationsLLM SDKs

Replicate (gateway)

Route Replicate calls through the Respan gateway only after the model is configured for your project and you have added your own Replicate credentials (BYOK). Provider usage is billed to your Replicate account, not Respan credits.

Setup

No Replicate models are currently listed in the public Gateway model catalog. Confirm that your project has a configured Replicate model before using this example. For direct Replicate calls with Respan observability, see the Replicate integration.

1

Install packages

pip install openai
2

Set environment variables

export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
3

Point an OpenAI-compatible client to the Respan gateway

import os
from openai import OpenAI
client = OpenAI(
    api_key=os.environ["RESPAN_API_KEY"],
    base_url="https://api.respan.ai/api",
)
response = client.chat.completions.create(
    model="YOUR_REPLICATE_MODEL_ID",
    messages=[{"role": "user", "content": "Say hello in three languages."}],
)
print(response.choices[0].message.content)

Switch models

Change the model parameter to use 1000+ models from different providers through the same gateway.

response = client.chat.completions.create(model="YOUR_REPLICATE_MODEL_ID", messages=messages)
response = client.chat.completions.create(model="openai/gpt-6-sol", messages=messages)

See the full model list.