GatewayLLM SDKs

Vertex AI (gateway)

Route Vertex AI requests through the Respan gateway after Vertex AI credentials and model access are configured in your Respan account. The client talks to Respan’s OpenAI-compatible endpoint with RESPAN_API_KEY; Respan handles provider authentication.

Setup

1

Install packages

pip install openai
2

Set environment variables

export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

Configure Vertex AI credentials in Respan before using Vertex-specific model slugs. The setup below uses a known gateway model so the route is copy-paste runnable with only RESPAN_API_KEY.

3

Point the OpenAI client to the Respan gateway

import os
from openai import OpenAI
client = OpenAI(
    api_key=os.environ["RESPAN_API_KEY"],
    base_url="https://api.respan.ai/api",
)
response = client.chat.completions.create(
    model="gpt-5-mini",
    messages=[{"role": "user", "content": "Say hello in one sentence."}],
)
print(response.choices[0].message.content)

Switch models

Change the model value to a Vertex AI model slug configured for your Respan gateway account, or to any other model available through your gateway key.

response = client.chat.completions.create(model="YOUR_VERTEX_AI_MODEL_ID", messages=messages)
response = client.chat.completions.create(model="gpt-5-mini", messages=messages)

See the full model list.