GatewayLLM SDKs

Vertex AI (gateway)

Route Vertex AI requests through the Respan gateway after Vertex AI credentials and model access are configured in your Respan account. The client talks to Respan’s OpenAI-compatible endpoint with RESPAN_API_KEY; Respan handles provider authentication.

Setup

1

Install packages

pip install openai
2

Set environment variables

export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

Configure Vertex AI credentials in Respan before using Vertex-specific model slugs. The setup below uses vertex_ai/claude-opus-4-8@default, which is marked Live in the model catalog. Enable the model in your Google Cloud project and configure a supported region in Respan.

3

Point the OpenAI client to the Respan gateway

import os
from openai import OpenAI
client = OpenAI(
    api_key=os.environ["RESPAN_API_KEY"],
    base_url="https://api.respan.ai/api",
)
response = client.chat.completions.create(
    model="vertex_ai/claude-opus-4-8@default",
    messages=[{"role": "user", "content": "Say hello in one sentence."}],
)
print(response.choices[0].message.content)

Switch models

Change the model value to a Vertex AI model slug configured for your Respan gateway account, or to any other model available through your gateway key.

response = client.chat.completions.create(model="vertex_ai/claude-opus-4-8@default", messages=messages)
response = client.chat.completions.create(model="openai/gpt-6-sol", messages=messages)

See the full model list.