GatewayLLM SDKs

Vertex AI (gateway)

Route Vertex AI requests through the Respan gateway after Vertex AI credentials and model access are configured in your Respan account. The client talks to Respan’s OpenAI-compatible endpoint with RESPAN_API_KEY; Respan handles provider authentication.

Setup

1

Install packages

$pip install openai
2

Set environment variables

$export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
$export RESPAN_BASE_URL="https://api.respan.ai/api"
$export RESPAN_MODEL="YOUR_VERTEX_AI_MODEL_ID"

Configure Vertex AI credentials in Respan before using this route.

3

Point the OpenAI client to the Respan gateway

1import os
2from openai import OpenAI
3
4client = OpenAI(
5    api_key=os.environ["RESPAN_API_KEY"],
6    base_url=os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api"),
7)
8
9response = client.chat.completions.create(
10    model=os.getenv("RESPAN_MODEL", "YOUR_VERTEX_AI_MODEL_ID"),
11    messages=[{"role": "user", "content": "Say hello in one sentence."}],
12)
13print(response.choices[0].message.content)

Switch models

Change RESPAN_MODEL to another Vertex AI model or to any other model configured for your Respan gateway account.

1response = client.chat.completions.create(model="YOUR_VERTEX_AI_MODEL_ID", messages=messages)
2response = client.chat.completions.create(model="gpt-5-mini", messages=messages)

See the full model list.