GatewayLLM SDKs

Vertex AI (gateway)

Route Vertex AI requests through the Respan gateway after Vertex AI credentials and model access are configured in your Respan account. The client talks to Respan’s OpenAI-compatible endpoint with RESPAN_API_KEY; Respan handles provider authentication.

Setup

1

Install packages

$pip install openai
2

Set environment variables

$export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

Configure Vertex AI credentials in Respan before using Vertex-specific model slugs. The setup below uses a known gateway model so the route is copy-paste runnable with only RESPAN_API_KEY.

3

Point the OpenAI client to the Respan gateway

1import os
2from openai import OpenAI
3
4client = OpenAI(
5    api_key=os.environ["RESPAN_API_KEY"],
6    base_url="https://api.respan.ai/api",
7)
8
9response = client.chat.completions.create(
10    model="gpt-5-mini",
11    messages=[{"role": "user", "content": "Say hello in one sentence."}],
12)
13print(response.choices[0].message.content)

Switch models

Change the model value to a Vertex AI model slug configured for your Respan gateway account, or to any other model available through your gateway key.

1response = client.chat.completions.create(model="YOUR_VERTEX_AI_MODEL_ID", messages=messages)
2response = client.chat.completions.create(model="gpt-5-mini", messages=messages)

See the full model list.