GatewayLLM SDKs

Vertex AI (gateway)

Route Vertex AI calls through the Respan gateway after Vertex AI credentials and a supported model are configured in your Respan account.

Setup

Live verification with the tested gateway key returned provider generation errors for Vertex AI models. Configure Vertex AI credentials/model access in Respan before using this setup.

1

Install packages

$pip install openai
2

Set environment variables

$export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

Configure Vertex AI credentials in Respan before using this route.

3

Point an OpenAI-compatible client to the Respan gateway

1import os
2from openai import OpenAI
3
4client = OpenAI(
5    api_key=os.environ["RESPAN_API_KEY"],
6    base_url="https://api.respan.ai/api",
7)
8
9response = client.chat.completions.create(
10    model="YOUR_VERTEX_AI_MODEL_ID",
11    messages=[{"role": "user", "content": "Say hello in three languages."}],
12)
13print(response.choices[0].message.content)

Switch models

Change the model parameter to use 250+ models from different providers through the same gateway.

1response = client.chat.completions.create(model="YOUR_VERTEX_AI_MODEL_ID", messages=messages)
2response = client.chat.completions.create(model="gpt-5.5", messages=messages)
3response = client.chat.completions.create(model="claude-sonnet-4-5-20250929", messages=messages)

See the full model list.