Skip to navigation
ModelsMore providers

Provider: Google Vertex AI

Call Google Vertex AI models through Respan Gateway with unified logs, cost, and latency.
This page is for Respan LLM Gateway users.

Use Respan Gateway to call Google Vertex AI models (vertex_ai/claude-opus-4-8@default and Gemini on Vertex) while keeping unified observability (logs, cost, latency, reliability) in Respan.

Google Vertex AI requires bring your own key (BYOK). Provider usage cannot be paid with Respan credits.

Quick setup

1

Get a Respan API key

Sign up and create a key on the API keys page.

2

Add your Google Vertex AI credentials

Configure your Google Vertex AI credentials on the Providers page before using the examples below. See Use your own Google Vertex AI key.

Send your first request

After configuring your Google Vertex AI credentials in Respan, pick the integration that matches your stack. The base URL is https://api.respan.ai/api and your application authenticates with RESPAN_API_KEY.

Call respan.chat(...) explicitly to use the Gateway Chat Completions endpoint.

Use @ai-sdk/openai and point createOpenAI at the Respan gateway. The example uses the live vertex_ai/claude-opus-4-8@default catalog entry. Enable that model in your Vertex project and choose a supported region.

TypeScript
import { createOpenAI } from "@ai-sdk/openai";
import { generateText } from "ai";
const respan = createOpenAI({
  apiKey: process.env.RESPAN_API_KEY!,
  baseURL: "https://api.respan.ai/api",
});
const result = await generateText({
  model: respan.chat("vertex_ai/claude-opus-4-8@default"),
  prompt: "Hello, Vertex AI!",
});
console.log(result.text);

More integrations

Vertex AI models work with every Respan gateway integration:

Switch models

Change the model parameter to call any supported model through the same client. Use the vertex_ai/ prefix to disambiguate when routing across providers. Browse the full list on the Models page.

client.chat.completions.create(model="vertex_ai/claude-opus-4-8@default", messages=messages)
client.chat.completions.create(model="vertex_ai/gemini-3.5-flash", messages=messages)
client.chat.completions.create(model="vertex_ai/gemini-3.1-flash-lite", messages=messages)
client.chat.completions.create(model="openai/gpt-6-sol", messages=messages)
client.chat.completions.create(model="anthropic/claude-opus-5-5", messages=messages)

Use your own Google Vertex AI key (BYOK)

Google Vertex AI credentials are required. Configure them in Respan or pass them with each request using one of the options below.

Google credentials can be tricky. Follow this walkthrough if you need help finding the required fields:

1

Open Providers

Go to the Providers page.

2

Add Google Vertex AI

Select Google Vertex AI and fill in the required credential fields:

  • vertex_ai_project (your Google Cloud project ID)
  • vertex_ai_location (the Vertex AI region to use)
  • vertex_ai_credentials (your Google service-account or application-default credential JSON object)
3

Load balancing (Optional)

Add multiple credential sets and use Load balancing weight to distribute traffic across them.

Override credentials per model (Optional)

Use credential_override when one model on a request should use a different Vertex project than the default.

{
  "customer_credentials": {
    "google_vertex_ai": {
      "vertex_ai_project": "your-project",
      "vertex_ai_location": "us-central1",
      "vertex_ai_credentials": {
        "type": "service_account",
        "project_id": "your-project"
      }
    }
  },
  "credential_override": {
    "vertex_ai/claude-opus-4-8@default": {
      "vertex_ai_project": "ANOTHER_VERTEX_AI_PROJECT",
      "vertex_ai_location": "ANOTHER_VERTEX_AI_LOCATION",
      "vertex_ai_credentials": {
        "type": "service_account",
        "project_id": "another-project"
      }
    }
  }
}

Log without proxying (Optional)

Already calling Vertex AI directly? Send logs to Respan asynchronously to track cost, latency, and performance for those external calls.

import requests
requests.post(
    "https://api.respan.ai/api/request-logs/create/",
    headers={
        "Authorization": "Bearer YOUR_RESPAN_API_KEY",
        "Content-Type": "application/json",
    },
    json={
        "model": "vertex_ai/claude-opus-4-8@default",
        "prompt_messages": [{"role": "user", "content": "Hello, how are you?"}],
        "completion_message": {"role": "assistant", "content": "Hello from Vertex AI through Respan."},
        "cost": 0.001,
        "generation_time": 1.2,
        "customer_params": {"customer_identifier": "user_123"},
    },
)

See the logging guide for the full setup.