Google Vertex AI (gateway)

1
{
2
  "mcpServers": {
3
    "respan-docs": {
4
      "url": "https://mcp.respan.ai/mcp/docs"
5
    }
6
  }
7
}

Use Respan Gateway to call Google Vertex AI models while keeping unified observability (logs, cost, latency, and reliability metrics) in Respan.

Prerequisites

• A Respan API key
• Google Vertex AI credentials (BYOK)

Supported SDKs / integrations

Configuration

There are 2 ways to add your Google Vertex AI credentials to your requests:

Via UI (Global)

Google credentials can be tricky. Follow this credential walkthrough video if you need help finding the required fields:

1

Navigate to Providers

Go to the Providers page. This page allows you to manage credentials for supported model providers.

2

Add your Google Vertex AI credentials

Select Google Vertex AI and add the required credential fields.

• vertex_ai_project — your Google Cloud project ID
• vertex_ai_location — the Vertex AI region to use
• vertex_ai_credentials — your Google service-account or application-default credential JSON object

3

Choose available models (Optional)

Copy the model ID from the Respan Models page, paste it into the available models field, and press Enter. Leave the field empty to apply the credentials to all Google Vertex AI models.

4

Configure Load Balancing (Optional)

Add multiple Google Vertex AI credential sets for redundancy. Use the Load balancing weight field to control traffic distribution.

Via code (Per-Request)

You can pass credentials dynamically in the request body. This is useful if you need to use your users’ own API keys (BYOK).

Add the customer_credentials parameter to your Gateway request:

1
{
2
  "customer_credentials": {
3
    "google_vertex_ai": {
4
      "vertex_ai_project": "your-project",
5
      "vertex_ai_location": "us-central1",
6
      "vertex_ai_credentials": {
7
        "type": "service_account",
8
        "project_id": "your-project"
9
      }
10
    }
11
  }
12
}

Override credentials for a particular model (Optional)

Use credential_override when one request or model should use different credentials than the default provider key.

1
{
2
  "customer_credentials": {
3
    "google_vertex_ai": {
4
      "vertex_ai_project": "your-project",
5
      "vertex_ai_location": "us-central1",
6
      "vertex_ai_credentials": {
7
        "type": "service_account",
8
        "project_id": "your-project"
9
      }
10
    }
11
  },
12
  "credential_override": {
13
    "vertex_ai/gemini-2.0-flash": {
14
      "vertex_ai_project": "ANOTHER_VERTEX_AI_PROJECT",
15
      "vertex_ai_location": "ANOTHER_VERTEX_AI_LOCATION",
16
      "vertex_ai_credentials": {
17
        "type": "service_account",
18
        "project_id": "another-project"
19
      }
20
    }
21
  }
22
}

Supported models

Find the complete and current list of Google Vertex AI model IDs on the Respan Models page. Use the exact model ID shown there in your gateway requests.

Log Google Vertex AI requests

If you are not using the Gateway to proxy requests, you can still log your Google Vertex AI requests to Respan asynchronously. This lets you track cost, latency, and performance metrics for external calls.

1
import requests
2
3
url = "https://api.respan.ai/api/request-logs/create/"
4
payload = {
5
  "model": "vertex_ai/gemini-2.0-flash",
6
  "prompt_messages": [
7
    {
8
      "role": "user",
9
      "content": "Hello, how are you?"
10
    }
11
  ],
12
  "completion_message": {
13
    "role": "assistant",
14
    "content": "Hello from Google Vertex AI through Respan."
15
  },
16
  "cost": 0.001,
17
  "generation_time": 1.2,
18
  "customer_params": {
19
    "customer_identifier": "user_123"
20
  }
21
}
22
headers = {
23
    "Authorization": "Bearer YOUR_RESPAN_API_KEY",
24
    "Content-Type": "application/json",
25
}
26
27
response = requests.post(url, headers=headers, json=payload)