Google Vertex AI

1
{
2
  "mcpServers": {
3
    "respan-docs": {
4
      "url": "https://mcp.respan.ai/mcp/docs"
5
    }
6
  }
7
}

Google Vertex AI models compatibility

Add Google Vertex AI API keys

There are 2 ways to add your Google Vertex AI credentials to your requests:

Via UI

Google credentials are little bit tricky. You can follow this video to get your credentials:

1

Go to the Providers page

Go to Providers page

2

Add your Vertex AI credentials.

3

(Optional) Choose models you want to use credits

You can choose the models you want to use your credits with. Just simply type the model ID from the Models page and copy the model id and paste it in the Available models. Press Enter to add the model.

Leave it empty to apply your credentials for all Vertex AI models.

Via Code

Add customer_credentials parameter in your request body to use your own Vertex AI credits.

1
{
2
  // Rest of the request body
3
  "customer_credentials": {
4
    "google_vertex_ai": {
5
      "vertex_ai_project": "your-project",
6
      "vertex_ai_location": "your-location",
7
      "vertex_ai_credentials": credentials // a JSON object, you could also pass an entire JSON object here
8
    }
9
  }
10
}

Override credentials for a particular model. (Optional)

One-off credential overrides. Instead of using what is uploaded for each provider, this targets credentials for individual models.

1
{
2
  // Rest of the request body
3
  "customer_credentials": {
4
    "google_vertex_ai": {
5
      "vertex_ai_project": "your-project",
6
      "vertex_ai_location": "your-location",
7
      "vertex_ai_credentials": credentials // a JSON object, you could also pass an entire JSON object here
8
    }
9
  },
10
  "credential_override": {
11
    "vertex_ai_beta/gemini-1.0-pro-001":{ // override for a specific model.
12
      "vertex_ai_project": "your-project",
13
      "vertex_ai_location": "your-location",
14
      "vertex_ai_credentials": another_credentials // a JSON object, you could also pass an entire JSON object here
15
    }
16
  }
17
}

Requirements

1. Ensure your deployment name matches those listed on our Models page.
2. Confirm that your models are available in the region specified by your credentials.
   Credentials validation is not supported. Failure to comply with these requirements will result in errors!

How to get your Vertex AI credentials

1. Run gcloud init to initialize the gcloud CLI.
2. Run gcloud info to get the path of your credential. You will see a line like
   User Config Directory: [/home/USERNAME/.config/gcloud]
3. You will find a file named application_default_credentials.json in the path you got from the previous step. This file contains your credentials.
4. Load this json file into the vertex_ai_credentials field in the request body.

For example:

1
with open("./credentials/google/application_default_credentials.json", "r") as f:
2
    credentials = json.load(f)

Full request example

1
from openai import OpenAI
2
3
client = OpenAI(
4
    base_url="https://api.respan.ai/api/",
5
    api_key="YOUR_RESPAN_API_KEY",
6
)
7
8
response = client.chat.completions.create(
9
    model="gpt-4o",
10
    messages=[{"role":"user", "content":"Tell me a long story"}],
11
    extra_body={"customer_credentials": {
12
                  "google_vertex_ai": {
13
                                "vertex_ai_project": "your-project",
14
                                "vertex_ai_location": "your-location",
15
                                "vertex_ai_credentials": credentials # a JSON object, you could also pass an entire JSON object here
16
                  }
17
                }
18
              }
19
)