Vertex AI (tracing) | Respan Docs

Vertex AI is Google Cloud’s managed platform for foundation models, including Gemini and third-party models. Respan captures Vertex AI generation, streaming, chat, prompt, completion, token, and workflow spans.

This tracing setup uses your Google Cloud credentials directly. To route requests through Respan instead, use the Vertex AI gateway setup.

Set up Respan

Create an account at platform.respan.ai and grab an API key.

Run npx @respan/cli setup to set up with your coding agent.

Use Respan Gateway

See Vertex AI gateway setup to route calls through the Respan gateway.

Example projects

Example projects root

Setup

Install packages

$ pip install respan-ai respan-instrumentation-vertexai google-cloud-aiplatform

Set environment variables

$ export GOOGLE_APPLICATION_CREDENTIALS="path/to/your/gcp-key.json"
$ export GOOGLE_CLOUD_PROJECT="your-gcp-project"
$ export GOOGLE_CLOUD_LOCATION="us-central1"
$ export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

Google Cloud credentials are used for Vertex AI requests. RESPAN_API_KEY exports traces to Respan.

Initialize and run

1 import vertexai
2 from vertexai.generative_models import GenerativeModel
3 from respan import Respan
4 from respan_instrumentation_vertexai import VertexAIInstrumentor
5 
6 respan = Respan(instrumentations=[VertexAIInstrumentor()])
7 
8 vertexai.init(project="your-gcp-project", location="us-central1")
9 
10 model = GenerativeModel("gemini-2.0-flash")
11 response = model.generate_content("Say hello in three languages.")
12 print(response.text)
13 respan.flush()

View your trace

Open the Traces page to see Vertex AI generate-content, streaming, and chat spans.

Configuration

Parameter	Type	Default	Description
`api_key` / `apiKey`	`str \| None` / `string \| undefined`	`RESPAN_API_KEY`	Respan API key used to export traces.
`base_url` / `baseURL`	`str \| None` / `string \| undefined`	`RESPAN_BASE_URL`	Optional Respan trace export API URL.
`instrumentations`	`list` / `RespanInstrumentation[]`	`[]`	Include `VertexAIInstrumentor()` or `new VertexAIInstrumentor()` to activate Vertex AI tracing.

Attributes

In Respan()

Set defaults at initialization. These apply to all spans.

1 from respan import Respan
2 from respan_instrumentation_vertexai import VertexAIInstrumentor
3 
4 respan = Respan(
5     instrumentations=[VertexAIInstrumentor()],
6     customer_identifier="user_123",
7     metadata={"service": "vertex-api", "version": "1.0.0"},
8 )

With propagate_attributes

Override per-request attributes using a context scope.

1 from respan import Respan, propagate_attributes
2 from respan_instrumentation_vertexai import VertexAIInstrumentor
3 
4 respan = Respan(instrumentations=[VertexAIInstrumentor()])
5 
6 async def handle_request(user_id: str, question: str):
7     with propagate_attributes(
8         customer_identifier=user_id,
9         thread_identifier="conv_abc_123",
10         metadata={"plan": "pro"},
11     ):
12         response = model.generate_content(question)
13         print(response.text)

Attribute	Type	Description
`customer_identifier`	`str` / `string`	Identifies the end user in Respan analytics.
`thread_identifier`	`str` / `string`	Groups related messages into a conversation.
`metadata`	`dict` / `Record<string, unknown>`	Custom key-value pairs.

Captured operations

GenerativeModel.generateContent()
GenerativeModel.generateContentStream()
ChatSession.sendMessage()
ChatSession.sendMessageStream()