Vertex AI (tracing)

Vertex AI is Google Cloud’s managed platform for foundation models, including Gemini and third-party models. Respan captures Vertex AI generation, streaming, chat, prompt, completion, token, and workflow spans.

This tracing setup uses your Google Cloud credentials directly. To route requests through Respan instead, use the Vertex AI gateway setup .

Set up Respan

Create an account at platform.respan.ai and grab an API key .

Paste this into your coding agent:


Help me set up Respan tracing with the Vertex AI integration.
Follow https://www.respan.ai/docs/integrations/vertex-ai.md

Setup

Install packages

Terminal


pip install respan-ai respan-instrumentation-vertexai google-cloud-aiplatform

Set environment variables

Terminal


export GOOGLE_APPLICATION_CREDENTIALS="path/to/your/gcp-key.json"
export GOOGLE_CLOUD_PROJECT="your-gcp-project"
export GOOGLE_CLOUD_LOCATION="us-central1"
export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

Google Cloud credentials are used for Vertex AI requests. RESPAN_API_KEY exports traces to Respan.

Initialize and run

Python


import vertexai
from vertexai.generative_models import GenerativeModel
from respan import Respan
from respan_instrumentation_vertexai import VertexAIInstrumentor

respan = Respan(instrumentations=[VertexAIInstrumentor()])

vertexai.init(project="your-gcp-project", location="us-central1")

model = GenerativeModel("gemini-2.0-flash")
response = model.generate_content("Say hello in three languages.")
print(response.text)

View your trace

Open the Traces page to see Vertex AI generate-content, streaming, and chat spans.

Configuration

Parameter	Type	Default	Description
`api_key` / `apiKey`	`str \| None` / `string \| undefined`	`RESPAN_API_KEY`	Respan API key used to export traces.
`base_url` / `baseURL`	`str \| None` / `string \| undefined`	`RESPAN_BASE_URL`	Optional Respan trace export API URL.
`instrumentations`	`list` / `RespanInstrumentation[]`	`[]`	Include `VertexAIInstrumentor()` or `new VertexAIInstrumentor()` to activate Vertex AI tracing.

Attributes

In Respan()

Set defaults at initialization. These apply to all spans.

Python


from respan import Respan
from respan_instrumentation_vertexai import VertexAIInstrumentor

respan = Respan(
    instrumentations=[VertexAIInstrumentor()],
    customer_identifier="user_123",
    metadata={"service": "vertex-api", "version": "1.0.0"},
)

With propagate_attributes

Override per-request attributes using a context scope.

Python


from respan import Respan, propagate_attributes
from respan_instrumentation_vertexai import VertexAIInstrumentor

respan = Respan(instrumentations=[VertexAIInstrumentor()])

async def handle_request(user_id: str, question: str):
    with propagate_attributes(
        customer_identifier=user_id,
        thread_identifier="conv_abc_123",
        metadata={"plan": "pro"},
    ):
        response = model.generate_content(question)
        print(response.text)

Attribute	Type	Description
`customer_identifier`	`str` / `string`	Identifies the end user in Respan analytics.
`thread_identifier`	`str` / `string`	Groups related messages into a conversation.
`metadata`	`dict` / `Record<string, unknown>`	Custom key-value pairs.

Captured operations

GenerativeModel.generateContent()
GenerativeModel.generateContentStream()
ChatSession.sendMessage()
ChatSession.sendMessageStream()