Ollama (tracing)

Ollama lets you run large language models locally. respan-instrumentation-ollama instruments the official Ollama Python client and emits chat, generation, and embedding spans into the Respan tracing pipeline.

Set up Respan

Sign up - Create an account at platform.respan.ai
Create an API key - Generate one on the API keys page

Paste this into your coding agent:


Help me set up Respan tracing with the Ollama integration.
Follow https://www.respan.ai/docs/integrations/ollama.md

Setup

Install packages

Terminal


pip install respan-ai respan-instrumentation-ollama ollama python-dotenv

ollama is the official Ollama Python client. A running Ollama server is required for real model calls.

Set environment variables

Terminal


export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

Optional:

Terminal


export RESPAN_BASE_URL="https://api.respan.ai/api"
export OLLAMA_HOST="http://localhost:11434"
export OLLAMA_MODEL="llama3.2"

Initialize and run

Python


import os

from dotenv import load_dotenv
from ollama import Client
from respan import Respan, workflow
from respan_instrumentation_ollama import OllamaInstrumentor

load_dotenv()

respan = Respan(
    api_key=os.environ["RESPAN_API_KEY"],
    base_url=os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api"),
    instrumentations=[OllamaInstrumentor()],
)
client = Client(host=os.getenv("OLLAMA_HOST"))


@workflow(name="ollama_chat")
def run_chat() -> str:
    response = client.chat(
        model=os.getenv("OLLAMA_MODEL", "llama3.2"),
        messages=[{"role": "user", "content": "Reply with one concise sentence."}],
    )
    return response["message"]["content"]


try:
    print(run_chat())
finally:
    respan.shutdown()

View your trace

Open the Traces page and search for the workflow name ollama_chat.

Configuration

OllamaInstrumentor() does not require additional arguments. It patches the official Ollama Client and AsyncClient chat, generate, embed, and embeddings methods.

Attributes

Attach customer identifiers, thread IDs, workflow names, and metadata to Ollama calls with propagate_attributes.

Python


from respan import propagate_attributes

with propagate_attributes(
    customer_identifier="user_123",
    thread_identifier="conversation_456",
    trace_group_identifier="ollama_support_chat.workflow",
    metadata={"plan": "pro", "workflow_name": "ollama_support_chat.workflow"},
):
    response = client.chat(
        model="llama3.2",
        messages=[{"role": "user", "content": "Summarize our support policy."}],
    )

Attribute	Type	Description
`customer_identifier`	`str`	Identifies the end user in Respan analytics.
`thread_identifier`	`str`	Groups related messages into a conversation.
`trace_group_identifier`	`str`	Groups spans by workflow name.
`metadata`	`dict`	Custom key-value pairs merged with default metadata.

Examples

Streaming generation

Python


stream = client.generate(
    model="llama3.2",
    prompt="Write a short haiku about observability.",
    stream=True,
)

for chunk in stream:
    print(chunk["response"], end="", flush=True)

Embeddings

Python


response = client.embed(
    model="nomic-embed-text",
    input=["Respan records traces for AI systems."],
)
print(len(response["embeddings"][0]))