TracingLLM SDKs

Ollama (tracing)

Ollama lets you run large language models locally. respan-instrumentation-ollama instruments the official Ollama Python client and emits chat, generation, and embedding spans into the Respan tracing pipeline.

  1. Sign up - Create an account at platform.respan.ai
  2. Create an API key - Generate one on the API keys page

See Ollama gateway setup to route Ollama model calls through the Respan gateway.

Setup

1

Install packages

pip install respan-ai respan-instrumentation-ollama ollama python-dotenv

ollama is the official Ollama Python client. A running Ollama server is required for real model calls.

2

Set environment variables

export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

Optional:

export RESPAN_BASE_URL="https://api.respan.ai/api"
export OLLAMA_HOST="http://localhost:11434"
export OLLAMA_MODEL="llama3.2"
3

Initialize and run

import os
from dotenv import load_dotenv
from ollama import Client
from respan import Respan, workflow
from respan_instrumentation_ollama import OllamaInstrumentor
load_dotenv()
respan = Respan(
    api_key=os.environ["RESPAN_API_KEY"],
    base_url=os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api"),
    instrumentations=[OllamaInstrumentor()],
)
client = Client(host=os.getenv("OLLAMA_HOST"))
@workflow(name="ollama_chat")
def run_chat() -> str:
    response = client.chat(
        model=os.getenv("OLLAMA_MODEL", "llama3.2"),
        messages=[{"role": "user", "content": "Reply with one concise sentence."}],
    )
    return response["message"]["content"]
try:
    print(run_chat())
finally:
    respan.shutdown()
4

View your trace

Open the Traces page and search for the workflow name ollama_chat.

Configuration

OllamaInstrumentor() does not require additional arguments. It patches the official Ollama Client and AsyncClient chat, generate, embed, and embeddings methods.

Attributes

Attach customer identifiers, thread IDs, workflow names, and metadata to Ollama calls with propagate_attributes.

from respan import propagate_attributes
with propagate_attributes(
    customer_identifier="user_123",
    thread_identifier="conversation_456",
    trace_group_identifier="ollama_support_chat.workflow",
    metadata={"plan": "pro", "workflow_name": "ollama_support_chat.workflow"},
):
    response = client.chat(
        model="llama3.2",
        messages=[{"role": "user", "content": "Summarize our support policy."}],
    )
AttributeTypeDescription
customer_identifierstrIdentifies the end user in Respan analytics.
thread_identifierstrGroups related messages into a conversation.
trace_group_identifierstrGroups spans by workflow name.
metadatadictCustom key-value pairs merged with default metadata.

Examples

Streaming generation

stream = client.generate(
    model="llama3.2",
    prompt="Write a short haiku about observability.",
    stream=True,
)
for chunk in stream:
    print(chunk["response"], end="", flush=True)

Embeddings

response = client.embed(
    model="nomic-embed-text",
    input=["Respan records traces for AI systems."],
)
print(len(response["embeddings"][0]))