TracingLLM SDKs

OpenAI SDK (tracing)

The OpenAI SDK is the official client for OpenAI’s APIs, available for both Python and TypeScript/JavaScript. It supports Chat Completions and the Responses API. Respan gives you full observability over every OpenAI call, streamed response, and tool invocation — and gateway routing to 250+ models with prompt management.

Create an account at platform.respan.ai and grab an API key.

Run npx @respan/cli setup to set up with your coding agent.

See OpenAI SDK gateway setup to route this integration through the Respan gateway.

Setup

1

Install packages

pip install respan-ai respan-instrumentation-openai openai
2

Set environment variables

export OPENAI_API_KEY="YOUR_OPENAI_API_KEY"
export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

OPENAI_API_KEY is used for OpenAI requests. RESPAN_API_KEY is used to export traces to Respan.

3

Initialize and run

from openai import OpenAI
from respan import Respan
from respan_instrumentation_openai import OpenAIInstrumentor
respan = Respan(instrumentations=[OpenAIInstrumentor()])
client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4.1-nano",
    messages=[{"role": "user", "content": "Say hello in three languages."}],
)
print(response.choices[0].message.content)
4

View your trace

Open the Traces page to see your auto-instrumented LLM spans.

Configuration

ParameterTypeDefaultDescription
api_keystr | NoneNoneFalls back to RESPAN_API_KEY env var.
base_urlstr | NoneNoneFalls back to RESPAN_BASE_URL env var.
instrumentationslist[]Plugin instrumentations to activate (e.g. OpenAIInstrumentor()).
customer_identifierstr | NoneNoneDefault customer identifier for all spans.
metadatadict | NoneNoneDefault metadata attached to all spans.
environmentstr | NoneNoneEnvironment tag (e.g. "production").

Attributes

In Respan()

Set defaults at initialization — these apply to all spans.

from respan import Respan
from respan_instrumentation_openai import OpenAIInstrumentor
respan = Respan(
    instrumentations=[OpenAIInstrumentor()],
    customer_identifier="user_123",
    metadata={"service": "chat-api", "version": "1.0.0"},
)

With propagate_attributes

Override per-request using a context scope.

from openai import OpenAI
from respan import Respan, propagate_attributes
from respan_instrumentation_openai import OpenAIInstrumentor
respan = Respan(instrumentations=[OpenAIInstrumentor()])
client = OpenAI()
def handle_request(user_id: str, question: str):
    with propagate_attributes(
        customer_identifier=user_id,
        thread_identifier="conv_abc_123",
        metadata={"plan": "pro"},
    ):
        response = client.chat.completions.create(
            model="gpt-4.1-nano",
            messages=[{"role": "user", "content": question}],
        )
        print(response.choices[0].message.content)
AttributeTypeDescription
customer_identifierstrIdentifies the end user in Respan analytics.
thread_identifierstrGroups related messages into a conversation.
metadatadictCustom key-value pairs. Merged with default metadata.

Decorators (optional)

Decorators are not required. All OpenAI calls are auto-traced by the instrumentor. Use @workflow and @task (Python) or withWorkflow and withTask (TypeScript) to add structure when you want to group related calls into a named workflow with nested tasks.

from openai import OpenAI
from respan import Respan, workflow, task
from respan_instrumentation_openai import OpenAIInstrumentor
respan = Respan(instrumentations=[OpenAIInstrumentor()])
client = OpenAI()
@task(name="generate_outline")
def outline(topic: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4.1-nano",
        messages=[
            {"role": "system", "content": "Create a brief outline."},
            {"role": "user", "content": topic},
        ],
    )
    return response.choices[0].message.content
@workflow(name="content_pipeline")
def pipeline(topic: str):
    plan = outline(topic)
    response = client.chat.completions.create(
        model="gpt-4.1-nano",
        messages=[
            {"role": "system", "content": "Write content from this outline."},
            {"role": "user", "content": plan},
        ],
    )
    print(response.choices[0].message.content)
pipeline("Benefits of API gateways")

Examples

Streaming

Streaming responses are auto-traced like regular completions.

stream = client.chat.completions.create(
    model="gpt-4.1-nano",
    messages=[{"role": "user", "content": "Write a haiku about Python."}],
    stream=True,
)
for chunk in stream:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

Tool calls

Function calling is auto-traced. Wrap the workflow with @workflow and @task decorators for a structured trace tree.

import json
from openai import OpenAI
from respan import Respan, workflow, task
from respan_instrumentation_openai import OpenAIInstrumentor
respan = Respan(instrumentations=[OpenAIInstrumentor()])
client = OpenAI()
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the weather for a city.",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    }
]
@task(name="get_weather")
def get_weather(city: str) -> str:
    return f"Sunny, 72F in {city}"
@workflow(name="weather_assistant")
def run(question: str):
    messages = [{"role": "user", "content": question}]
    response = client.chat.completions.create(
        model="gpt-4.1-nano",
        messages=messages,
        tools=tools,
    )
    message = response.choices[0].message
    if message.tool_calls:
        messages.append(message)
        for tc in message.tool_calls:
            args = json.loads(tc.function.arguments)
            result = get_weather(**args)
            messages.append(
                {"role": "tool", "tool_call_id": tc.id, "content": result}
            )
        final = client.chat.completions.create(
            model="gpt-4.1-nano",
            messages=messages,
            tools=tools,
        )
        print(f"Answer: {final.choices[0].message.content}")
run("What's the weather in Paris?")

Structured output

JSON mode with Pydantic models is auto-traced.

from pydantic import BaseModel
from openai import OpenAI
client = OpenAI()
class MovieReview(BaseModel):
    title: str
    rating: int
    summary: str
    pros: list[str]
    cons: list[str]
response = client.beta.chat.completions.parse(
    model="gpt-4.1-nano",
    messages=[
        {"role": "system", "content": "You are a film critic. Rate movies 1-10."},
        {"role": "user", "content": "Review: The Matrix"},
    ],
    response_format=MovieReview,
)
result = response.choices[0].message.parsed
print(f"{result.title} - {result.rating}/10")