Braintrust

Braintrust is an AI evaluation and observability platform for experiments, spans, scores, and datasets. Respan’s Braintrust instrumentation registers as Braintrust’s background logger, translates Braintrust records into Respan’s tracing model, and sends them through the standard Respan OTLP pipeline.

Set up Respan

Create an account at platform.respan.ai and grab an API key .

Paste this into your coding agent:


Help me set up Respan tracing with the Braintrust integration.
Follow https://www.respan.ai/docs/integrations/braintrust.md

Setup

Install packages

Terminal


pip install respan-ai respan-instrumentation-braintrust braintrust

Set environment variables

Terminal


export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
export BRAINTRUST_API_KEY="YOUR_BRAINTRUST_API_KEY"

RESPAN_API_KEY exports traces to Respan. BRAINTRUST_API_KEY is used by Braintrust when you initialize a Braintrust project.

Initialize and run

Python


import braintrust
from respan import Respan
from respan_instrumentation_braintrust import BraintrustInstrumentor

WORKFLOW_NAME = "Braintrust Support Evaluation"

respan = Respan(
    app_name="braintrust-support-eval",
    instrumentations=[BraintrustInstrumentor()],
)

logger = braintrust.init_logger(
    project="Support Evaluation",
    async_flush=False,
)

try:
    with respan.propagate_attributes(
        group_identifier=WORKFLOW_NAME,
        custom_identifier="support-eval-001",
        metadata={"workflow_name": WORKFLOW_NAME, "framework": "braintrust"},
    ):
        with logger.start_span(name=f"{WORKFLOW_NAME}.workflow", type="eval") as root:
            with root.start_span(name="draft_answer", type="llm") as span:
                span.log(
                    input=[{
                        "role": "user",
                        "content": "Can I get a refund after 21 days?",
                    }],
                    output="Yes. Refunds are available within 30 days with a receipt.",
                    metadata={"model": "gpt-4o-mini"},
                    metrics={"prompt_tokens": 15, "completion_tokens": 16},
                    scores={"helpfulness": 0.95},
                )

            root.log(
                input={"case_id": "case-1024"},
                output={"decision": "eligible"},
                metadata={"workflow_name": WORKFLOW_NAME},
            )

        logger.flush()
finally:
    respan.shutdown()

View your trace

Open the Traces page to see your Braintrust workflow, child spans, scores, token usage, and metadata.

Workflow names

For traces that are easy to find in Respan, use a readable root span name and propagate the same workflow name:

Python


WORKFLOW_NAME = "Braintrust Scored Evaluation Workflow"

with respan.propagate_attributes(
    group_identifier=WORKFLOW_NAME,
    metadata={"workflow_name": WORKFLOW_NAME},
):
    with logger.start_span(name=f"{WORKFLOW_NAME}.workflow", type="eval") as root:
        ...

The instrumentation maps that value to trace_group_identifier and span_workflow_name, so root and child spans are recognizable by workflow name.

Span mapping

Braintrust span type	Respan log type
`llm`, `chat`	`chat`
`tool`, `function`	`tool`
`eval`, `automation`	`workflow`
`task`, `score`, `facet`, `preprocessor`	`task`

LLM spans include prompt messages, completion messages, model, and token usage when those fields are present in the Braintrust record.

Configuration

Parameter	Type	Default	Description
`instrumentations`	`list`	`[]`	Pass `[BraintrustInstrumentor()]` to activate Braintrust tracing.
`api_key`	`str \| None`	`None`	Respan API key. Falls back to `RESPAN_API_KEY`.
`base_url`	`str \| None`	`None`	Respan base URL. Falls back to `RESPAN_BASE_URL`.
`customer_identifier`	`str \| None`	`None`	Default customer identifier for all Braintrust spans.
`metadata`	`dict \| None`	`None`	Default metadata attached to all spans.
`environment`	`str \| None`	`None`	Environment tag, such as `"production"`.

Notes

BraintrustInstrumentor captures Braintrust records when they are logged and exports them when Braintrust flushes.
Use logger.flush() so buffered Braintrust records reach the Respan pipeline before the Python SDK drains spans on shutdown or process exit.
The older respan-exporter-braintrust package sends legacy payloads directly. Use respan-instrumentation-braintrust for the current tracing pipeline.