Haystack (tracing)

Haystack is a Python framework for building production-ready LLM pipelines with retrieval, generation, routing, and evaluation components. Respan traces Haystack runs with respan-instrumentation-haystack, which activates the OpenInference Haystack instrumentor and exports pipeline, component, and LLM spans through the Respan tracing pipeline.

Set up Respan

Create an account at platform.respan.ai and grab an API key .

Paste this into your coding agent:


Help me set up Respan tracing with the Haystack integration.
Follow https://www.respan.ai/docs/integrations/haystack.md

Setup

Install packages

Terminal


pip install respan-ai respan-instrumentation-haystack haystack-ai

Set environment variables

Terminal


export OPENAI_API_KEY="YOUR_OPENAI_API_KEY"
export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"
export HAYSTACK_CONTENT_TRACING_ENABLED="true"

OPENAI_API_KEY is used by Haystack’s OpenAI component. RESPAN_API_KEY exports traces to Respan. HAYSTACK_CONTENT_TRACING_ENABLED lets Haystack include prompt and response content in spans.

Initialize and run

Python


from haystack import Pipeline
from haystack.components.builders import PromptBuilder
from haystack.components.generators import OpenAIGenerator
from respan import Respan
from respan_instrumentation_haystack import HaystackInstrumentor

respan = Respan(instrumentations=[HaystackInstrumentor()])

pipeline = Pipeline()
pipeline.add_component(
    "prompt_builder",
    PromptBuilder(template="Answer the following question: {{question}}"),
)
pipeline.add_component("generator", OpenAIGenerator(model="gpt-4o-mini"))
pipeline.connect("prompt_builder", "generator")

result = pipeline.run(
    {"prompt_builder": {"question": "What is the capital of France?"}}
)
print(result["generator"]["replies"][0])

View your trace

Open the Traces page to see your pipeline run with pipeline spans, component spans, LLM calls, and inputs/outputs.

Configuration

Parameter	Type	Default	Description
`api_key`	`str \| None`	`None`	Falls back to `RESPAN_API_KEY` env var.
`base_url`	`str \| None`	`None`	Falls back to `RESPAN_BASE_URL` env var.
`instrumentations`	`list`	`[]`	Plugin instrumentations to activate, such as `HaystackInstrumentor()`.
`customer_identifier`	`str \| None`	`None`	Default customer identifier for all spans.
`metadata`	`dict \| None`	`None`	Default metadata attached to all spans.
`environment`	`str \| None`	`None`	Environment tag, such as `"production"`.

HaystackInstrumentor(**kwargs) passes keyword arguments through to the underlying OpenInference Haystack instrumentor.

Attributes

In Respan()

Set defaults at initialization. These apply to all spans.

Python


from respan import Respan
from respan_instrumentation_haystack import HaystackInstrumentor

respan = Respan(
    instrumentations=[HaystackInstrumentor()],
    customer_identifier="user_123",
    metadata={"service": "rag-api", "version": "1.0.0"},
)

With propagate_attributes

Override per-request using a context scope.

Python


from respan import propagate_attributes

with propagate_attributes(
    customer_identifier="user_123",
    thread_identifier="conversation_abc",
    metadata={"plan": "pro"},
):
    result = pipeline.run(
        {"prompt_builder": {"question": "What is retrieval-augmented generation?"}}
    )
    print(result["generator"]["replies"][0])

Attribute	Type	Description
`customer_identifier`	`str`	Identifies the end user in Respan analytics.
`thread_identifier`	`str`	Groups related messages into a conversation.
`metadata`	`dict`	Custom key-value pairs. Merged with default metadata.

Decorators (optional)

Decorators are not required. Pipeline runs, components, and LLM calls are auto-traced by the instrumentor. Use @workflow and @task when you want to group multiple Haystack runs or add application-level steps around a pipeline.

Python


from respan import Respan, task, workflow
from respan_instrumentation_haystack import HaystackInstrumentor

respan = Respan(instrumentations=[HaystackInstrumentor()])

@task(name="answer_question")
def answer_question(question: str) -> str:
    result = pipeline.run({"prompt_builder": {"question": question}})
    return result["generator"]["replies"][0]

@workflow(name="haystack_qa")
def haystack_qa(question: str) -> str:
    return answer_question(question)

print(haystack_qa("What is the capital of France?"))

Examples

RAG pipeline

Retrieval and generation components are captured as separate spans.

Python


from haystack import Document, Pipeline
from haystack.components.builders import PromptBuilder
from haystack.components.generators import OpenAIGenerator
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore

document_store = InMemoryDocumentStore()
document_store.write_documents(
    [
        Document(content="Python was created by Guido van Rossum."),
        Document(content="Rust is focused on safety and performance."),
    ]
)

template = """
Documents:
{% for document in documents %}
- {{ document.content }}
{% endfor %}

Question: {{question}}
Answer:
"""

pipeline = Pipeline()
pipeline.add_component("retriever", InMemoryBM25Retriever(document_store=document_store))
pipeline.add_component("prompt_builder", PromptBuilder(template=template))
pipeline.add_component("generator", OpenAIGenerator(model="gpt-4o-mini"))
pipeline.connect("retriever.documents", "prompt_builder.documents")
pipeline.connect("prompt_builder", "generator")

result = pipeline.run(
    {
        "retriever": {"query": "Who created Python?"},
        "prompt_builder": {"question": "Who created Python?"},
    }
)
print(result["generator"]["replies"][0])

Gateway

See Haystack gateway setup to route this integration through the Respan gateway.