HuggingFace | Respan Docs

HuggingFace is the leading platform for machine learning models, datasets, and tools. The Transformers library provides access to thousands of pretrained models for text generation, classification, translation, and more. Respan gives you full observability over every pipeline run, model inference, and token output — and gateway routing through the OpenAI-compatible Respan endpoint when you want to mix HuggingFace with other providers.

Set up Respan

Create an account at platform.respan.ai and grab an API key. For gateway, also add credits or a provider key.

Run npx @respan/cli setup to set up with your coding agent.

Example projects

Python examples

Tracing

Gateway

Setup

Install packages

$ pip install respan-ai opentelemetry-instrumentation-transformers transformers

Set environment variables

$ export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

RESPAN_API_KEY is used to export traces to Respan.

Initialize and run

1 from transformers import pipeline
2 from respan import Respan
3 from opentelemetry.instrumentation.transformers import TransformersInstrumentor
4 
5 respan = Respan(instrumentations=[TransformersInstrumentor()])
6 
7 generator = pipeline("text-generation", model="gpt2")
8 
9 output = generator("Say hello in three languages:", max_new_tokens=100)
10 print(output[0]["generated_text"])
11 respan.flush()

View your trace

Open the Traces page to see your auto-instrumented inference spans with model names, tokens, and latency.

Configuration

Parameter	Type	Default	Description
`api_key`	`str \| None`	`None`	Falls back to `RESPAN_API_KEY` env var.
`base_url`	`str \| None`	`None`	Falls back to `RESPAN_BASE_URL` env var.
`instrumentations`	`list`	`[]`	Plugin instrumentations to activate (e.g. `TransformersInstrumentor()`).
`customer_identifier`	`str \| None`	`None`	Default customer identifier for all spans.
`metadata`	`dict \| None`	`None`	Default metadata attached to all spans.
`environment`	`str \| None`	`None`	Environment tag (e.g. `"production"`).

Attributes

In Respan()

1 from respan import Respan
2 from opentelemetry.instrumentation.transformers import TransformersInstrumentor
3 
4 respan = Respan(
5     instrumentations=[TransformersInstrumentor()],
6     customer_identifier="user_123",
7     metadata={"service": "hf-api", "version": "1.0.0"},
8 )

With propagate_attributes

1 from respan import Respan, propagate_attributes
2 from opentelemetry.instrumentation.transformers import TransformersInstrumentor
3 
4 respan = Respan(instrumentations=[TransformersInstrumentor()])
5 
6 def handle_request(user_id: str, prompt: str):
7     with propagate_attributes(
8         customer_identifier=user_id,
9         thread_identifier="conv_abc_123",
10         metadata={"plan": "pro"},
11     ):
12         output = generator(prompt, max_new_tokens=100)
13         print(output[0]["generated_text"])

Attribute	Type	Description
`customer_identifier`	`str`	Identifies the end user in Respan analytics.
`thread_identifier`	`str`	Groups related messages into a conversation.
`metadata`	`dict`	Custom key-value pairs. Merged with default metadata.