Integrate LiteLLM with Respan for observability and logging
Overview
LiteLLM provides a unified interface for calling multiple LLM providers. The Respan integration offers two approaches: route requests through the Respan gateway for centralized management, or export logs through LiteLLM callbacks for async monitoring.
The gateway mode provides automatic logging, fallbacks, and load balancing. Pass tracking data using extra_body parameters for customer identification, workflow naming, and custom metadata.
How it works
Install litellm and keywordsai-exporter-litellm. For gateway mode, set api_base to https://api.keywordsai.co/api and use your Respan API key.
Pass Respan parameters like span_workflow_name, span_name, and customer_identifier through extra_body. Every request is logged with token usage, cost, and latency in the Respan dashboard.
python
import litellm
response = litellm.completion(
api_key="YOUR_RESPAN_API_KEY",
api_base="https://api.keywordsai.co/api",
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello!"}],
extra_body={
"span_workflow_name": "simple_logging",
"span_name": "single_log",
"customer_identifier": "user-123",
},
)