LiteLLM (gateway)

Route LiteLLM requests through the Respan gateway to use 250+ models from different providers. Only your RESPAN_API_KEY is needed - no separate provider keys required.

Setup

Install packages

$ pip install respan-ai respan-instrumentation-litellm litellm

Set environment variables

$ export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

No provider key needed - the Respan gateway handles provider authentication.

Point LiteLLM to the Respan gateway

1 import os
2 import litellm
3 from respan import Respan
4 from respan_instrumentation_litellm import LiteLLMInstrumentor
5 
6 respan = Respan(instrumentations=[LiteLLMInstrumentor()])
7 
8 response = litellm.completion(
9     api_key=os.environ["RESPAN_API_KEY"],
10     api_base="https://api.respan.ai/api",
11     model="gpt-4.1-nano",
12     messages=[{"role": "user", "content": "Hello!"}],
13 )
14 print(response.choices[0].message.content)
15 respan.flush()

Switch models

Change the model parameter to use 250+ models from different providers through the same gateway.

1 litellm.completion(api_key=..., api_base="https://api.respan.ai/api", model="claude-sonnet-4-5-20250929", messages=messages)
2 litellm.completion(api_key=..., api_base="https://api.respan.ai/api", model="gemini-2.5-flash", messages=messages)

See the full model list.

Respan parameters

Pass additional Respan parameters via extra_body for gateway features.

1 response = litellm.completion(
2     api_key=os.environ["RESPAN_API_KEY"],
3     api_base="https://api.respan.ai/api",
4     model="gpt-4.1-nano",
5     messages=[{"role": "user", "content": "Hello!"}],
6     extra_body={
7         "customer_identifier": "user-123",
8         "metadata": {"session_id": "abc123"},
9         "thread_identifier": "conversation_456",
10     },
11 )

See Respan params & metadata for the full list.

1	import os
2	import litellm
3	from respan import Respan
4	from respan_instrumentation_litellm import LiteLLMInstrumentor
5
6	respan = Respan(instrumentations=[LiteLLMInstrumentor()])
7
8	response = litellm.completion(
9	api_key=os.environ["RESPAN_API_KEY"],
10	api_base="https://api.respan.ai/api",
11	model="gpt-4.1-nano",
12	messages=[{"role": "user", "content": "Hello!"}],
13	)
14	print(response.choices[0].message.content)
15	respan.flush()

1	litellm.completion(api_key=..., api_base="https://api.respan.ai/api", model="claude-sonnet-4-5-20250929", messages=messages)
2	litellm.completion(api_key=..., api_base="https://api.respan.ai/api", model="gemini-2.5-flash", messages=messages)