LangChain (gateway)

Route LangChain’s underlying LLM calls through the Respan gateway to use 250+ models from different providers. Only your Respan API key is needed — no separate provider keys required.

Setup

Install packages

$ pip install respan-tracing respan-instrumentation-langchain langchain langchain-openai python-dotenv

Set environment variables

$ export RESPAN_API_KEY="YOUR_RESPAN_API_KEY"

No OPENAI_API_KEY needed — the Respan gateway handles provider authentication.

Point LangChain to the Respan gateway

1 import os
2 from langchain_openai import ChatOpenAI
3 
4 llm = ChatOpenAI(
5     api_key=os.environ["RESPAN_API_KEY"],
6     base_url=os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api"),
7     model="gpt-4.1-nano",
8 )
9 
10 response = llm.invoke("Say hello through the Respan gateway.")
11 print(response.content)

Switch models

Change the model parameter to use 250+ models from different providers through the same gateway.

1 llm = ChatOpenAI(api_key=..., base_url="https://api.respan.ai/api", model="claude-sonnet-4-5-20250929")
2 llm = ChatOpenAI(api_key=..., base_url="https://api.respan.ai/api", model="gemini-2.5-flash")

See the full model list.

Respan parameters

Pass additional Respan parameters via extra_body for gateway features.

1 llm = ChatOpenAI(
2     base_url="https://api.respan.ai/api/",
3     api_key=os.environ["RESPAN_API_KEY"],
4     model="gpt-4.1-nano",
5     extra_body={
6         "customer_identifier": "user_123",
7         "fallback_models": ["gpt-3.5-turbo"],
8         "metadata": {"session_id": "abc123"},
9         "thread_identifier": "conversation_456",
10     },
11 )

See Respan params & metadata for the full list.

1	import os
2	from langchain_openai import ChatOpenAI
3
4	llm = ChatOpenAI(
5	api_key=os.environ["RESPAN_API_KEY"],
6	base_url=os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api"),
7	model="gpt-4.1-nano",
8	)
9
10	response = llm.invoke("Say hello through the Respan gateway.")
11	print(response.content)

1	llm = ChatOpenAI(api_key=..., base_url="https://api.respan.ai/api", model="claude-sonnet-4-5-20250929")
2	llm = ChatOpenAI(api_key=..., base_url="https://api.respan.ai/api", model="gemini-2.5-flash")

1	llm = ChatOpenAI(
2	base_url="https://api.respan.ai/api/",
3	api_key=os.environ["RESPAN_API_KEY"],
4	model="gpt-4.1-nano",
5	extra_body={
6	"customer_identifier": "user_123",
7	"fallback_models": ["gpt-3.5-turbo"],
8	"metadata": {"session_id": "abc123"},
9	"thread_identifier": "conversation_456",
10	},
11	)