Custom Provider (gateway)

Set up a custom or self-hosted LLM provider with the Respan gateway.

Custom Provider (gateway)

Set up a custom or self-hosted LLM provider with the Respan gateway.

Set up Respan

Sign up — Create an account at platform.respan.ai
Create an API key — Generate one on the API keys page
Add credits or a provider key — Add credits on the Credits page or connect your own provider key on the Integrations page

Use AI

Add the Docs MCP to your AI coding tool to get help building with Respan. No API key needed.

1 {
2   "mcpServers": {
3     "respan-docs": {
4       "url": "https://mcp.respan.ai/mcp/docs"
5     }
6   }
7 }

This section is for Respan LLM gateway users.

Use Respan Gateway to route requests to your own self-hosted or custom LLM provider while keeping unified observability (logs, cost, latency, and reliability metrics) in Respan.

Prerequisites

A Respan API key
A running LLM endpoint that exposes an OpenAI-compatible API

Create a custom provider

Add your provider base URL, API key, and custom models in Respan.

Supported SDKs / integrations

✅ Supported Frameworks

❌ Unsupported Frameworks

Configuration

There are 2 ways to use your custom provider through Respan:

Via UI (Global)

Navigate to Providers

Go to the Providers page and click Add Custom Provider.

Add your custom provider

Provide a provider name, base URL, and API key if your endpoint requires one.

Add a custom model

Create a model entry that uses your custom provider. Use the model name your endpoint expects.

Via code (Gateway request)

After you create the custom provider and model in Respan, call the model through the gateway with your Respan API key.

OpenAI Python SDK

1 from openai import OpenAI
2 
3 client = OpenAI(
4     api_key="YOUR_RESPAN_API_KEY",
5     base_url="https://api.respan.ai/api/",
6 )
7 
8 response = client.chat.completions.create(
9     model="MY_CUSTOM_MODEL_ID",
10     messages=[{"role": "user", "content": "Hello!"}],
11 )
12 print(response.choices[0].message.content)

Supported models

Custom model IDs are managed on the Respan Models page. Use the exact model ID you create there in your gateway requests.

Log custom provider requests

If you are not using the Gateway to proxy requests, you can still log requests to Respan asynchronously.

Python

1 import requests
2 
3 url = "https://api.respan.ai/api/request-logs/create/"
4 payload = {
5   "model": "MY_CUSTOM_MODEL_ID",
6   "prompt_messages": [
7     {
8       "role": "user",
9       "content": "Hello!"
10     }
11   ],
12   "completion_message": {
13     "role": "assistant",
14     "content": "Hello from my custom provider."
15   },
16   "cost": 0.001,
17   "generation_time": 1.2,
18   "customer_params": {
19     "customer_identifier": "user_123"
20   }
21 }
22 headers = {
23     "Authorization": "Bearer YOUR_RESPAN_API_KEY",
24     "Content-Type": "application/json",
25 }
26 
27 response = requests.post(url, headers=headers, json=payload)

Custom Providers API

Create, list, update, and delete custom providers programmatically.

Set up Respan

Sign up — Create an account at platform.respan.ai
Create an API key — Generate one on the API keys page
Add credits or a provider key — Add credits on the Credits page or connect your own provider key on the Integrations page

Use AI

Add the Docs MCP to your AI coding tool to get help building with Respan. No API key needed.

1 {
2   "mcpServers": {
3     "respan-docs": {
4       "url": "https://mcp.respan.ai/mcp/docs"
5     }
6   }
7 }

This section is for Respan LLM gateway users.

Use Respan Gateway to route requests to your own self-hosted or custom LLM provider while keeping unified observability (logs, cost, latency, and reliability metrics) in Respan.

Prerequisites

A Respan API key
A running LLM endpoint that exposes an OpenAI-compatible API

Create a custom provider

Add your provider base URL, API key, and custom models in Respan.

Supported SDKs / integrations

✅ Supported Frameworks

❌ Unsupported Frameworks

Configuration

There are 2 ways to use your custom provider through Respan:

Via UI (Global)

Navigate to Providers

Go to the Providers page and click Add Custom Provider.

Add your custom provider

Provide a provider name, base URL, and API key if your endpoint requires one.

Add a custom model

Create a model entry that uses your custom provider. Use the model name your endpoint expects.

Via code (Gateway request)

After you create the custom provider and model in Respan, call the model through the gateway with your Respan API key.

OpenAI Python SDK

1 from openai import OpenAI
2 
3 client = OpenAI(
4     api_key="YOUR_RESPAN_API_KEY",
5     base_url="https://api.respan.ai/api/",
6 )
7 
8 response = client.chat.completions.create(
9     model="MY_CUSTOM_MODEL_ID",
10     messages=[{"role": "user", "content": "Hello!"}],
11 )
12 print(response.choices[0].message.content)

Supported models

Custom model IDs are managed on the Respan Models page. Use the exact model ID you create there in your gateway requests.

Log custom provider requests

If you are not using the Gateway to proxy requests, you can still log requests to Respan asynchronously.

Python

1 import requests
2 
3 url = "https://api.respan.ai/api/request-logs/create/"
4 payload = {
5   "model": "MY_CUSTOM_MODEL_ID",
6   "prompt_messages": [
7     {
8       "role": "user",
9       "content": "Hello!"
10     }
11   ],
12   "completion_message": {
13     "role": "assistant",
14     "content": "Hello from my custom provider."
15   },
16   "cost": 0.001,
17   "generation_time": 1.2,
18   "customer_params": {
19     "customer_identifier": "user_123"
20   }
21 }
22 headers = {
23     "Authorization": "Bearer YOUR_RESPAN_API_KEY",
24     "Content-Type": "application/json",
25 }
26 
27 response = requests.post(url, headers=headers, json=payload)

Custom Providers API

Create, list, update, and delete custom providers programmatically.

1	{
2	"mcpServers": {
3	"respan-docs": {
4	"url": "https://mcp.respan.ai/mcp/docs"
5	}
6	}
7	}

1	from openai import OpenAI
2
3	client = OpenAI(
4	api_key="YOUR_RESPAN_API_KEY",
5	base_url="https://api.respan.ai/api/",
6	)
7
8	response = client.chat.completions.create(
9	model="MY_CUSTOM_MODEL_ID",
10	messages=[{"role": "user", "content": "Hello!"}],
11	)
12	print(response.choices[0].message.content)

1	import requests
2
3	url = "https://api.respan.ai/api/request-logs/create/"
4	payload = {
5	"model": "MY_CUSTOM_MODEL_ID",
6	"prompt_messages": [
7	{
8	"role": "user",
9	"content": "Hello!"
10	}
11	],
12	"completion_message": {
13	"role": "assistant",
14	"content": "Hello from my custom provider."
15	},
16	"cost": 0.001,
17	"generation_time": 1.2,
18	"customer_params": {
19	"customer_identifier": "user_123"
20	}
21	}
22	headers = {
23	"Authorization": "Bearer YOUR_RESPAN_API_KEY",
24	"Content-Type": "application/json",
25	}
26
27	response = requests.post(url, headers=headers, json=payload)