Provider: Custom / Self-hosted

Bring your own OpenAI-compatible endpoint and call it through Respan Gateway to get unified observability (logs, cost, latency, reliability) alongside the rest of your models.

Quick setup

Call your custom model

The base URL is https://api.respan.ai/api and the only key needed is your RESPAN_API_KEY. Replace MY_CUSTOM_MODEL_ID with the model ID you created.

1
from openai import OpenAI
2
3
client = OpenAI(
4
    api_key="YOUR_RESPAN_API_KEY",
5
    base_url="https://api.respan.ai/api",
6
)
7
8
response = client.chat.completions.create(
9
    model="MY_CUSTOM_MODEL_ID",
10
    messages=[{"role": "user", "content": "Hello!"}],
11
)
12
print(response.choices[0].message.content)

More integrations

Custom models work with every Respan gateway integration over the OpenAI-compatible API:

• OpenAI SDK
• OpenAI Agents SDK
• LangChain
• LangGraph
• LlamaIndex
• Pydantic AI
• Respan native (OTel)

Override credentials per model (Optional)

Use credential_override when a single request should route to a different endpoint or API key than the one configured on your custom provider.

1
{
2
  "credential_override": {
3
    "MY_CUSTOM_MODEL_ID": {
4
      "api_key": "YOUR_CUSTOM_API_KEY",
5
      "api_base": "https://your-endpoint.example.com/v1"
6
    }
7
  }
8
}

Log without proxying (Optional)

Already calling your endpoint directly? Send logs to Respan asynchronously to track cost, latency, and performance for those external calls.

1
import requests
2
3
requests.post(
4
    "https://api.respan.ai/api/request-logs/create/",
5
    headers={
6
        "Authorization": "Bearer YOUR_RESPAN_API_KEY",
7
        "Content-Type": "application/json",
8
    },
9
    json={
10
        "model": "MY_CUSTOM_MODEL_ID",
11
        "prompt_messages": [{"role": "user", "content": "Hello!"}],
12
        "completion_message": {"role": "assistant", "content": "Hello from my custom provider."},
13
        "cost": 0.001,
14
        "generation_time": 1.2,
15
        "customer_params": {"customer_identifier": "user_123"},
16
    },
17
)

See the logging guide for the full setup.