Open-source gateway
Call any model from the SDK you already use.
The gateway sits between your code and the AI APIs you call. Point the OpenAI, Anthropic or Gemini SDK at it, ask for any model, and it translates the request to that model’s provider and the answer back.
from anthropic import Anthropic
client = Anthropic(base_url="http://localhost:8080", api_key="unused")
reply = client.messages.create(
model="gpt-6-luna", max_tokens=256,
messages=[{"role": "user", "content": "Hello from the Anthropic SDK"}])
print(reply.content[0].text)It is licensed MIT or Apache-2.0. It runs as one binary, as a Python package (a server, or inside your process), or as a Docker image.
What it does
| Feature | For example |
|---|---|
| Any SDK, any model | The Anthropic SDK calls GPT, and the OpenAI SDK calls Claude. Streaming, tools, images and reasoning are translated. |
| Images, speech, video, embeddings | One OpenAI client generates images, speech and transcriptions, and client.videos.create makes a Google Veo video. |
| Realtime voice | An OpenAI Realtime client talks to Gemini Live, and a Gemini Live client talks to OpenAI. |
| Tools and MCP | Put MCP servers and built-in tool vendors (web search, browsers, code sandboxes, memory) behind the gateway. It runs their tools inside a chat request, so any model can use them. |
| Routing and fallback | Clients ask for a route such as smart. The gateway falls back to the next candidate on errors, picks the cheapest or fastest, or races a slow call against a second provider. |
| Request log with cost | The web UI at /ui/ lists every request with its provider, latency, tokens and cost. |
| Respan tracing | The respan plugin sends each request to Respan as a trace. |
16 vendors are built in, including OpenAI, Anthropic, Gemini, Groq, DeepSeek, Mistral, xAI, OpenRouter and Ollama. Any OpenAI-compatible endpoint works with a base URL. Every feature is a plugin on a public Rust API, and you can write your own.
How it fits with Respan
| Hosted Respan gateway | Respan Gateway (open source) | |
|---|---|---|
| Runs | At https://api.respan.ai/api/ | On your machine or server |
| Provider keys | Respan credits, or your keys stored in Respan | Your keys, in your environment or config file |
| Setup | Change your client’s base URL | Install, add keys, run |
You can use both. Add your Respan API key as a provider (vendor = "respan", or RESPAN_API_KEY in the environment), and turn on the respan plugin to trace every request in Respan. See Providers and Plugins.