Realtime

Run voice and text sessions over WebSocket with OpenAI Realtime, Gemini Live and Mistral.

Realtime sessions are long-lived WebSocket connections. The gateway serves three realtime formats and translates between them, so an OpenAI Realtime client can talk to a Gemini Live model and the other way round. Each session is charged and logged once it ends.

Pick a model: filter by output type realtime or input type audio on the model catalog , or read Models.

Quickstart

import asyncio
import os
from openai import AsyncOpenAI

async def main():
    client = AsyncOpenAI(
        base_url="https://gateway.respan.ai/v1",
        api_key=os.environ["RESPAN_API_KEY"],
    )
    async with client.realtime.connect(model="gpt-realtime") as connection:
        await connection.session.update(
            session={"type": "realtime", "output_modalities": ["text"]}
        )
        await connection.conversation.item.create(
            item={
                "type": "message",
                "role": "user",
                "content": [{"type": "input_text", "text": "Say hello."}],
            }
        )
        await connection.response.create()

        async for event in connection:
            if event.type == "response.output_text.delta":
                print(event.delta, end="", flush=True)
            elif event.type == "response.done":
                break

asyncio.run(main())

import OpenAI from "openai";
import { OpenAIRealtimeWS } from "openai/realtime/ws"; // needs the `ws` package

const client = new OpenAI({
  baseURL: "https://gateway.respan.ai/v1",
  apiKey: process.env.RESPAN_API_KEY,
});

const rt = new OpenAIRealtimeWS({ model: "gpt-realtime" }, client);

rt.socket.on("open", () => {
  rt.send({ type: "session.update", session: { type: "realtime", output_modalities: ["text"] } });
  rt.send({
    type: "conversation.item.create",
    item: { type: "message", role: "user", content: [{ type: "input_text", text: "Say hello." }] },
  });
  rt.send({ type: "response.create" });
});

rt.on("response.output_text.delta", (event) => process.stdout.write(event.delta));
rt.on("response.done", () => rt.close());

The SDK opens wss://gateway.respan.ai/v1/realtime?model=gpt-realtime. The model in the URL is required: the gateway routes the session from it before any message arrives.

Endpoints

Any of the three formats can reach models on OpenAI, Gemini or Mistral. The session’s model is routed like a chat model, trying its attempts in turn.

Gemini Live

Gemini Live clients must put the model in the URL too. The google-genai SDK names the model only in its setup message, which arrives after the gateway has routed the session, so point your client at the full URL with ?model= set.

Gemini fixes its setup when the session starts, so a session.update that changes it closes the session with code 1008. See OSS realtime formats for other translation details.

Mistral realtime transcription

Mistral’s realtime endpoint only transcribes. An OpenAI Realtime transcription session can be answered by a Mistral model, and a Mistral client can transcribe with OpenAI or Gemini models. A conversation session sent to Mistral is refused.

Authentication

Send your Respan API key in any one of these:

Browsers can’t set headers on a WebSocket, so a browser client uses the subprotocol or the query string. Either exposes the key to the page, so use a key with tight limits.

Respan parameters

A WebSocket has no request body, so set the session’s customer and logging in the X-Data-Respan-Params header: a base64-encoded JSON object.

import base64, json

params = base64.b64encode(json.dumps({
    "customer_identifier": "user_123",
    "disable_log": False,
}).encode()).decode()

async with client.realtime.connect(
    model="gpt-realtime",
    extra_headers={"x-data-respan-params": params},
) as connection:
    ...

To scope a session to its key’s environment’s integrations, send x-respan-beta: env-scoped-integrations-2026-03-28, as for chat. See Respan parameters.

Realtime sessions are not redacted, so an organization with PII redaction on is refused.

Session limits

Open sessions are capped per API key, per organization and per gateway server: 10, 50 and 500. A handshake over a cap is refused with 429 realtime_session_limit and writes no row. When a gateway server shuts down, it closes its sessions with code 1001.

Pricing and logs

• A session is charged and logged once it ends, as one streamed chat row priced from the provider’s usage reports, with audio at the model’s audio rates. Its messages are the conversation.
• If a response was cut off when the session ended, its usage is estimated and the row is marked is_token_count_estimated.
• A handshake refused before the session opens (400, 403, 404) writes a failed row.
• On Respan’s credits, the model must state a token price and both audio rates, or the handshake is refused with 403 model_unpriced. The session passes chat’s checks at the handshake.
• On Respan’s credits, each response is held before it runs. A response your credits can’t cover closes the session with code 1008 insufficient_credits, and so does spending past the customer’s budget.
• When your client closes, responses still running are cancelled and their usage is read for up to 3 seconds; a response asked for and never seen is charged an estimate.

Next steps

• Audio: one-off speech, transcription and audio chat.
• OSS realtime formats: how sessions translate between formats.
• Budgets and limits: cap what a key or customer can spend.
• Logs: find each session’s row.