---
title: "Realtime"
subtitle: "Run voice and text sessions over WebSocket with OpenAI Realtime, Gemini Live and Mistral."
---

Realtime sessions are long-lived WebSocket connections. The gateway serves three realtime formats and translates between them, so an OpenAI Realtime client can talk to a Gemini Live model and the other way round. Each session is charged and logged once it ends.

**Pick a model:** filter by output type `realtime` or input type `audio` on the [model catalog](https://www.respan.ai/models), or read [Models](/docs/ai-gateway/models/overview).

## Quickstart

<Tabs>
<Tab title="Python">
```python
import asyncio
import os
from openai import AsyncOpenAI

async def main():
    client = AsyncOpenAI(
        base_url="https://gateway.respan.ai/v1",
        api_key=os.environ["RESPAN_API_KEY"],
    )
    async with client.realtime.connect(model="gpt-realtime") as connection:
        await connection.session.update(
            session={"type": "realtime", "output_modalities": ["text"]}
        )
        await connection.conversation.item.create(
            item={
                "type": "message",
                "role": "user",
                "content": [{"type": "input_text", "text": "Say hello."}],
            }
        )
        await connection.response.create()

        async for event in connection:
            if event.type == "response.output_text.delta":
                print(event.delta, end="", flush=True)
            elif event.type == "response.done":
                break

asyncio.run(main())
```
</Tab>
<Tab title="TypeScript">
```typescript
import OpenAI from "openai";
import { OpenAIRealtimeWS } from "openai/realtime/ws"; // needs the `ws` package

const client = new OpenAI({
  baseURL: "https://gateway.respan.ai/v1",
  apiKey: process.env.RESPAN_API_KEY,
});

const rt = new OpenAIRealtimeWS({ model: "gpt-realtime" }, client);

rt.socket.on("open", () => {
  rt.send({ type: "session.update", session: { type: "realtime", output_modalities: ["text"] } });
  rt.send({
    type: "conversation.item.create",
    item: { type: "message", role: "user", content: [{ type: "input_text", text: "Say hello." }] },
  });
  rt.send({ type: "response.create" });
});

rt.on("response.output_text.delta", (event) => process.stdout.write(event.delta));
rt.on("response.done", () => rt.close());
```
</Tab>
</Tabs>

The SDK opens `wss://gateway.respan.ai/v1/realtime?model=gpt-realtime`. The `model` in the URL is required: the gateway routes the session from it before any message arrives.

## Endpoints

| Format | WebSocket endpoint |
| --- | --- |
| OpenAI Realtime | `GET /v1/realtime?model=...` |
| Gemini Live | `GET /ws/google.ai.generativelanguage.v1beta.GenerativeService.BidiGenerateContent?model=...` |
| Mistral realtime transcription | `GET /v1/audio/transcriptions/realtime?model=...` |

Any of the three formats can reach models on OpenAI, Gemini or Mistral. The session's model is routed like a chat model, trying its attempts in turn.

### Gemini Live

Gemini Live clients must put the model in the URL too. The google-genai SDK names the model only in its setup message, which arrives after the gateway has routed the session, so point your client at the full URL with `?model=` set.

Gemini fixes its setup when the session starts, so a `session.update` that changes it closes the session with code `1008`. See [OSS realtime formats](/docs/oss-gateway/api-formats#realtime) for other translation details.

### Mistral realtime transcription

Mistral's realtime endpoint only transcribes. An OpenAI Realtime transcription session can be answered by a Mistral model, and a Mistral client can transcribe with OpenAI or Gemini models. A conversation session sent to Mistral is refused.

## Authentication

Send your Respan API key in any one of these:

| Where | Form |
| --- | --- |
| `Authorization` header | `Bearer YOUR_RESPAN_API_KEY` |
| WebSocket subprotocol | `openai-insecure-api-key.YOUR_RESPAN_API_KEY` |
| `x-goog-api-key` header | `YOUR_RESPAN_API_KEY` |
| Query string | `?key=YOUR_RESPAN_API_KEY` |

Browsers can't set headers on a WebSocket, so a browser client uses the subprotocol or the query string. Either exposes the key to the page, so use a key with tight [limits](/docs/ai-gateway/spend/budgets-and-limits).

## Respan parameters

A WebSocket has no request body, so set the session's customer and logging in the `X-Data-Respan-Params` header: a base64-encoded JSON object.

```python
import base64, json

params = base64.b64encode(json.dumps({
    "customer_identifier": "user_123",
    "disable_log": False,
}).encode()).decode()

async with client.realtime.connect(
    model="gpt-realtime",
    extra_headers={"x-data-respan-params": params},
) as connection:
    ...
```

To scope a session to its key's environment's integrations, send `x-respan-beta: env-scoped-integrations-2026-03-28`, as for chat. See [Respan parameters](/docs/ai-gateway/customers/respan-params).

Realtime sessions are not redacted, so an organization with [PII redaction](/docs/ai-gateway/features/pii-redaction) on is refused.

{/* REVIEW: realtime reads only customer_identifier and disable_log from the header today; confirm whether metadata and thread fields are read before launch. */}

## Session limits

Open sessions are capped per API key, per organization and per gateway server: 10, 50 and 500. A handshake over a cap is refused with `429 realtime_session_limit` and writes no row. When a gateway server shuts down, it closes its sessions with code `1001`.

{/* REVIEW: confirm 10/50/500 (code defaults) are the production caps, and whether customers can request higher ones. */}

## Pricing and logs

- A session is charged and logged once it ends, as one streamed `chat` row priced from the provider's usage reports, with audio at the model's audio rates. Its messages are the conversation.
- If a response was cut off when the session ended, its usage is estimated and the row is marked `is_token_count_estimated`.
- A handshake refused before the session opens (`400`, `403`, `404`) writes a failed row.
- On Respan's credits, the model must state a token price and both audio rates, or the handshake is refused with `403 model_unpriced`. The session passes chat's checks at the handshake.
- On Respan's credits, each response is held before it runs. A response your credits can't cover closes the session with code `1008 insufficient_credits`, and so does spending past the customer's budget.
- When your client closes, responses still running are cancelled and their usage is read for up to 3 seconds; a response asked for and never seen is charged an estimate.

## Next steps

- [Audio](/docs/ai-gateway/modalities/audio): one-off speech, transcription and audio chat.
- [OSS realtime formats](/docs/oss-gateway/api-formats#realtime): how sessions translate between formats.
- [Budgets and limits](/docs/ai-gateway/spend/budgets-and-limits): cap what a key or customer can spend.
- [Logs](/docs/ai-gateway/observability/logs-and-traces): find each session's row.