Audio

Turn text into speech, transcribe audio, and send audio to chat models.

The gateway serves three kinds of audio call. Text to speech uses OpenAI’s Speech API (/v1/audio/speech). Transcription uses OpenAI’s Transcriptions API (/v1/audio/transcriptions). Audio in chat sends audio to an audio chat model and can get a spoken answer back. For live voice over a WebSocket, see Realtime.

Pick a model: filter by output type speech or transcription, or input type audio, on the model catalog , or read Models.

Text to speech

Terminal


curl https://gateway.respan.ai/v1/audio/speech \
  -H "Authorization: Bearer $RESPAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini-tts",
    "voice": "alloy",
    "input": "Your order has shipped.",
    "response_format": "mp3"
  }' \
  -o speech.mp3

Python


import os
from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.respan.ai/v1",
    api_key=os.environ["RESPAN_API_KEY"],
)

with client.audio.speech.with_streaming_response.create(
    model="gpt-4o-mini-tts",
    voice="alloy",
    input="Your order has shipped.",
    response_format="mp3",
) as response:
    response.stream_to_file("speech.mp3")

TypeScript


import fs from "node:fs";
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.respan.ai/v1",
  apiKey: process.env.RESPAN_API_KEY,
});

const response = await client.audio.speech.create({
  model: "gpt-4o-mini-tts",
  voice: "alloy",
  input: "Your order has shipped.",
  response_format: "mp3",
});
fs.writeFileSync("speech.mp3", Buffer.from(await response.arrayBuffer()));

Parameter	Type	Default	Description
`model`	string	required	The speech model.
`input`	string	required	The text to speak.
`voice`	string	required	The provider’s voice name.
`response_format`	string	`mp3`	`mp3`, `wav` or `pcm`.

The gateway prices speech by the length of the audio it makes, so it accepts only the formats it can measure: mp3, wav and pcm. Any other format (such as opus, aac or flac) is refused before the request is sent, on any key.

Transcription

Transcriptions take a multipart upload, as OpenAI’s API does.

Terminal


curl https://gateway.respan.ai/v1/audio/transcriptions \
  -H "Authorization: Bearer $RESPAN_API_KEY" \
  -F model="gpt-4o-transcribe" \
  -F file="@speech.mp3"

Python


with open("speech.mp3", "rb") as audio:
    transcript = client.audio.transcriptions.create(
        model="gpt-4o-transcribe",
        file=audio,
    )
print(transcript.text)

TypeScript


const transcript = await client.audio.transcriptions.create({
  model: "gpt-4o-transcribe",
  file: fs.createReadStream("speech.mp3"),
});
console.log(transcript.text);

Parameter	Type	Default	Description
`file`	file	required	The audio to transcribe.
`model`	string	required	The transcription model, such as `gpt-4o-transcribe` or Groq’s `whisper-large-v3-turbo`.
`language`	string	detected	Language hint, where the provider takes it.
`response_format`	string	`json`	Output format, where the provider takes it.

For live transcription over a WebSocket, see Realtime.

Providers

Provider	Speech	Transcription
OpenAI	Yes	Yes
Groq	Yes	Yes

Audio in chat

Audio chat models take audio in a message and can answer in audio. Send audio as an input_audio content part (base64 only; audio parts don’t take URLs), and ask for spoken output with modalities and audio.

Terminal


curl https://gateway.respan.ai/v1/chat/completions \
  -H "Authorization: Bearer $RESPAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-audio",
    "modalities": ["text", "audio"],
    "audio": {"voice": "alloy", "format": "wav"},
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "Answer the question in this recording."},
        {"type": "input_audio", "input_audio": {"data": "UklGRi...", "format": "wav"}}
      ]
    }]
  }'

Python


import base64

with open("question.wav", "rb") as f:
    audio_in = base64.b64encode(f.read()).decode()

response = client.chat.completions.create(
    model="gpt-audio",
    modalities=["text", "audio"],
    audio={"voice": "alloy", "format": "wav"},
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Answer the question in this recording."},
            {"type": "input_audio", "input_audio": {"data": audio_in, "format": "wav"}},
        ],
    }],
)

message = response.choices[0].message
print(message.audio.transcript)
with open("answer.wav", "wb") as f:
    f.write(base64.b64decode(message.audio.data))

TypeScript


const audioIn = fs.readFileSync("question.wav").toString("base64");

const response = await client.chat.completions.create({
  model: "gpt-audio",
  modalities: ["text", "audio"],
  audio: { voice: "alloy", format: "wav" },
  messages: [{
    role: "user",
    content: [
      { type: "text", text: "Answer the question in this recording." },
      { type: "input_audio", input_audio: { data: audioIn, format: "wav" } },
    ],
  }],
});

const message = response.choices[0].message;
console.log(message.audio?.transcript);
fs.writeFileSync("answer.wav", Buffer.from(message.audio!.data, "base64"));

Parameter	Type	Default	Description
`input_audio.data`	string	required	The audio, base64-encoded, in a user message’s content.
`input_audio.format`	string	required	The audio’s format, such as `wav` or `mp3`.
`modalities`	array	`["text"]`	Output types to return. Add `"audio"` for a spoken answer.
`audio.voice`	string	none	The voice of the spoken answer.
`audio.format`	string	none	The format of the spoken answer.

The answer’s audio is in choices[0].message.audio.data (base64), with its text in audio.transcript. audio and modalities are forwarded only to models whose catalog entry lists audio in its modalities, such as gpt-audio and gpt-audio-mini.

Respan parameters

Respan parameters work on all three calls. Speech and audio chat take them in the JSON body (or extra_body). Transcription is multipart, so send them in the X-Data-Respan-Params header (base64 JSON). See Respan parameters.

Speech and transcription requests are not redacted, so an organization with PII redaction on is refused for them.

Pricing and logs

Text to speech writes one speech row, priced per minute of audio made at the model’s unit price, never per token.
Transcription writes one transcription row, priced per minute of audio sent. A transcription whose duration neither the gateway nor the provider can read costs nothing.
Audio in chat writes a chat row, billed from the provider’s usage report, with audio tokens at the model’s audio rates where the catalog states them.
Respan’s credits pay for speech and transcription on any provider whose catalog entry states a unit price. Calls on Respan’s credits are held before they are sent.
All three pass the same checks as chat (limit policies, RPM, key usage cap, customer budgets).

Next steps

Realtime: stream voice both ways over WebSocket.
Text: chat request basics.
Logs: find speech and transcription rows.
Pricing: how per-minute pricing works.