Rate limits

How many requests the gateway admits per minute, and what happens over the limit.

Every request is checked against these limits before any provider is called. Over a limit, the gateway answers 429 and the request costs nothing.

Plan limits

Your plan sets how many requests per minute your organization can send through the gateway, across all its API keys.

There is no plan limit on tokens per minute.

Limits you set

A request has to fit every limit that applies, so the tightest one wins. To cap spend rather than speed, use budgets and limits.

Realtime sessions

Open realtime sessions are capped at 10 per API key, 50 per organization, and 500 per gateway instance. A new session over a cap is refused with 429 realtime_session_limit.

Provider limits

Calls on Respan credits share Respan’s capacity with each provider. If a provider rate limits that capacity, you get the provider’s 429. Calls on your own key use your provider account’s quotas instead, and managed fallback can move them onto Respan’s keys when your key is rate limited.

Over the limit

{
  "error": {
    "message": "Rate Limit - You have reached the rate limit for your Respan API key: prod-key with rate limit 60.0 per minute, try again shortly or modify your rate limit at ...",
    "type": "rate_limit_exceeded",
    "code": "rate_limit_exceeded"
  }
}

A tokens-per-minute refusal includes a retry-after header in seconds; wait that long. Other refusals have none, so retry with exponential backoff.