Limits
Cap requests, tokens, and spend per plan, API key, and customer.
Limits are checked before any provider is called. A request over a limit is refused at once, so no provider, retry, or fallback runs and nothing is charged.
| Limit | Scope | Set it in | Over it |
|---|---|---|---|
| Plan rate limit | Organization | Your plan | 429 |
| Key rate limits | API key | API keys | 429 |
| Key spend and usage limits | API key | API keys | 429 |
| Customer budget | One customer | Limits tab, or the customer | 402 |
| Customer rate limit | One customer | Limits tab | 429 |
Plan rate limits
The Limits tab of the Gateway page shows your plan’s limits, and the customer limits below.
| Plan | Limit |
|---|---|
| Free | 412 requests per minute, 100k per day |
| Team | 4.2k requests per minute, 1M per day |
| Enterprise | Custom |
API key limits
Open API keys and click the key. Each limit shows its usage, what’s left, and when it resets.
| Limit | Detail |
|---|---|
| Rate limits | Requests per minute and tokens per minute for this key. |
| Lifetime limit | A cost cap that never resets, counted from what the key has already spent. |
| Recurring limits | Cost, requests, or tokens per minute, hour, day, week, or month. Stack rows for a daily and a monthly cap. |
| Alert at | A warning threshold below Block at. The request goes through, and Respan sends an email or Slack alert. |
Cost and tokens are counted when a call ends, so the call that crosses a cap completes and the ones after it are refused.
Customer limits
Customer limits apply to requests tagged with a customer (see Users & identifiers). Set the defaults for every customer on the Limits tab:
| Setting | Detail |
|---|---|
| Customer monthly budget | Spend per customer per month, in USD. Over it, the customer’s requests are refused until the next period. |
| Customer rate limit (requests/min) | Chat requests per minute per customer. |
To give one customer a different budget, open them under Observability › Users & identifiers and edit Period budget.
Over a limit
| Event | What happens |
|---|---|
| Alert at reached | The request goes through. Respan sends an alert and a limit_policy_soft_triggered webhook, once per threshold per period. |
| Block at reached | 429 until the window resets, and a limit_policy_hard_triggered webhook. |
| Customer budget reached | 402 until the next period, and a customer_budget_limit_reached webhook. |
The Rate limited chart on the Gateway page’s Overview counts every 429, from your limits or a provider’s.