Create a chat completion
Sends a chat completion request through the Respan gateway with automatic logging. Accepts OpenAI chat completion parameters and Respan options for fallbacks, caching, and prompt management.
Pass Respan parameters in top-level body fields, under respan_params, or as base64-encoded JSON in the X-Data-Respan-Params header. If the same field is sent more than one way, respan_params takes precedence over the header, and both take precedence over a top-level field. The exception is variables: top-level variables are merged key by key over the variables from respan_params or the header. respan_params, keywordsai_params, and the decoded header must each be a JSON object; a top-level respan_params may also be a JSON string that decodes to one. Any other value is ignored, and so are the header’s parameters: the request is served, but none of those parameters apply. Fields sent at the top level still apply. With the OpenAI SDK, use extra_body.
For legacy compatibility, keywordsai_params is merged into respan_params, and X-Data-Keywordsai-Params is still accepted and renamed internally.
Authentication
Use your Respan API key for Respan API authentication. Enter only the Respan API key value; clients send Authorization: Bearer <RESPAN_API_KEY>. For /api/responses, provider credentials such as Perplexity, OpenAI, or Azure OpenAI go in Settings -> Providers or respan_params.credential_override in the request body, not in this authentication field.
Headers
Base64-encoded JSON object of Respan parameters. Legacy X-Data-Keywordsai-Params is still accepted.
Pin the request to a specific provider without changing the model slug. Example: vertex_ai routes a claude-sonnet-4-5-20250929 request to Vertex AI Claude.
Comma-separated beta feature flags. Available: token-breakdown-2026-03-26, env-scoped-integrations-2026-03-28
Request
Array of messages in the conversation. Each message has role (system, user, assistant, tool) and content.
Model to use. See Models for available options.
Stream back partial progress token by token as server-sent events.
Controls tool selection. "none" = no tools, "auto" = model decides, or specify a tool object.
Penalizes tokens based on frequency in text so far (-2 to 2).
Sampling temperature (0-2). Higher = more random.
Number of completions to generate. Note: costs multiply with n.
Penalizes tokens already present in text (-2 to 2).
Output format. Set {"type": "json_schema", "json_schema": {...}} for structured output, or {"type": "json_object"} for JSON mode.
Load balance group selection. Use {"group_id": "..."} to route through a configured group.
Backup models (ranked by priority) if the primary model fails.
Per-customer LLM provider credentials. Keys are provider names, values are API keys.
One-off credential overrides per provider. Overrides uploaded provider keys for this request only.
Enable response caching. See Caching.
How long a cached response is served, in seconds. A response stored without cache_ttl is served from the cache for at most 30 minutes.
Cache behavior options. Properties: cache_by_customer, is_cached_by_model, omit_log.
Prompt template config. Properties: prompt_id (required), variables (template variables), version (number, or "latest" for draft), echo (return rendered prompt), override (use override_params), override_params (OpenAI params to override), schema_version (1 = legacy, 2 = prompt config wins). See Prompt management.
Has no effect on this endpoint: chat completions always uses your organization's retry settings. Per-request retry_params (retry_enabled, num_retries, retry_after) applies only to POST /api/responses. See Retries and fallback.
When true, omits input/output from the log. Metrics (tokens, cost, latency) are still recorded.
Custom key-value metadata attached to the span.
Customer details. Properties: customer_identifier (takes precedence over the top-level customer_identifier), name and email (logged with the request, and saved on the customer when Respan first sees it), and rate_limit (requests per minute for this customer, overriding your organization's customer rate limit; requests over it get 429). Budget fields sent here aren't saved or enforced. Set budgets with Update a user.
User feedback. true = liked, false = disliked.
Inline load balancing options. Each item can include model, weight, and optional credentials.
Conversation thread ID. Spans with the same thread_identifier are grouped together.
Typed metadata preserving native types (numbers, booleans, nested objects). Unlike metadata which coerces to strings.
Has no effect: chat completions always uses your organization's retry settings. See Retries and fallback.