Create a chat completion

POSThttps://api.respan.ai/api/chat/completions

Sends a chat completion request through the Respan gateway with automatic logging. Accepts OpenAI chat completion parameters and Respan options for fallbacks, caching, and prompt management.

Pass Respan parameters in top-level body fields, under respan_params, or as base64-encoded JSON in the X-Data-Respan-Params header. If the same field is sent more than one way, respan_params takes precedence over the header, and both take precedence over a top-level field. The exception is variables: top-level variables are merged key by key over the variables from respan_params or the header. respan_params, keywordsai_params, and the decoded header must each be a JSON object; a top-level respan_params may also be a JSON string that decodes to one. Any other value is ignored, and so are the header’s parameters: the request is served, but none of those parameters apply. Fields sent at the top level still apply. With the OpenAI SDK, use extra_body.

For legacy compatibility, keywordsai_params is merged into respan_params, and X-Data-Keywordsai-Params is still accepted and renamed internally.

Servers

https://api.respan.ai — Respan API Server

Authentication

RespanApiKey

Use your Respan API key for Respan API authentication. Enter only the Respan API key value; clients send Authorization: Bearer <RESPAN_API_KEY>. For /api/responses, provider credentials such as Perplexity, OpenAI, or Azure OpenAI go in Settings -> Providers or respan_params.credential_override in the request body, not in this authentication field.

HTTP scheme: bearer; format: Respan API key.


Authorization: Bearer $RESPAN_API_KEY

Headers

Parameter Type Required Description and constraints

X-Data-Respan-Params string Optional

Base64-encoded JSON object of Respan parameters. Legacy X-Data-Keywordsai-Params is still accepted.

X-Respan-Route-Provider string Optional

Pin the request to a specific provider without changing the model slug. Use the provider ID shown for that model in the model catalog.
Example: "vertex_ai"

X-Respan-Beta string Optional

Comma-separated beta feature flags. Available: token-breakdown-2026-03-26, env-scoped-integrations-2026-03-28

Request body

A request body is required.

application/json

Field Type Requirement Description and constraints

$ object Required body

Example:

{"model":"gpt-6-luna","messages":[{"role":"user","content":"Reply with exactly ok."}],"max_tokens":32,"temperature":0,"reasoning_effort":"none"}

messages array<object> Required in parent

Array of messages in the conversation. Each message has role (system, user, assistant, tool) and content.
Example: [{"role":"user","content":"Reply with exactly ok."}]

messages[] object Array item

—

messages[].role string Required in parent

Message role.
Allowed: "system", "user", "assistant", "tool"

messages[].content string or array<object> Required in parent

Message content. Use a string for text-only requests, or an array of content parts for multimodal requests.

messages[].content (oneOf 1) string Schema variant

—

messages[].content (oneOf 2) array<object> Schema variant

—

messages[].content (oneOf 2)[] object Array item

—

messages[].name string Optional

Optional participant name.

messages[].tool_call_id string Optional

Required for tool response messages.

model string Required in parent

Model to use. See Models for available options.
Example: "gpt-6-luna"

stream boolean Optional

Stream back partial progress token by token as server-sent events.
Default: false
Example: false

tools array<object> Optional

Tools the model may call. Currently only functions are supported.

tools[] object Array item

—

tool_choice object Optional

Controls tool selection. "none" = no tools, "auto" = model decides, or specify a tool object.

frequency_penalty number Optional

Penalizes tokens based on frequency in text so far (-2 to 2).
Example: 0

max_tokens number Optional

Maximum tokens to generate.
Example: 1024

temperature number Optional

Sampling temperature (0-2). Higher = more random.
Default: 1
Example: 0.7

n number Optional

Number of completions to generate. Note: costs multiply with n.
Default: 1
Example: 1

logprobs boolean Optional

Return log probabilities of output tokens.
Default: false

echo boolean Optional

Echo back the prompt in addition to the completion

stop array<string> Optional

Stop sequences where generation halts.

stop[] string Array item

—

presence_penalty number Optional

Penalizes tokens already present in text (-2 to 2).
Example: 0

logit_bias object Optional

Used to modify the probability of tokens appearing in the response

response_format object Optional

Output format. Set {"type": "json_schema", "json_schema": {...}} for structured output, or {"type": "json_object"} for JSON mode.

parallel_tool_calls boolean Optional

Enable parallel function calling during tool use.

load_balance_group object Optional

Load balance group selection. Use {"group_id": "..."} to route through a configured group.

fallback_models array<string> Optional

Backup models (ranked by priority) if the primary model fails.
Example: ["gpt-6-luna","claude-haiku-5-5"]

fallback_models[] string Array item

—

customer_credentials object Optional

Per-customer LLM provider credentials. Keys are provider names, values are API keys.

credential_override object Optional

One-off credential overrides per provider. Overrides uploaded provider keys for this request only.

cache_enabled boolean Optional

Enable response caching. See Caching.
Example: false

cache_ttl number Optional

How long a cached response is served, in seconds. A response stored without cache_ttl is served from the cache for at most 30 minutes.
Example: 3600

cache_options object Optional

Cache behavior options. Properties: cache_by_customer, is_cached_by_model, omit_log.

cache_options.cache_by_customer boolean Optional

Partition cache entries by customer identifier.
Default: false

cache_options.is_cached_by_model boolean Optional

Partition cache entries by model name.
Default: false

cache_options.omit_log boolean Optional

Suppress log creation for cache hits.
Default: false

prompt object Optional

Prompt template config. Properties: prompt_id (required), variables (template variables), version (number, or "latest" for draft), echo (return rendered prompt), override (use override_params), override_params (OpenAI params to override), schema_version (1 = legacy, 2 = prompt config wins). See Prompt management.

retry_params object Optional

Has no effect on this endpoint: chat completions always uses your organization’s retry settings. Per-request retry_params (retry_enabled, num_retries, retry_after) applies only to POST /api/responses. See Retries and fallback.

disable_log boolean Optional

When true, omits input/output from the log. Metrics (tokens, cost, latency) are still recorded.
Example: false

model_name_map object Optional

Azure deployment name mapping. Maps your custom Azure deployment names to standard model names.

models array<string> Optional

Model list for LLM router selection.

models[] string Array item

—

exclude_providers array<string> Optional

Providers to exclude from routing. All models under excluded providers are skipped.
Default: []

exclude_providers[] string Array item

—

exclude_models array<string> Optional

Specific models to exclude from routing.
Default: []

exclude_models[] string Array item

—

metadata object Optional

Custom key-value metadata attached to the span.
Example: {"feature":"chat","env":"production"}

custom_identifier string Optional

Indexed custom tag for fast querying.
Example: "order_456"

customer_identifier string Optional

End user identifier for analytics and budgets.
Maximum length: 254
Example: "user_123"

customer_params object Optional

Customer details. Properties: customer_identifier (takes precedence over the top-level customer_identifier), name and email (logged with the request, and saved on the customer when Respan first sees it), and rate_limit (requests per minute for this customer, overriding your organization’s customer rate limit; requests over it get 429). Budget fields sent here aren’t saved or enforced. Set budgets with Update a user.

request_breakdown boolean Optional

Return response metrics summary in the response body. For streaming, metrics appear in the final chunk.
Default: false

positive_feedback boolean Optional

User feedback. true = liked, false = disliked.

load_balance_models array<object> Optional

Inline load balancing options. Each item can include model, weight, and optional credentials.

load_balance_models[] object Array item

—

thread_identifier string Optional

Conversation thread ID. Spans with the same thread_identifier are grouped together.
Example: "thread_abc"

properties object Optional

Typed metadata preserving native types (numbers, booleans, nested objects). Unlike metadata which coerces to strings.

retries integer Optional

Has no effect: chat completions always uses your organization’s retry settings. See Retries and fallback.
Default: 0
Example: 2

weight number Optional

Load balancing weight.
Example: 1

span_name string Optional

Custom span name for tracing.
Example: "chat_completion"

respan_params object Optional

Namespaced container for all Respan parameters. Alternative to passing them at top level. Must be a JSON object, or a JSON string that decodes to one. A list, number or boolean is ignored and none of its params apply.

OpenAPI request schema

{
  "type": "object",
  "properties": {
    "messages": {
      "type": "array",
      "items": {
        "type": "object",
        "required": [
          "role",
          "content"
        ],
        "properties": {
          "role": {
            "type": "string",
            "enum": [
              "system",
              "user",
              "assistant",
              "tool"
            ],
            "description": "Message role."
          },
          "content": {
            "description": "Message content. Use a string for text-only requests, or an array of content parts for multimodal requests.",
            "oneOf": [
              {
                "type": "string"
              },
              {
                "type": "array",
                "items": {
                  "type": "object"
                }
              }
            ]
          },
          "name": {
            "type": "string",
            "description": "Optional participant name."
          },
          "tool_call_id": {
            "type": "string",
            "description": "Required for tool response messages."
          }
        }
      },
      "description": "Array of messages in the conversation. Each message has `role` (`system`, `user`, `assistant`, `tool`) and `content`.",
      "example": [
        {
          "role": "user",
          "content": "Reply with exactly ok."
        }
      ]
    },
    "model": {
      "type": "string",
      "description": "Model to use. See [Models](https://platform.respan.ai/platform/models) for available options.",
      "example": "gpt-6-luna"
    },
    "stream": {
      "type": "boolean",
      "description": "Stream back partial progress token by token as server-sent events.",
      "default": "false",
      "example": false
    },
    "tools": {
      "type": "array",
      "items": {
        "type": "object"
      },
      "description": "Tools the model may call. Currently only functions are supported."
    },
    "tool_choice": {
      "type": "object",
      "description": "Controls tool selection. `\"none\"` = no tools, `\"auto\"` = model decides, or specify a tool object."
    },
    "frequency_penalty": {
      "type": "number",
      "description": "Penalizes tokens based on frequency in text so far (-2 to 2).",
      "example": 0
    },
    "max_tokens": {
      "type": "number",
      "description": "Maximum tokens to generate.",
      "example": 1024
    },
    "temperature": {
      "type": "number",
      "description": "Sampling temperature (0-2). Higher = more random.",
      "default": 1,
      "example": 0.7
    },
    "n": {
      "type": "number",
      "description": "Number of completions to generate. Note: costs multiply with `n`.",
      "default": 1,
      "example": 1
    },
    "logprobs": {
      "type": "boolean",
      "description": "Return log probabilities of output tokens.",
      "default": "false"
    },
    "echo": {
      "type": "boolean",
      "description": "Echo back the prompt in addition to the completion"
    },
    "stop": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "Stop sequences where generation halts."
    },
    "presence_penalty": {
      "type": "number",
      "description": "Penalizes tokens already present in text (-2 to 2).",
      "example": 0
    },
    "logit_bias": {
      "type": "object",
      "description": "Used to modify the probability of tokens appearing in the response"
    },
    "response_format": {
      "type": "object",
      "description": "Output format. Set `{\"type\": \"json_schema\", \"json_schema\": {...}}` for structured output, or `{\"type\": \"json_object\"}` for JSON mode."
    },
    "parallel_tool_calls": {
      "type": "boolean",
      "description": "Enable parallel function calling during tool use."
    },
    "load_balance_group": {
      "type": "object",
      "description": "Load balance group selection. Use `{\"group_id\": \"...\"}` to route through a configured group."
    },
    "fallback_models": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "Backup models (ranked by priority) if the primary model fails.",
      "example": [
        "gpt-6-luna",
        "claude-haiku-5-5"
      ]
    },
    "customer_credentials": {
      "type": "object",
      "description": "Per-customer LLM provider credentials. Keys are provider names, values are API keys."
    },
    "credential_override": {
      "type": "object",
      "description": "One-off credential overrides per provider. Overrides uploaded provider keys for this request only."
    },
    "cache_enabled": {
      "type": "boolean",
      "description": "Enable response caching. See [Caching](/docs/documentation/features/gateway/caching).",
      "example": false
    },
    "cache_ttl": {
      "type": "number",
      "description": "How long a cached response is served, in seconds. A response stored without `cache_ttl` is served from the cache for at most 30 minutes.",
      "example": 3600
    },
    "cache_options": {
      "type": "object",
      "description": "Cache behavior options. Properties: `cache_by_customer`, `is_cached_by_model`, `omit_log`.",
      "properties": {
        "cache_by_customer": {
          "type": "boolean",
          "default": false,
          "description": "Partition cache entries by customer identifier."
        },
        "is_cached_by_model": {
          "type": "boolean",
          "default": false,
          "description": "Partition cache entries by model name."
        },
        "omit_log": {
          "type": "boolean",
          "default": false,
          "description": "Suppress log creation for cache hits."
        }
      }
    },
    "prompt": {
      "type": "object",
      "description": "Prompt template config. Properties: `prompt_id` (required), `variables` (template variables), `version` (number, or `\"latest\"` for draft), `echo` (return rendered prompt), `override` (use override_params), `override_params` (OpenAI params to override), `schema_version` (`1` = legacy, `2` = prompt config wins). See [Prompt management](/docs/documentation/features/prompt-management/advanced)."
    },
    "retry_params": {
      "type": "object",
      "description": "Has no effect on this endpoint: chat completions always uses your organization's retry settings. Per-request `retry_params` (`retry_enabled`, `num_retries`, `retry_after`) applies only to `POST /api/responses`. See [Retries and fallback](/docs/documentation/features/gateway/retries)."
    },
    "disable_log": {
      "type": "boolean",
      "description": "When `true`, omits input/output from the log. Metrics (tokens, cost, latency) are still recorded.",
      "example": false
    },
    "model_name_map": {
      "type": "object",
      "description": "Azure deployment name mapping. Maps your custom Azure deployment names to standard model names."
    },
    "models": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "Model list for LLM router selection."
    },
    "exclude_providers": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "Providers to exclude from routing. All models under excluded providers are skipped.",
      "default": "[]"
    },
    "exclude_models": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "Specific models to exclude from routing.",
      "default": "[]"
    },
    "metadata": {
      "type": "object",
      "description": "Custom key-value metadata attached to the span.",
      "example": {
        "feature": "chat",
        "env": "production"
      }
    },
    "custom_identifier": {
      "type": "string",
      "description": "Indexed custom tag for fast querying.",
      "example": "order_456"
    },
    "customer_identifier": {
      "type": "string",
      "description": "End user identifier for analytics and budgets.",
      "maxLength": 254,
      "example": "user_123"
    },
    "customer_params": {
      "type": "object",
      "description": "Customer details. Properties: `customer_identifier` (takes precedence over the top-level `customer_identifier`), `name` and `email` (logged with the request, and saved on the customer when Respan first sees it), and `rate_limit` (requests per minute for this customer, overriding your organization's customer rate limit; requests over it get `429`). Budget fields sent here aren't saved or enforced. Set budgets with [Update a user](/docs/apis/users/update-user)."
    },
    "request_breakdown": {
      "type": "boolean",
      "description": "Return response metrics summary in the response body. For streaming, metrics appear in the final chunk.",
      "default": "false"
    },
    "positive_feedback": {
      "type": "boolean",
      "description": "User feedback. `true` = liked, `false` = disliked."
    },
    "load_balance_models": {
      "type": "array",
      "items": {
        "type": "object"
      },
      "description": "Inline load balancing options. Each item can include `model`, `weight`, and optional `credentials`."
    },
    "thread_identifier": {
      "type": "string",
      "description": "Conversation thread ID. Spans with the same `thread_identifier` are grouped together.",
      "example": "thread_abc"
    },
    "properties": {
      "type": "object",
      "description": "Typed metadata preserving native types (numbers, booleans, nested objects). Unlike `metadata` which coerces to strings."
    },
    "retries": {
      "type": "integer",
      "description": "Has no effect: chat completions always uses your organization's retry settings. See [Retries and fallback](/docs/documentation/features/gateway/retries).",
      "default": 0,
      "example": 2
    },
    "weight": {
      "type": "number",
      "description": "Load balancing weight.",
      "example": 1
    },
    "span_name": {
      "type": "string",
      "description": "Custom span name for tracing.",
      "example": "chat_completion"
    },
    "respan_params": {
      "type": "object",
      "description": "Namespaced container for all Respan parameters. Alternative to passing them at top level. Must be a JSON object, or a JSON string that decodes to one. A list, number or boolean is ignored and none of its params apply."
    }
  },
  "required": [
    "messages",
    "model"
  ],
  "example": {
    "model": "gpt-6-luna",
    "messages": [
      {
        "role": "user",
        "content": "Reply with exactly ok."
      }
    ],
    "max_tokens": 32,
    "temperature": 0,
    "reasoning_effort": "none"
  }
}

Example


{
  "model": "gpt-6-luna",
  "messages": [
    {
      "role": "user",
      "content": "Reply with exactly ok."
    }
  ],
  "max_tokens": 32,
  "temperature": 0,
  "reasoning_effort": "none"
}

Examples

Chat completion

Request


{
  "model": "gpt-6-luna",
  "messages": [
    {
      "role": "user",
      "content": "Reply with exactly ok."
    }
  ],
  "max_tokens": 32,
  "temperature": 0,
  "reasoning_effort": "none"
}

Response


{
  "body": {
    "id": "chatcmpl_abc123",
    "object": "chat.completion",
    "created": 1709155200,
    "model": "gpt-6-luna",
    "choices": [
      {
        "index": 0,
        "message": {
          "role": "assistant",
          "content": "ok"
        },
        "finish_reason": "stop"
      }
    ],
    "usage": {
      "prompt_tokens": 12,
      "completion_tokens": 1,
      "total_tokens": 13
    }
  }
}

cURL


curl --request POST 'https://api.respan.ai/api/chat/completions' \
  --header "Authorization: Bearer $RESPAN_API_KEY" \
  --header 'Content-Type: application/json' \
  --data @- <<'JSON'
{
  "model": "gpt-6-luna",
  "messages": [
    {
      "role": "user",
      "content": "Reply with exactly ok."
    }
  ],
  "max_tokens": 32,
  "temperature": 0,
  "reasoning_effort": "none"
}
JSON

Python


import json
import os
import urllib.request

payload = json.loads(r'''{
  "model": "gpt-6-luna",
  "messages": [
    {
      "role": "user",
      "content": "Reply with exactly ok."
    }
  ],
  "max_tokens": 32,
  "temperature": 0,
  "reasoning_effort": "none"
}''')

request = urllib.request.Request(
    "https://api.respan.ai/api/chat/completions",
    data=json.dumps(payload).encode(),
    headers={
        "Authorization": f"Bearer {os.environ['RESPAN_API_KEY']}",
        "Content-Type": "application/json",
    },
    method="POST",
)
with urllib.request.urlopen(request) as response:
    print(json.load(response))

TypeScript


const response = await fetch("https://api.respan.ai/api/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.RESPAN_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    "model": "gpt-6-luna",
    "messages": [
      {
        "role": "user",
        "content": "Reply with exactly ok."
      }
    ],
    "max_tokens": 32,
    "temperature": 0,
    "reasoning_effort": "none"
  }),
});
if (!response.ok) throw new Error(await response.text());
console.log(await response.json());

Python SDK (respan-api)


import json
import os
from respan import RespanClient

client = RespanClient(respan_api_key=os.environ["RESPAN_API_KEY"])
payload = json.loads(r'''{
  "model": "gpt-6-luna",
  "messages": [
    {
      "role": "user",
      "content": "Reply with exactly ok."
    }
  ],
  "max_tokens": 32,
  "temperature": 0,
  "reasoning_effort": "none"
}''')
response = client.gateway.create_chat_completion(
    **payload,
)
print(response)

TypeScript SDK (@respan/respan-api)


import { RespanClient } from "@respan/respan-api";

const apiKey = process.env.RESPAN_API_KEY!;
const client = new RespanClient({
  respanApiKey: apiKey,
  authorization: `Bearer ${apiKey}`,
});
const response = await client.gateway.createChatCompletion({
  "model": "gpt-6-luna",
  "messages": [
    {
      "role": "user",
      "content": "Reply with exactly ok."
    }
  ],
  "max_tokens": 32,
  "temperature": 0,
  "reasoning_effort": "none"
});
console.log(response);

Responses

200

Successful response for Create chat completion

application/json

Field Type Requirement Description and constraints

$ object Body

—

id string Required in parent

Chat completion ID.

object string Required in parent

Example: "chat.completion"

created integer Required in parent

Unix timestamp for when the completion was created.

model string Required in parent

Model used for the completion.

choices array<object> Required in parent

—

choices[] object Array item

—

choices[].index integer Optional

—

choices[].message object Optional

—

choices[].message.role string Optional

Example: "assistant"

choices[].message.content string Optional

Example: "ok"

choices[].finish_reason string Optional

Example: "stop"

usage object Optional

—

usage.prompt_tokens integer Optional

—

usage.completion_tokens integer Optional

—

usage.total_tokens integer Optional

—

OpenAPI response schema

{
  "type": "object",
  "required": [
    "id",
    "object",
    "created",
    "model",
    "choices"
  ],
  "properties": {
    "id": {
      "type": "string",
      "description": "Chat completion ID."
    },
    "object": {
      "type": "string",
      "example": "chat.completion"
    },
    "created": {
      "type": "integer",
      "description": "Unix timestamp for when the completion was created."
    },
    "model": {
      "type": "string",
      "description": "Model used for the completion."
    },
    "choices": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "index": {
            "type": "integer"
          },
          "message": {
            "type": "object",
            "properties": {
              "role": {
                "type": "string",
                "example": "assistant"
              },
              "content": {
                "type": "string",
                "example": "ok"
              }
            }
          },
          "finish_reason": {
            "type": "string",
            "example": "stop"
          }
        }
      }
    },
    "usage": {
      "type": "object",
      "properties": {
        "prompt_tokens": {
          "type": "integer"
        },
        "completion_tokens": {
          "type": "integer"
        },
        "total_tokens": {
          "type": "integer"
        }
      }
    }
  }
}

Example


{
  "id": "chatcmpl_abc123",
  "object": "chat.completion",
  "created": 1709155200,
  "model": "gpt-6-luna",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "ok"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 1,
    "total_tokens": 13
  }
}

text/event-stream

Field Type Requirement Description and constraints

$ string Body

OpenAI-compatible server-sent events when stream is true.

OpenAPI response schema

{
  "type": "string",
  "description": "OpenAI-compatible server-sent events when `stream` is true."
}

400

Invalid request or preprocessing failure.

application/json

Field Type Requirement Description and constraints

$ object Body

—

error object Required in parent

—

error.message string Required in parent

—

error.type string Required in parent

—

error.param any or null Optional

—

error.code any or null Optional

—

OpenAPI response schema

{
  "type": "object",
  "required": [
    "error"
  ],
  "properties": {
    "error": {
      "type": "object",
      "required": [
        "message",
        "type"
      ],
      "properties": {
        "message": {
          "type": "string"
        },
        "type": {
          "type": "string"
        },
        "param": {
          "nullable": true
        },
        "code": {
          "nullable": true
        }
      }
    }
  }
}

401

No usable key for the model. The code is activation_required when your organization has no credits and no provider key for the model.

403

The API key is missing, invalid or expired.

application/json

Field Type Requirement Description and constraints

$ object Body

—

detail string Required in parent

—

OpenAPI response schema

{
  "type": "object",
  "required": [
    "detail"
  ],
  "properties": {
    "detail": {
      "type": "string"
    }
  }
}

Example


{
  "detail": "Your API key is invalid or expired, please check your API key in the platform settings."
}

404

The model isn’t available to your organization.

application/json

Field Type Requirement Description and constraints

$ object Body

—

error object Required in parent

—

error.message string Required in parent

—

error.type string Required in parent

—

error.param any or null Optional

—

error.code any or null Optional

—

OpenAPI response schema

{
  "type": "object",
  "required": [
    "error"
  ],
  "properties": {
    "error": {
      "type": "object",
      "required": [
        "message",
        "type"
      ],
      "properties": {
        "message": {
          "type": "string"
        },
        "type": {
          "type": "string"
        },
        "param": {
          "nullable": true
        },
        "code": {
          "nullable": true
        }
      }
    }
  }
}

424

The upstream model provider failed.

application/json

Field Type Requirement Description and constraints

$ object Body

—

error object Required in parent

—

error.message string Required in parent

—

error.type string Required in parent

—

error.param any or null Optional

—

error.code any or null Optional

—

OpenAPI response schema

{
  "type": "object",
  "required": [
    "error"
  ],
  "properties": {
    "error": {
      "type": "object",
      "required": [
        "message",
        "type"
      ],
      "properties": {
        "message": {
          "type": "string"
        },
        "type": {
          "type": "string"
        },
        "param": {
          "nullable": true
        },
        "code": {
          "nullable": true
        }
      }
    }
  }
}

OpenAPI source

The operation and referenced components below retain the complete OpenAPI definitions, including constraints and examples.

OpenAPI operation

{
  "summary": "Create a chat completion",
  "operationId": "createChatCompletion",
  "tags": [
    "Gateway"
  ],
  "description": "Sends a chat completion request through the Respan gateway with automatic logging. Accepts [OpenAI chat completion parameters](https://platform.openai.com/docs/apis/chat) and Respan options for fallbacks, caching, and prompt management.\n\nPass Respan parameters in top-level body fields, under `respan_params`, or as base64-encoded JSON in the `X-Data-Respan-Params` header. If the same field is sent more than one way, `respan_params` takes precedence over the header, and both take precedence over a top-level field. The exception is `variables`: top-level `variables` are merged key by key over the `variables` from `respan_params` or the header. `respan_params`, `keywordsai_params`, and the decoded header must each be a JSON object; a top-level `respan_params` may also be a JSON string that decodes to one. Any other value is ignored, and so are the header's parameters: the request is served, but none of those parameters apply. Fields sent at the top level still apply. With the OpenAI SDK, use `extra_body`.\n\nFor legacy compatibility, `keywordsai_params` is merged into `respan_params`, and `X-Data-Keywordsai-Params` is still accepted and renamed internally.",
  "requestBody": {
    "content": {
      "application/json": {
        "schema": {
          "type": "object",
          "properties": {
            "messages": {
              "type": "array",
              "items": {
                "type": "object",
                "required": [
                  "role",
                  "content"
                ],
                "properties": {
                  "role": {
                    "type": "string",
                    "enum": [
                      "system",
                      "user",
                      "assistant",
                      "tool"
                    ],
                    "description": "Message role."
                  },
                  "content": {
                    "description": "Message content. Use a string for text-only requests, or an array of content parts for multimodal requests.",
                    "oneOf": [
                      {
                        "type": "string"
                      },
                      {
                        "type": "array",
                        "items": {
                          "type": "object"
                        }
                      }
                    ]
                  },
                  "name": {
                    "type": "string",
                    "description": "Optional participant name."
                  },
                  "tool_call_id": {
                    "type": "string",
                    "description": "Required for tool response messages."
                  }
                }
              },
              "description": "Array of messages in the conversation. Each message has `role` (`system`, `user`, `assistant`, `tool`) and `content`.",
              "example": [
                {
                  "role": "user",
                  "content": "Reply with exactly ok."
                }
              ]
            },
            "model": {
              "type": "string",
              "description": "Model to use. See [Models](https://platform.respan.ai/platform/models) for available options.",
              "example": "gpt-6-luna"
            },
            "stream": {
              "type": "boolean",
              "description": "Stream back partial progress token by token as server-sent events.",
              "default": "false",
              "example": false
            },
            "tools": {
              "type": "array",
              "items": {
                "type": "object"
              },
              "description": "Tools the model may call. Currently only functions are supported."
            },
            "tool_choice": {
              "type": "object",
              "description": "Controls tool selection. `\"none\"` = no tools, `\"auto\"` = model decides, or specify a tool object."
            },
            "frequency_penalty": {
              "type": "number",
              "description": "Penalizes tokens based on frequency in text so far (-2 to 2).",
              "example": 0
            },
            "max_tokens": {
              "type": "number",
              "description": "Maximum tokens to generate.",
              "example": 1024
            },
            "temperature": {
              "type": "number",
              "description": "Sampling temperature (0-2). Higher = more random.",
              "default": 1,
              "example": 0.7
            },
            "n": {
              "type": "number",
              "description": "Number of completions to generate. Note: costs multiply with `n`.",
              "default": 1,
              "example": 1
            },
            "logprobs": {
              "type": "boolean",
              "description": "Return log probabilities of output tokens.",
              "default": "false"
            },
            "echo": {
              "type": "boolean",
              "description": "Echo back the prompt in addition to the completion"
            },
            "stop": {
              "type": "array",
              "items": {
                "type": "string"
              },
              "description": "Stop sequences where generation halts."
            },
            "presence_penalty": {
              "type": "number",
              "description": "Penalizes tokens already present in text (-2 to 2).",
              "example": 0
            },
            "logit_bias": {
              "type": "object",
              "description": "Used to modify the probability of tokens appearing in the response"
            },
            "response_format": {
              "type": "object",
              "description": "Output format. Set `{\"type\": \"json_schema\", \"json_schema\": {...}}` for structured output, or `{\"type\": \"json_object\"}` for JSON mode."
            },
            "parallel_tool_calls": {
              "type": "boolean",
              "description": "Enable parallel function calling during tool use."
            },
            "load_balance_group": {
              "type": "object",
              "description": "Load balance group selection. Use `{\"group_id\": \"...\"}` to route through a configured group."
            },
            "fallback_models": {
              "type": "array",
              "items": {
                "type": "string"
              },
              "description": "Backup models (ranked by priority) if the primary model fails.",
              "example": [
                "gpt-6-luna",
                "claude-haiku-5-5"
              ]
            },
            "customer_credentials": {
              "type": "object",
              "description": "Per-customer LLM provider credentials. Keys are provider names, values are API keys."
            },
            "credential_override": {
              "type": "object",
              "description": "One-off credential overrides per provider. Overrides uploaded provider keys for this request only."
            },
            "cache_enabled": {
              "type": "boolean",
              "description": "Enable response caching. See [Caching](/docs/documentation/features/gateway/caching).",
              "example": false
            },
            "cache_ttl": {
              "type": "number",
              "description": "How long a cached response is served, in seconds. A response stored without `cache_ttl` is served from the cache for at most 30 minutes.",
              "example": 3600
            },
            "cache_options": {
              "type": "object",
              "description": "Cache behavior options. Properties: `cache_by_customer`, `is_cached_by_model`, `omit_log`.",
              "properties": {
                "cache_by_customer": {
                  "type": "boolean",
                  "default": false,
                  "description": "Partition cache entries by customer identifier."
                },
                "is_cached_by_model": {
                  "type": "boolean",
                  "default": false,
                  "description": "Partition cache entries by model name."
                },
                "omit_log": {
                  "type": "boolean",
                  "default": false,
                  "description": "Suppress log creation for cache hits."
                }
              }
            },
            "prompt": {
              "type": "object",
              "description": "Prompt template config. Properties: `prompt_id` (required), `variables` (template variables), `version` (number, or `\"latest\"` for draft), `echo` (return rendered prompt), `override` (use override_params), `override_params` (OpenAI params to override), `schema_version` (`1` = legacy, `2` = prompt config wins). See [Prompt management](/docs/documentation/features/prompt-management/advanced)."
            },
            "retry_params": {
              "type": "object",
              "description": "Has no effect on this endpoint: chat completions always uses your organization's retry settings. Per-request `retry_params` (`retry_enabled`, `num_retries`, `retry_after`) applies only to `POST /api/responses`. See [Retries and fallback](/docs/documentation/features/gateway/retries)."
            },
            "disable_log": {
              "type": "boolean",
              "description": "When `true`, omits input/output from the log. Metrics (tokens, cost, latency) are still recorded.",
              "example": false
            },
            "model_name_map": {
              "type": "object",
              "description": "Azure deployment name mapping. Maps your custom Azure deployment names to standard model names."
            },
            "models": {
              "type": "array",
              "items": {
                "type": "string"
              },
              "description": "Model list for LLM router selection."
            },
            "exclude_providers": {
              "type": "array",
              "items": {
                "type": "string"
              },
              "description": "Providers to exclude from routing. All models under excluded providers are skipped.",
              "default": "[]"
            },
            "exclude_models": {
              "type": "array",
              "items": {
                "type": "string"
              },
              "description": "Specific models to exclude from routing.",
              "default": "[]"
            },
            "metadata": {
              "type": "object",
              "description": "Custom key-value metadata attached to the span.",
              "example": {
                "feature": "chat",
                "env": "production"
              }
            },
            "custom_identifier": {
              "type": "string",
              "description": "Indexed custom tag for fast querying.",
              "example": "order_456"
            },
            "customer_identifier": {
              "type": "string",
              "description": "End user identifier for analytics and budgets.",
              "maxLength": 254,
              "example": "user_123"
            },
            "customer_params": {
              "type": "object",
              "description": "Customer details. Properties: `customer_identifier` (takes precedence over the top-level `customer_identifier`), `name` and `email` (logged with the request, and saved on the customer when Respan first sees it), and `rate_limit` (requests per minute for this customer, overriding your organization's customer rate limit; requests over it get `429`). Budget fields sent here aren't saved or enforced. Set budgets with [Update a user](/docs/apis/users/update-user)."
            },
            "request_breakdown": {
              "type": "boolean",
              "description": "Return response metrics summary in the response body. For streaming, metrics appear in the final chunk.",
              "default": "false"
            },
            "positive_feedback": {
              "type": "boolean",
              "description": "User feedback. `true` = liked, `false` = disliked."
            },
            "load_balance_models": {
              "type": "array",
              "items": {
                "type": "object"
              },
              "description": "Inline load balancing options. Each item can include `model`, `weight`, and optional `credentials`."
            },
            "thread_identifier": {
              "type": "string",
              "description": "Conversation thread ID. Spans with the same `thread_identifier` are grouped together.",
              "example": "thread_abc"
            },
            "properties": {
              "type": "object",
              "description": "Typed metadata preserving native types (numbers, booleans, nested objects). Unlike `metadata` which coerces to strings."
            },
            "retries": {
              "type": "integer",
              "description": "Has no effect: chat completions always uses your organization's retry settings. See [Retries and fallback](/docs/documentation/features/gateway/retries).",
              "default": 0,
              "example": 2
            },
            "weight": {
              "type": "number",
              "description": "Load balancing weight.",
              "example": 1
            },
            "span_name": {
              "type": "string",
              "description": "Custom span name for tracing.",
              "example": "chat_completion"
            },
            "respan_params": {
              "type": "object",
              "description": "Namespaced container for all Respan parameters. Alternative to passing them at top level. Must be a JSON object, or a JSON string that decodes to one. A list, number or boolean is ignored and none of its params apply."
            }
          },
          "required": [
            "messages",
            "model"
          ],
          "example": {
            "model": "gpt-6-luna",
            "messages": [
              {
                "role": "user",
                "content": "Reply with exactly ok."
              }
            ],
            "max_tokens": 32,
            "temperature": 0,
            "reasoning_effort": "none"
          }
        },
        "example": {
          "model": "gpt-6-luna",
          "messages": [
            {
              "role": "user",
              "content": "Reply with exactly ok."
            }
          ],
          "max_tokens": 32,
          "temperature": 0,
          "reasoning_effort": "none"
        }
      }
    },
    "required": true
  },
  "responses": {
    "200": {
      "description": "Successful response for Create chat completion",
      "content": {
        "application/json": {
          "schema": {
            "type": "object",
            "required": [
              "id",
              "object",
              "created",
              "model",
              "choices"
            ],
            "properties": {
              "id": {
                "type": "string",
                "description": "Chat completion ID."
              },
              "object": {
                "type": "string",
                "example": "chat.completion"
              },
              "created": {
                "type": "integer",
                "description": "Unix timestamp for when the completion was created."
              },
              "model": {
                "type": "string",
                "description": "Model used for the completion."
              },
              "choices": {
                "type": "array",
                "items": {
                  "type": "object",
                  "properties": {
                    "index": {
                      "type": "integer"
                    },
                    "message": {
                      "type": "object",
                      "properties": {
                        "role": {
                          "type": "string",
                          "example": "assistant"
                        },
                        "content": {
                          "type": "string",
                          "example": "ok"
                        }
                      }
                    },
                    "finish_reason": {
                      "type": "string",
                      "example": "stop"
                    }
                  }
                }
              },
              "usage": {
                "type": "object",
                "properties": {
                  "prompt_tokens": {
                    "type": "integer"
                  },
                  "completion_tokens": {
                    "type": "integer"
                  },
                  "total_tokens": {
                    "type": "integer"
                  }
                }
              }
            }
          },
          "example": {
            "id": "chatcmpl_abc123",
            "object": "chat.completion",
            "created": 1709155200,
            "model": "gpt-6-luna",
            "choices": [
              {
                "index": 0,
                "message": {
                  "role": "assistant",
                  "content": "ok"
                },
                "finish_reason": "stop"
              }
            ],
            "usage": {
              "prompt_tokens": 12,
              "completion_tokens": 1,
              "total_tokens": 13
            }
          }
        },
        "text/event-stream": {
          "schema": {
            "type": "string",
            "description": "OpenAI-compatible server-sent events when `stream` is true."
          }
        }
      }
    },
    "400": {
      "description": "Invalid request or preprocessing failure.",
      "content": {
        "application/json": {
          "schema": {
            "type": "object",
            "required": [
              "error"
            ],
            "properties": {
              "error": {
                "type": "object",
                "required": [
                  "message",
                  "type"
                ],
                "properties": {
                  "message": {
                    "type": "string"
                  },
                  "type": {
                    "type": "string"
                  },
                  "param": {
                    "nullable": true
                  },
                  "code": {
                    "nullable": true
                  }
                }
              }
            }
          }
        }
      }
    },
    "401": {
      "description": "No usable key for the model. The code is `activation_required` when your organization has no credits and no provider key for the model."
    },
    "403": {
      "$ref": "#/components/responses/InvalidApiKey"
    },
    "404": {
      "description": "The model isn't available to your organization.",
      "content": {
        "application/json": {
          "schema": {
            "type": "object",
            "required": [
              "error"
            ],
            "properties": {
              "error": {
                "type": "object",
                "required": [
                  "message",
                  "type"
                ],
                "properties": {
                  "message": {
                    "type": "string"
                  },
                  "type": {
                    "type": "string"
                  },
                  "param": {
                    "nullable": true
                  },
                  "code": {
                    "nullable": true
                  }
                }
              }
            }
          }
        }
      }
    },
    "424": {
      "description": "The upstream model provider failed.",
      "content": {
        "application/json": {
          "schema": {
            "type": "object",
            "required": [
              "error"
            ],
            "properties": {
              "error": {
                "type": "object",
                "required": [
                  "message",
                  "type"
                ],
                "properties": {
                  "message": {
                    "type": "string"
                  },
                  "type": {
                    "type": "string"
                  },
                  "param": {
                    "nullable": true
                  },
                  "code": {
                    "nullable": true
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  "parameters": [
    {
      "name": "X-Data-Respan-Params",
      "in": "header",
      "required": false,
      "schema": {
        "type": "string"
      },
      "description": "Base64-encoded JSON object of Respan parameters. Legacy `X-Data-Keywordsai-Params` is still accepted."
    },
    {
      "name": "X-Respan-Route-Provider",
      "in": "header",
      "required": false,
      "schema": {
        "type": "string"
      },
      "description": "Pin the request to a specific provider without changing the model slug. Use the provider ID shown for that model in the model catalog.",
      "example": "vertex_ai"
    },
    {
      "name": "X-Respan-Beta",
      "in": "header",
      "required": false,
      "schema": {
        "type": "string"
      },
      "description": "Comma-separated beta feature flags. Available: token-breakdown-2026-03-26, env-scoped-integrations-2026-03-28"
    }
  ],
  "security": [
    {
      "RespanApiKey": []
    }
  ]
}

OpenAPI path metadata

{}

Referenced OpenAPI components

{
  "responses": {
    "InvalidApiKey": {
      "description": "The API key is missing, invalid or expired.",
      "content": {
        "application/json": {
          "schema": {
            "type": "object",
            "required": [
              "detail"
            ],
            "properties": {
              "detail": {
                "type": "string"
              }
            }
          },
          "example": {
            "detail": "Your API key is invalid or expired, please check your API key in the platform settings."
          }
        }
      }
    }
  },
  "securitySchemes": {
    "RespanApiKey": {
      "type": "http",
      "scheme": "bearer",
      "bearerFormat": "Respan API key",
      "description": "Use your Respan API key for Respan API authentication. Enter only the Respan API key value; clients send Authorization: Bearer <RESPAN_API_KEY>. For /api/responses, provider credentials such as Perplexity, OpenAI, or Azure OpenAI go in Settings -> Providers or respan_params.credential_override in the request body, not in this authentication field.",
      "x-fern-bearer": {
        "name": "respanApiKey",
        "env": "RESPAN_API_KEY"
      }
    }
  }
}

POST/api/chat/completions

Server

AuthenticationRespan API key

Headers

X-Data-Respan-Params Include

X-Respan-Route-Provider Include

X-Respan-Beta Include

Request body

messagesrequired

modelrequired

max_tokensnumber

temperaturenumber

reasoning_effortoptional · string

Add optional field

Language

curl --request POST 'https://api.respan.ai/api/chat/completions' \
  --header 'Authorization: Bearer YOUR_RESPAN_API_KEY' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "model": "gpt-6-luna",
  "messages": [
    {
      "role": "user",
      "content": "Reply with exactly ok."
    }
  ],
  "max_tokens": 32,
  "temperature": 0,
  "reasoning_effort": "none"
}'