Question 1

What is Respan?

Accepted Answer

Respan is an LLM engineering platform that unifies observability, evals, prompt optimization, and a unified LLM gateway. It helps teams trace agent workflows, evaluate behavior, and continuously improve quality in production. Backed by Y Combinator (W24) and Gradient Ventures. Processing 80+ trillion tokens for world-class founders, engineers, and product teams.

Question 2

Is Respan the same as Keywords AI?

Accepted Answer

Keywords AI rebranded to Respan in 2026.

Question 3

Who is Respan for?

Accepted Answer

Respan is for engineering teams running LLM applications and agents in production. It works for a solo developer running one agent and for a large team with compliance requirements and multiple applications in production.

Question 4

Is Respan an AI gateway or an observability platform?

Accepted Answer

Both. Respan is an LLM engineering platform that includes an AI gateway, observability, evaluations, and prompt management in one product. The gateway routes requests across models and providers. Observability traces what your agents did in production. Evals score output quality. All of it reads from the same data, so a request routed through the gateway is automatically traced and can be evaluated without extra instrumentation.

Question 5

Is Respan SOC 2 compliant?

Accepted Answer

Yes. Respan is SOC 2 Type II certified. Data is encrypted in transit with TLS 1.2+ and at rest with AES-256, access follows least-privilege controls, and Respan runs regular penetration testing and security audits. SOC 2 reports are available on the Team plan and above. Full policies and control documentation are in the Respan Trust Center.

Question 6

How long does Respan retain logs and traces?

Accepted Answer

Retention depends on your plan. Free retains logs and traces for 7 days, Team for 30 days, and Enterprise is custom. Team and Enterprise also support custom retention periods and conditional retention, so you can keep specific data longer than the default. Enterprise adds data retention management, PII masking, and the ability to omit logs entirely.

Question 7

Can Respan be self-hosted?

Accepted Answer

Respan does not currently support self-hosting, though it is on our roadmap to be added in the future.

Question 8

Does Respan support HIPAA / sign a BAA?

Accepted Answer

Yes. HIPAA compliance is available as a $249/month add-on, and Respan signs a BAA with Enterprise customers.

Question 9

Which models and providers does Respan support?

Accepted Answer

Respan's gateway provides a single OpenAI-compatible endpoint for 1,000+ models across every major provider, including OpenAI, Anthropic, Google Gemini, and Google Vertex AI, Azure OpenAI, AWS Bedrock, Groq, Fireworks, Together AI, Perplexity, OpenRouter, Nebius, and Novita. A full list of supported models is at respan.ai/models.

Question 10

How much latency does the Respan gateway add?

Accepted Answer

The Respan gateway adds roughly 50 to 150ms of latency per request. For most applications that is negligible against model response times. If latency is critical to your application, you can skip the gateway and use Respan's tracing SDK instead, which sits outside the request path and adds no latency. Gateway traffic is logged automatically either way, so routing through the gateway means you get observability without separate instrumentation.

Question 11

Does Respan support OpenTelemetry / OTLP?

Accepted Answer

Yes. Respan accepts traces over OTLP, so you can send data from any OpenTelemetry-compatible SDK or collector without using a Respan-specific SDK. OpenTelemetry support is available on all plans, including Free. If you already run OpenTelemetry instrumentation, you can point your exporter at Respan and keep your existing setup. Traces can also be sent through the JSON API or the Respan SDKs.

Question 12

Which languages and frameworks have SDKs?

Accepted Answer

Respan has official SDKs for Python and TypeScript/JavaScript. Framework integrations are available for OpenAI SDK, OpenAI Agents SDK, Vercel AI SDK, LangChain, LangGraph, LlamaIndex, Mastra, and Google GenAI. If you work in a language without an official SDK, you can send traces over OTLP or the JSON API from any OpenTelemetry-compatible library.

Question 13

Does Respan offer integrations?

Accepted Answer

Yes. Respan's integrations help you connect to LLM providers including OpenAI, Anthropic, Google Gemini, Google Vertex AI, Azure OpenAI, AWS Bedrock, Groq, Fireworks, Together AI, Perplexity, OpenRouter, Nebius, and Novita for gateway routing, using Respan credits or your own provider keys.

Question 14

Does Respan have an MCP server?

Accepted Answer

Yes. Respan has two MCP servers. The Platform MCP server gives your AI coding tools direct access to your observability data, covering logs, traces, customers, prompts, experiments, evaluators, evaluation pipelines, workflows, and datasets. The Docs MCP server gives those same tools access to Respan documentation and needs no API key. Both work with Claude Code, Cursor, and Codex CLI, and connect over HTTP at mcp.respan.ai. Enterprise customers get a dedicated endpoint.

Question 15

How do I get started with Respan?

Accepted Answer

Create an account at platform.respan.ai, then generate an API key from the API keys page. Add credits or connect your own provider key on the Integrations page. From there, pick the path that matches what you need: the gateway quickstart to route requests across models, the tracing quickstart to instrument an existing application, or the evals quickstart to start scoring output quality.

Question 16

How do I create a new Respan API key?

Accepted Answer

Go to platform.respan.ai, open the API keys page under settings, and create a new key. Copy it when it's shown, since keys are only displayed once. Respan API keys authenticate your requests to the gateway, the tracing SDKs, and the MCP server.

Question 17

How can I demo Respan?

Accepted Answer

There are two ways. To try it yourself, sign up and run npx @respan/cli setup. It detects your project language, installs the SDK, instruments your code, and creates an API key, so you're looking at your own traces in a few minutes. To see it applied to your use case with the team, book a demo.

Question 18

What is Respan's pricing?

Accepted Answer

Respan has three plans. Free is $0 and includes 100k logs, 1k scores, 5 datasets, 2 evaluators, 5 prompts, and unlimited seats on one workspace. Team is $199 per month billed yearly, and adds unlimited datasets, evaluators, and prompts, 10k scores, 5 members with additional seats at $15 each, 30-day retention, a private Slack channel, and a SOC 2 report. Enterprise is custom priced and adds SAML SSO, custom SLAs, a dedicated support engineer, and volume discounts. Usage beyond plan limits is billed at $8 per 100k logs and $1 per 1k scores.

Question 19

How do I add model fallback to Respan?

Accepted Answer

Set fallback models in the Respan platform under fallback settings, or pass fallback_models in your gateway request. Respan tries them in the order you specify, moving to the next model once the current one exhausts its eligible attempts. Retries run against the current model first, then the gateway advances to the next fallback, so longer chains and higher retry counts increase worst-case latency. See Retries and fallback order for the full execution sequence. Fallback reduces the impact of a provider failure but doesn't guarantee availability. Preflight rejections and fail-fast failures don't advance through the list, and if every eligible model fails, the gateway returns the terminal error.

Question 20

How do I add load balancers to Respan?

Accepted Answer

Create a load balancer on the Load balancing page, add the models you want to distribute across, and set a weight for each. Copy the group ID into your codebase and pass it as load_balance_group in your requests. Note that the model parameter overrides load_balance_group, so leave it out when you want the balancer to pick. You can also balance across deployments rather than models, where a deployment means one set of provider credentials, so adding two OpenAI keys gives you two deployments, each with its own weight. Load balancing only selects which route to try; see Retries and fallback order for what happens when that route fails.

Question 21

How do I add cost and rate limits to Respan?

Accepted Answer

Respan applies limits at three scopes. Per-API-key limits are set on each key's detail page and cover a lifetime cost cap, recurring caps on cost, requests, or tokens, and an expiry date. Organization spend caps and per-customer budgets are both under the limits page, where you set a billing period, a hard spend cap, and an optional warning threshold. Customer limits cover a monthly budget and a requests-per-minute rate limit, applied per customer_identifier. Every limit either warns, which fires a notification and lets the request through, or blocks, which rejects it in real time. Blocked requests never reach the provider, so retries and fallback don't run for them.

Question 22

How long does it take for traces to appear in Respan?

Accepted Answer

Traces appear in Respan within seconds of the request completing. If you don't see a trace, the most common cause is a process exiting before the SDK flushes its buffer, which happens with scripts and serverless functions.

Question 23

How do I track LLM cost and tokens in Respan?

Accepted Answer

Cost and token tracking is automatic. Every span records tokens, cost, latency, and the model used, whether the request came through the gateway, the tracing SDK, or a framework integration. On the metrics dashboard you can break requests, tokens, and cost down by model, user, and API key. To track spend by feature or by customer, tag spans with customer_identifier, metadata, or custom attributes, then filter and group on those. Setup details are in the metrics dashboards docs.

Question 24

How do I set up error tracking in Respan?

Accepted Answer

Error tracking works automatically once your traffic is flowing into Respan. Failed spans are grouped into error groups by error class, provider, endpoint, and HTTP status, and Respan detects incidents on its own when error rates spike or drift above their baseline. Open the errors page to see the timeline, filter by environment, error type, provider, fault domain, or HTTP status, and drill into an incident to see contributing groups and affected models and customers. You can acknowledge incidents, add internal notes, and mark groups resolved, ignored, or reopened. Incident detection is separate from monitors, which alert you when a metric crosses a threshold you define.

Question 25

How do I use Respan monitors?

Accepted Answer

A monitor is a saved rule that aggregates a metric over a time window and alerts you when it crosses a threshold. Create one at the monitors page from scratch or from a cost-spike, high-error-rate, or model-error template. The trigger reads as a sentence: when a metric of a source crosses an operator and threshold over a window. You can monitor count, errors, tokens, cost, latency, or limits, over windows from five minutes to seven days, and scope it with filters on model, provider, customer ID, prompt, API key, error type, or custom properties. Notifications go to email, Slack, Microsoft Teams, or a webhook. Send a test alert to confirm delivery, then deploy.

Question 26

How do I set up behaviors in Respan?

Accepted Answer

Behaviors detect semantic patterns in agent interactions, such as user frustration, agent laziness, jailbreak attempts, or a successful outcome. Unlike keyword or regex matching, a behavior captures meaning in context. Open Behaviors and select 'Enable behaviors' to turn them on, then use the status control in the behaviors table to choose which ones your organization tracks. Enablement is an organization-level setting and can require an admin. Respan ships six built-in behaviors: user frustration, laziness, unsafe, jailbreak, win, and escalation. You can create a custom behavior for a pattern specific to your product. Classification runs asynchronously, so a missing result means an interaction isn't classified yet, not that it passed.

Question 27

How do I create graphs in Respan?

Accepted Answer

Go to the metrics page and click 'New graph'. Templates are the fastest route: browse categories for models, latency, tokens, prompts, users, caches, storage, keys, providers, and scores, select the graphs you want, and save. For something a template doesn't cover, switch to 'Custom', give the chart a title, pick a metric and calculation such as P95 latency, optionally add span filters, and group the series by a dimension like model, provider, or customer ID. Check the preview, then create. Graphs can be dragged, resized, expanded, or removed, and the whole dashboard can be exported.

Question 28

How do I create prompts in Respan?

Accepted Answer

Go to the prompts page and click 'New prompt'. Add a title and description, then write your system and user prompts, using {{variable_name}} for dynamic content. Choose a default model and any fallback models, and adjust settings like temperature and max output tokens in the sidebar. Add values in the Variables tab and click 'Run' to test. When it's working, 'Commit' with a message to save the version, then 'Deploy' to push it to production. Call it from your code by prompt ID, which you'll find in the Overview panel on the Prompts page.

Question 29

How do I create evaluations in Respan?

Accepted Answer

Evals have four parts: something to test (a prompt or model), a dataset of test inputs, an evaluator that scores the outputs, and an experiment that runs them together. Build a dataset on the datasets page from a CSV or by sampling production spans and traces. Then create an evaluator on the evaluators page: start with a grader, which is the scoring unit and can be an LLM judge, a code check, or human review, then chain graders into a workflow using the visual block builder and deploy it. Finally, go to experiments, select your dataset, prompt version, and evaluators, and run it. Compare score distributions across experiments in the Analytics tab, then deploy the version that scores best.

Question 30

How can I use Respan for red-teaming?

Accepted Answer

Respan Red Team sends adversarial messages to your agent and evaluates its replies. In the platform, open the campaigns page and pick one of three connection methods: connect an agent using a Python adapter that runs alongside it, which works for private, stateful, or custom-auth agents; connect an endpoint if your agent exposes a browser-reachable OpenAI-compatible Chat Completions URL; or use the hosted ShopBot sandbox to try the workflow without touching your own systems. For the adapter route, install with pip install respan-redteam, authenticate, and run respan-redteam scan adapter.py --output report.json. Only assess agents you own or have explicit authorization to test.

Question 31

How can I use Respan with my team?

Accepted Answer

Invite your team from the members page: click 'Invite member', enter their email, and assign a role. They get an email invitation. Respan has four roles: Owner has full access including billing, Admin manages members, API keys, and project settings, Member can use the platform and view logs, traces, prompts, and metrics, and Annotator is scoped to annotating and evaluating spans, which suits QA reviewers and graders. Only Owners and Admins can change roles.

Question 32

How do I create automations in Respan?

Accepted Answer

Go to automations and choose a type. 'Online evals' runs an evaluator against live traffic: select a condition, select the evaluator, and set a sampling rate, starting around 0.1 for 10% of traffic keeps cost down. 'Alert' fires on aggregated metrics, for example when an average evaluation score drops below a threshold over the last hour, and sends to a webhook such as Slack or PagerDuty. Conditions are created separately under 'Conditions', where you set a type such as single log and a filter like metadata.agent = "support".

Question 33

How do I create recurring reports with Respan?

Accepted Answer

Go to the reports page, toggle the usage report active, then set a schedule by selecting which days of the week to send on and a time in your timezone. Under Metrics, choose what the digest includes (spans, errors, users, cost, tokens, latency, and Pulse) then group by API key, model, customer, or prompt, and set how many top results to show per group. Under Delivery, send by email to one or more addresses. The report covers activity since the last one and shows period-over-period changes.

Question 34

How do I add webhooks to Respan?

Accepted Answer

Go to the webhooks page and select 'Create webhook'. Give it a name that identifies the receiving system and environment, enter the full HTTPS endpoint URL, choose the 'span' event type, and select the API key whose traffic should be forwarded. Data webhooks are scoped by event type and API key, so using a separate key per application or environment keeps the scope easy to reason about. Note that this is different from selecting a webhook inside a monitor, which is a notification destination rather than a data feed.

Question 35

How do I export data from Respan?

Accepted Answer

Start from the table you want to export, such as Spans or Traces, and use the download control in the toolbar. Set your time range and filters, choose a sampling percentage, select which columns to include, and pick CSV for tabular analysis or JSONL for one JSON record per line. Choose Recurring export instead of a one-time job when a downstream system needs a refreshed copy (recurring exports currently run every 15 minutes). Manage and review jobs on the exports page, which collects jobs started from logs, traces, users, or datasets.

Question 36

Does Respan have a status page?

Accepted Answer

Yes. Respan's status page shows current status and 90-day history for platform web, logging services, and gateway services, with separate sections for cloud and enterprise.

Question 37

How do I contact Respan for support?

Accepted Answer

The fastest route for technical questions is the Respan Discord, where the team and community answer setup and debugging questions. Team plans and above get a private Slack channel with direct access to the Respan team. For account, billing, or anything you'd rather not post publicly, you can email us. Sales questions, demos, and enterprise inquiries schedule a demo. Most setup and troubleshooting questions are already answered in the docs.

Frequently Asked Questions

About Respan

Security & compliance

Platform & compatibility

Getting started

Gateway & routing

Observability & evals

Workspace & data