FeaturesMonitoring

Monitors

Set up metric-based alerts to catch regressions in your LLM applications.

  1. Sign up — Create an account at platform.respan.ai
  2. Create an API key — Generate one on the API keys page
  3. Add credits or a provider key — Add credits on the Credits page or connect your own provider key on the Integrations page

Add the Docs MCP to your AI coding tool to get help building with Respan. No API key needed.

1{
2  "mcpServers": {
3    "respan-docs": {
4      "url": "https://mcp.respan.ai/mcp/docs"
5    }
6  }
7}

What is a monitor?

A monitor watches a metric over time and sends you a notification when it crosses a threshold. You can use monitors to track error rates, latency, cost, token usage, and more across your LLM applications.

For example:

  • Alert when error rate spikes above 10%
  • Alert when total cost exceeds $1.00 in a 5-minute window
  • Alert when average latency crosses a threshold

Monitor templates

When creating a new monitor, you can start from a pre-built template to get up and running quickly:

TemplateDescription
Cost spike monitorAlert when total cost exceeds a threshold over a 5-minute window
Average cost per request monitorAlert when average cost per request exceeds a threshold over a 10-minute window
High error rate alertAlert when error rate exceeds a threshold over a 10-minute window

Select a template to pre-fill the metric, interval, and threshold — then adjust as needed.

How to create a monitor

Go to Monitors and click + New Monitor.

1. Define your metric

Select the metric you want to track:

MetricDescription
CountTotal number of events
ErrorsError count or error rate
TokensTotal token usage
CostDollar cost of LLM calls
LatencyResponse time

Then configure:

  • Source — Whether to monitor individual logs or full traces.
  • Interval — The time window for aggregation (e.g., every 5 minutes, 1 hour).

2. Filter events

Narrow down which events the monitor watches. You can filter by model, project, environment, tags, users, or any other span attributes.

3. Set thresholds

Define when the monitor should fire. Set a threshold.

4. Configure notifications

Select how and where you want to be notified when the threshold is breached.

Supported channels:

  • Email
  • Slack
  • Webhook

You can customize the notification subject and message content, and add multiple notification actions per monitor.

Once saved, deploy the monitor. When the metric crosses your threshold, alerts are sent automatically through your configured channels.

Quick setup from dashboard

You can also create a monitor directly from any metric chart on the dashboard. Click the monitor action on a chart to pre-fill the metric, source, and interval — then adjust thresholds and notifications as needed.