FeaturesMonitoring

Monitors

Set up metric-based alerts to catch regressions in your LLM applications.

  1. Sign up. Create an account at platform.respan.ai
  2. Create an API key. Generate one on the API keys page
  3. Add credits or a provider key. Add credits on the Credits page or connect your own provider key on the Integrations page

Add the Docs MCP to your AI coding tool to get help building with Respan. No API key needed.

1{
2  "mcpServers": {
3    "respan-docs": {
4      "url": "https://mcp.respan.ai/mcp/docs"
5    }
6  }
7}

What is a monitor?

A monitor watches a metric over time and sends a notification when it crosses a threshold. Use monitors to catch regressions in error rate, cost, token usage, and latency across your LLM applications.

For example, create a monitor when error rate rises above 5% over 10 minutes, when total cost goes above $1.00 over 5 minutes, or when average latency stays above 3 seconds.

Set up a monitor

Go to Monitors and click New monitor.

The monitor editor has two parts: Trigger and Notifications.

Trigger

Use Trigger to define the condition that should fire the alert:

When [metric] of [source] is [threshold condition] over [time window]

Choose the metric, source, threshold, and time window. Add Where conditions to narrow the monitor to a model, project, environment, user, or other event attributes.

Notifications

Use Notifications to choose where alerts are sent. Send alerts to email, Slack, or a webhook. Select an existing destination or create one while editing. For email and Slack, you can keep the default message or customize the subject and body. For webhooks, enter the destination URL directly. A monitor can send alerts to more than one destination.

Deploy and manage monitors

Use Send test alert to validate the configuration. When the monitor is ready, click Deploy to make it live.

The monitors list shows status and version information for each monitor.

Open a monitor and switch to Overview to view activity and monitor details.

Open the versions panel to review earlier versions. From there, you can pause or resume a deployed monitor and restore an earlier version as a new draft.

Create a monitor from a dashboard chart

You can also create a monitor from a metric chart on the dashboard. The chart action prefills the metric, source, and interval. Adjust the trigger and notification settings, then deploy the monitor.