Basic
$0
- Includes: Up to 5,000 events, agent-agnostic SDK, LLM cost tracking across 400+ LLMs, replay analytics
Respan Gateway: up to 55% off tokens on Gemini, GPT, Claude and 1,000+ more models. One API key, 0% fee.
AgentOps is an observability platform for AI agents built around the session rather than the single request. A Python SDK, with a TypeScript SDK in alpha, records LLM calls, tool calls, and multi-agent interactions, and the dashboard replays each session step by step so you can rewind to the point where an agent went wrong.
The SDK and the dashboard app are both open source under the MIT license, and the app can be self-hosted. Native integrations cover CrewAI, AG2 (formerly AutoGen), the OpenAI Agents SDK, LangChain, LlamaIndex, and others, and CrewAI ships an AgentOps option built in. Cost and token tracking run per session, with prices kept current across 400+ LLMs.
Evaluation is the thinner side of the product. The SDK supports custom eval metrics, while agent scorecards and an evaluation playground are still listed as upcoming on AgentOps' own roadmap. Prompt management and model routing aren't part of the product, so those jobs fall to other LLMOps tools.
Core capabilities this platform advertises.
What this tool does well, and the limitations to keep in mind.
Pros
Cons
What's included in each plan, and how the tiers compare.
$0
$40
Per month, pay as you go
Custom
Contact sales for a quote
Developers building multi-agent systems with frameworks like CrewAI, AG2, and the OpenAI Agents SDK who want session-level debugging
Respan has a native AgentOps integration. The respan-instrumentation-agentops package takes the spans your AgentOps decorators already create and exports them through Respan, without starting AgentOps' own exporter, so you keep your instrumentation and add Respan's evals, monitors, and gateway on top.
Respan's AgentOps integration exports the spans your decorators already create, so every session lands in Respan without re-instrumenting. Score sessions on live traffic and get alerted the moment a metric trips, with routing across 1,000+ models built in. Start free.
Top companies in Observability, Prompts & Evals you can use instead of AgentOps.
Respan
Nested traces for every LLM call, tool run, and agent turn
LangSmith
Native tracing for LangChain and LangGraph
MLflow
OpenTelemetry-based tracing for LLMs and agents
Weights & Biases
ML experiment tracking
Arize AI
ML observability with LLM support
Langfuse
Tracing for LLM calls, tool runs, and agent steps
Helicone
Traceloop
OpenTelemetry
Datadog LLM
LLM monitoring within Datadog platform
Braintrust
Experiments comparing prompts and models
HoneyHive
Prompt management
Promptfoo
Patronus AI
Automated LLM evaluation platform
Phoenix
OpenTelemetry-based LLM and agent tracing
Humanloop
Portkey
DeepEval
Ragas
RAG-specific evaluation framework
Sentry
LangWatch
Multi-turn agent simulation testing
Galileo AI
LLM output quality evaluation
PromptLayer
Maxim AI
Distributed tracing for LLM and agent apps
Confident AI
DeepEval open-source evaluation framework
Opik
Agenta
Future AGI
Multimodal evaluation (text, image, audio, video)
Lunary
Parea AI
Sentrial
Agent failure root cause analysis
Chamber
ML infrastructure automation
Moda
Hallucination detection
Athina AI
Ashr
Multi-modal synthetic testing
Prefactor
Traces and spans for every agent run, with cost and data risk per step
Fiddler AI
Agentic observability across multi-step agent workflows
Laminar
Agent tracing with a readable transcript view
OpenObserve
Logs, metrics, traces, RUM, and session replay
Side-by-side comparisons with other tools in this category.
Companies from adjacent layers in the AI stack that work well with AgentOps.