Skip to main content
MLflow

What Is MLflow? GenAI Tracing, Evals & AI Gateway (2026)

Founded 2018|San Francisco, CA (Linux Foundation governed)|900+ contributors people|Free and open source (Apache 2.0), with managed versions billed through Databricks, Amazon SageMaker AI, or Azure Machine Learning

Respan Gateway: up to 55% off tokens on Gemini, GPT, Claude and 1,000+ more models. One API key, 0% fee.

What is MLflow?

Key features

Core capabilities this platform advertises.

  • OpenTelemetry-based tracing for LLMs and agents
  • 50+ built-in eval metrics and LLM judges
  • Prompt versioning and automatic prompt optimization
  • AI Gateway with routing, rate limiting, and fallbacks
  • Agent server for deploying agents on FastAPI

Strengths and tradeoffs

What this tool does well, and the limitations to keep in mind.

Pros

  • Apache 2.0 license with Linux Foundation governance
  • Tracing, evals, prompts, and a gateway in one open-source project
  • Covers classic ML and GenAI in the same platform
  • Managed options on Databricks, SageMaker AI, and Azure ML
  • Auth and role-based access built into the open-source server

Cons

  • Self-hosting means running the tracking server, database, and artifact store yourself
  • Auth and role-based access are opt-in and need configuring on self-hosted servers
  • Managed versions bill through a cloud platform's compute rather than a standalone plan
  • The broader ML scope can mean more to learn for teams that only need LLM tooling

Plans & pricing

What's included in each plan, and how the tiers compare.

Open Source

Free

 

  • Full platform under Apache 2.0, self-hosted, all GenAI features

Managed on Amazon SageMaker AI

$0.60

Per hour

  • Hosted MLflow tracking server ($0.60/hr small, $1.40/hr medium), plus $0.10/GB-month for metadata storage

Managed on Azure Machine Learning

No added charge

Usage-based

  • MLflow tracking in Azure ML at no extra fee, with compute, storage, and other Azure services billed separately

Managed on Databricks

Usage-based

(DBUs)

  • Hosted MLflow billed through Databricks compute, with rates set by cloud and workload in Databricks' pricing calculator
View official pricing page

Common use cases

ML engineers and AI teams, especially those in the Databricks ecosystem

  • Tracing and evaluating LLM apps and agents
  • Versioning and optimizing prompts
  • Routing LLM traffic through a self-hosted gateway
  • Tracking classic ML experiments alongside GenAI work
  • Running AI tooling inside Databricks

Using MLflow with Respan

Respan covers the same LLM work as MLflow's GenAI features as a managed platform, with the AI router, tracing, evals, and prompt management built together. There's no tracking server, database, or artifact store to run, and every score links back to the trace behind it.

  • Nothing to host: tracing, evals, and prompts are ready on signup
  • Nested traces for every LLM call, tool run, and agent turn
  • Evals on test sets before shipping and on live production traffic after
  • Prompt versions that ship without a code deploy
  • One API for 1,000+ models with automatic fallbacks

Trace and evaluate every LLM call, with nothing to host

Up to 55%Off tokens
1,000+Models
0%Fee

Respan traces every LLM call and agent run, scores output on live production traffic, and routes across 1,000+ models in one managed platform. Start free.

Similar tools in Observability, Prompts & Evals

Top companies in Observability, Prompts & Evals you can use instead of MLflow.

View all MLflow alternatives

Compare MLflow

Side-by-side comparisons with other tools in this category.

Best integrations for MLflow

Companies from adjacent layers in the AI stack that work well with MLflow.

Last verified: March 27, 2026