What Is LiteLLM? Proxy Features, Pricing, Pros & Cons

Founded 2023|San Francisco, California|11-50 people|Free and open source (MIT), with an Enterprise license priced by sales

What is LiteLLM?

LiteLLM is an open-source LLM gateway and Python SDK from BerriAI that puts 140+ providers behind one OpenAI-compatible API, licensed under MIT. You run the proxy yourself with Docker, Kubernetes, or Terraform, and it handles virtual keys, budgets, rate limits, fallbacks, load balancing, and spend tracking per user and team.

The open-source proxy is free at any volume, and an Enterprise license adds SSO, RBAC, audit logs, and support SLAs at a price quoted by sales. Because it's self-hosted, uptime, scaling, and security of the proxy sit with your team, and in March 2026 two LiteLLM releases on PyPI were compromised in a supply chain attack before a clean version shipped.

For tracing and evaluation, LiteLLM sends request logs to integrations like Langfuse, Arize Phoenix, and LangSmith, so proxy logs and the traces from the rest of an agent end up in separate tools, which is why teams often compare LiteLLM alternatives that keep routing and evaluation together.

Key features

Core capabilities this platform advertises.

OpenAI-compatible proxy for 140+ providers
Virtual keys scoped to users and teams
Budgets, rate limits, and spend tracking
Fallbacks, retries, and load balancing
Guardrails for PII masking and prompt injection

Strengths and tradeoffs

What this tool does well, and the limitations to keep in mind.

Pros

MIT-licensed proxy, free at any volume
140+ providers behind one OpenAI-compatible API
Virtual keys and budgets per user and team
Runs inside your own infrastructure

Cons

You run, scale, and secure the proxy yourself
Two PyPI releases were compromised in a March 2026 supply chain attack
SSO, RBAC, and audit logs require the Enterprise license
Tracing and evals depend on separate tools

Plans & pricing

What's included in each plan, and how the tiers compare.

Open Source

Free

MIT-licensed proxy and SDK, 140+ providers, virtual keys, budgets, rate limits, fallbacks, guardrails, self-hosted

Enterprise

Custom

Contact sales for a quote

Everything in open source, plus SSO, RBAC, audit logs, dedicated support, custom SLAs, billed annually

View official pricing page

Common use cases

Engineering teams who want a self-hosted, open-source LLM proxy for managing providers, keys, and budgets

Self-hosted LLM gateway for data control
Standardizing LLM access across teams
Budget enforcement per team or project
Switching providers without code changes
Issuing scoped API keys to internal users

Using LiteLLM with Respan

Respan has a native LiteLLM integration. The respan-instrumentation-litellm package registers a LiteLLM callback, so every completion across every provider LiteLLM calls is traced in Respan, and you can optionally route through Respan's gateway instead of running your own proxy.

Trace every LiteLLM completion with a native callback
All providers logged through the same callback
Evals on live production traffic, linked to each trace
Optional routing through Respan's gateway, with no proxy to run
Hard spend limits per key, per customer, or org-wide

One API for every provider, with no proxy to run.

10KFree traces/mo

1,000+Models

5 minSetup

Respan's LLM gateway gives you automatic failover, hard spend limits, and every call logged as a span with cost and latency attached. Routing and debugging in the same place, running in minutes.

Try Respan free

Best LiteLLM alternatives & competitors

Top companies in LLM Gateways you can use instead of LiteLLM.

Respan

One API for 1,000+ models