8 Best LiteLLM Alternatives in 2026 (Post-Breach)

On March 24, 2026, a supply chain attack hit LiteLLM, one of the most widely used open-source AI gateways in production. A threat group called TeamPCP exploited a cascading CI/CD compromise that started with Aqua Security's Trivy scanner, stole a PyPI publishing token, and pushed two malicious package versions containing a multi-stage credential stealer.

The malware harvested SSH keys, cloud credentials, Kubernetes secrets, .env files, and, most relevant here, the LLM API keys for every provider configured on the machine. Wiz found LiteLLM present in 36% of the cloud environments it analyzed, and plenty of those teams never chose it directly, since DSPy, CrewAI, MLflow, and OpenHands all pull it as a transitive dependency.

The incident didn't make LiteLLM a bad tool. It made the operating cost visible. Self-hosting an OpenAI-compatible proxy is free to install and expensive to run, and the bill arrives as dependency management, patching, scaling, incident response, and the observability stack you have to bolt on separately because LiteLLM doesn't ship one.

Below are the 8 best LiteLLM alternatives, compared on what they cost to run, what they cover beyond routing, and where your provider credentials actually live.

What is LiteLLM?

LiteLLM is an open-source proxy that translates OpenAI-format API calls into the request shape each provider expects. Point your client at it, change a model string, and the same code reaches Anthropic, Bedrock, Vertex, Azure, or any of 100+ providers without a rewrite. The core is MIT-licensed and maintained by BerriAI, and it runs either as a Python library imported into your application or as a standalone proxy server in front of it.

What it does not include is a way to see what happened inside a request. LiteLLM has no built-in tracing, so observability means configuring an external callback handler and operating a second system alongside the gateway. There is also no support for distributed tracing, which means client-side agent traces and proxy-side request logs stay disconnected. Debugging a multi-step agent across that gap is manual work. The same is true for evaluations and prompt management, neither of which LiteLLM attempts. Those are separate tools you adopt and integrate yourself.

LiteLLM pricing: what it actually costs to run

LiteLLM is $0. The open-source proxy is free forever, self-hosted, with no license fee at any request volume, and it covers more than most teams expect from a free tier: 100+ providers behind one OpenAI-compatible endpoint, virtual keys scoped to users and teams, budgets, rate limits, spend tracking, fallbacks, request and response logging, and Prometheus metrics.

Enterprise is where the number stops being public. It's quoted by sales, billed annually, and sized to your gateway request capacity and deployment architecture rather than per token, across two tiers called Standard and SCALE. What you're buying is governance: SSO and SCIM, OIDC and JWT auth, audit logs, secret manager integration with key rotation, org and team admins, a multi-region control plane, and support with SLAs up to 24/7. Both tiers stay self-hosted, and both can run air-gapped.

The threshold most teams hit first is SSO, which is free up to five users and licensed above that.

The March 2026 LiteLLM supply chain attack

The attack didn't start at LiteLLM. TeamPCP spent March 19 through 27 working through the tools engineers install to stay safe, force-pushing malicious commits across Trivy's GitHub Actions and Checkmarx KICS before reaching PyPI. LiteLLM ran Trivy in its own CI security scanning workflow. The compromised scanner could read the runner's memory, and the PyPI publishing token was in that memory.

Two releases went out with that token, thirteen minutes apart. Version 1.82.7 hid a payload in proxy_server.py that fired when the proxy started. Version 1.82.8 kept that and added a .pth file, so the payload also ran on every Python interpreter start whether or not anything imported LiteLLM. It gated itself on the GITHUB_ACTIONS variable, activating specifically in CI, where secrets concentrate. PyPI quarantined both versions at 11:25 UTC, inside an hour of the first upload.

The stealer took SSH keys, cloud credentials, Kubernetes secrets, .env files, and the LLM API keys for every provider configured on the machine. That last one is the point. A gateway holds credentials for everything you route to, so compromising it hands over every provider at once instead of one. It was also caught by accident, when a fork bomb in the payload spiked CPU and a researcher hit it while testing a plugin that pulled LiteLLM in as a transitive dependency. He never chose LiteLLM. Neither have a lot of teams running it.

BerriAI pulled the packages, engaged Mandiant, published a timeline the same day, and froze releases until the pipeline was rebuilt. Teams on the official Docker image were never exposed, because that path pins dependencies. If you installed 1.82.7 or 1.82.8, treat it as full credential exposure rather than a package you can uninstall, and rotate provider keys, cloud credentials, SSH keys, and Kubernetes secrets.

Here's what actually changed:

• A pip-installed gateway inherits the security posture of every dependency in its build chain, and that chain now includes the scanners.
• Pinning versions, reviewing upgrades before they ship, and keeping publishing scope off shared runners are ordinary practices most teams do halfway.

March priced what halfway costs. If your gateway is a package you install, that work is yours, permanently, and it belongs in the cost of running it.

What to look for in a LiteLLM replacement

The incident sharpened a set of questions that were already worth asking of any gateway. These are the ones that separate the options below.

• Credential location - Whether provider keys sit in your environment variables and CI secrets, in a managed key vault, or in a vendor's infrastructure. This determines what a compromise anywhere in your build chain can reach.
• Built-in observability - Whether traces, cost, latency, and errors come with the gateway or require a second platform and a callback integration to maintain.
• Deployment control - Whether you can run it in your own VPC, on-prem, or air-gapped when data residency or compliance requires it.
• Certifications and access controls - SOC 2, HIPAA, GDPR, ISO 27001, SSO, RBAC, and PII handling, and which tier each one is gated behind.
• Total operating cost - License plus infrastructure plus the engineering time to run it, measured against a subscription rather than against zero.
• Evaluation and prompt tooling - Whether quality measurement lives in the same system as the traffic it scores, or in a separate tool with no link back to the request.

Teams whose only real requirement is routing across providers can stop at a thinner tool than the ones below.

LiteLLM alternatives compared

8 best LiteLLM alternatives

1. Respan

Respan runs the gateway and everything you would otherwise attach to it on one platform. One endpoint reaches 1,000+ models across every major provider, with automatic retries, failover, load balancing, semantic caching, and per-key spend and rate limits. Because traffic and telemetry share a system, a cost spike or a latency regression opens directly into the trace tree of the requests that caused it.

Provider credentials live in a managed key vault rather than in environment variables on machines your CI can reach, so a compromised dependency has nothing to steal. Evaluations run against production traffic in real time by attaching an evaluator to calls flowing through the gateway, so quality is scored where it happens instead of in a separate tool with no path back to the request.

Free covers 100k logs and 1k scores with 7-day retention and 412 requests per minute of proxy throughput. Team lifts that to 30-day retention and 8,400 requests per minute. HIPAA is a $249/mo add-on on Team and included on Enterprise, and self-hosting is Enterprise-only.

Security features:

• Managed key vault - Provider credentials stay out of your environment variables, CI secrets, and application config.
• SOC 2, HIPAA, GDPR, ISO 27001 - Full certification coverage, with a signed BAA available for healthcare workloads.
• PII masking and log omission - Strip sensitive fields before they are stored, or skip storage entirely for specific calls.
• SSO and admin roles - Google SSO on Free and Team, SAML on Enterprise, with advanced admin roles from Team.
• Red teaming - Test deployed agents to see what they block and what gets through.
• Spend and rate controls - Per-key budgets and rate limits enforced at the gateway.

Pricing: Free is $0 with 100k logs, 1k scores, and unlimited seats. Team is $199/mo billed yearly for 5 members at $15 each after that, and Enterprise is custom with self-hosting, HIPAA BAA, and a 99.99% uptime SLA.

2. Portkey

Portkey pairs routing with logging, guardrails, and governance in one control plane. A universal API covers a large model catalog with fallbacks, load balancing, retries, timeouts, and both simple and semantic caching. With Portkey and similar tools, guardrails run at the request layer, including PII anonymization and evaluation templates.

The $49 tier is not the product most teams leaving LiteLLM would actually buy. SSO, VPC hosting, compliance certificates, custom retention, and private tenancy all sit at Enterprise, and the free Developer tier is labeled by Portkey itself as unsuitable for production. Logs are the billing unit, so cost tracks observability volume rather than request volume, and paid overage stops at 3M requests before you are into a sales conversation regardless.

Security features:

• Guardrails - PII anonymizer and deterministic checks, with custom hooks on Enterprise.
• RBAC and virtual keys - Role-based access and budget-scoped keys from the paid tier.
• SSO with Okta - Enterprise only.
• Compliance certificates - SOC 2, ISO 27001, GDPR, HIPAA with custom BAAs, all Enterprise.
• VPC hosting and private tenancy - Managed deployment inside your own cloud, Enterprise only.

Pricing: The open-source gateway is free to self-host and Developer is free with 10k logs a month. Production is $49/mo for 100k logs with $9 per additional 100k, and Enterprise is custom above 10M.

Note that Palo Alto Networks acquired Portkey in 2026, and it now ships as the Prisma AIRS AI Gateway, and the buying process runs through Palo Alto rather than Portkey.

3. TrueFoundry

If self-hosting was the requirement and the operations were the problem, TrueFoundry splits the difference. The gateway runs inside your own Kubernetes cluster rather than as a hosted proxy, so prompts and keys stay inside your network boundary, with fallbacks, rate limiting, and RBAC applied per model. Around it sit model serving, GPU orchestration, agent deployment, and an MCP gateway with schema validation for agent tools.

However, self-hosted does not mean cheaper with TrueFoundry. Running the gateway plane in your own cloud adds roughly $600 to $1,000 a month in infrastructure on top of the license, so you are paying a subscription and an infra bill rather than trading one for the other. The platform also assumes Kubernetes, which is either already true for you or a serious adoption project, and the jump from Pro to the tier carrying SSO and compliance certificates is six times the price.

Security features:

• VPC, on-prem, and air-gapped deployment - Prompts, data, and models stay in your environment.
• RBAC and audit logging - Per-model access control with immutable audit logs.
• SOC 2, HIPAA, GDPR - Certificates available from Pro Plus.
• Real-time policy enforcement - Data residency, usage quotas, and spend controls at the gateway.
• Scoped access tokens - Personal and virtual tokens with granular permissions.

Pricing: Developer is free to 50k requests a month. Pro is $499/mo for 1M requests, Pro Plus is $2,999/mo and adds SSO, compliance certificates, and air-gapped deployment, and Enterprise is custom above 10M.

4. Bifrost

Migration is a base URL change. Bifrost speaks the same OpenAI-compatible format LiteLLM does, ships as a binary, Docker container, or Kubernetes workload, and is Apache 2.0 with no separately licensed components. It handles automatic failover, weighted load balancing across keys, semantic caching, native MCP as both client and server, and a budget hierarchy running from customer down to individual provider config. Maxim AI reports 11 microseconds of overhead at 5,000 requests per second, which is a vendor benchmark and worth testing against your own traffic.

Check your provider list before anything else. Bifrost supports 20+ providers against LiteLLM's 100+, and if you depend on a long-tail model the migration ends there. It is also a younger project with a smaller community and fewer production incidents behind it, and self-hosting means the operational ownership that made LiteLLM expensive does not go away. The dependency surface is smaller, not absent.

Security features:

• Self-hosted only - Keys and prompts never leave your infrastructure.
• Budget hierarchy - Spend and rate limits enforced per customer, team, virtual key, and provider config.
• Five auth types - None, headers, OAuth 2.0, per-user OAuth, and per-user headers.
• Air-gapped deployment - Enterprise tier.
• Apache 2.0 - Full source auditability, no dual-licensed components.

Pricing: Free and open source at any scale. Enterprise is custom and adds air-gapped support, SLAs, a private support channel, and custom plugin work.

5. OpenRouter

OpenRouter removes the gateway from your operations budget entirely. One key and a prepaid credit balance reach hundreds of models across dozens of providers, with automatic fallback on provider outages and token rates passed through at each provider's published price. Nothing to deploy, patch, or page anyone about.

OpenRouter charges 5.5% on credit purchases with a $0.80 minimum, so a $10 top-up effectively costs 8% while a $500 one costs 5.5. Observability stops at per-model spend and usage rather than traces, there is no RBAC or guardrail layer, and every request routes through third-party infrastructure, which is the specific arrangement most GDPR and HIPAA constraints rule out before any of the rest matters. If that doesn't work for you, there are OpenRouter alternatives that have different pricing structures that might work better.

Security features:

• Vendor-managed infrastructure - No proxy in your build chain to compromise.
• BYOK - Route on your own provider keys rather than through OpenRouter credits.
• SSO and SAML - Enterprise only.
• Contractual SLAs - Negotiated at Enterprise; no public SLA below it.

Pricing: Free to start with a set of no-cost models under rate limits. Paid usage is pass-through token pricing plus 5.5% on credit purchases with a $0.80 minimum, and Enterprise is custom.

6. MLflow

MLflow is the only option here where tracing arrives with the gateway rather than bolted to it. Requests routed through its AI Gateway become traces automatically once usage tracking is on, and W3C TraceContext links client-side agent spans to gateway spans in the same view, so the callback handler and the second observability bill both go away. That is the LiteLLM gap closed without leaving Apache 2.0.

Provider coverage runs well short of LiteLLM's long tail, so check your model list before committing. MLflow also carries a decade of classical ML surface area, experiment tracking, model registry, fine-tuning workflows, none of which has anything to do with routing LLM calls but all of which you reason about during setup. Self-hosting means the same patching, scaling, and on-call you already own. What you get in exchange is a project governed by the Linux Foundation rather than a single vendor, so the license terms and the roadmap aren't one acquisition away from moving.

Security features:

• Self-hosted - Runs inside your infrastructure, inheriting your existing controls.
• Linux Foundation governance - Apache 2.0 with no separately licensed enterprise tier.
• Guardrails in the open-source gateway - Not gated behind a paid plan.
• Large contributor base - More review surface on the code than a single-vendor project.

Pricing: Free and open source under Apache 2.0. You pay for the infrastructure you run it on and the tokens you consume.

7. Helicone

Helicone is a logging proxy. Point your base URL at it and requests appear on a dashboard with prompt and response bodies, token counts, latency, error rates, and cost per call, searchable through its own query language. It is open source and self-hostable, so teams that need control over log retention can run tools like Helicone themselves.

The issue is that Helicone's roadmap is frozen, as it was acquired by Mintlify in March 2026 and the team has said the product is in maintenance mode: security patches, new model support, and bug fixes continue, feature work has stopped. A logging proxy that works today does not break when development stops, so if Helicone does exactly what you need, that may be fine for a long time. If you are choosing a platform to grow into, it is the wrong shape. Routing is also thinner than a purpose-built gateway, which makes it a better observability layer than a full LiteLLM replacement.

Security features:

• Self-hostable - Run the whole stack yourself for full control over log retention.
• SOC 2 and HIPAA - Team tier and above.
• Open source - Full code auditability.
• Dedicated support channel - Team tier and above.

Pricing: Hobby is free with 10,000 requests, 1 GB of storage, and one seat. Pro is $79/mo with unlimited seats and Team is $799/mo adding SOC 2 and HIPAA, both with usage-based charges on top.

8. Vercel AI Gateway

Zero markup is true for tokens and false for the controls a compliance-driven team would turn on. The Vercel AI Gateway passes tokens through at provider list price with no fee, including on BYOK, and gives you a unified endpoint with load balancing, automatic failover, and a dashboard covering requests by model, time to first token, token counts, and spend. Team-wide zero data retention and the provider allowlist each cost $0.10 per 1,000 requests, and custom reporting is billed separately.

The free tier also ends the moment you outgrow it, since buying credits permanently stops the $5 monthly allowance rather than supplementing it. Beyond pricing, the gateway is coupled to the Vercel platform, observability stops at gateway metrics rather than agent traces, and, like similar tools to Vercel's gateway, there is no evaluation or prompt management layer.

Security features:

• Zero data retention routing - Routes only through providers with ZDR agreements and fails closed when none is available.
• Provider allowlist - Restrict which providers receive traffic, billed per request team-wide.
• BYOK - Use your own provider credentials with no added fee.
• Vendor-managed infrastructure - No proxy to operate or patch.

Pricing: Every Vercel team account gets $5 in gateway credits monthly from first use. Past that it is pay-as-you-go at provider list price, though ZDR and allowlist controls add $0.10 per 1,000 requests each.

Frequently asked questions

What is the best LiteLLM alternative?

It depends on what LiteLLM was doing for you and what you were paying to make up the difference. Respan is the strongest fit for teams running LiteLLM alongside a separate observability tool, because the gateway, tracing, evaluations, and prompt management sit in one platform and a metric always ties back to the run behind it. Teams that need Kubernetes-native self-hosting will find TrueFoundry closer to their architecture, and teams who want to stay open source with the fewest changes should test Bifrost, which speaks the same OpenAI-compatible format.

Is there a free LiteLLM alternative?

Several, though "free" means different things. Respan's Free tier is a hosted gateway with 100k logs, 1k scores, unlimited seats, and no infrastructure to run, so nothing sits in your cloud bill. Bifrost and MLflow are free under Apache 2.0 but self-hosted, which means you own the compute, patching, and on-call the same way you did with LiteLLM. Portkey, TrueFoundry, Helicone, and Vercel all offer free tiers with request or log caps.

Is LiteLLM safe to use after the supply chain attack?

The compromised versions, 1.82.7 and 1.82.8, were quarantined by PyPI at 11:25 UTC, inside an hour of the first upload, and 1.83.0 shipped afterward through a rebuilt CI/CD pipeline with isolated environments and stronger release gates. Exposure depended entirely on install path, since the official Docker image pinned its dependencies and never pulled the compromised wheels. If you installed either affected version during the window, treat it as a full credential exposure rather than a package problem: rotate provider keys, cloud credentials, SSH keys, and Kubernetes secrets, and investigate for persistence and outbound traffic.

How much does it cost to self-host LiteLLM?

The license is $0. The rest depends on your scale and is not published anywhere because it lands in your own cloud bill: compute for the proxy, a Postgres instance, Redis if you cache, load balancers, and multi-region redundancy if the gateway is on your critical path. Add the engineering time for dependency review, patching, and incident response, and a separate observability subscription since LiteLLM has no built-in tracing.

Does LiteLLM have built-in observability?

No. LiteLLM logs requests and responses and exposes Prometheus metrics, but there is no tracing layer. Gaining visibility means configuring an external callback handler such as Datadog or Langfuse and operating that system alongside the gateway. It also has no distributed tracing support, so client-side agent traces and proxy-side request logs stay disconnected and end-to-end debugging is manual.

Can you migrate off LiteLLM without rewriting your code?

Usually yes. LiteLLM's value is its OpenAI-compatible interface, and most alternatives here speak the same format, so migration is typically a base URL change and a key swap rather than a refactor. Respan, Bifrost, Portkey, OpenRouter, and Vercel all accept OpenAI-format requests. What needs planning is everything around the calls: virtual keys and budget rules have to be recreated, and any callback-based logging integration is replaced rather than ported. Run both paths in parallel for a day and reconcile cost and token counts against your provider invoice before cutting over.

Respan gives you one endpoint for 1,000+ models with tracing, evaluations, and cost controls in the same platform, so the observability stack you bolted onto LiteLLM comes included rather than as a second bill.