8 Best LiteLLM Alternatives in 2026 (Post-Breach)

On March 24, 2026, a supply chain attack hit LiteLLM, a widely used open-source AI gateway in production. A threat group called TeamPCP exploited a cascading CI/CD compromise that started with Aqua Security's Trivy scanner, stole a PyPI publishing token, and pushed two malicious package versions containing a multi-stage credential stealer.

The malware harvested SSH keys, cloud credentials, Kubernetes secrets, .env files, and, most relevant here, the LLM API keys for every provider configured on the machine. Plenty of the teams affected never chose LiteLLM directly, since DSPy, CrewAI, MLflow, and OpenHands all pull it in as a transitive dependency.

The incident didn't make LiteLLM a bad tool, but it did make the operating cost visible. Self-hosting an OpenAI-compatible proxy is free to install and expensive to run, and the bill arrives as dependency management, patching, scaling, incident response, and the observability stack you have to bolt on separately because LiteLLM doesn't ship one.

Below are the 8 best LiteLLM alternatives, compared on what they cost to run and how much of the stack around the gateway they replace.

LiteLLM alternatives at a glance

What is LiteLLM?

LiteLLM is an open-source proxy that translates OpenAI-format API calls into the request shape each provider expects. Point your client at it, change a model string, and the same code reaches Anthropic, Bedrock, Vertex, Azure, or any of 100+ providers without a rewrite. The core is MIT-licensed and maintained by BerriAI, and it runs either as a Python library imported into your application or as a standalone proxy server in front of it.

What it does not include is a way to see what happened inside a request. LiteLLM has no built-in tracing, so observability means configuring an external callback handler and operating a second system alongside the gateway. There is also no support for distributed tracing, which means client-side agent traces and proxy-side request logs stay disconnected. Debugging a multi-step agent across that gap is manual work. The same is true for evaluations and prompt management, neither of which LiteLLM attempts. Those are separate tools you adopt and integrate yourself.

LiteLLM pricing: what it actually costs to run

LiteLLM is $0. The open-source proxy is free forever, self-hosted, with no license fee at any request volume, and it covers a lot for a free tier: 100+ providers behind one OpenAI-compatible endpoint, virtual keys scoped to users and teams, budgets, rate limits, spend tracking, fallbacks, request and response logging, and Prometheus metrics.

Enterprise is where the number stops being public. It's quoted by sales, billed annually, and sized to your gateway request capacity and deployment architecture rather than per token, across two tiers called Standard and SCALE. What you're buying is governance: SSO and SCIM, OIDC and JWT auth, audit logs, secret manager integration with key rotation, org and team admins, a multi-region control plane, and support with SLAs up to 24/7. Both tiers stay self-hosted, and both can run air-gapped.

The threshold you're likely to hit first is SSO, which is free up to five users and licensed above that.

The March 2026 LiteLLM supply chain attack

The attack didn't start at LiteLLM. TeamPCP spent March 19 through 27 working through the tools engineers install to stay safe, force-pushing malicious commits across Trivy's GitHub Actions and Checkmarx KICS before reaching PyPI. LiteLLM ran Trivy in its own CI security scanning workflow. The compromised scanner could read the runner's memory, and the PyPI publishing token was in that memory.

Two releases went out with that token, thirteen minutes apart. Version 1.82.7 hid a payload in proxy_server.py that fired when the proxy started. Version 1.82.8 kept that and added a .pth file, so the payload also ran on every Python interpreter start whether or not anything imported LiteLLM. It gated itself on the GITHUB_ACTIONS variable, activating specifically in CI, where secrets concentrate. PyPI quarantined both versions at 11:25 UTC, inside an hour of the first upload.

The stealer took SSH keys, cloud credentials, Kubernetes secrets, .env files, and the LLM API keys for every provider configured on the machine. That last one is the point. A gateway holds credentials for everything you route to, so compromising it hands over every provider at once instead of one. It was also caught by accident, when a fork bomb in the payload spiked CPU and a researcher hit it while testing a plugin that pulled LiteLLM in as a transitive dependency. He never chose LiteLLM, and plenty of teams running it are in the same position, since Wiz found LiteLLM present in 36% of the cloud environments it analyzed.

BerriAI pulled the packages, engaged Mandiant, published a timeline the same day, and froze releases until the pipeline was rebuilt. Teams on the official Docker image were never exposed, because that path pins dependencies. If you installed 1.82.7 or 1.82.8, treat it as full credential exposure rather than a package you can uninstall, and rotate provider keys, cloud credentials, SSH keys, and Kubernetes secrets.

Here's what actually changed:

• A pip-installed gateway inherits the security posture of every dependency in its build chain, and that chain now includes the scanners.
• Pinning versions, reviewing upgrades before they ship, and keeping publishing scope off shared runners are ordinary practices that are easy to do halfway.

March priced what halfway costs. If your gateway is a package you install, that work is yours, permanently, and it belongs in the cost of running it.

What to look for in a LiteLLM replacement

The incident sharpened questions that were already worth asking of any LLM gateway, and they're what separates the options below.

• Credential location - Whether provider keys sit in your environment variables and CI secrets, in a managed service, or in a vendor's infrastructure. This determines what a compromise anywhere in your build chain can reach.
• Built-in observability - Whether traces, cost, latency, and errors come with the gateway or require a second platform and a callback integration to maintain.
• Data residency and retention - Where prompts, logs, and keys are processed and stored, and whether you can mask PII or skip storing sensitive calls entirely.
• Certifications and access controls - SOC 2, HIPAA, GDPR, ISO 27001, SSO, RBAC, and PII handling, and which tier each one is gated behind.
• Total operating cost - License plus infrastructure plus the engineering time to run it, measured against a subscription rather than against zero.
• Evaluation and prompt tooling - Whether quality measurement lives in the same system as the traffic it scores, or in a separate tool with no link back to the request.

Teams whose only real requirement is routing across providers can stop at a thinner tool than the ones below.

8 best LiteLLM alternatives

1. Respan

Respan runs the gateway and everything you would otherwise attach to it on one platform. One endpoint reaches 1,000+ models across every major provider, with automatic retries, fallbacks, load balancing, caching for exact repeat requests, and per-key spend and rate limits. Because traffic and telemetry share a system, a cost spike or a latency regression opens directly into the trace tree of the requests that caused it.

Evaluations run against production traffic in real time by attaching an evaluator to calls flowing through the gateway, so quality is scored where it happens instead of in a separate tool with no path back to the request.

Free covers 100k logs and 1k scores with 7-day retention and 412 requests per minute of proxy throughput. Team lifts that to 10k scores, 30-day retention, and 8,400 requests per minute. HIPAA is a $249/mo add-on on Team and included on Enterprise.

Security features:

• Managed infrastructure - No gateway package in your build chain to patch, pin, or compromise.
• SOC 2, HIPAA, GDPR, ISO 27001 - Full certification coverage, with a signed BAA available for healthcare workloads.
• PII masking and log omission - Strip sensitive fields before they are stored, or skip storage entirely for specific calls.
• SSO and admin roles - Google SSO on Free and Team, SAML on Enterprise, with advanced admin roles from Team.
• Red teaming - Test deployed agents to see what they block and what gets through.
• Guardrails - Block requests in categories like prompt injection and system prompt leak at the gateway before they reach the model, currently in early access.
• Spend and rate controls - Per-key budgets and rate limits enforced at the gateway.

Pricing: Free is $0 with 100k logs, 1k scores, and unlimited seats. Team is $199/mo billed yearly for 5 members at $15 each after that, and Enterprise is custom with a HIPAA BAA and a 99.99% uptime SLA.

2. Portkey

Portkey pairs routing with logging, guardrails, and governance in one control plane. A universal API covers a large model catalog with fallbacks, load balancing, retries, timeouts, and both simple and semantic caching. Guardrails run at the request layer, including PII anonymization and evaluation templates.

If you're leaving LiteLLM over security, the $49 tier isn't the plan you'd end up buying. SSO, VPC hosting, compliance certificates, custom retention, and private tenancy all sit at Enterprise, and the free Developer tier is labeled by Portkey itself as unsuitable for production. Logs are the billing unit, so cost tracks observability volume rather than request volume, and paid overage stops at 3M requests before you are into a sales conversation regardless.

Security features:

• Guardrails - PII anonymizer and deterministic checks, with custom hooks on Enterprise.
• RBAC and virtual keys - Role-based access and budget-scoped keys from the paid tier.
• SSO with Okta - Enterprise only.
• Compliance certificates - SOC 2, ISO 27001, GDPR, HIPAA with custom BAAs, all Enterprise.
• VPC hosting and private tenancy - Managed deployment inside your own cloud, Enterprise only.

Pricing: The open-source gateway is free to self-host and Developer is free with 10k logs a month. Production is $49/mo for 100k logs with $9 per additional 100k, and Enterprise is custom above 10M.

Palo Alto Networks acquired Portkey in 2026, and it now ships as the Prisma AIRS AI Gateway with the buying process running through Palo Alto, which is worth knowing before you compare it against Portkey alternatives that are still sold independently.

3. TrueFoundry

If self-hosting was the requirement and the operations were the problem, TrueFoundry splits the difference. On Enterprise, the gateway runs inside your own cloud, on-prem, or air-gapped rather than as a hosted proxy, so prompts and keys stay inside your network boundary, with fallbacks, rate limiting, and RBAC applied per model. Around it sit model serving, GPU orchestration, agent deployment, and an MCP gateway with schema validation for agent tools.

TrueFoundry prices Pro per seat with a request allowance per user, so the bill grows with headcount as well as traffic. Private deployment, SSO, and audit logs all sit at Enterprise, and those are the features that make TrueFoundry a LiteLLM replacement for a security-driven migration, so the Pro price is unlikely to be the one you pay. The platform also assumes Kubernetes, which is either already true for you or a serious adoption project.

Security features:

• VPC, on-prem, and air-gapped deployment - Enterprise, with prompts, data, and models staying in your environment.
• RBAC and audit logging - Per-model access control, with audit logs at Enterprise.
• SSO and custom roles - Enterprise only.
• Real-time policy enforcement - Data residency, usage quotas, and spend controls at the gateway.
• Scoped access tokens - Personal and virtual tokens with granular permissions.

Pricing: Developer is free for up to 3 users at 10k requests each. Pro is $25/user/month with 20k requests per user and $20 per additional 100k requests, and Enterprise is custom, adding SSO, audit logs, and VPC, on-prem, or air-gapped deployment.

4. Bifrost

Migration is a base URL change. Bifrost speaks the same OpenAI-compatible format LiteLLM does, ships as a binary, Docker container, or Kubernetes workload, and is Apache 2.0 with no separately licensed components. It handles automatic failover, weighted load balancing across keys, semantic caching, native MCP as both client and server, and a budget hierarchy running from customer down to individual provider config. Maxim AI reports 20 microseconds of added latency at 5,000 requests per second, which is a vendor benchmark and worth testing against your own traffic.

Check your provider list before anything else. Bifrost lists 8+ providers covering 1,000+ models, against LiteLLM's 100+ providers, so the major models carry over, but if you depend on a long-tail provider, confirm it's supported before planning the migration. It is also a younger project with a smaller community and fewer production incidents behind it, and self-hosting means the operational ownership that made LiteLLM expensive does not go away. The dependency surface is smaller, not absent.

Security features:

• Self-hosted only - Keys and prompts never leave your infrastructure.
• Budget hierarchy - Spend and rate limits enforced per customer, team, virtual key, and provider config.
• Five auth types - None, headers, OAuth 2.0, per-user OAuth, and per-user headers.
• Air-gapped deployment - Enterprise tier.
• Apache 2.0 - Full source auditability, no dual-licensed components.

Pricing: Free and open source at any scale. Enterprise is custom and adds air-gapped support, SLAs, a private support channel, and custom plugin work.

5. OpenRouter

OpenRouter removes the gateway from your operations budget entirely. One key and a prepaid credit balance reach hundreds of models across dozens of providers, with automatic fallback on provider outages and token rates passed through at each provider's published price. Nothing to deploy, patch, or page anyone about.

OpenRouter charges 5.5% on credit purchases with a $0.80 minimum, so a $10 top-up effectively costs 8% while a $500 one costs 5.5%. Observability stops at per-model spend and usage rather than traces, there is no RBAC or guardrail layer, and every request routes through third-party infrastructure, which is the arrangement GDPR and HIPAA constraints often rule out before any of the rest matters. If compliance is your constraint, OpenRouter alternatives with different hosting models are worth comparing.

Security features:

• Vendor-managed infrastructure - No proxy in your build chain to compromise.
• BYOK - Route on your own provider keys rather than through OpenRouter credits.
• SSO and SAML - Enterprise only.
• Contractual SLAs - Negotiated at Enterprise; no public SLA below it.

Pricing: Free to start with a set of no-cost models under rate limits. Paid usage is pass-through token pricing plus 5.5% on credit purchases with a $0.80 minimum, and Enterprise is custom.

Stripe agreed to acquire OpenRouter in August 2026, and OpenRouter has said the product, name, and roadmap stay the same.

6. MLflow

MLflow brings tracing into the gateway rather than bolting it on. Requests routed through its AI Gateway become traces automatically once usage tracking is on, and W3C TraceContext links client-side agent spans to gateway spans in the same view, so the callback handler and the second observability bill both go away. That is the LiteLLM gap closed without leaving Apache 2.0.

Provider coverage runs well short of LiteLLM's long tail, so check your model list before committing. MLflow also carries a decade of classical ML surface area, experiment tracking, model registry, fine-tuning workflows, none of which has anything to do with routing LLM calls but all of which you reason about during setup. Self-hosting means the same patching, scaling, and on-call you already own. What you get in exchange is a project governed by the Linux Foundation rather than a single vendor, so the license terms and the roadmap aren't one acquisition away from moving.

Security features:

• Self-hosted - Runs inside your infrastructure, inheriting your existing controls.
• Linux Foundation governance - Apache 2.0 with no separately licensed enterprise tier.
• Guardrails in the open-source gateway - Not gated behind a paid plan.
• Large contributor base - More review surface on the code than a single-vendor project.

Pricing: Free and open source under Apache 2.0. You pay for the infrastructure you run it on and the tokens you consume.

7. Kong AI Gateway

Kong AI Gateway extends Kong's API gateway to LLM traffic, so teams already running Kong for their APIs can route model calls through the same control plane. One API reaches the major providers, including OpenAI, Anthropic, Azure AI, Bedrock, and Gemini, with routing and load balancing across them, automatic failover, and semantic caching for repeated or near-duplicate prompts.

Guardrails cover keyword blocking, jailbreak detection, PII redaction before a request reaches the provider, and checks on unsafe responses, and it also handles MCP and agent-to-agent traffic. Observability runs through metrics exporters and OpenTelemetry, with Konnect dashboards on top, so how deep you can debug an agent run depends on the tool you export to.

Kong prices per control plane rather than per request, so cost scales with how many environments and regions you run, and AI capabilities add a separate meter per model routed. If you aren't already on Kong, adopting an API management platform to get an LLM gateway can be a larger project than the LiteLLM swap you started with.

Security features:

• PII redaction - Strips PII from prompts before a request reaches the provider.
• Prompt and response guards - Keyword blocking, semantic jailbreak detection, and checks on unsafe responses.
• Cost-based rate limiting - Spend limits enforced against the calculated cost of each request.
• Self-hosted data plane - Data plane nodes run in your environment, with the control plane on Konnect or self-hosted.

Pricing: Free trial, then $500/month per control plane, with separate environments or regions each paying again and AI capabilities metered per model routed.

8. Vercel AI Gateway

Zero markup is true for tokens and false for the controls a compliance-driven team would turn on. The Vercel AI Gateway passes tokens through at provider list price with no fee, including on BYOK, and gives you a unified endpoint with load balancing, automatic failover, and a dashboard covering requests by model, time to first token, token counts, and spend. Team-wide zero data retention and the provider allowlist each cost $0.10 per 1,000 requests and require a Vercel Pro or Enterprise plan, while the per-request versions of both are free. Custom reporting is billed separately.

The free tier also ends the moment you outgrow it, since buying credits moves you to the paid tier and the $5 monthly allowance stops applying, and free credits only cover a subset of models at lower rate limits. Beyond pricing, the gateway is coupled to the Vercel platform, and observability stops at gateway metrics and per-request OpenTelemetry traces you can forward to your own tool rather than agent traces. There is also no evaluation or prompt management layer, which is one of the main things to weigh across Vercel AI Gateway alternatives.

Security features:

• Zero data retention routing - Routes only through providers with ZDR agreements and fails closed when none is available.
• Provider allowlist - Restrict which providers receive traffic, billed per request team-wide.
• BYOK - Use your own provider credentials with no added fee.
• Vendor-managed infrastructure - No proxy to operate or patch.

Pricing: Every Vercel team account gets $5 in gateway credits monthly from first use. Past that it is pay-as-you-go at provider list price, though team-wide ZDR and allowlist controls add $0.10 per 1,000 requests each.

Frequently asked questions

What is the best LiteLLM alternative?

It depends on what LiteLLM was doing for you and what you were paying to make up the difference. Respan fits teams running LiteLLM alongside a separate observability tool, because the gateway, tracing, evaluations, and prompt management sit in one platform and a metric always ties back to the run behind it. Teams that need Kubernetes-native self-hosting will find TrueFoundry closer to their architecture, and teams who want to stay open source with the fewest changes should test Bifrost, which speaks the same OpenAI-compatible format.

Is there a free LiteLLM alternative?

Several, though "free" means different things. Respan's Free tier is a hosted gateway with 100k logs, 1k scores, unlimited seats, and no infrastructure to run, so nothing sits in your cloud bill. Bifrost and MLflow are free under Apache 2.0 but self-hosted, which means you own the compute, patching, and on-call the same way you did with LiteLLM. Portkey, TrueFoundry, and Vercel all offer free tiers with request, log, or credit caps.

Is LiteLLM safe to use after the supply chain attack?

The compromised versions, 1.82.7 and 1.82.8, were quarantined by PyPI at 11:25 UTC, inside an hour of the first upload, and 1.83.0 shipped afterward through a rebuilt CI/CD pipeline with isolated environments and stronger release gates. Exposure depended entirely on install path, since the official Docker image pinned its dependencies and never pulled the compromised wheels. If you installed either affected version during the window, treat it as a full credential exposure rather than a package problem: rotate provider keys, cloud credentials, SSH keys, and Kubernetes secrets, and investigate for persistence and outbound traffic.

Is LiteLLM production ready?

Yes, for teams willing to run it as production infrastructure. The open-source proxy covers fallbacks, budgets, rate limits, and Prometheus metrics, and Enterprise adds SSO, audit logs, multi-region deployment, and support SLAs. What you take on is running it: keeping the build chain safe after March, scaling the proxy, and carrying the on-call when it's on your critical path, with debugging limited to logs and metrics unless you add a tracing tool. Respan runs the gateway as managed infrastructure with tracing and evaluations built in, so that work moves off your team.

How much does LiteLLM cost?

The open-source proxy is $0 with no license fee at any volume, and SSO is free up to five users. Enterprise is quoted by sales and billed annually, sized to your gateway request capacity, deployment architecture, and support needs rather than per token. The rest depends on your scale and lands in your own cloud bill: compute for the proxy, a Postgres instance, Redis if you cache, load balancers, and multi-region redundancy if the gateway is on your critical path. Add the engineering time for dependency review, patching, and incident response.

Does LiteLLM have built-in observability?

No. LiteLLM logs requests and responses and exposes Prometheus metrics, but there is no tracing layer. Gaining visibility means configuring an external callback handler such as Datadog or Langfuse and operating that system alongside the gateway. It also has no distributed tracing support, so client-side agent traces and proxy-side request logs stay disconnected and end-to-end debugging is manual.

Can you migrate off LiteLLM without rewriting your code?

Usually yes. LiteLLM's value is its OpenAI-compatible interface, and most alternatives here speak the same format, so migration is typically a base URL change and a key swap rather than a refactor. Respan, Bifrost, Portkey, OpenRouter, and Vercel all accept OpenAI-format requests. What needs planning is everything around the calls: virtual keys and budget rules have to be recreated, and any callback-based logging integration is replaced rather than ported. Run both paths in parallel for a day and reconcile cost and token counts against your provider invoice before cutting over. With Respan, that cutover also brings tracing, evaluations, and cost controls for 1,000+ models into the same platform, so the observability stack you bolted onto LiteLLM comes included rather than as a second bill.