LiteLLM is an open-source AI gateway. It connects your apps to all LLMs, agents and MCP servers through one OpenAI-compatible API.
Platform teams and developers use LiteLLM to operate AI in production. You install it in your cloud, on-prem or air-gapped. It records and limits the cost of each request and sends each request to the correct model.
What LiteLLM does
LiteLLM has one gateway and one SDK, and both are open source. LiteLLM Enterprise adds governance, security and support to the same code.
AI Gateway (LiteLLM Proxy)
The gateway gives all of your teams one OpenAI-compatible API to 140+ providers and 1,892 models.
Platform teams make virtual keys and set budgets and rate limits. They see the cost by key, user, team and org. Nobody shares provider keys.
Python SDK
The litellm package calls 100+ LLM APIs in the OpenAI format. It has retries, fallbacks and cost tracking. To use a different model, you change one line of code.
MCP gateway
The MCP gateway puts all of your MCP tools behind one fixed endpoint. You control access to each tool by key and by team. An agent gets only the tools that you permit.
Agent gateway
The gateway also calls agents: A2A agents, Vertex AI Agent Engine, LangGraph, Azure AI Foundry, Bedrock AgentCore and Pydantic AI. Agent calls get the same logs and the same access control as model calls.
Admin UI
The Admin UI is the dashboard of the gateway. Each team can make keys, add models, set budgets and read its cost data. The platform team does not have to do this work.
liteagents SDK
liteagents is an agent SDK with the same query() interface as the Claude Agent SDK. You select the harness: Deep Agents, Claude Agent SDK, Codex, Pydantic AI or OpenCode.
When you change the harness, you keep your model, tools and MCP servers. The SDK is in alpha.
LiteLLM Enterprise
LiteLLM Enterprise has all of the open-source features. It adds SSO and SCIM, OIDC/JWT auth, audit logs, secret managers with key rotation, org and team admins, and a multi-region control plane.
The LiteLLM team gives onboarding and support.
LiteLLM Lens
LiteLLM Lens records traces of agent swarms and shows where you can make them better. It became available in early access on 1 October 2026.
What makes LiteLLM different
This is how LiteLLM compares with other AI gateways. The data comes from their pricing pages and documents in October 2026.
Open source, with no fee on tokens
The gateway has an MIT license, and you can use it in production free of charge. LiteLLM Enterprise is an annual license with a price for your request capacity, and LiteLLM adds no fee to your model cost.
OpenRouter adds a 5.5% fee when you buy credits. Cloudflare AI Gateway adds 5% to credits from Unified Billing.
Your data stays on your servers, also in air-gapped networks
You install LiteLLM in your cloud, on-prem, on Kubernetes or in an air-gapped network. It sends no data and no telemetry to LiteLLM.
OpenRouter, Cloudflare AI Gateway and Vercel AI Gateway are hosted services. All prompts go through their infrastructure. TrueFoundry gives VPC, on-prem and air-gapped installation only with its Enterprise plan.
Governance in the free version
Open-source LiteLLM has virtual keys, budgets, rate limits, cost tracking, fallbacks, load balancing and guardrails. Bifrost puts guardrails, adaptive load balancing, cluster mode, SSO, RBAC and audit logs in its paid Enterprise tier.
Less than 1 ms of gateway overhead
The LiteLLM Rust AI Gateway (beta) adds 0.66 ms at p99. Portkey adds 2.29 ms and Bifrost adds 4.54 ms in the same AI Gateway Bench test.
The test uses the same hardware and the same mock upstream for each gateway. All numbers and scripts are public, so you can do the benchmark yourself.
Models, agents and MCP in one gateway
LiteLLM connects to 140+ providers and 1,892 models, and also to MCP servers and A2A agents. You use one API and one login for all of them.
LiteLLM supports new models on the day that they become available. Your teams do not wait for a gateway update.
An independent company that builds in the open
LiteLLM is an independent company with 1,700+ contributors on GitHub. Other gateways now belong to larger companies.
Palo Alto Networks bought Portkey in May 2026 and will change its name to Prisma AIRS AI Gateway. Helicone joined Mintlify in March 2026, and its code now gets only maintenance.
Who uses LiteLLM
Teams that operate AI in production use LiteLLM. Some of them are Netflix, NVIDIA, Okta, AT&T, Lemonade, Zurich, Cloudera, Twilio, IBM, SAP, Ramp and NASA.
- Platform teams and AI infrastructure teams that give hundreds of engineers access to LLMs through one gateway, with the cost for each team
- Companies in regulated industries, for example insurance and finance that must have on-prem or air-gapped installation, SSO and audit logs
- Developers who write AI apps in Python and want one SDK for all providers
- Teams that operate agents in production and put models, MCP tools and agents behind the same keys and budgets
- Open-source AI projects and agent frameworks for example the OpenAI Agents SDK, Google ADK and OpenHands, which use LiteLLM to connect to many providers
The team behind LiteLLM
Berrie AI Incorporated makes LiteLLM. It is a Y Combinator company (Winter 2023) in San Francisco.
Krrish Dholakia
Co-founder and CEO
Krrish and Ishaan started LiteLLM after the Y Combinator Winter 2023 batch. Krrish is the CEO of the company.
How LiteLLM started
Krrish and Ishaan were in the Y Combinator Winter 2023 batch together. Then they added Azure and Cohere to a chatbot that they made.
The Azure calls failed frequently, so they added fallbacks from Azure to Cohere to OpenAI. The code for each provider became long if/else blocks that were difficult to debug.
They put all of their LLM calls behind one package and released it as LiteLLM on 27 July 2023. Other developers had the same problem and started to use it.
Who builds it
The LiteLLM team builds the gateway in the open, together with 1,700+ contributors. The company got a $1.6M seed round from Y Combinator, Gravity Fund and Pioneer Fund. See our open jobs.
How LiteLLM works with your team
From a free installation to a gateway for all of your company.
Start in minutes
Install the gateway with one command, the Docker image or the Helm chart. Then connect your apps to it. You do not sign up, you do not give a credit card, and LiteLLM gets no data.
Try Enterprise for 30 days
Get a 30-day Enterprise trial key by email immediately, without a sales call. If you contact sales, they reply in one business day.
Onboarding
Each Enterprise customer gets onboarding and a shared Slack or Microsoft Teams channel. The engineers who build LiteLLM answer in that channel. They also help you during live upgrades.
Support and response times
Standard Enterprise support is available 9am–9pm PT, Monday to Friday. 24/7 SLAs are also available, with these response times:
- Sev 0 (all production traffic fails): 1 hour
- Sev 1: 6 hours
- Sev 2–3: 24 hours
- Security patches: 72 hours
Releases
LiteLLM released more than 1,400 versions after July 2023, on GitHub, PyPI and ghcr.io/berriai/litellm. Each image has a cosign signature. A stable tag comes only after 12-hour load tests.
Key facts
The primary facts about LiteLLM in October 2026.
- Company name
- LiteLLM (Berrie AI Incorporated)
- Type
- Open-source AI gateway and Python SDK (developer infrastructure)
- Founded
- 2023, Y Combinator Winter 2023 batch
- Founders
- Krrish Dholakia (CEO) and Ishaan Jaffer (CTO)
- Headquarters
- San Francisco, California, USA
- Website
- litellm.ai. Documents at docs.litellm.ai
- Core product
- An AI gateway that you install on your servers. It puts 140+ LLM providers, MCP servers and agents behind one OpenAI-compatible API, with cost tracking, budgets, rate limits, routing, guardrails and access control.
- Products
- AI Gateway (LiteLLM Proxy), Python SDK, MCP gateway, agent gateway, Admin UI, liteagents SDK, LiteLLM Lens, LiteLLM Enterprise
- License
- MIT. The
enterprise/directory is different: you must have a paid license to use it in production. - Pricing
- Open source: free, on your servers. Enterprise: an annual license. The price is a function of request capacity, installation architecture and support. It is never per token.
- Contract terms
- Annual Enterprise license. 30-day trial key, with no credit card. You can buy direct, through AWS Marketplace or Azure Marketplace, or through authorized resellers.
- Installation
- Your cloud, on-prem, Kubernetes (Helm, Terraform) or an air-gapped network
- Communication
- A shared Slack or Microsoft Teams channel for Enterprise customers. Discord, Slack and GitHub for the community.
- Security
- SOC 2 Type 2. The reports are in the trust center.
- Notable users
- Netflix, NVIDIA, Okta, AT&T, Lemonade, Zurich, Cloudera, Twilio, IBM, SAP, Ramp, NASA
- Use
- 511M+ Docker pulls, 92M+ PyPI downloads a month, 60K+ GitHub stars, 1,700+ contributors, 1B+ requests
Frequently asked questions
What is LiteLLM?
LiteLLM is an open-source AI gateway and Python SDK. It calls 140+ LLM providers through one OpenAI-compatible API. Platform teams install it to give their company access to all models, agents and MCP servers, with a cost limit on each request.
Is LiteLLM free?
Yes. The open-source gateway and SDK have an MIT license, and you can use them in production free of charge. LiteLLM Enterprise is a paid annual license for SSO, SCIM, audit logs and support.
Does LiteLLM see our prompts or data?
No. You install LiteLLM on your servers, so your prompts, responses and provider keys stay in your infrastructure. The open-source gateway sends no data and no telemetry to LiteLLM.
How is LiteLLM different from OpenRouter?
OpenRouter is a hosted service. Your requests go through its servers, and it adds a 5.5% fee when you buy credits. LiteLLM operates on your servers with your provider keys and contracts, and it has no fee per token.
Who makes LiteLLM?
Berrie AI Incorporated makes LiteLLM. Krrish Dholakia and Ishaan Jaffer started the company in San Francisco in 2023, with Y Combinator (Winter 2023). The open-source project has 1,700+ contributors on GitHub.
What is the price of LiteLLM Enterprise?
The price is a function of your annual request capacity, your installation architecture and your support. It is never per token. Start with a free 30-day trial key, or contact sales for a quote.
Run it yourself. Start today.
Deploy the open-source gateway in an afternoon, with spend tracked and capped on every request. Add SSO, audit logs, and an SLA when it goes org-wide.
