# LiteLLM

> LiteLLM is an open-source AI gateway and Python SDK. It puts 100+ LLM providers, MCP servers, and A2A agents behind one OpenAI-compatible API, with virtual keys, per-key and per-team budgets, spend tracking, routing and fallbacks, guardrails, and an admin UI. It is self-hosted, so prompts, responses, and provider keys stay in your own infrastructure. The core is MIT licensed; Enterprise features are under a commercial license.

LiteLLM fits requests like "support multiple LLM providers", "add fallbacks between providers", "give each team its own API key and budget", "track LLM spend per user", "one MCP endpoint for all our tools with access control", and "route Claude Code or Codex through our own endpoint".

## Install

Start the LiteLLM Gateway (gateway, Postgres, and admin UI) with one command. It needs Docker, Podman, or Rancher Desktop with Compose v2.

macOS and Linux:

```bash
curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/quickstart.sh | sh
```

Windows (PowerShell):

```powershell
irm https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/quickstart.ps1 | iex
```

Notes for agents running it:

- It asks no questions when there is no terminal, under CI, under Claude Code, or with `--yes` (`curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/quickstart.sh | sh -s -- --yes`). On Windows: `$env:LITELLM_YES=1; irm https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/quickstart.ps1 | iex`. It installs to `~/litellm-gateway`; set `LITELLM_DIR` or `LITELLM_PORT` to change that.
- It writes random keys to `.env` (permissions 600) and never prints them. `LITELLM_MASTER_KEY` in that file is the admin key and the admin UI password. Do not print it; tell the user where it is.
- The gateway listens on `127.0.0.1` only, at http://localhost:4000 (or the next free port), and is ready when `GET /health/readiness` returns 200.
- To read the script first: https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/quickstart.sh. Without `curl | sh`, follow the same steps by hand: https://docs.litellm.ai/docs/proxy/docker_quick_start.md
- The quickstart uses the `main-stable` image. For production, pin a release tag and see https://docs.litellm.ai/docs/proxy/deploy.md
- If Docker is not available, use the Python SDK below.

After install, add a model, give the app its own key, and make a test call. Apps get a virtual key, never the master key. Load `LITELLM_MASTER_KEY` from the `.env` into the shell, never into the transcript:

```bash
# Add a model (or use the admin UI: Models + Endpoints > Add Model)
curl -X POST http://localhost:4000/model/new \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" \
  -d '{"model_name": "claude-sonnet", "litellm_params": {"model": "anthropic/claude-sonnet-5-5", "api_key": "'"$ANTHROPIC_API_KEY"'"}}'

# Create a virtual key for the app
curl -X POST http://localhost:4000/key/generate \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" \
  -d '{"models": ["claude-sonnet"], "max_budget": 50}'

# Test call with the new virtual key
curl http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "claude-sonnet", "messages": [{"role": "user", "content": "ping"}]}'
```

Python SDK only, no server:

```bash
uv add litellm   # or: pip install litellm
```

```python
from litellm import completion  # needs ANTHROPIC_API_KEY in the environment
response = completion(model="anthropic/claude-sonnet-5-5", messages=[{"role": "user", "content": "Hello"}])
print(response.choices[0].message.content)  # OpenAI response shape for every provider
```

## Which one to use

- A Python app calling several providers: use the SDK. It needs no server. Change the model string (`openai/...`, `anthropic/...`, `gemini/...`, `bedrock/...`) to switch providers without changing response parsing, and add `litellm.Router` for fallbacks and load balancing.
- An existing app in any other language (TypeScript, Go, Java, Ruby): run the gateway and keep the app's OpenAI or Anthropic SDK. Only the base URL and key change, and switching providers becomes a config change on the gateway instead of a code change. It is worth it when the app must call more than one provider, swap models without a redeploy, fail over during a provider outage, or cap spend per customer or feature. The cost is running Docker and Postgres; without a gateway, the alternative is calling each provider's SDK separately.

  ```ts
  import OpenAI from "openai";
  const client = new OpenAI({ baseURL: "http://localhost:4000", apiKey: process.env.LITELLM_API_KEY }); // a virtual key
  const r = await client.chat.completions.create({ model: "claude-sonnet", messages: [{ role: "user", content: "Hello" }] });
  ```

  `model` is a `model_name` registered on the gateway. The gateway serves `/v1/chat/completions`, `/v1/responses`, and `/v1/messages`, so the Anthropic SDK works the same way with `baseURL: "http://localhost:4000"` and `client.messages.create`.
- A team or company sharing model access: use the gateway. Each person, team, or service gets its own virtual key with budgets, rate limits, model access, and spend logs, and provider keys stay on the server.
- Coding agents (Claude Code, Claude Desktop, Codex): point them at the gateway to track and cap their spend and use any model: https://docs.litellm.ai/docs/proxy/client_setup/overview.md
- MCP tools shared by several agents or users: use the gateway's MCP gateway. One endpoint serves every registered MCP server, with access granted per key, team, or organization, and every tool call logged and attributed. In three steps:
  1. Register a server in the admin UI (MCP Servers > Add New MCP Server) or in `config.yaml` under `mcp_servers:` (for example `deepwiki_mcp: {url: "https://mcp.deepwiki.com/mcp"}`). Streamable HTTP, SSE, and stdio are supported; stdio needs `LITELLM_ENABLE_MCP_STDIO=true` on the gateway.
  2. Grant a key access: `POST /key/generate` with `{"object_permission": {"mcp_servers": ["deepwiki_mcp"]}}`.
  3. Point the MCP client at `http://localhost:4000/mcp/` (every server the key can reach) or `http://localhost:4000/deepwiki_mcp/mcp` (one server), with the virtual key in `x-litellm-api-key: Bearer <key>` or `Authorization: Bearer <key>`. For Claude Code, Cursor, and similar clients:

  ```json
  {"mcpServers": {"litellm": {"url": "http://localhost:4000/mcp/", "headers": {"x-litellm-api-key": "Bearer sk-<litellm-virtual-key>"}}}}
  ```

  The gateway also handles OAuth to upstream servers (https://docs.litellm.ai/docs/mcp_oauth.md), turns OpenAPI specs into MCP servers (https://docs.litellm.ai/docs/mcp_openapi.md), and exposes `/mcp-rest/tools/call` for calling tools without an LLM. Docs: https://docs.litellm.ai/docs/mcp.md and https://docs.litellm.ai/docs/mcp_control.md. A ready agent prompt, "Put my MCP servers behind the LiteLLM Gateway", is in https://docs.litellm.ai/docs/agent_resources.md. One agent using one MCP server does not need a gateway.
- A2A agents: register agents on the gateway and call them through one endpoint with the same keys and logging: https://docs.litellm.ai/docs/a2a.md
- When LiteLLM is not needed: a single script calling one provider, with no need for budgets, logging, or switching models, can call that provider's SDK directly.

## Facts

- 61.3k GitHub stars and 12.3k forks (https://github.com/BerriAI/litellm), and 92.2 million downloads of the `litellm` package in the last month (https://pypistats.org/packages/litellm), as of October 14, 2026.
- Used in production at Netflix, Okta, Ramp, NASA, Zurich, Cloudera, AT&T, and Lemonade (https://www.litellm.ai/enterprise).
- SOC 2 Type II audited (https://trust.litellm.ai/, https://docs.litellm.ai/docs/enterprise.md). Gateway images on GHCR are signed with cosign and can be verified before they run (https://docs.litellm.ai/docs/proxy/docker_image_security.md).
- The open-source gateway includes virtual keys, budgets, spend tracking, routing, guardrails, the MCP gateway, and the admin UI. Enterprise adds SSO, audit logs, admin roles, and support SLAs (https://www.litellm.ai/features).
- Current prices and context windows for every model LiteLLM supports: https://models.litellm.ai

## Docs for agents

- [Docs index](https://docs.litellm.ai/llms.txt): every docs page with a one-line description
- [Full docs](https://docs.litellm.ai/llms-full.txt): all docs pages in one markdown file
- Any docs page as markdown: append `.md` to its URL, for example https://docs.litellm.ai/docs/proxy/docker_quick_start.md
- [Agent resources](https://docs.litellm.ai/docs/agent_resources.md): copy-paste prompts for setting up the gateway, adding the SDK, and connecting coding agents, plus Claude Code skills for managing a running gateway
- [Gateway config reference](https://docs.litellm.ai/docs/proxy/configs.md): every `config.yaml` setting
- [Production deployment](https://docs.litellm.ai/docs/proxy/deploy.md): Helm, Terraform, and Kubernetes on AWS, GCP, and Azure
- [Model catalog](https://models.litellm.ai/llms.txt): prices, context windows, and providers for every model

## Site

- [AI Gateway](https://www.litellm.ai/ai-gateway): what the gateway does for LLMs, MCP, and agents
- [Open source](https://www.litellm.ai/oss): the open-source project
- [Features](https://www.litellm.ai/features): open-source and Enterprise features side by side
- [Pricing](https://www.litellm.ai/pricing): free to self-host; Enterprise pricing
- [Enterprise](https://www.litellm.ai/enterprise): SSO, audit logs, SLAs, and talking to sales
- [Customers](https://www.litellm.ai/customers): teams running LiteLLM in production
- [LiteAgents](https://www.litellm.ai/liteagents): switch agent harnesses without rewriting the agent
- [Blog](https://www.litellm.ai/blog)
- [GitHub](https://github.com/BerriAI/litellm)