• MCP Hub - Publish and discover MCP servers within your organization

• Model Compare UI - Side-by-side model comparison interface for testing

• Gemini 3 - Day-0 support for Gemini 3 models with thought signatures

• Batch API Spend Tracking - Granular spend tracking with custom metadata for batch and file creation requests

• AWS IAM Secret Manager - IAM role authentication support for AWS Secret Manager

• Logging Callback Controls - Admin-level controls to prevent callers from disabling logging callbacks in compliance environments

• Proxy CLI JWT Authentication - Enable developers to authenticate to LiteLLM AI Gateway using the Proxy CLI

• Batch API Routing - Route batch operations to different provider accounts using model-specific credentials from your config.yaml

• Prompt Management - Full prompt versioning support with UI for editing, testing, and version history

• Agent Hub - Register and make agents public for your organization with a central discovery page

• Performance Improvements - /embeddings API 92% Lower p95 Latency

• OpenAI GPT-5.1 - Support for OpenAI gpt-5.1 and gpt-5.1-codex models on OpenAI and Azure OpenAI

• RunwayML Integration - Complete integration for Gen-4 models supporting video generation, image generation, and text-to-speech

• FAL AI Image Generation - Native support for FAL AI image generation models with cost tracking

• Container API Support - End-to-end OpenAI Container API support with proxy integration, logging, and cost tracking

• UI Enhancements - Guardrail Playground, Cache Settings, Tag Routing, SSO Settings

• Batch API Rate Limiting - Input-based rate limits support for Batch API requests

• Vector Store Expansion - Milvus vector store support and Azure AI virtual indexes

• Memory Leak Fixes - Resolved issues accounting for 90% of memory leaks on Python SDK & AI Gateway

• Search API - Unified /v1/search endpoint supporting 6 providers (Exa AI, Perplexity, Tavily etc) with cost tracking and fallbacks.

• Videos API - Unified /v1/videos endpoint to use sora-2 models through OpenAI and Azure OpenAI.

• Azure AI Speech (TTS) - Use Azure AI Speech services through /audio/speech.

• Vertex AI Search - Use Vertex AI Search as a vector store provider.

• MCP Gateway - Control Tool Access by Team, Key - Control MCP tool access by team/key.

• Performance - This release cuts p99 latency by 70% on LiteLLM AI Gateway, making it even better for low-latency use cases.

• GPT-5 Pro & GPT-Image-1-Mini - Day 0 support for OpenAI's GPT-5 Pro (400K context) and gpt-image-1-mini image generation

• EnkryptAI Guardrails - New guardrail integration for content moderation

• Tag-Based Budgets - Support for setting budgets based on request tags

• Major Performance Improvements - 2.9x lower median latency at 1,000 concurrent users.

• AMD Lemonade & Nvidia NIM - New provider support for AMD Lemonade and Nvidia NIM Rerank

• GitLab Prompt Management - GitLab-based prompt management integration

• Dynamic Rate Limiter v3 - Automatically maximizes throughput when capacity is available (< 80% saturation) by allowing lower-priority requests to use unused capacity, then switches to fair priority-based allocation under high load (≥ 80%) to prevent blocking

This release delivers 2.9x faster median latency on LiteLLM AI Gateway. This is great for low latency use cases running through LiteLLM.

We benchmarked at 1K concurrent requests and saw the following improvements from v1.775 to v1.77.7

(Note: latency was measured as end to end latency of calling a fake endpoint.)

• Median latency: 320 ms → 110 ms (−65.6%)

• p95 latency: 850 ms → 440 ms (−48.2%)

• p99 latency: 1,400 ms → 810 ms (−42.1%)

• Average latency: 864 ms → 310 ms (−64.1%)

Here are the key highlights of this release:

• Priority Quota Reservation - Proxy admins can now reserve model capacity for different use cases.

• +550 RPS Performance Improvements - Optimizations in request handling and object initialization.

• Bedrock Twelve Labs Support - Generate high-quality embeddings for video, text, audio, and image with Bedrock Twelve Labs.

• Major Performance Improvements - 6.5x faster LiteLLM Python SDK completion with fastuuid integration.

• New Model Support - Gemini 2.5 Flash Image Preview, Grok Code Fast, and GPT Realtime models

• Enhanced Provider Support - DeepSeek-v3.1 pricing on Fireworks AI, Vercel AI Gateway, and improved Anthropic/GitHub Copilot integration

• MCP Improvements - Better connection testing and SSE MCP tools bug fixes

• Team Member Rate Limits - Individual rate limiting for team members with JWT authentication support.

• Performance Improvements - New experimental HTTP handler flag for 100+ RPS improvement on OpenAI calls.

• GPT-5 Model Family Support - Full support for OpenAI's GPT-5 models with reasoning_effort parameter and Azure OpenAI integration.

• Azure AI Flux Image Generation - Support for Azure AI's Flux image generation models.

• Prompt Management - Use Git-Ops style prompt management with prompt templates.

• User Agent Activity Tracking - Track spend for each AI Coding Tool connected to LiteLLM.

• MCP Gateway: Guardrails - Support for running guardrails before MCP calls.

• Google AI Studio Imagen4 - Support for using Imagen4 models on Google AI Studio.

LiteLLM v1.70.1-stable is live now. Here are the key highlights of this release:

• Gemini Realtime API: You can now call Gemini's Live API via the OpenAI /v1/realtime API

• Spend Logs Retention Period: Enable deleting spend logs older than a certain period.

• PII Masking 2.0: Easily configure masking or blocking specific PII/PHI entities on the UI

This release brings the following key improvements:

• Load balance Batch API Models: Easily load balance across multiple azure batch deployments using LiteLLM Managed Files

• Email Invites 2.0: Send new users onboarded to LiteLLM an email invite.

• Nscale: LLM API for compliance with European regulations.

• Bedrock /v1/messages: Use Bedrock Anthropic models with Anthropic's /v1/messages.

This release brings:

• Bedrock Knowledge Base: You can now call query your Bedrock Knowledge Base with all LiteLLM models via `/chat/completion` or `/responses` API.

• Rate Limits: This release brings accurate rate limiting across multiple instances, reducing spillover to at most 10 additional requests in high traffic.

• Meta Llama API: Added support for Meta Llama API

• LlamaFile: Support for Mozilla's new local LLM API

​v1.67.4-stable is live now. Here are the key highlights of this release:

• Improved User Management: This release enables search and filtering across users, keys, teams, and models.

• Responses API Load Balancing: Route requests across provider regions and ensure session continuity.

• UI Session Logs: Group several requests to LiteLLM into a session.

v1.67.0-stable is live now, here are the key highlights of this release​

• SCIM Integration: Enables identity providers (Okta, Azure AD, OneLogin, etc.) to automate user and team (group) provisioning, updates, and deprovisioning

• Team and Tag based usage tracking: You can now see usage and spend by team and tag at 1M+ spend logs.

• Unified Responses API: Support for calling Anthropic, Gemini, Groq, etc. via OpenAI's new Responses API.

v1.66.0-stable is live now, here are the key highlights of this release:​

• Realtime API Cost Tracking: Track cost of realtime API calls

• Microsoft SSO Auto-sync: Auto-sync groups and group members from Azure Entra ID to LiteLLM

• Unified File IDs: Use the same file id across LLM API providers.

• xAI grok-3: Added support for xai/grok-3 models

LiteLLM v1.65.4-stable is live. Here are the key highlights of this release:

• Preventing DB Deadlocks: Enables spend tracking by key/team/user to work at 10k+ RPS.

• Usage Tab: Enables viewing spend by model and customizing date range.

v1.65.0-stable is live now. Here are the key highlights of this release:

• MCP Support: Support for adding and using MCP servers on the LiteLLM proxy.

• UI view total usage after 1M+ logs: You can now view usage analytics after crossing 1M+ logs in DB.

These are the changes since -v1.63.11-stable:

• LLM Translation Improvements (MCP Support and Bedrock Application Profiles)

• Perf improvements for Usage-based Routing

• Streaming guardrail support via websockets

• Azure OpenAI client perf fix (from previous release)

These are the changes since v1.63.2-stable.

This covers:

• [Beta] Responses API Support

• Snowflake Cortex Support, Amazon Nova Image Generation

• UI - Credential Management, re-use credentials when adding new models

• UI - Test Connection to LLM Provider before adding a model

These are the changes since v1.61.20-stable.

This release is primarily focused on:

• LLM Translation improvements (more thinking content improvements)

• UI improvements (Error logs now shown on UI)

v1.63.0 fixes Anthropic 'thinking' response on streaming to return the signature block. Github Issue

It also moves the response structure from signature_delta to signature to be the same as Anthropic. Anthropic Docs

These are the changes since v1.61.13-stable.

This release is primarily focused on:

• LLM Translation improvements (claude-3-7-sonnet + 'thinking'/'reasoning_content' support)

• UI improvements (add model flow, user management, etc)

alerting, prometheus, secret management, management endpoints, ui, prompt management, finetuning, batch

langfuse, management endpoints, ui, prometheus, secret management

docker image, security, vulnerability

deepgram, fireworks ai, vision, admin ui, dependency upgrades

guardrails, logging, virtual key management, new models

key management, budgets/rate limits, logging, guardrails

batches, guardrails, team management, custom auth

A new LiteLLM Stable release just went out. Here are 5 updates since v1.52.2-stable.

langfuse, fallbacks, new models, azure_storage