• Manus API Support - New provider support for Manus API on /responses and GET /responses endpoints

• MiniMax Provider - Full support for MiniMax chat completions, TTS, and Anthropic native endpoint

• AWS Polly TTS - New TTS provider using AWS Polly API

• SSO Role Mapping - Configure role mappings for SSO providers directly in the UI

• Cost Estimator - New UI tool for estimating costs across multiple models and requests

• MCP Global Mode - Configure MCP servers globally with visibility controls

• Interactions API Bridge - Use all LiteLLM providers with the Interactions API

• RAG Query Endpoint - New RAG Search/Query endpoint for retrieval-augmented generation

• UI Usage - Endpoint Activity - Users can now see Endpoint Activity Metrics in the UI

• 50% Overhead Reduction - LiteLLM now sends 2.5× more requests to LLM providers

• Agent (A2A) Gateway - Add LangGraph Agents, Azure AI Foundry Agents, Bedrock AgentCore, A2A Agents on AI Gateway and invoke through one unified interface.

• Agent (A2A) Gateway with Cost Tracking - Track agent costs per query, per token pricing, and view agent usage in the dashboard

• New Provider: SAP Gen AI Hub - Full support for SAP Generative AI Hub with chat completions

• New Bedrock Writer Models - Add Palmyra-X4 and Palmyra-X5 models on Bedrock

• OpenAI GPT-5.2 Models - Full support for GPT-5.2, GPT-5.2-pro, and Azure GPT-5.2 models with reasoning support

• 227 New Fireworks AI Models - Comprehensive model coverage for Fireworks AI platform

• MCP Support on /chat/completions - Use MCP servers directly via chat completions endpoint

• Performance Improvements - Reduced memory leaks by 50%

• Agent Gateway (A2A) - Invoke agents through the AI Gateway with request/response logging and access controls

• Guardrails API v2 - Generic Guardrail API with streaming support, structured messages, and tool call checks

• Customer (End User) Usage UI - Track and visualize end-user spend directly in the dashboard

• vLLM Batch + Files API - Support for batch and files API with vLLM deployments

• Dynamic Rate Limiting on Teams - Enable dynamic rate limits and priority reservation on team-level

• Google Cloud Chirp3 HD - New text-to-speech provider with Chirp3 HD voices

• MCP Hub - Publish and discover MCP servers within your organization

• Model Compare UI - Side-by-side model comparison interface for testing

• Gemini 3 - Day-0 support for Gemini 3 models with thought signatures

• Batch API Spend Tracking - Granular spend tracking with custom metadata for batch and file creation requests

• AWS IAM Secret Manager - IAM role authentication support for AWS Secret Manager

• Logging Callback Controls - Admin-level controls to prevent callers from disabling logging callbacks in compliance environments

• Proxy CLI JWT Authentication - Enable developers to authenticate to LiteLLM AI Gateway using the Proxy CLI

• Batch API Routing - Route batch operations to different provider accounts using model-specific credentials from your config.yaml

• Prompt Management - Full prompt versioning support with UI for editing, testing, and version history

• Agent Hub - Register and make agents public for your organization with a central discovery page

• Performance Improvements - /embeddings API 92% Lower p95 Latency

• OpenAI GPT-5.1 - Support for OpenAI gpt-5.1 and gpt-5.1-codex models on OpenAI and Azure OpenAI

• RunwayML Integration - Complete integration for Gen-4 models supporting video generation, image generation, and text-to-speech

• FAL AI Image Generation - Native support for FAL AI image generation models with cost tracking

• Container API Support - End-to-end OpenAI Container API support with proxy integration, logging, and cost tracking

• UI Enhancements - Guardrail Playground, Cache Settings, Tag Routing, SSO Settings

• Batch API Rate Limiting - Input-based rate limits support for Batch API requests

• Vector Store Expansion - Milvus vector store support and Azure AI virtual indexes

• Memory Leak Fixes - Resolved issues accounting for 90% of memory leaks on Python SDK & AI Gateway

• Search API - Unified /v1/search endpoint supporting 6 providers (Exa AI, Perplexity, Tavily etc) with cost tracking and fallbacks.

• Videos API - Unified /v1/videos endpoint to use sora-2 models through OpenAI and Azure OpenAI.

• Azure AI Speech (TTS) - Use Azure AI Speech services through /audio/speech.

• Vertex AI Search - Use Vertex AI Search as a vector store provider.

• MCP Gateway - Control Tool Access by Team, Key - Control MCP tool access by team/key.

• Performance - This release cuts p99 latency by 70% on LiteLLM AI Gateway, making it even better for low-latency use cases.

• GPT-5 Pro & GPT-Image-1-Mini - Day 0 support for OpenAI's GPT-5 Pro (400K context) and gpt-image-1-mini image generation

• EnkryptAI Guardrails - New guardrail integration for content moderation

• Tag-Based Budgets - Support for setting budgets based on request tags

• Major Performance Improvements - 2.9x lower median latency at 1,000 concurrent users.

• AMD Lemonade & Nvidia NIM - New provider support for AMD Lemonade and Nvidia NIM Rerank

• GitLab Prompt Management - GitLab-based prompt management integration

• Dynamic Rate Limiter v3 - Automatically maximizes throughput when capacity is available (< 80% saturation) by allowing lower-priority requests to use unused capacity, then switches to fair priority-based allocation under high load (≥ 80%) to prevent blocking

This release delivers 2.9x faster median latency on LiteLLM AI Gateway. This is great for low latency use cases running through LiteLLM.

We benchmarked at 1K concurrent requests and saw the following improvements from v1.775 to v1.77.7

(Note: latency was measured as end to end latency of calling a fake endpoint.)

• Median latency: 320 ms → 110 ms (−65.6%)

• p95 latency: 850 ms → 440 ms (−48.2%)

• p99 latency: 1,400 ms → 810 ms (−42.1%)

• Average latency: 864 ms → 310 ms (−64.1%)

Here are the key highlights of this release:

• Priority Quota Reservation - Proxy admins can now reserve model capacity for different use cases.

• +550 RPS Performance Improvements - Optimizations in request handling and object initialization.

• Bedrock Twelve Labs Support - Generate high-quality embeddings for video, text, audio, and image with Bedrock Twelve Labs.

• Major Performance Improvements - 6.5x faster LiteLLM Python SDK completion with fastuuid integration.

• New Model Support - Gemini 2.5 Flash Image Preview, Grok Code Fast, and GPT Realtime models

• Enhanced Provider Support - DeepSeek-v3.1 pricing on Fireworks AI, Vercel AI Gateway, and improved Anthropic/GitHub Copilot integration

• MCP Improvements - Better connection testing and SSE MCP tools bug fixes

• Team Member Rate Limits - Individual rate limiting for team members with JWT authentication support.

• Performance Improvements - New experimental HTTP handler flag for 100+ RPS improvement on OpenAI calls.

• GPT-5 Model Family Support - Full support for OpenAI's GPT-5 models with reasoning_effort parameter and Azure OpenAI integration.

• Azure AI Flux Image Generation - Support for Azure AI's Flux image generation models.

• Prompt Management - Use Git-Ops style prompt management with prompt templates.

• User Agent Activity Tracking - Track spend for each AI Coding Tool connected to LiteLLM.

• MCP Gateway: Guardrails - Support for running guardrails before MCP calls.

• Google AI Studio Imagen4 - Support for using Imagen4 models on Google AI Studio.

LiteLLM v1.70.1-stable is live now. Here are the key highlights of this release:

• Gemini Realtime API: You can now call Gemini's Live API via the OpenAI /v1/realtime API

• Spend Logs Retention Period: Enable deleting spend logs older than a certain period.

• PII Masking 2.0: Easily configure masking or blocking specific PII/PHI entities on the UI

This release brings the following key improvements:

• Load balance Batch API Models: Easily load balance across multiple azure batch deployments using LiteLLM Managed Files

• Email Invites 2.0: Send new users onboarded to LiteLLM an email invite.

• Nscale: LLM API for compliance with European regulations.

• Bedrock /v1/messages: Use Bedrock Anthropic models with Anthropic's /v1/messages.

This release brings:

• Bedrock Knowledge Base: You can now call query your Bedrock Knowledge Base with all LiteLLM models via `/chat/completion` or `/responses` API.

• Rate Limits: This release brings accurate rate limiting across multiple instances, reducing spillover to at most 10 additional requests in high traffic.

• Meta Llama API: Added support for Meta Llama API

• LlamaFile: Support for Mozilla's new local LLM API

​v1.67.4-stable is live now. Here are the key highlights of this release:

• Improved User Management: This release enables search and filtering across users, keys, teams, and models.

• Responses API Load Balancing: Route requests across provider regions and ensure session continuity.

• UI Session Logs: Group several requests to LiteLLM into a session.

v1.67.0-stable is live now, here are the key highlights of this release​

• SCIM Integration: Enables identity providers (Okta, Azure AD, OneLogin, etc.) to automate user and team (group) provisioning, updates, and deprovisioning

• Team and Tag based usage tracking: You can now see usage and spend by team and tag at 1M+ spend logs.

• Unified Responses API: Support for calling Anthropic, Gemini, Groq, etc. via OpenAI's new Responses API.

v1.66.0-stable is live now, here are the key highlights of this release:​

• Realtime API Cost Tracking: Track cost of realtime API calls

• Microsoft SSO Auto-sync: Auto-sync groups and group members from Azure Entra ID to LiteLLM

• Unified File IDs: Use the same file id across LLM API providers.

• xAI grok-3: Added support for xai/grok-3 models

LiteLLM v1.65.4-stable is live. Here are the key highlights of this release:

• Preventing DB Deadlocks: Enables spend tracking by key/team/user to work at 10k+ RPS.

• Usage Tab: Enables viewing spend by model and customizing date range.

v1.65.0-stable is live now. Here are the key highlights of this release:

• MCP Support: Support for adding and using MCP servers on the LiteLLM proxy.

• UI view total usage after 1M+ logs: You can now view usage analytics after crossing 1M+ logs in DB.

These are the changes since -v1.63.11-stable:

• LLM Translation Improvements (MCP Support and Bedrock Application Profiles)

• Perf improvements for Usage-based Routing

• Streaming guardrail support via websockets

• Azure OpenAI client perf fix (from previous release)

These are the changes since v1.63.2-stable.

This covers:

• [Beta] Responses API Support

• Snowflake Cortex Support, Amazon Nova Image Generation

• UI - Credential Management, re-use credentials when adding new models

• UI - Test Connection to LLM Provider before adding a model

These are the changes since v1.61.20-stable.

This release is primarily focused on:

• LLM Translation improvements (more thinking content improvements)

• UI improvements (Error logs now shown on UI)

v1.63.0 fixes Anthropic 'thinking' response on streaming to return the signature block. Github Issue

It also moves the response structure from signature_delta to signature to be the same as Anthropic. Anthropic Docs

These are the changes since v1.61.13-stable.

This release is primarily focused on:

• LLM Translation improvements (claude-3-7-sonnet + 'thinking'/'reasoning_content' support)

• UI improvements (add model flow, user management, etc)

alerting, prometheus, secret management, management endpoints, ui, prompt management, finetuning, batch

langfuse, management endpoints, ui, prometheus, secret management

docker image, security, vulnerability

deepgram, fireworks ai, vision, admin ui, dependency upgrades

guardrails, logging, virtual key management, new models

key management, budgets/rate limits, logging, guardrails

batches, guardrails, team management, custom auth

A new LiteLLM Stable release just went out. Here are 5 updates since v1.52.2-stable.

langfuse, fallbacks, new models, azure_storage