GiorgioAresu/home-ops

Watch

Fork 0

feat(container): update image ghcr.io/berriai/litellm (v1.108.2 ➔ v1.109.1) #2608

Open

renovate-bot wants to merge 1 commit from renovate/ghcr.io-berriai-litellm-1.x into main

renovate-bot commented

2026-11-08 00:08:11 +01:00

Collaborator

This PR contains the following updates:

Package	Update	Change
ghcr.io/berriai/litellm (source)	minor	`v1.108.2` → `v1.109.1`

⚠️ Warning

Some dependencies could not be looked up. Check the Dependency Dashboard for more information.

Release Notes

BerriAI/litellm (ghcr.io/berriai/litellm)

`v1.109.1`

Compare Source

Verify Docker Image Signature

All LiteLLM Docker images are signed with cosign. This release is signed with the key introduced in commit e82a3c6.

Verify using the pinned commit hash (recommended):

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the published signing key:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \
  ghcr.io/berriai/litellm:v1.109.1

Verify using the release tag (convenience):

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.109.1/cosign.pub \
  ghcr.io/berriai/litellm:v1.109.1

Expected output:

The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

What's Changed

security(proxy): backport #51254 to stable/1.109.x by @shrey-berri in #51264
chore: bump litellm 1.109.0 -> 1.109.1 by @yuneng-berri in #51298

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.109.0...v1.109.1

`v1.109.0`

Compare Source

Verify this release

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \
  ghcr.io/berriai/litellm:v1.109.0

Same command for litellm-database, litellm-non_root and the docker.io/litellm/* images

What's new in 1.109.0

982 PRs. 144 added, 424 fixed, 60 changed, 17 improved, 9 removed. 5 need action

Action required

Routing and fallbacks: Change string fallback targets to lists of model names (#48943)
Caching: Switch cache_params.type from disk to local or redis, since the disk cache backend (diskcache) has been removed (#48114)
Routing and fallbacks: Add optional_pre_call_checks: ["enforce_model_rate_limits"] to keep a hard rpm cap on deployments (#48873)
Vertex AI and Gemini: Set drop_params: true or stop sending temperature, top_p and top_k to Gemini 3.6 Flash, 3.5 Flash-Lite and newer (#47645)
Deployment: Deploy Lens from its own chart and set lens.* in your values (#49169)

Highlights

Admin UI: Added a Chat UI with a model hub, image generation and usage metrics (#48108)
Routing and fallbacks: Added mid-stream fallback continuation so a streaming chat cut off mid-answer continues on the configured fallback (#41127)
Deployment: Added LITELLM_OFFLINE_MODE to turn off every fetch LiteLLM makes on its own with one switch (#48096)
Cost tracking and pricing: Added model_info.pricing_provider to price a deployment from another provider's catalog rates instead of $0 (#47620)
Vertex AI and Gemini: Added Gemini 3.1 Flash TTS support, including multi-speaker voices and mp3, opus, alaw and mulaw output (#31915)
SDK: Fixed tool calls on completion() and responses() failing with No module named 'mcp' on base SDK installs (#45961) ^{Also in 1.108.4}

All changes

OpenAI (19)

Added per-deployment workload identity on non-chat routes, realtime and the moderation guardrail (#39966)
Added the 12-hour cache write rate for gpt-5.6-cyber (#48246)
Added OpenAI gpt-6.1-terra with pricing (#48429)
Added Batch and Ultrafast 12-hour cache write rates for OpenAI models (#48433)
Added gpt-6-rosalind to the model cost map with its published prices (#49268)
Added gpt-6-astra-law with pricing from the OpenAI pricing page (#49444)
Fixed 400 errors and $0 spend logs on gpt-transcribe calls without response_format (#44021)
Fixed nonstreaming aiohttp_openai responses dropping service_tier (#47263)
Fixed prompt_cache_retention such as 24h being silently dropped instead of sent upstream (#47480)
Fixed /health marking OpenAI and Azure audio chat models unhealthy (#47730)
Fixed OpenAI context_length_exceeded errors raising BadRequestError instead of ContextWindowExceededError (#47954)
Fixed 12-hour prompt cache writes on openai/gpt-5.6 and newer billing at the 30-minute rate (#48088)
Fixed uncatalogued ChatGPT models hitting an unsupported Chat Completions endpoint (#48244)
Fixed the whole reasoning_effort dict not being sent when chat is bridged to Responses (#48807)
Fixed 400 invalid_beta on Agents API calls through the OpenAI passthrough (#49009)
Fixed empty e.response.text on OpenAI and Azure errors so the upstream error body is readable (#49079)
Changed OpenAI 401 and 404 invalid_request_error to raise matching exceptions, not BadRequestError (#32537)
Changed the ChatGPT session id to derive from prompt_cache_key for stable cache affinity (#42014)
Changed gpt-6-luna Decisions cache rates to the standard cached and write rates (#48066)

Azure (18)

Added Entra ID auth, including keyless auth, for Azure Decisions on /v1/decisions and /v1/systemone (#46539, #48529)
Added access to Foundry agents through the project Responses API for azure_ai/agents/<name> (#47717)
Added azure_ai/MAI-Code-1.1-Flash (#47779)
Added max_input_tokens of 32768 for azure_ai/Microsoft-Decision-1 (#48434)
Added outputContentFormat=markdown support on /ocr for Azure Document Intelligence (#48926)
Fixed Foundry chat requests losing OpenAI prompt cache breakpoints (#45299)
Fixed /v1/responses 404 on dated api_version deployments and azure_ai Grok reasoning token billing (#46377)
Fixed under-billed Grok reasoning tokens on azure_ai when the deployment name lacks "grok" (#46969)
Fixed Azure realtime over WebRTC failing with a 404 on client_secrets and an invalid token on the SDP exchange (#47316)
Fixed 400 from Foundry when tool results contain tool_reference parts on chat (#48032)
Fixed 400 on azure_ai/claude-* when using thinking.block_binding (#48033)
Fixed Responses API calls that use an explicit v1 Azure base URL (#48118)
Fixed 404 on Azure vector store retrieve, update and delete (#48224)
Fixed null audio capability metadata on two Azure audio snapshot models (#48475)
Fixed Azure context_length_exceeded raising BadRequestError so context_window_fallbacks never ran (#48616)
Fixed batch calls on unversioned azure/ o3, o4-mini, o3-mini, gpt-4o and gpt-4o-mini billing at the standard rate (#49182)
Changed max_input_tokens to 128000 for gpt-chat-latest on Azure (#48252)
Improved Entra auth by reusing credentials and tokens across requests and refreshing them off the event loop (#46421, #47348, #48769)

Anthropic (16)

Fixed a fake [System: ...] message being added to tool-call-only assistant turns sent to Anthropic (#34822)
Fixed PDFs sent as chat image_url being rejected with 400 by sending them as document blocks (#46984)
Fixed Claude json_schema answers losing keys listed only in required and escaping non-ASCII characters (#47478)
Fixed replayed redacted_thinking blocks being dropped, and Bedrock Converse 400s when sent twice (#47489)
Fixed clear_tool_uses input_tokens trigger ignoring the system prompt, which delayed clearing (#47857)
Fixed native Anthropic responses omitting a null stop_sequence (#47987)
Fixed streamed responses leaving native web search fees out of the reconstructed usage cost (#48528)
Fixed drop_params not dropping speed on claude-opus-4-7, which Anthropic rejects with 400 (#48764)
Fixed 400 from beta-gated body fields when drop_params: true, including Claude Code and Codex traffic (#48793)
Fixed model_context_window_exceeded being reported as stop instead of length (#48874)
Fixed 400 on /v1/messages with many strict tools under drop_params by retrying without strict (#49102)
Fixed Claude pause_turn reported as finished and refusal explanations dropped on chat and Responses (#49192)
Fixed 500 on a non-string role or tool call id, now a 400 that names the field (#49237)
Fixed metadata.user_id being dropped on Anthropic /v1/messages pass-through routes (#49333)
Changed the default max_tokens for unmapped Anthropic models from 4096 to 128000 (#48502)
Changed the opening prompt usage estimate in the Chat Completions adapter to be off by default (#48633)

AWS Bedrock (28)

Added context and output token limits for the Nova Sonic models (#48247)
Added pricing for native guardrailConfig trace usage to spend and x-litellm-response-cost (#48278)
Added global.openai.gpt-6.1-terra and us.openai.gpt-6.1-terra Bedrock models with pricing (#48441)
Added token counting for three Mantle-native Bedrock Claude models (#48682)
Added published Priority, Batch and Flex prices to 46 Bedrock regional rows (#49113)
Fixed Bedrock Voxtral transcription returning 400, logging $0 spend and ignoring language (#41565)
Fixed Unknown provider=None 404 for Bedrock models missing from the cost map (#43290)
Fixed 400 errors on Bedrock Kimi K3 Converse calls with max_tokens below 16 (#43875)
Fixed 400 invalid beta flag on Claude Code auto mode requests that carry no safeguards (#44471)
Fixed a malformed Bedrock batch JSONL line failing the upload with a 500 instead of a 400 (#45992)
Fixed post_call guardrails not scanning Claude output on invoke-with-response-stream passthrough (#46524)
Fixed Claude json_schema streams dropping reasoning and thinking blocks and ending on tool_calls with no tool calls (#46863)
Fixed JSON schema requests failing on Fable 5 and Sonnet 4.6 and losing adaptive thinking on Haiku 5.5 (#46870)
Fixed bedrock_request_metadata_fields being ignored on the /bedrock pass-through routes (#47040)
Fixed Bedrock Opus 4.8 and Sonnet 5 dropping reasoning on json_schema requests by trying native output first (#47100)
Fixed Bedrock batch jobs ignoring a configured control-plane endpoint (#47260)
Fixed AWS region lookups through Vault blocking the event loop and stalling other requests on the worker (#47331)
Fixed $0 spend and ModelNotMappedError for bedrock/claude_platform/<id> calls (#48150)
Fixed max_input_tokens to 922000 on Bedrock GPT models and added Mantle token counting for claude-opus-5-5 (#48479)
Fixed Bedrock rejecting strict tool schemas by retrying without strict under drop_params (#48733)
Fixed 400 on Bedrock when Codex repeats its tool list across turns (#49006)
Fixed region-prefixed Bedrock embedding models not routing to their region or billing its price (#49072)
Fixed bedrock_mantle/openai.gpt-5.6-cyber token limits and endpoints to match its model card (#49350)
Fixed max_output_tokens for Bedrock GovCloud West Llama 3 models, now 8192 (#49543)
Changed converse-stream pass-through to stream frames through post_call guardrails instead of buffering the reply (#46307)
Changed /bedrock pass-through routes to apply a model's guardrailConfig, returning 400 if Bedrock cannot use it (#47429)
Changed Bedrock pass-through to use request metadata as the session id, fixing 400s with missing_session_id: reject (#49193)
Changed Bedrock Invoke and Mantle Claude to use the model's own default max_tokens (#49724)

Vertex AI and Gemini (34)

Added Gemini 3.1 Flash TTS support, including multi-speaker voices and mp3, opus, alaw and mulaw output (#31915)
Added a setting to drop chosen anthropic-beta values on the Vertex AI pass-through routes (#47185)
Added LITELLM_GEMINI_DEFAULT_MAX_OUTPUT_TOKENS to set the output limit for requests that set none (#47542)
Added thought_signature_in_tool_call_id to keep Gemini tool call ids short for clients that cap them (#47692)
Added a 2026-06-30 deprecation date to vertex_ai/veo-2.0-generate-001 (#47847)
Added Batch rates for two Llama 4 MaaS models so Vertex batch jobs no longer bill at on-demand rates (#48085)
Added Vertex batch prices above 200k tokens and image output prices for Gemini models (#48945)
Fixed the Gemini 3 sampling deprecation warning logging on every request, now once per model per worker (#40868)
Fixed vertex_ai/* keys not listing partner, Veo and embedding models and dropping publisher prefixes (#41959)
Fixed Gemini TTS /v1/audio/speech calling Google twice and logging 0 tokens in spend (#44551)
Fixed 400 from compaction, thinking display and tool change betas on Claude via Vertex AI and Azure AI (#44747)
Fixed context caching on Vertex AI failing when a custom api_base is set (#44783)
Fixed streamed Vertex AI pass-through calls missing the regional 10% uplift and custom deployment prices (#44795)
Fixed Gemini not receiving integer enum values in tool parameters (#44908)
Fixed Vertex Claude batches returning 404 at create (#45715)
Fixed cache-marked requests listing all cachedContents on every call and creating duplicate caches (#46857)
Fixed Vertex AI rate limit errors returned as 403 when the error text contains 403 (#47134)
Fixed Gemini audio input and cached audio billed at the Standard rate on flex, priority and batch tiers (#48086)
Fixed gemini-2.5-flash-native-audio video input billed at the text rate instead of the audio/video rate (#48248)
Fixed Gemini 2.5 rejecting tool results with images and the history replay that follows (#48428)
Fixed Vertex AI shared credential cache growing without bound across credential and project pairs (#48442)
Fixed Vertex AI image sizes outside the old lookup table silently producing square output (#48463)
Fixed slow Google credential requests blocking the proxy worker on Claude /v1/messages (#48523)
Fixed explicit Gemini and Vertex embedding timeouts silently becoming 300 seconds (#48690)
Fixed Gemini grounding keeping only the first web source for a passage (#48745)
Fixed labels on vertex_ai/claude-* returning 400 by sending them as X-Vertex-AI-Labels (#48753)
Fixed 400 on countTokens and generateContent when repeated fields are a single object or null (#48771)
Fixed Vertex Claude /v1/messages returning 400 when the SDK sends anthropic-version as a header (#48814)
Fixed Claude web search billing per request on Vertex AI deployments priced in model_info (#49160)
Fixed batch image output billed at the Standard rate on gemini-nano-banana-2.1 and gemini-3.1-flash-lite-image (#49317, #49358)
Changed Gemini 3.6 Flash, 3.5 Flash-Lite and newer to refuse temperature, top_p and top_k without drop_params (#47645)
Improved rerank latency by reusing cached Google credentials on every request (#48430)
Improved failure logs for Gemini and Vertex pass-through calls by adding the URL model and declared provider (#49075)
Removed the floor deprecation_date from gemini-3.1-flash-lite-image (#48587)

Other providers (3)

Added 11 new providers: ai&, BigModel, CLF AI Gateway, Hostinger Router, Kyma API, Nace, OpenCode Go, OpenInfer, Refuseless, StreamLake and ZeroGPU (#38958, #39324, #41006, #41034, #43874, #44863, #46523, #48140, #48497, #48563, #48756)
Added models, endpoints and options for smaller providers (31 PRs) (#41021, #41522, #46156, #46875, #47401, #47473, #47610, #47690, #47785, #47812, #48158, #48251, #48335, #48448, #48450, #48453, #48498, #48621, #48651, #48676, #48677, #48735, #48871, #48998, #49252, #49280, #49281, #49282, #49287, #49297, #49542)
Fixed issues and updated behavior on smaller providers (41 PRs) (#38736, #40152, #43390, #44723, #45092, #45567, #46091, #46861, #46945, #46959, #47091, #47423, #47424, #47432, #47495, #47784, #47794, #48121, #48254, #48265, #48268, #48270, #48455, #48522, #48527, #48668, #48684, #48706, #48707, #48778, #48820, #48837, #49068, #49158, #49278, #49344, #49458, #49462, #49498, #49502, #49616)

Chat completions (17)

Fixed streams with logprobs ending in a 500 error frame before the finish chunk (#33549)
Fixed 500 errors when a prompt template omits initial_prompt_value or final_prompt_value (#39762)
Fixed rebuilt streamed text completions returning null finish_reason and zero prompt_tokens (#44349)
Fixed Foundry deployments and Claude fallbacks getting the wrong cache options, which caused 400s (#45295)
Fixed non-streaming chat dropping upstream response headers on bedrock_mantle/ and other OpenAI-like providers (#46928)
Fixed chat streams failing with a proxy error when progress chunks have no choice index (#46947)
Fixed the overhead header under-reporting on chat requests when guardrails add litellm_metadata (#47790)
Fixed streamed chunks containing [DONE] being read as end of stream and dropped (#47832)
Fixed completions streams including usage when it was not requested (#48052)
Fixed an upstream abort on streamed chat completions returning a clean HTTP 200 instead of a server error (#48328)
Fixed unbalanced think tags in merged streams and SSE lines splitting on characters other than CR/LF (#48755)
Fixed 400 on system-first models like Qwen3.5 when a later developer message appears (#48897)
Fixed streamed chat completions with n>1 rebuilding each choice (#49002)
Fixed streaming clients missing tool execution when OpenAI-compatible providers end tool calls with stop (#49388)
Changed the include_usage stream chunk to send an empty choices array like OpenAI (#47109)
Changed stalled stream reads to raise a 408 litellm.Timeout on chat, messages and responses (#47198)
Changed streams that end without a finish reason to fail as errors instead of reporting a normal completion (#47665)

Responses API (34)

Added max_agentic_loops support to raise the streamed MCP auto-execute round cap above 5 (#47688)
Added resuming background response streams with stream and starting_after on retrieve (#48819)
Added Codex client tool_search support on Anthropic and other bridged providers, cutting input tokens (#48972)
Fixed Responses API logging tasks being dropped by garbage collection and cut off at shutdown (#31074)
Fixed non-stream MCP auto-execute on /v1/responses logging and billing only one of its two model calls (#43667)
Fixed 400 from malformed prompt_cache_breakpoint on /v1/responses with drop_params: true (#45330)
Fixed later turns on an open Responses WebSocket skipping deployment checks and not counting toward budgets (#45727)
Fixed streamed /v1/responses events showing the backend deployment name in nested response.model (#46662)
Fixed /v1/responses on Claude dropping returned and replayed compaction items (#46754)
Fixed Responses WebSocket running pre_call guardrails on turn one only and rebilling that cost every turn (#47013)
Fixed OpenAI pass-through GET /responses/{id} being billed again on every call (#47171)
Fixed background spend not counting toward the key's project or per-model budgets (#47504)
Fixed the billed cost of background responses not reaching key and team loggers (#47506)
Fixed failed Responses streams billing 0 tokens and bridged /v1/messages stream failures not being reported (#47537)
Fixed clear_tool_uses_20250919 being silently dropped for openai/ models on the Responses API (#47861)
Fixed per-call input_cost_per_token and output_cost_per_token being ignored by responses() (#48119)
Fixed 500 when replaying web_search_call input items that have no action (#48417)
Fixed audio, PDF and image tool results and history media being dropped in Responses and Gemini conversions (#48484)
Fixed retries from previous_response_id returning 400 on Bedrock after a failed turn (#48647)
Fixed Responses params the chat bridge cannot translate causing 400s like max_tool_calls on Bedrock Claude (#48655)
Fixed typed streaming text access breaking on /v1/responses when logprobs is omitted (#48686)
Fixed moderation on /v1/responses being dropped before the OpenAI call (#48708)
Fixed later citation batches and pending citations being dropped when streaming Responses (#48747)
Fixed replayed Responses output items being rejected by OpenAI because of a null created_by (#48783)
Fixed an empty function_call_output reaching models as serialized JSON instead of an empty tool result (#48845)
Fixed Responses compaction by dropping params it rejects and sending Azure compaction to the v1 route (#48876)
Fixed Responses item order, phase and boundaries not being replayed on the next chat turn (#48912)
Fixed bridged Responses requests losing the requested text verbosity when sent to chat completions (#49049)
Fixed deployment itpm and otpm limits not being checked or counted on later Responses WebSocket turns (#49069)
Fixed litellm.responses() through litellm_proxy/ leaving e.body, e.code and e.type empty on errors (#49073)
Fixed dict reasoning_effort losing mode and context on Azure, ChatGPT, Perplexity, xAI, Fireworks and Copilot (#49082)
Fixed repeated Codex tools being merged for all callers, now only merged when drop_params is on (#49112)
Fixed native /v1/responses WebSocket turns not counting toward key and tag tpm_limit (#49257)
Changed a lone Responses built-in tool on Claude to return 400 before token counting (#48798)

Messages API (14)

Fixed web-search-only requests coming back without native search blocks on streamed replies (#40967)
Fixed retried /v1/messages attempts mixing up usage and spend logs, and exhausted retries billing only the last (#45194)
Fixed safeguard_results and stop_details missing from streamed /v1/messages message_delta events (#46096)
Fixed the clear_tool_uses polyfill ignoring clear_at_least, exclude_tools and clear_tool_inputs (#47123)
Fixed /v1/messages PDF documents being sent to Azure OpenAI models as images instead of file parts (#47166)
Fixed 500 on /v1/messages and count_tokens for null or numeric tool_result text, now a 400 (#47347)
Fixed 400 Tool reference not found for tool names over 64 chars when a guardrail rewrites /v1/messages (#47434)
Fixed canceled /v1/messages streams not closing the upstream request (#48220)
Fixed stop_sequences being ignored for OpenAI models on /v1/messages (#48534)
Fixed the client's anthropic-beta replacing the deployment's on /v1/messages (#48594)
Fixed /v1/messages streams reporting zero input_tokens in message_start when bridged to Chat Completions (#48618)
Fixed Anthropic-format tool_choice crashing input token counting and causing false budget refusals (#48624)
Fixed 400 on /v1/messages caused by a non-spec custom object in tool definitions (#48893)
Fixed Mistral thinking text being lost on streaming /v1/messages (#49059)

Realtime (4)

Added Soniox realtime speech-to-text on /v1/realtime with stt-rt-v5 (#44822)
Fixed Gemini Live ignoring session.update instructions and tools, and answering text clients with audio (#39864)
Fixed /health reporting OpenAI realtime deployments with a missing or invalid key as healthy (#41233)
Fixed sessions going silent when a transcript guardrail sent turn_detection with no type to OpenAI (#44939)

Embeddings, images and audio (8)

Added a per-request input batch limit on /v1/embeddings to cap how many inputs one call can send (#44574)
Fixed video status and content polling failing with per-model keys by keeping model_id in video ids (#44614)
Fixed aws_polly/standard, long-form and generative being sent with the neural engine (#47963)
Fixed image generation ignoring max_retries and Router retries multiplying upstream attempts (#48705)
Fixed async image edits with no image returning HTTP 500 instead of 400 (#48872)
Fixed key and team router_settings being ignored on /v1/images/generations (#49286)
Changed unsupported transcription providers to return 400 naming the provider and model instead of 500 (#47971)
Removed dall-e-2 and dall-e-3 support, and image generation without a model now returns 400 (#48703)

Batches and files (8)

Added routing of POST /v1/batches to batch deployments and supported_endpoints on /model_group/info (#47376)
Fixed 400 on GET /files/{id}/content with raw gs:// file ids from the unmanaged Vertex batch flow (#40621)
Fixed fine-tuning jobs failing with 400 for files uploaded with target_model_names (#46552)
Fixed completed batches skipping success callbacks and losing usage under redaction (#47192)
Fixed file lists showing every file on the shared key and ignoring limit and after with require_managed_files (#47682)
Fixed batch retrieve showing zero prompt and completion tokens (#47740)
Fixed unbillable batches blocking file deletes and new-batch polling (#48752)
Fixed unmanaged gs:// Vertex batches being costed through the wrong deployment and reading zero spend (#49301)

Vector stores (5)

Added OpenSearch k-NN search as a vector store provider (#47759)
Fixed /rag/ingest and /rag/query ignoring team provider credentials and denying viewers and team JWTs (#45497)
Fixed team key vector store search failing when the team's first model is from a different provider (#46894)
Fixed team and key Allowed Vector Stores being ignored on /v1/responses file_search without a registry (#48915)
Improved Vertex AI Search vector store searches by reusing HTTP/2 connections and OAuth tokens (#40631)

Token counting (10)

Added one log line per token count call with its id, model and count (#47657)
Fixed /v1/messages/count_tokens and /utils/token_counter failing on histories with a compaction block (#44784)
Fixed DeepSeek V4 token counts missing Azure's billed tokens by using DeepSeek's own tokenizer (#47173, #47353)
Fixed count_tokens overcounting PDFs about 12x for OpenAI and Azure models (#47554)
Fixed token counts retrying a failed HuggingFace tokenizer download and adding about 23s per request (#47586)
Fixed HTTP 500 when counting tokens for Anthropic search results and unsupported blocks (#48699)
Fixed token counts for OpenAI models newer than the tiktoken table by using o200k_base (#48728)
Fixed a loaded Hugging Face tokenizer being hidden when a later download fails (#48916)
Changed Responses input token counts to match inference, fixing Claude counts with tools and text.format (#47363)
Improved token_counter for inline auto images, including images under 768 pixels, with no HTTP requests (#48526)

MCP (27)

Added upstream status and response body to the log when an MCP health check fails (#44125)
Added support for redeeming an ID-JAG as an RFC 7523 jwt-bearer grant on the gateway token endpoint (#46748)
Added Authorization URL, Token URL and Scopes fields to OAuth Delegate and True Passthrough server forms (#46941)
Added tool search and multi-select to the MCP toolset editor in the Admin UI (#48601)
Added failure logging for MCP SDK request rejections, shown as MCP: request_rejected (#49184)
Fixed LITELLM_MCP_TOOL_LISTING_TIMEOUT not applying to paginated tools/list on /mcp (#46513)
Fixed tools/list returning an empty list when servers answer 429, now retried with an error if all fail (#46649)
Fixed 412 on MCP calls with id_jag_subject_source: refresh_token after an Okta OIDC login (#46745, #46747, #47004)
Fixed the caller's LiteLLM key being forwarded to upstream MCP servers when Authorization is in extra_headers (#46932)
Fixed VS Code sign-in to DCR bridge MCP servers when the upstream rejects loopback redirect URIs (#46940)
Fixed 403 when proxy_admin_viewer stores its own MCP credentials (#46994)
Fixed large MCP SSE tool results failing, and invalid size limits breaking MCP discovery and tool calls (#47034)
Fixed streamed MCP auto-execute billing only the first of its two model calls (#47422, #47445, #48899)
Fixed an upstream MCP 429 on session handshake being reported as an internal error (#47569)
Fixed MCP OAuth sign-in callback returning 403 behind AWS WAF by sealing the litellm_cp_return_to cookie (#47595)
Fixed per-user auth MCP servers missing from /mcp-rest/tools/list and /mcp tools/list (#47677)
Fixed streaming MCP follow-ups on /v1/responses dropping all but the latest tool round (#47698)
Fixed slow MCP tool calls delaying response headers until the SSE ping (#47897)
Fixed MCP management routes to follow the gateway's existing access rules (#48080)
Fixed a readonly MCP toolset URL returning every tool instead of only the toolset's tools (#48095)
Fixed oversized SSE tool catalogs returning a generic 500, now 502 with the size limit and pagination advice (#48424)
Fixed MCP toolset OAuth login advertising a resource URL that clients reject (#48474)
Fixed revoked or expired browser sessions still being accepted during gateway authorization (#48494)
Fixed MCP OpenAPI loading to accept valid YAML specs (#48622)
Fixed MCP tools in both allowed_tools and disallowed_tools still being callable (#48625)
Fixed MCP tools hidden by discovery guardrails still being callable by name (#48669)
Changed admin-owned team keys to reach only their team's MCP servers and toolsets (#49161)

Agents (10)

Added the pi harness to litellm.agent() and fixed harness turn cost ignoring prompt cache prices (#45467)
Added request log entries for rejected A2A calls to forbidden or unknown agents (#49492)
Fixed OpenCode via litellm.agent() using the script's folder instead of the sandbox, and hanging on resume (#47530)
Fixed Gemini Managed Agents create returning 400 for name and interactions dropping inline agent_config (#48750)
Fixed /a2a calls to completion-bridge agents logging $0 on send and no spend row on stream (#49150)
Fixed 404 on /a2a/{agent_id}/message/stream and /v1/a2a/{agent_id}/message/stream (#49471)
Fixed agent litellm_params.api_key not being sent as bearer auth on A2A calls (#49474)
Fixed agent list and detail responses showing non-admins the backend URL instead of the proxy URL (#49483)
Fixed a blank search embedding model returning 503 instead of the *_not_configured 400 on agent and skill search (#49485)
Changed proxy refusals on /a2a POST routes to return JSON-RPC errors (#49476)

Guardrails (21)

Added opt-in response_masked_data on PANW Prisma AIRS post_call blocks (#46219)
Added num_retries to generic_guardrail_api for checks lost after connecting (#46965)
Added a native Reco guardrail that blocks requests on Reco errors unless fail_on_error: false (#48243)
Added Ovalix guardrail scanning of files, tool calls and tool results (#49309)
Added litellm.guardrails.apply SDK helper to invoke configured guardrails and get native findings (#49460, #49469, #49470)
Fixed guardrail masks on messages and tools not reaching Bedrock Converse requests (#45874)
Fixed generic_guardrail_api rejecting Gemini native tools that have no type key (#46338)
Fixed corrupted numbered PII tokens in Presidio when one value matches several entities with output_parse_pii (#47024)
Fixed hide-secrets missing secrets in tool results, tool inputs, tool_calls arguments, and system prompts (#47122)
Fixed generic guardrails with mode: pre_mcp_call being rejected at startup so MCP calls went unchecked (#48060)
Fixed pass-through guardrails scanning litellm_metadata fields added by LiteLLM, which caused false matches (#48276)
Fixed hide-secrets ignoring skip_tool_message_in_guardrail (#48689)
Fixed guardrails chosen in a Responses WebSocket frame not running, and a 500 with other guardrail settings (#48725)
Fixed Responses tool results not reaching texts guardrails (#48827)
Fixed google_text_moderation scanning embeddings input (#48934)
Fixed guardrail blocks on streamed /v1/messages not returning Anthropic events (#49010)
Fixed streamed /v1/completions skipping post_call guardrails for masking and blocking (#49035)
Fixed mid-stream upstream errors on /v1/messages being turned into a guardrail 400 (#49088)
Changed guardrail blocks mid-stream to end chat completions streams with an error event, not a cut connection (#46953)
Changed lakera_v2 on_flagged: block to reject PII-only detections and added on_flagged: mask (#48944)
Improved streaming by running sampled guardrail scans in the background so chunks no longer pause (#48952)

Caching (4)

Fixed streamed cache hits dropping reasoning, citations and audio and reordering tool calls (#32096)
Fixed cache_control being stripped from caller messages and tools, which broke caching on Claude fallbacks (#35602)
Fixed the cache key's model group being read from caller metadata instead of the router's metadata (#48917)
Removed the disk cache backend, so cache_params.type: disk now stops proxy startup (#48114)

Routing and fallbacks (44)

Added mid-stream fallback continuation so a streaming chat cut off mid-answer continues on the configured fallback (#41127)
Added Kubernetes pod discovery to spread load across pods behind a headless service api_base (#44342, #44393, #44666)
Added opt-in filtering that keeps chat requests off deployments flagged as non-chat (#46257)
Added the x-litellm-priority header to set request priority (#47618)
Added opt-in session repinning to the replacement deployment after a successful failover (#48206)
Added a clamp mode for complexity routing that keeps the caller's smaller output token limit (#48632)
Added deployment rpd and tpd daily limits to enforce_model_rate_limits (#48875, #48957)
Added inferring custom classifier tiers from model mappings, and fixed a crash on plan-mode requests (#48987)
Fixed weighted failover skipping healthy same-group deployments under affinity (#32309)
Fixed content_policy_fallbacks not applying after POST /fallback or /config/update changes (#45428)
Fixed an explicit allowed_fails: 3 not counting failures and fallbacks cooling down healthy deployments (#46183)
Fixed a bedrock_mantle deployment's mode switching sibling deployments between Chat Completions and Responses (#46263)
Fixed prompt caching affinity ignoring tools and the /v1/messages system prompt when choosing a deployment (#46411)
Fixed all-cooled router 429 rows in SpendLogs not recording which deployments were cooling (#47120)
Fixed stream_timeout cutting off slow answers when Bedrock serves a stream=True call without streaming (#47162)
Fixed key and team router_settings reaching the provider and dropping retries, fallbacks and timeout (#47257)
Fixed 500s from router_settings.default_priority or an integer priority, and now reject a malformed priority (#47498)
Fixed fallback retries logging a negative completion time and dropping latency-routed groups (#47541)
Fixed prompt caching affinity raising on non-dict tools entries instead of returning the provider's 400 (#47565)
Fixed cut Responses streams returning server_error instead of continuing on a fallback without prefill (#47672)
Fixed IndexError and wrong deployment lookups when deployments are deleted or added concurrently (#47680)
Fixed native Responses latency using different units than bridged responses in latency-based routing (#47727)
Fixed the router sending image requests to supports_vision: false deployments in mixed model groups (#48149, #48152, #48241)
Fixed Decisions calls routing to deployments whose provider does not support them (#48175)
Fixed 500s on every request to a model group after one negative or NaN latency sample under latency routing (#48216)
Fixed tag fail-open groups rejecting unmatched tags after affinity or cooldown filtering (#48470)
Fixed one caller's exhausted forwarded Claude subscription putting the shared model into cooldown for others (#48485)
Fixed proxy-wide wildcard deployments shadowing a team's own wildcard deployment for that team's keys (#48490)
Fixed complexity router replacing the held user_turn route on Codex compaction requests (#48627)
Fixed session tier pins diverging between proxy workers when using Redis (#48645)
Fixed nested environment references in router classifier connections being sent literally (#48717)
Fixed lifecycle events not passing through on background OpenAI Responses streams that cannot fall back (#48817)
Fixed new fallback rules erasing older ones when Redis refuses DEL (#48999)
Fixed a deployment's own model_id being overridden by the request body's (#49045)
Fixed 429s on requests without max_tokens and token counters not dropping to real usage after calls (#49054)
Fixed streamed context window errors skipping fallbacks when they occur before any output (#49293)
Fixed large requests silently falling back to heuristic routing when they exceed the classifier's input limit (#49306)
Fixed queued requests waiting until a 408 when a dead replica left expired scheduler queue entries (#49387)
Changed per-deployment reasoning_effort_levels to apply on every provider and route, including azure_ai (#46601)
Changed tag routing to use only prefixed caller tags once tag_routing_prefix is set (#47428)
Changed router allowed_fails to keep a lone deployment with no fallback in rotation instead of returning 429 (#48644)
Changed user_turn routing so child agent requests keep their own tier pin (#48701)
Changed Router init to reject string fallback targets that never routed (#48943)
Removed the deployment rpm filter from enable_pre_call_checks (#48873)

Cost tracking and pricing (36)

Added s3:// URI support for LITELLM_MODEL_COST_MAP_URL (#46972)
Added model_info.pricing_provider to price a deployment from another provider's catalog rates instead of $0 (#47620)
Added x-litellm-upstream-model header so pass-through upstreams can report the model name for spend (#48275)
Added a per-guardrail spend breakdown to user, team and organization daily activity (#48702)
Fixed Responses-style bodies sent to /cursor/chat/completions not being logged or billed (#30277)
Fixed cached and reasoning token counts dropped from streamed usage, pricing streams at the full input rate (#36678)
Fixed per-second deployment rates billing per token on native /v1/messages (#42417)
Fixed streams with no backend usage being recorded at 0 tokens and $0 instead of the token estimate (#44230)
Fixed background interactions going unbilled after a crash, a late restart or on models with their own api_key (#45743)
Fixed daily tag spend being lost when the Redis tag buffer write failed (#47307)
Fixed deployments with model_info.base_model such as azure/eu/o3 billing $0 on every request (#47497)
Fixed unpriced Decisions model versions billing $0 and malformed replies failing with an unclear error (#47508)
Fixed non-streamed requests adding double their recorded cost to key spend (#47510)
Fixed $0 billing for undated azure/eu/* and azure/us/* models and added Volcengine doubao-seed-2-1 rows (#47561)
Fixed live budget counters missing the partial cost of streams that fail mid-flight (#47741)
Fixed key, team, user and daily spend rows being dropped when the proxy stops right after a request (#48131)
Fixed Azure AI Sonnet 5.5 cache read pricing, Azure web search cost and retired model dates in the cost map (#48188)
Fixed batch costs billing 12-hour cache writes at the plain batch write rate (#48259)
Fixed Price Data Reload wiping custom pricing of DB models, which logged $0 spend on /v1/messages (#48486)
Fixed extra_spend_tag_headers not matching mixed-case header names (#48620)
Fixed gpt-5-nano priority pricing using an input price OpenAI does not sell (#48763)
Fixed a NaN or infinite cost blocking daily spend writes and returning 500 on non-streaming calls (#48829)
Fixed SDK calls sharing one metadata dict being billed for earlier calls' Bedrock guardrail traces (#48923)
Fixed a gateway's usage.cost being billed as 0 on chat, Responses and streamed calls (#48953)
Fixed streamed usage.cost disagreeing with the spend row the proxy writes (#48955)
Fixed response cost differing between the header, spend log and budget counters (#48993)
Fixed wrong rates for long-prompt Gemini Pro batch and long-context Azure GPT-6 Astra Flex requests (#49114)
Fixed explicit zero prices being discarded in batch cache, speech and Perplexity search costs (#49294)
Fixed rerank token usage missing from spend logs (#49308)
Fixed provider-reported totals booking all spend on output, now split by the cost map rate ratio (#49357)
Fixed streamed bridged /v1/messages spend booked as input 0 and discounts applied twice (#49365)
Fixed /openai pass-through /v1/decisions calls logged at $0 with no tokens (#49392)
Changed streamed /v1/completions without stream_options to bill from provider usage like chat (#45366)
Changed daily activity reports and exports to show missing model and key values under fallback buckets (#48925)
Removed the legacy max_tokens cost map field in favor of max_input_tokens and max_output_tokens (#48471)
Updated cost map prices for models on smaller providers (17 PRs) (#47411, #47623, #48694, #48825, #48844, #48849, #48858, #48879, #48884, #48891, #48900, #48978, #49123, #49124, #49125, #49146, #49148)

Logging and observability (39)

Added key alias, user and end user to Langfuse generation metadata on OTEL v2 (#42697)
Added maximum_spend_logs_size to delete the oldest spend logs first once they exceed a size limit (#47450, #47451, #47500)
Added a warning log when drop_params drops a requested reasoning level (#47761, #48933)
Added disable_pre_routing_error_logs to skip spend rows for pre-routing rejections (#48239)
Added is_silent_experiment marker on mirrored silent_model calls in spend logs and callbacks (#48888)
Added diagnostics that identify aiohttp connection pool saturation behind request timeouts (#49499)
Fixed 404 when deleting callbacks configured under callbacks or failure_callback (#42740)
Fixed Responses instructions still reaching request and response logs when message logging is turned off (#43315)
Fixed OTel spans reporting full cost for cache hits and added a cache-hit flag (#43604)
Fixed failed OTel spans showing an empty status message instead of the error message (#44185)
Fixed spend logs ignoring the caller's litellm_session_id when request_correlation_in_logs: true (#44199)
Fixed Langfuse showing no time to first token for langfuse_otel calls (#45597)
Fixed provider-side 401s missing from litellm_deployment_failure_responses and litellm_deployment_state (#46668)
Fixed Prometheus /metrics not reporting a value for the in-memory spend queue size (#46956)
Fixed the proxy printing plaintext at startup with json_logs on, so the banner and model list are now JSON (#47035)
Fixed mid-stream failure alerts reporting https://api.openai.com and dropping deployment, key and team (#47045)
Fixed CustomLogger receiving both sync and async failure hooks on async call failures (#47124)
Fixed a Prometheus-enabled backend returning 404 on /metrics during startup (#47129)
Fixed legacy s3 success callback writing no object for streamed /v1/messages calls (#47156)
Fixed root logger INFO lines being dropped when json_logs is on and LITELLM_LOG is set (#47175)
Fixed Prometheus deployment failure metrics exporting api_provider="None" on many failure paths (#47384)
Fixed spend log size-based cleanup over-deleting rows after a Postgres statistics reset (#47449, #47501)
Fixed per-team Arize sampling to decide by trace id so every pod keeps or drops the same traces (#47468)
Fixed dropped streams being logged with 0 prompt and completion tokens instead of the usage already sent (#47838)
Fixed spend_logs_metadata being null on /v1beta/interactions spend log rows (#47848)
Fixed key and team Langfuse destinations losing user, tags and session on OTel v2 without global Langfuse keys (#48718)
Fixed excluded requests like /model/info still exporting auth and database spans on OTEL v2 (#48720)
Fixed proxy logs staying red after a failed request under --detailed_debug (#48966)
Fixed /v1/models requests missing from the caller's OTEL trace (#49141)
Fixed llm_exceptions alerts firing on every BudgetExceededError rejection (#49290)
Fixed image generation requests refused by the router missing from failure logging (#49349)
Fixed failure metrics losing deployment ids for blocked requests and Ollama metadata lookups calling the wrong server (#49727)
Changed langfuse_otel to set trace-level name, input, output and tags on traces LiteLLM starts (#42705)
Changed OTel spans to omit cost when the price lookup failed instead of reporting 0 (#44187)
Changed Prometheus to count successful fallbacks under the originally requested model (#45555)
Changed proxy 401 auth rejections to log one WARNING line without a traceback (#48613)
Changed pass-through routes to trace as litellm.passthrough instead of chat with OTel v2 or GenAI semconv (#48675)
Changed OTel v2 to stop printing span JSON to stdout unless the operator asks (#48723)
Improved --detailed_debug logs by printing the request body only where it enters and leaves the proxy (#48973)

Keys, teams and auth (33)

Added assigning a project to a key that has none through /key/update, without regenerating the key (#41700)
Added an opt-in setting for key owners to edit max_budget and budget_limits on /key/update (#44763)
Added mapping of propagated baggage values to budget tags (#47358)
Added per-provider and per-model rpm/tpm requirements for enforce_project_model_quota (#47693)
Added per-request resolution of oidc/google/<audience> api_key references (#47703)
Added general_settings.claude_code_marketplace_auth_required to put the Claude Code marketplace behind a key (#48980)
Added choosing the session team for Claude Code gateway users (#49001)
Fixed /team/block and /team/unblock not applying to cached teams until the cache expired (#35643)
Fixed default_key_generate_params.allowed_routes being ignored on key generation (#41236)
Fixed SCIM PATCH dropping enterprise extension attributes, now stored in metadata.scim_enterprise (#46738)
Fixed 401 when an admin-configured api_key is an oidc/file/ or oidc/env/ reference (#47341)
Fixed SCIM group delete leaving memberships and members' team keys active, and removing members one at a time (#47544)
Fixed deleted keys staying usable on other workers until the auth cache TTL expired (#47609)
Fixed 403 on /typesafe Jev calls with typesafe/* allowlists and added Jev to Admin UI model pickers (#47697)
Fixed model family and access group entries in model_max_budget never being enforced (#47722)
Fixed team member total_spend showing lower than current cycle spend (#48260)
Fixed negative RPM and TPM limits being stored, they now return 422 (#48444)
Fixed concurrency slots staying occupied after finished streams, so keys got rejected with SSE keepalive on (#48452)
Fixed team members' spend never resetting after a temporary budget increase on a member using the team default (#48495)
Fixed 500 on /key/generate and /user/new with get_spend_routes and no license, now 403 (#48623)
Fixed key edit and regenerate sending duration: "" when the expiry field is blank (#48634)
Fixed max_end_user_budget_id not applying to new end users on cached keys (#48704)
Fixed per-model end user budgets not being enforced for master-key calls that send user (#48815)
Fixed TPM reservations ignoring chat media and overcounting Responses tool images as base64 text (#48865)
Fixed credential PATCH overwriting the stored secret with its masked read-back value (#48950)
Fixed CLI SSO and Claude Code gateway team lists changing order after team writes (#49033)
Fixed /health probing models a key or team is not allowed to use, including wildcard key models (#49155)
Fixed team_id_upsert not creating the team_id_jwt_field team on UI SSO login (#49256)
Fixed exhausted key or member budgets blocking model discovery and key and user info calls (#49463)
Changed /key/info to return a generic 500 for unexpected database errors (#46190)
Changed personal keys to inherit MCP and agent access from owner teams under personal_key_model_access_from_teams (#46293)
Changed key budget_duration and budget_id updates to need admin rights, like max_budget (#47681)
Changed free installs to refuse new SSO accounts past five users without locking out existing SSO users (#48097)

Admin UI (57)

Added shareable URL state for Usage views and links from Internal Users to Usage (#43425)
Added a general settings option to enable wildcard model listings from the dashboard (#43624, #47979)
Added separate display of subscription-covered providers and account fees on the Usage page (#44772)
Added your own member spend and budget per team to the Your Teams page (#45674)
Added Delete to the guardrail detail page with a typed-name confirmation shared with the table row (#45893)
Added admin-defined custom tags and metadata from model_info on the Model Hub (#46064)
Added a 1 hour cache write cost field to the add model and model edit forms (#46830)
Added LITELLM_UI_USE_SI_TERMINOLOGY to show SI instead of AI in the Admin UI (#47440)
Added a failed state with Retry on the Logs table, Usage totals and Caching chart when fetches fail (#47671)
Added client usage inspection for users and team members (#47974)
Added sorting of virtual keys by expiry, activity, spend and limits (#48074, #48105, #48106, #48107, #48109)
Added usage charts and PDF export to Aria answers (#48094)
Added a Chat UI with a model hub, image generation and usage metrics (#48108)
Added PDF, text, .docx and .xlsx document uploads to Chat (#48137)
Added an admin setting for the default Usage time range (#48207)
Added budget period tooltips to key, team, user and organization spend tiles (#48487, #48488)
Added sorting and persistent display ordering to the Model Group Alias table (#48500)
Added the authorization MCP auth type to the create and edit server forms (#48629)
Added Chat replies that keep running after the browser closes, with Stop to cancel (#49461)
Added an optional deep research mode in Chat with persistent source citations (#49508, #49531)
Fixed the key edit form not prefilling Organization from the key's org_id (#39535)
Fixed UI request logs not showing the internal user's email (#41150)
Fixed the upgrade notice undercounting new features and fixes for patch releases (#46942)
Fixed negative prices being saved on the model edit form (#46978)
Fixed Model Hub showing stale Unhealthy status instead of the newest health check (#47552)
Fixed Admin UI snippets still showing the retired example master key (#47612)
Fixed team per-model rate limit and budget pickers missing access-group models (#47630)
Fixed Tool Policies page and /v1/tool/list staying empty with use_redis_transaction_buffer enabled (#47653)
Fixed empty headers blocking pass-through endpoint creation and authentication showing off for free users (#48081)
Fixed MCP results being dropped on Chat follow-up messages (#48124)
Fixed the Download CSV Template button in bulk invite doing nothing (#48199)
Fixed copy icons in key and log details silently failing over HTTP (#48425)
Fixed empty value cells for String settings in general settings (#48431)
Fixed the Aria button blocking Logs pagination by moving it into the dashboard header (#48454)
Fixed deleted model parameters returning after saving and refreshing (#48456)
Fixed denied or failed search tool calls showing No results found instead of the error (#48464, #48478)
Fixed Decisions Playground hiding chat models that also declare decision endpoints (#48504, #48505)
Fixed Connect showing the light logo in dark mode and Chat omitting configured branding (#48532)
Fixed the spend log detail drawer breaking for request IDs containing #, ? or % (#48630)
Fixed guardrail wizard navigation buttons being hidden when topics are selected (#48688)
Fixed the Playground not showing reasoning for /v1/responses (#48850)
Fixed Simple auto router setup hiding Responses models that Advanced setup offers (#48851)
Fixed the Aria Agent avatar showing as a broken image (#48890)
Fixed budget-only team saves silently revoking an unrestricted team's direct model access (#49258)
Fixed Z.AI credential fields missing from the Add Model provider form (#49314, #49325)
Fixed auto router templates not resolving when a * catch-all lists models over a partial wildcard (#49331)
Changed the Chat page to be on by default, and enable_chat_ui: false still hides it (#46918)
Changed the Playground chat to a single composer with a Settings popover and a reply inspector panel (#46934)
Changed the unset Budget Reset Period label from n/a to Never resets (#47065)
Changed budget dialogs to show spend limit and rate limits without a collapsed Optional Settings section (#47662)
Changed the upgrade notice to follow the changelog instead of the latest GitHub release (#47998)
Changed gateway and usage assistance into a single Aria Agent (#48082)
Changed the Aria empty model state to show in the model picker instead of a banner (#48223)
Changed auto router setup to start with two models and move classifier options under Advanced (#48451)
Changed keyboard hint chips across the Admin UI to one shared style (#48885)
Removed the separate hosted telemetry request sent on Chat submissions (#48225)
Removed the deprecated Old Usage page at /ui/old-usage, as /ui/usage shows everything it did (#49284)

Deployment (18)

Added a boot warning when config environment_variables replaces a key already set in the process environment (#47165)
Added suppress_startup_output to turn off startup prints and stop repeated Search Tools lists on reload (#47708)
Added a litellm-setup agent skill for self-hosting setup (#47731)
Added LITELLM_DISABLE_LATEST_RELEASE_CHECK to turn off the outbound latest release check (#48065)
Added LITELLM_OFFLINE_MODE to turn off every fetch LiteLLM makes on its own with one switch (#48096)
Added a pre-upgrade check for old spend-log rows that would fail the (api_key, startTime) index build (#48493)
Added serviceName and servicePort to Helm ingress.extraPaths to route paths to custom Services (#48674)
Fixed workers staying stale on configuration changes after a Redis reconnect (#43313)
Fixed litellm --num_workers 2 crash-looping on slow hosts (#45039)
Fixed --reload ignoring --num_workers, now honored on uvicorn 0.50.0 or newer (#45909)
Fixed environment variables removed from the database staying active on running workers until restart (#46709)
Fixed rare lock hangs by requiring anyio>=4.14.0 (#47378)
Fixed proxy startup being blocked by compressible api_key values that fit in the Postgres index (#48911)
Fixed tag spend and queued success callbacks being lost on proxy shutdown (#48988)
Changed the Terraform module to default its component images to the module release tag (#46960)
Changed litellm-helm to stop installing placeholder models into every install (#47056)
Changed the pre-migration check to a rolled-back dry run of pending migrations (#48961)
Removed the bundled Lens subchart from both Helm charts, which now connect to a separately deployed Lens (#49169)

SDK (8)

Added support for Python 3.15 (#48359)
Fixed tool calls on completion() and responses() failing with No module named 'mcp' on base SDK installs (#45961) ^{Also in 1.108.4}
Fixed mock_response="" sending a real provider request instead of returning an empty mock (#48266)
Fixed deprecated MCP helper imports requiring optional MCP dependencies (#48615)
Fixed Unclosed client session warnings from evicted aiohttp SDK clients (#48650)
Fixed httpx.ReadError: Connection closed on streams using the aiohttp transport's own session (#48920)
Changed import litellm to raise ImportError when the _native extension is missing (#48852)
Improved provider timeout errors to name the expired phase and limit instead of after None seconds (#48250)

Performance (12)

Fixed memory growth on proxies without Redis, which queued a budget spend increment per request (#46831)
Fixed the first /openapi.json request blocking a worker for about 6 seconds (#47352)
Fixed LiteLLM_ToolTable writes scaling with workers, and call_count now counts real tool calls (#48089)
Improved lite command startup time by no longer importing the full SDK (#46803)
Improved streaming so long buffered streams no longer stall other requests on the same worker (#46925)
Improved proxy responsiveness on very large OpenAI chat completions requests (#46967)
Improved streaming speed on OpenAI-compatible providers by cutting per-chunk overhead (#47030)
Improved proxy latency by freezing the startup heap so gen-2 GC passes no longer pause requests for 0.6 to 0.8 s (#47137)
Improved routing by resolving deployment usage limits once instead of on every request (#47393)
Improved savings rollup speed by reading deployment pricing from the router's cached model info (#47400)
Improved proxy CPU use on streamed OpenAI and Azure responses by removing caught exceptions on every chunk (#49166)
Improved request logging speed by skipping the base64 image scan on plain text (#49228)

Other (33)

Added lite claude --subscription to keep the claude.ai login while routing through the proxy (#47385)
Added lite codex --subscription to keep the ChatGPT login (#48741)
Added each model group's reasoning efforts to pi, OpenCode and Codex when launched through lite (#48949)
Fixed agentic web search follow-ups losing the provider prefix for Bedrock Mantle, OpenRouter and Bedrock models (#38885)
Fixed proxy errors returning the string "None" for type and param instead of null (#39540)
Fixed Claude Code web searches sending the instructional prefix to the search provider (#43458)
Fixed partial wildcards like cloudflare/@cf/* listing no models on /v1/models and /model/info (#46390)
Fixed POST /model/update dropping the model_info fields sent in the request (#46902)
Fixed 400 Invalid JSON on request bodies sent with Content-Encoding: zstd or gzip (#47418)
Fixed an api_base ending in / adding // to the provider path and causing 404 errors (#47535)
Fixed lite pi sending no session id, so sessions now group and session_affinity can pin them (#47700)
Fixed lite login tokens being treated as expired after a day when the proxy still accepts them (#47712)
Fixed WebSocket connections ignoring SSL verification, custom CA bundles, client certificates and proxies (#47968)
Fixed 404 on model discovery when /claude_code_gateway is the client base URL (#48221)
Fixed missing usage on native decisions calls being reported as zero tokens and $0 cost (#48261)
Fixed non-object JSON request bodies crashing parsing and auth errors, they now return 400 (#48438)
Fixed Decisions responses to keep missing probabilities absent while preserving explicit 0.0 (#48524)
Fixed /config/field/update storing boolean settings as text instead of JSON booleans (#48595)
Fixed concurrency slots not being released when synthetic streams complete (#48628)
Fixed client disconnects during body upload returning misleading errors, they now abort with 499 (#48640)
Fixed Decisions providers returning empty or non-JSON 200 bodies raising an unhandled validation error (#48662)
Fixed lite codex aliases of Codex-known models missing that model's catalog entry (#48894)
Fixed new model rows not applying on every worker as soon as they are written (#49034)
Fixed partial wildcard health checks probing models outside the wildcard (#49060)
Fixed model listing for wildcard aliases over a provider org showing models that cannot be routed (#49152)
Fixed Headroom compression and retrieval reaching different pods by forwarding tenant-scoped session affinity (#49255)
Fixed HTTP 500 on unrelated model edits when a model has saved access windows (#49316)
Changed shared-handler errors to report the right type instead of always internal_server_error (#39555)
Changed customer list routes to paginate, defaulting to 50 customers with page sizes from 1 to 100 (#48700)
Changed management and health route errors to stop reporting type: auth_error and to use fitting status codes (#48739)
Changed litellm_settings.drop_params to apply to Claude Code and Codex requests (#49058)
Changed /v1/decisions to honor a deployment's own supports_audio_input and supports_video_input flags (#49207)
Removed the System One question count cap in Decisions (#47985)

Full changelog

`v1.108.8`: 1.108.8 (LTS)

Compare Source

[!TIP]
The lts image tag always points to the latest LTS release. Pull ghcr.io/berriai/litellm:lts to stay on the newest LTS patch

Verify this release

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \
  ghcr.io/berriai/litellm:v1.108.8

Same command for litellm-database, litellm-non_root and the docker.io/litellm/* images

What's new in 1.108.8 (LTS)

4 PRs. 3 fixed. No action needed to upgrade

Changes

Azure: Fixed 400 on /v1/decisions for azure/ deployments (#51460)
Batches and files: Fixed file ownership checks missing other spellings of a Gemini file id, such as files/<id> or its URL (#51503)
MCP: Fixed sign-in loop ending in Authentication incomplete on servers that answer mcp_all with detailed scopes, broken since v1.107.0 (#51578)

Backport #51640 · Full changelog

`v1.108.7`: 1.108.7 (LTS)

Compare Source

[!TIP]
The lts image tag always points to the latest LTS release. Pull ghcr.io/berriai/litellm:lts to stay on the newest LTS patch

Verify this release

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \
  ghcr.io/berriai/litellm:v1.108.7

Same command for litellm-database, litellm-non_root and the docker.io/litellm/* images

What's new in 1.108.7 (LTS)

14 PRs. 3 added, 8 fixed, 1 changed. 1 needs action

Action required

Responses API: Set drop_params: true or stop sending Responses-only fields such as max_tool_calls to providers bridged through chat completions (#48655) ^{Also in 1.109.0}

Changes

Azure: Added gpt-image edits on Azure AI Foundry deployments, including deployments under a custom name (#49650)
AWS Bedrock: Added image search on Bedrock knowledge bases with query_image, including image hits in the results (#51062)
Vertex AI and Gemini: Fixed Gemini image calls billing text, thinking and Vertex Priority tokens at the wrong rates (#51017)
Vertex AI and Gemini: Fixed /gemini batchGenerateContent batches logging $0 with model unknown instead of billing the calling key (#51279)
Other providers: Fixed reasoning_effort being dropped or rejected for Baseten models (#40152) ^{Also in 1.109.0}
Other providers: Fixed baseten/* listing retired deployment IDs instead of the current Baseten Model APIs (#46945) ^{Also in 1.109.0}
Responses API: Fixed missing or repeated sequence_number on bridged /v1/responses streaming events (#44944)
Responses API: Fixed streamed MCP follow-ups keeping only the latest tool round, so models repeated tool calls (#47698) ^{Also in 1.109.0}
Responses API: Fixed 400 on Bedrock when a previous_response_id chain contains a failed turn (#48647) ^{Also in 1.109.0}
Responses API: Changed Responses-only fields such as max_tool_calls to answer 400 on bridged providers unless drop_params is on (#48655) ^{Also in 1.109.0}
Batches and files: Added Gemini batches on /v1/batches, with output download through /v1/files and spend tracking (#51032)
Routing and fallbacks: Fixed Decisions calls on a mixed model group hitting a Bedrock deployment and answering 400 (#48175) ^{Also in 1.109.0}

Backport #51262 · Backport #51266 · Backport #51271 · Backport #51340 · Full changelog

`v1.108.6`: 1.108.6 (LTS)

Compare Source

[!TIP]
The lts image tag always points to the latest LTS release. Pull ghcr.io/berriai/litellm:lts to stay on the newest LTS patch

Verify this release

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \
  ghcr.io/berriai/litellm:v1.108.6

Same command for litellm-database, litellm-non_root and the docker.io/litellm/* images

What's new in 1.108.6 (LTS)

16 PRs. 7 fixed, 2 changed. 1 needs action

Action required

Keys, teams and auth: Move tenant_id, client_id, client_secret, azure_username, azure_password and azure_scope request tags into metadata (#51254) ^{Also in 1.106.9}

Changes

OpenAI: Fixed gpt-5.4 and gpt-5.5 function-tool calls without reasoning_effort being bridged to /v1/responses (#50968)
Vertex AI and Gemini: Fixed Gemini image edits being billed at a flat per-image price instead of by tokens (#51038)
Chat completions: Fixed 400 on chat and bridged /v1/responses calls that send an allowed_tools tool choice (#50726)
MCP: Fixed Slack and GitHub MCP reconnects looping between the provider and the Connect page, broken since v1.107.0 (#51045)
Guardrails: Changed Zscaler AI Guard to scan only the latest user message on requests, not the whole conversation (#50139)
Cost tracking and pricing: Fixed 500 on usage routes when the database drops during key, user or team detail lookups (#50253)
Keys, teams and auth: Fixed key, user and team spend going negative after a scheduled budget reset on multi-pod deployments, broken since v1.103.0 (#51120)
Keys, teams and auth: Changed model and pass-through routes to refuse body client_id, client_secret, tenant_id and Azure login fields (#51254) ^{Also in 1.106.9}
Admin UI: Fixed the empty Spend Per User card and missing per-user data on /user/daily/activity/aggregated (#51043)

Backport #50435 · Backport #51064 · Backport #51122 · Backport #51229 · Backport #51240 · Backport #51260 · Full changelog

`v1.108.5`: 1.108.5 (LTS)

Compare Source

[!TIP]
The lts image tag always points to the latest LTS release. Pull ghcr.io/berriai/litellm:lts to stay on the newest LTS patch

Verify this release

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \
  ghcr.io/berriai/litellm:v1.108.5

Same command for litellm-database, litellm-non_root and the docker.io/litellm/* images

What's new in 1.108.5 (LTS)

5 PRs. 4 fixed. No action needed to upgrade

Changes

OpenAI: Fixed keyless workload identity deployments answering 401 on image edits, audio, files, batches and other routes (#50962)
Other providers: Fixed ollama_chat/ streams giving every parallel tool call index 0, so clients ran only the first call (#50718)
Messages API: Fixed /v1/messages answering 400 for a param listed in the deployment's additional_drop_params (#50755)
Cost tracking and pricing: Fixed incorrect transcription cost for some FLAC files, which could push a key over its budget (#50860)

Full changelog

`v1.108.4`: 1.108.4 (LTS)

Compare Source

[!TIP]
The lts image tag always points to the latest LTS release. Pull ghcr.io/berriai/litellm:lts to stay on the newest LTS patch

Verify this release

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \
  ghcr.io/berriai/litellm:v1.108.4

Same command for litellm-database, litellm-non_root and the docker.io/litellm/* images

What's new in 1.108.4 (LTS)

10 PRs. 7 fixed, 1 changed. No action needed to upgrade

Changes

AWS Bedrock: Fixed 400s from GLM, GPT-6 Luna and Nova 2 on Bedrock when the conversation carries Claude's signed thinking (#50399)
AWS Bedrock: Changed /bedrock pass-through routes to enforce the model's guardrailConfig over any guardrail the caller sends (#47429)
Vertex AI and Gemini: Fixed Vertex AI Claude batches failing with 404 at create (#50365)
Messages API: Fixed chatgpt/ models on /v1/messages dropping encrypted reasoning between turns (#50456)
Messages API: Fixed /v1/messages to OpenAI sending background: true upstream, which billed full runs that came back empty (#50462)
Embeddings, images and audio: Fixed /v1/videos polls failing with per-model keys or hitting the wrong provider account with a 404 (#44614)
Routing and fallbacks: Fixed team keys being served by a proxy-wide wildcard instead of the team's own BYOK wildcard deployment (#48490)
SDK: Fixed completion(), responses() and messages with tools failing with No module named 'mcp' on a base install, broken since v1.106.0 (#45961)

Backport #50568 · Backport #50605 · Backport #50693 · Full changelog

`v1.108.3`: 1.108.3 (LTS)

Compare Source

[!TIP]
The lts image tag always points to the latest LTS release. Pull ghcr.io/berriai/litellm:lts to stay on the newest LTS patch

Verify this release

cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \
  ghcr.io/berriai/litellm:v1.108.3

Same command for litellm-database, litellm-non_root and the docker.io/litellm/* images

What's new in 1.108.3 (LTS)

11 PRs. 3 fixed. No action needed to upgrade

Changes

AWS Bedrock: Fixed every request failing on a Bedrock deployment with api_base: "", which answered 500 or 400 (#50449)
MCP: Fixed tool results over 1 MiB failing with SSE stream ended without a response, broken since v1.103.0 (#47034)
Guardrails: Fixed model-level post_call guardrails checking each response twice and calling the moderation provider twice (#50463)

Backport #50484 · Backport #50487 · Full changelog

Configuration

📅 Schedule: (in timezone Europe/Rome)

Branch creation
- At any time (no schedule defined)
Automerge
- At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate CLI.

This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | minor | `v1.108.2` → `v1.109.1` | --- > ⚠️ **Warning** > > Some dependencies could not be looked up. Check the [Dependency Dashboard](issues/6) for more information. --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.109.1`](https://github.com/BerriAI/litellm/releases/tag/v1.109.1) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.109.0...v1.109.1) #### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). This release is signed with the key introduced in [commit `e82a3c6`](https://github.com/BerriAI/litellm/commit/e82a3c646908ba0a6ee0beddc4310d7a6cac4065). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the published signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \ ghcr.io/berriai/litellm:v1.109.1 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.109.1/cosign.pub \ ghcr.io/berriai/litellm:v1.109.1 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** #### What's Changed - security(proxy): backport [#51254](https://github.com/BerriAI/litellm/issues/51254) to stable/1.109.x by [@shrey-berri](https://github.com/shrey-berri) in [#51264](https://github.com/BerriAI/litellm/pull/51264) - chore: bump litellm 1.109.0 -> 1.109.1 by [@yuneng-berri](https://github.com/yuneng-berri) in [#51298](https://github.com/BerriAI/litellm/pull/51298) **Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.109.0...v1.109.1> ### [`v1.109.0`](https://github.com/BerriAI/litellm/releases/tag/v1.109.0) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.108.8...v1.109.0) #### Verify this release ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \ ghcr.io/berriai/litellm:v1.109.0 ``` Same command for `litellm-database`, `litellm-non_root` and the `docker.io/litellm/*` images #### What's new in 1.109.0 982 PRs. 144 added, 424 fixed, 60 changed, 17 improved, 9 removed. 5 need action ##### Action required - **Routing and fallbacks:** Change string fallback targets to lists of model names ([#48943](https://github.com/BerriAI/litellm/issues/48943)) - **Caching:** Switch `cache_params.type` from `disk` to `local` or `redis`, since the `disk` cache backend (diskcache) has been removed ([#48114](https://github.com/BerriAI/litellm/issues/48114)) - **Routing and fallbacks:** Add `optional_pre_call_checks: ["enforce_model_rate_limits"]` to keep a hard `rpm` cap on deployments ([#48873](https://github.com/BerriAI/litellm/issues/48873)) - **Vertex AI and Gemini:** Set `drop_params: true` or stop sending `temperature`, `top_p` and `top_k` to Gemini 3.6 Flash, 3.5 Flash-Lite and newer ([#47645](https://github.com/BerriAI/litellm/issues/47645)) - **Deployment:** Deploy Lens from its own chart and set `lens.*` in your values ([#49169](https://github.com/BerriAI/litellm/issues/49169)) ##### Highlights - **Admin UI:** Added a Chat UI with a model hub, image generation and usage metrics ([#48108](https://github.com/BerriAI/litellm/issues/48108)) - **Routing and fallbacks:** Added mid-stream fallback continuation so a streaming chat cut off mid-answer continues on the configured fallback ([#41127](https://github.com/BerriAI/litellm/issues/41127)) - **Deployment:** Added `LITELLM_OFFLINE_MODE` to turn off every fetch LiteLLM makes on its own with one switch ([#48096](https://github.com/BerriAI/litellm/issues/48096)) - **Cost tracking and pricing:** Added `model_info.pricing_provider` to price a deployment from another provider's catalog rates instead of $0 ([#47620](https://github.com/BerriAI/litellm/issues/47620)) - **Vertex AI and Gemini:** Added Gemini 3.1 Flash TTS support, including multi-speaker voices and `mp3`, `opus`, `alaw` and `mulaw` output ([#31915](https://github.com/BerriAI/litellm/issues/31915)) - **SDK:** Fixed tool calls on `completion()` and `responses()` failing with `No module named 'mcp'` on base SDK installs ([#45961](https://github.com/BerriAI/litellm/issues/45961)) Also in [1.108.4](https://github.com/BerriAI/litellm/releases/tag/v1.108.4) ##### All changes <details><summary>OpenAI (19)</summary> - Added per-deployment workload identity on non-chat routes, realtime and the moderation guardrail ([#39966](https://github.com/BerriAI/litellm/issues/39966)) - Added the 12-hour cache write rate for `gpt-5.6-cyber` ([#48246](https://github.com/BerriAI/litellm/issues/48246)) - Added OpenAI `gpt-6.1-terra` with pricing ([#48429](https://github.com/BerriAI/litellm/issues/48429)) - Added Batch and Ultrafast 12-hour cache write rates for OpenAI models ([#48433](https://github.com/BerriAI/litellm/issues/48433)) - Added `gpt-6-rosalind` to the model cost map with its published prices ([#49268](https://github.com/BerriAI/litellm/issues/49268)) - Added `gpt-6-astra-law` with pricing from the OpenAI pricing page ([#49444](https://github.com/BerriAI/litellm/issues/49444)) - Fixed 400 errors and $0 spend logs on `gpt-transcribe` calls without `response_format` ([#44021](https://github.com/BerriAI/litellm/issues/44021)) - Fixed nonstreaming `aiohttp_openai` responses dropping `service_tier` ([#47263](https://github.com/BerriAI/litellm/issues/47263)) - Fixed `prompt_cache_retention` such as `24h` being silently dropped instead of sent upstream ([#47480](https://github.com/BerriAI/litellm/issues/47480)) - Fixed `/health` marking OpenAI and Azure audio chat models unhealthy ([#47730](https://github.com/BerriAI/litellm/issues/47730)) - Fixed OpenAI `context_length_exceeded` errors raising `BadRequestError` instead of `ContextWindowExceededError` ([#47954](https://github.com/BerriAI/litellm/issues/47954)) - Fixed 12-hour prompt cache writes on `openai/gpt-5.6` and newer billing at the 30-minute rate ([#48088](https://github.com/BerriAI/litellm/issues/48088)) - Fixed uncatalogued ChatGPT models hitting an unsupported Chat Completions endpoint ([#48244](https://github.com/BerriAI/litellm/issues/48244)) - Fixed the whole `reasoning_effort` dict not being sent when chat is bridged to Responses ([#48807](https://github.com/BerriAI/litellm/issues/48807)) - Fixed 400 `invalid_beta` on Agents API calls through the OpenAI passthrough ([#49009](https://github.com/BerriAI/litellm/issues/49009)) - Fixed empty `e.response.text` on OpenAI and Azure errors so the upstream error body is readable ([#49079](https://github.com/BerriAI/litellm/issues/49079)) - Changed OpenAI 401 and 404 `invalid_request_error` to raise matching exceptions, not `BadRequestError` ([#32537](https://github.com/BerriAI/litellm/issues/32537)) - Changed the ChatGPT session id to derive from `prompt_cache_key` for stable cache affinity ([#42014](https://github.com/BerriAI/litellm/issues/42014)) - Changed `gpt-6-luna` Decisions cache rates to the standard cached and write rates ([#48066](https://github.com/BerriAI/litellm/issues/48066)) </details> <details><summary>Azure (18)</summary> - Added Entra ID auth, including keyless auth, for Azure Decisions on `/v1/decisions` and `/v1/systemone` ([#46539](https://github.com/BerriAI/litellm/issues/46539), [#48529](https://github.com/BerriAI/litellm/issues/48529)) - Added access to Foundry agents through the project Responses API for `azure_ai/agents/<name>` ([#47717](https://github.com/BerriAI/litellm/issues/47717)) - Added `azure_ai/MAI-Code-1.1-Flash` ([#47779](https://github.com/BerriAI/litellm/issues/47779)) - Added `max_input_tokens` of 32768 for `azure_ai/Microsoft-Decision-1` ([#48434](https://github.com/BerriAI/litellm/issues/48434)) - Added `outputContentFormat=markdown` support on `/ocr` for Azure Document Intelligence ([#48926](https://github.com/BerriAI/litellm/issues/48926)) - Fixed Foundry chat requests losing OpenAI prompt cache breakpoints ([#45299](https://github.com/BerriAI/litellm/issues/45299)) - Fixed `/v1/responses` 404 on dated `api_version` deployments and `azure_ai` Grok reasoning token billing ([#46377](https://github.com/BerriAI/litellm/issues/46377)) - Fixed under-billed Grok reasoning tokens on `azure_ai` when the deployment name lacks "grok" ([#46969](https://github.com/BerriAI/litellm/issues/46969)) - Fixed Azure realtime over WebRTC failing with a 404 on `client_secrets` and an invalid token on the SDP exchange ([#47316](https://github.com/BerriAI/litellm/issues/47316)) - Fixed 400 from Foundry when tool results contain `tool_reference` parts on chat ([#48032](https://github.com/BerriAI/litellm/issues/48032)) - Fixed 400 on `azure_ai/claude-*` when using `thinking.block_binding` ([#48033](https://github.com/BerriAI/litellm/issues/48033)) - Fixed Responses API calls that use an explicit v1 Azure base URL ([#48118](https://github.com/BerriAI/litellm/issues/48118)) - Fixed 404 on Azure vector store retrieve, update and delete ([#48224](https://github.com/BerriAI/litellm/issues/48224)) - Fixed null audio capability metadata on two Azure audio snapshot models ([#48475](https://github.com/BerriAI/litellm/issues/48475)) - Fixed Azure `context_length_exceeded` raising `BadRequestError` so `context_window_fallbacks` never ran ([#48616](https://github.com/BerriAI/litellm/issues/48616)) - Fixed batch calls on unversioned `azure/` o3, o4-mini, o3-mini, gpt-4o and gpt-4o-mini billing at the standard rate ([#49182](https://github.com/BerriAI/litellm/issues/49182)) - Changed `max_input_tokens` to 128000 for `gpt-chat-latest` on Azure ([#48252](https://github.com/BerriAI/litellm/issues/48252)) - Improved Entra auth by reusing credentials and tokens across requests and refreshing them off the event loop ([#46421](https://github.com/BerriAI/litellm/issues/46421), [#47348](https://github.com/BerriAI/litellm/issues/47348), [#48769](https://github.com/BerriAI/litellm/issues/48769)) </details> <details><summary>Anthropic (16)</summary> - Fixed a fake `[System: ...]` message being added to tool-call-only assistant turns sent to Anthropic ([#34822](https://github.com/BerriAI/litellm/issues/34822)) - Fixed PDFs sent as chat `image_url` being rejected with 400 by sending them as document blocks ([#46984](https://github.com/BerriAI/litellm/issues/46984)) - Fixed Claude `json_schema` answers losing keys listed only in `required` and escaping non-ASCII characters ([#47478](https://github.com/BerriAI/litellm/issues/47478)) - Fixed replayed `redacted_thinking` blocks being dropped, and Bedrock Converse 400s when sent twice ([#47489](https://github.com/BerriAI/litellm/issues/47489)) - Fixed `clear_tool_uses` `input_tokens` trigger ignoring the system prompt, which delayed clearing ([#47857](https://github.com/BerriAI/litellm/issues/47857)) - Fixed native Anthropic responses omitting a null `stop_sequence` ([#47987](https://github.com/BerriAI/litellm/issues/47987)) - Fixed streamed responses leaving native web search fees out of the reconstructed usage cost ([#48528](https://github.com/BerriAI/litellm/issues/48528)) - Fixed `drop_params` not dropping `speed` on `claude-opus-4-7`, which Anthropic rejects with 400 ([#48764](https://github.com/BerriAI/litellm/issues/48764)) - Fixed 400 from beta-gated body fields when `drop_params: true`, including Claude Code and Codex traffic ([#48793](https://github.com/BerriAI/litellm/issues/48793)) - Fixed `model_context_window_exceeded` being reported as `stop` instead of `length` ([#48874](https://github.com/BerriAI/litellm/issues/48874)) - Fixed 400 on `/v1/messages` with many strict tools under `drop_params` by retrying without `strict` ([#49102](https://github.com/BerriAI/litellm/issues/49102)) - Fixed Claude `pause_turn` reported as finished and refusal explanations dropped on chat and Responses ([#49192](https://github.com/BerriAI/litellm/issues/49192)) - Fixed 500 on a non-string role or tool call id, now a 400 that names the field ([#49237](https://github.com/BerriAI/litellm/issues/49237)) - Fixed `metadata.user_id` being dropped on Anthropic `/v1/messages` pass-through routes ([#49333](https://github.com/BerriAI/litellm/issues/49333)) - Changed the default `max_tokens` for unmapped Anthropic models from 4096 to 128000 ([#48502](https://github.com/BerriAI/litellm/issues/48502)) - Changed the opening prompt usage estimate in the Chat Completions adapter to be off by default ([#48633](https://github.com/BerriAI/litellm/issues/48633)) </details> <details><summary>AWS Bedrock (28)</summary> - Added context and output token limits for the Nova Sonic models ([#48247](https://github.com/BerriAI/litellm/issues/48247)) - Added pricing for native `guardrailConfig` trace usage to spend and `x-litellm-response-cost` ([#48278](https://github.com/BerriAI/litellm/issues/48278)) - Added `global.openai.gpt-6.1-terra` and `us.openai.gpt-6.1-terra` Bedrock models with pricing ([#48441](https://github.com/BerriAI/litellm/issues/48441)) - Added token counting for three Mantle-native Bedrock Claude models ([#48682](https://github.com/BerriAI/litellm/issues/48682)) - Added published Priority, Batch and Flex prices to 46 Bedrock regional rows ([#49113](https://github.com/BerriAI/litellm/issues/49113)) - Fixed Bedrock Voxtral transcription returning 400, logging $0 spend and ignoring `language` ([#41565](https://github.com/BerriAI/litellm/issues/41565)) - Fixed `Unknown provider=None` 404 for Bedrock models missing from the cost map ([#43290](https://github.com/BerriAI/litellm/issues/43290)) - Fixed 400 errors on Bedrock Kimi K3 Converse calls with `max_tokens` below 16 ([#43875](https://github.com/BerriAI/litellm/issues/43875)) - Fixed 400 `invalid beta flag` on Claude Code auto mode requests that carry no `safeguards` ([#44471](https://github.com/BerriAI/litellm/issues/44471)) - Fixed a malformed Bedrock batch JSONL line failing the upload with a 500 instead of a 400 ([#45992](https://github.com/BerriAI/litellm/issues/45992)) - Fixed `post_call` guardrails not scanning Claude output on `invoke-with-response-stream` passthrough ([#46524](https://github.com/BerriAI/litellm/issues/46524)) - Fixed Claude `json_schema` streams dropping reasoning and thinking blocks and ending on `tool_calls` with no tool calls ([#46863](https://github.com/BerriAI/litellm/issues/46863)) - Fixed JSON schema requests failing on Fable 5 and Sonnet 4.6 and losing adaptive thinking on Haiku 5.5 ([#46870](https://github.com/BerriAI/litellm/issues/46870)) - Fixed `bedrock_request_metadata_fields` being ignored on the `/bedrock` pass-through routes ([#47040](https://github.com/BerriAI/litellm/issues/47040)) - Fixed Bedrock Opus 4.8 and Sonnet 5 dropping reasoning on `json_schema` requests by trying native output first ([#47100](https://github.com/BerriAI/litellm/issues/47100)) - Fixed Bedrock batch jobs ignoring a configured control-plane endpoint ([#47260](https://github.com/BerriAI/litellm/issues/47260)) - Fixed AWS region lookups through Vault blocking the event loop and stalling other requests on the worker ([#47331](https://github.com/BerriAI/litellm/issues/47331)) - Fixed $0 spend and `ModelNotMappedError` for `bedrock/claude_platform/<id>` calls ([#48150](https://github.com/BerriAI/litellm/issues/48150)) - Fixed `max_input_tokens` to 922000 on Bedrock GPT models and added Mantle token counting for `claude-opus-5-5` ([#48479](https://github.com/BerriAI/litellm/issues/48479)) - Fixed Bedrock rejecting `strict` tool schemas by retrying without `strict` under `drop_params` ([#48733](https://github.com/BerriAI/litellm/issues/48733)) - Fixed 400 on Bedrock when Codex repeats its tool list across turns ([#49006](https://github.com/BerriAI/litellm/issues/49006)) - Fixed region-prefixed Bedrock embedding models not routing to their region or billing its price ([#49072](https://github.com/BerriAI/litellm/issues/49072)) - Fixed `bedrock_mantle/openai.gpt-5.6-cyber` token limits and endpoints to match its model card ([#49350](https://github.com/BerriAI/litellm/issues/49350)) - Fixed `max_output_tokens` for Bedrock GovCloud West Llama 3 models, now 8192 ([#49543](https://github.com/BerriAI/litellm/issues/49543)) - Changed `converse-stream` pass-through to stream frames through post_call guardrails instead of buffering the reply ([#46307](https://github.com/BerriAI/litellm/issues/46307)) - Changed `/bedrock` pass-through routes to apply a model's `guardrailConfig`, returning 400 if Bedrock cannot use it ([#47429](https://github.com/BerriAI/litellm/issues/47429)) - Changed Bedrock pass-through to use request metadata as the session id, fixing 400s with `missing_session_id: reject` ([#49193](https://github.com/BerriAI/litellm/issues/49193)) - Changed Bedrock Invoke and Mantle Claude to use the model's own default `max_tokens` ([#49724](https://github.com/BerriAI/litellm/issues/49724)) </details> <details><summary>Vertex AI and Gemini (34)</summary> - Added Gemini 3.1 Flash TTS support, including multi-speaker voices and `mp3`, `opus`, `alaw` and `mulaw` output ([#31915](https://github.com/BerriAI/litellm/issues/31915)) - Added a setting to drop chosen `anthropic-beta` values on the Vertex AI pass-through routes ([#47185](https://github.com/BerriAI/litellm/issues/47185)) - Added `LITELLM_GEMINI_DEFAULT_MAX_OUTPUT_TOKENS` to set the output limit for requests that set none ([#47542](https://github.com/BerriAI/litellm/issues/47542)) - Added `thought_signature_in_tool_call_id` to keep Gemini tool call ids short for clients that cap them ([#47692](https://github.com/BerriAI/litellm/issues/47692)) - Added a 2026-06-30 deprecation date to `vertex_ai/veo-2.0-generate-001` ([#47847](https://github.com/BerriAI/litellm/issues/47847)) - Added Batch rates for two Llama 4 MaaS models so Vertex batch jobs no longer bill at on-demand rates ([#48085](https://github.com/BerriAI/litellm/issues/48085)) - Added Vertex batch prices above 200k tokens and image output prices for Gemini models ([#48945](https://github.com/BerriAI/litellm/issues/48945)) - Fixed the Gemini 3 sampling deprecation warning logging on every request, now once per model per worker ([#40868](https://github.com/BerriAI/litellm/issues/40868)) - Fixed `vertex_ai/*` keys not listing partner, Veo and embedding models and dropping publisher prefixes ([#41959](https://github.com/BerriAI/litellm/issues/41959)) - Fixed Gemini TTS `/v1/audio/speech` calling Google twice and logging 0 tokens in spend ([#44551](https://github.com/BerriAI/litellm/issues/44551)) - Fixed 400 from compaction, thinking display and tool change betas on Claude via Vertex AI and Azure AI ([#44747](https://github.com/BerriAI/litellm/issues/44747)) - Fixed context caching on Vertex AI failing when a custom `api_base` is set ([#44783](https://github.com/BerriAI/litellm/issues/44783)) - Fixed streamed Vertex AI pass-through calls missing the regional 10% uplift and custom deployment prices ([#44795](https://github.com/BerriAI/litellm/issues/44795)) - Fixed Gemini not receiving integer enum values in tool parameters ([#44908](https://github.com/BerriAI/litellm/issues/44908)) - Fixed Vertex Claude batches returning 404 at create ([#45715](https://github.com/BerriAI/litellm/issues/45715)) - Fixed cache-marked requests listing all `cachedContents` on every call and creating duplicate caches ([#46857](https://github.com/BerriAI/litellm/issues/46857)) - Fixed Vertex AI rate limit errors returned as 403 when the error text contains 403 ([#47134](https://github.com/BerriAI/litellm/issues/47134)) - Fixed Gemini audio input and cached audio billed at the Standard rate on flex, priority and batch tiers ([#48086](https://github.com/BerriAI/litellm/issues/48086)) - Fixed `gemini-2.5-flash-native-audio` video input billed at the text rate instead of the audio/video rate ([#48248](https://github.com/BerriAI/litellm/issues/48248)) - Fixed Gemini 2.5 rejecting tool results with images and the history replay that follows ([#48428](https://github.com/BerriAI/litellm/issues/48428)) - Fixed Vertex AI shared credential cache growing without bound across credential and project pairs ([#48442](https://github.com/BerriAI/litellm/issues/48442)) - Fixed Vertex AI image sizes outside the old lookup table silently producing square output ([#48463](https://github.com/BerriAI/litellm/issues/48463)) - Fixed slow Google credential requests blocking the proxy worker on Claude `/v1/messages` ([#48523](https://github.com/BerriAI/litellm/issues/48523)) - Fixed explicit Gemini and Vertex embedding timeouts silently becoming 300 seconds ([#48690](https://github.com/BerriAI/litellm/issues/48690)) - Fixed Gemini grounding keeping only the first web source for a passage ([#48745](https://github.com/BerriAI/litellm/issues/48745)) - Fixed `labels` on `vertex_ai/claude-*` returning 400 by sending them as `X-Vertex-AI-Labels` ([#48753](https://github.com/BerriAI/litellm/issues/48753)) - Fixed 400 on `countTokens` and `generateContent` when repeated fields are a single object or null ([#48771](https://github.com/BerriAI/litellm/issues/48771)) - Fixed Vertex Claude `/v1/messages` returning 400 when the SDK sends `anthropic-version` as a header ([#48814](https://github.com/BerriAI/litellm/issues/48814)) - Fixed Claude web search billing per request on Vertex AI deployments priced in `model_info` ([#49160](https://github.com/BerriAI/litellm/issues/49160)) - Fixed batch image output billed at the Standard rate on `gemini-nano-banana-2.1` and `gemini-3.1-flash-lite-image` ([#49317](https://github.com/BerriAI/litellm/issues/49317), [#49358](https://github.com/BerriAI/litellm/issues/49358)) - Changed Gemini 3.6 Flash, 3.5 Flash-Lite and newer to refuse `temperature`, `top_p` and `top_k` without `drop_params` ([#47645](https://github.com/BerriAI/litellm/issues/47645)) - Improved rerank latency by reusing cached Google credentials on every request ([#48430](https://github.com/BerriAI/litellm/issues/48430)) - Improved failure logs for Gemini and Vertex pass-through calls by adding the URL model and declared provider ([#49075](https://github.com/BerriAI/litellm/issues/49075)) - Removed the floor `deprecation_date` from `gemini-3.1-flash-lite-image` ([#48587](https://github.com/BerriAI/litellm/issues/48587)) </details> <details><summary>Other providers (3)</summary> - Added 11 new providers: ai&, BigModel, CLF AI Gateway, Hostinger Router, Kyma API, Nace, OpenCode Go, OpenInfer, Refuseless, StreamLake and ZeroGPU ([#38958](https://github.com/BerriAI/litellm/issues/38958), [#39324](https://github.com/BerriAI/litellm/issues/39324), [#41006](https://github.com/BerriAI/litellm/issues/41006), [#41034](https://github.com/BerriAI/litellm/issues/41034), [#43874](https://github.com/BerriAI/litellm/issues/43874), [#44863](https://github.com/BerriAI/litellm/issues/44863), [#46523](https://github.com/BerriAI/litellm/issues/46523), [#48140](https://github.com/BerriAI/litellm/issues/48140), [#48497](https://github.com/BerriAI/litellm/issues/48497), [#48563](https://github.com/BerriAI/litellm/issues/48563), [#48756](https://github.com/BerriAI/litellm/issues/48756)) - Added models, endpoints and options for smaller providers (31 PRs) ([#41021](https://github.com/BerriAI/litellm/issues/41021), [#41522](https://github.com/BerriAI/litellm/issues/41522), [#46156](https://github.com/BerriAI/litellm/issues/46156), [#46875](https://github.com/BerriAI/litellm/issues/46875), [#47401](https://github.com/BerriAI/litellm/issues/47401), [#47473](https://github.com/BerriAI/litellm/issues/47473), [#47610](https://github.com/BerriAI/litellm/issues/47610), [#47690](https://github.com/BerriAI/litellm/issues/47690), [#47785](https://github.com/BerriAI/litellm/issues/47785), [#47812](https://github.com/BerriAI/litellm/issues/47812), [#48158](https://github.com/BerriAI/litellm/issues/48158), [#48251](https://github.com/BerriAI/litellm/issues/48251), [#48335](https://github.com/BerriAI/litellm/issues/48335), [#48448](https://github.com/BerriAI/litellm/issues/48448), [#48450](https://github.com/BerriAI/litellm/issues/48450), [#48453](https://github.com/BerriAI/litellm/issues/48453), [#48498](https://github.com/BerriAI/litellm/issues/48498), [#48621](https://github.com/BerriAI/litellm/issues/48621), [#48651](https://github.com/BerriAI/litellm/issues/48651), [#48676](https://github.com/BerriAI/litellm/issues/48676), [#48677](https://github.com/BerriAI/litellm/issues/48677), [#48735](https://github.com/BerriAI/litellm/issues/48735), [#48871](https://github.com/BerriAI/litellm/issues/48871), [#48998](https://github.com/BerriAI/litellm/issues/48998), [#49252](https://github.com/BerriAI/litellm/issues/49252), [#49280](https://github.com/BerriAI/litellm/issues/49280), [#49281](https://github.com/BerriAI/litellm/issues/49281), [#49282](https://github.com/BerriAI/litellm/issues/49282), [#49287](https://github.com/BerriAI/litellm/issues/49287), [#49297](https://github.com/BerriAI/litellm/issues/49297), [#49542](https://github.com/BerriAI/litellm/issues/49542)) - Fixed issues and updated behavior on smaller providers (41 PRs) ([#38736](https://github.com/BerriAI/litellm/issues/38736), [#40152](https://github.com/BerriAI/litellm/issues/40152), [#43390](https://github.com/BerriAI/litellm/issues/43390), [#44723](https://github.com/BerriAI/litellm/issues/44723), [#45092](https://github.com/BerriAI/litellm/issues/45092), [#45567](https://github.com/BerriAI/litellm/issues/45567), [#46091](https://github.com/BerriAI/litellm/issues/46091), [#46861](https://github.com/BerriAI/litellm/issues/46861), [#46945](https://github.com/BerriAI/litellm/issues/46945), [#46959](https://github.com/BerriAI/litellm/issues/46959), [#47091](https://github.com/BerriAI/litellm/issues/47091), [#47423](https://github.com/BerriAI/litellm/issues/47423), [#47424](https://github.com/BerriAI/litellm/issues/47424), [#47432](https://github.com/BerriAI/litellm/issues/47432), [#47495](https://github.com/BerriAI/litellm/issues/47495), [#47784](https://github.com/BerriAI/litellm/issues/47784), [#47794](https://github.com/BerriAI/litellm/issues/47794), [#48121](https://github.com/BerriAI/litellm/issues/48121), [#48254](https://github.com/BerriAI/litellm/issues/48254), [#48265](https://github.com/BerriAI/litellm/issues/48265), [#48268](https://github.com/BerriAI/litellm/issues/48268), [#48270](https://github.com/BerriAI/litellm/issues/48270), [#48455](https://github.com/BerriAI/litellm/issues/48455), [#48522](https://github.com/BerriAI/litellm/issues/48522), [#48527](https://github.com/BerriAI/litellm/issues/48527), [#48668](https://github.com/BerriAI/litellm/issues/48668), [#48684](https://github.com/BerriAI/litellm/issues/48684), [#48706](https://github.com/BerriAI/litellm/issues/48706), [#48707](https://github.com/BerriAI/litellm/issues/48707), [#48778](https://github.com/BerriAI/litellm/issues/48778), [#48820](https://github.com/BerriAI/litellm/issues/48820), [#48837](https://github.com/BerriAI/litellm/issues/48837), [#49068](https://github.com/BerriAI/litellm/issues/49068), [#49158](https://github.com/BerriAI/litellm/issues/49158), [#49278](https://github.com/BerriAI/litellm/issues/49278), [#49344](https://github.com/BerriAI/litellm/issues/49344), [#49458](https://github.com/BerriAI/litellm/issues/49458), [#49462](https://github.com/BerriAI/litellm/issues/49462), [#49498](https://github.com/BerriAI/litellm/issues/49498), [#49502](https://github.com/BerriAI/litellm/issues/49502), [#49616](https://github.com/BerriAI/litellm/issues/49616)) </details> <details><summary>Chat completions (17)</summary> - Fixed streams with `logprobs` ending in a 500 error frame before the finish chunk ([#33549](https://github.com/BerriAI/litellm/issues/33549)) - Fixed 500 errors when a prompt template omits `initial_prompt_value` or `final_prompt_value` ([#39762](https://github.com/BerriAI/litellm/issues/39762)) - Fixed rebuilt streamed text completions returning null `finish_reason` and zero `prompt_tokens` ([#44349](https://github.com/BerriAI/litellm/issues/44349)) - Fixed Foundry deployments and Claude fallbacks getting the wrong cache options, which caused 400s ([#45295](https://github.com/BerriAI/litellm/issues/45295)) - Fixed non-streaming chat dropping upstream response headers on `bedrock_mantle/` and other OpenAI-like providers ([#46928](https://github.com/BerriAI/litellm/issues/46928)) - Fixed chat streams failing with a proxy error when progress chunks have no choice index ([#46947](https://github.com/BerriAI/litellm/issues/46947)) - Fixed the overhead header under-reporting on chat requests when guardrails add `litellm_metadata` ([#47790](https://github.com/BerriAI/litellm/issues/47790)) - Fixed streamed chunks containing `[DONE]` being read as end of stream and dropped ([#47832](https://github.com/BerriAI/litellm/issues/47832)) - Fixed completions streams including `usage` when it was not requested ([#48052](https://github.com/BerriAI/litellm/issues/48052)) - Fixed an upstream `abort` on streamed chat completions returning a clean HTTP 200 instead of a server error ([#48328](https://github.com/BerriAI/litellm/issues/48328)) - Fixed unbalanced think tags in merged streams and SSE lines splitting on characters other than CR/LF ([#48755](https://github.com/BerriAI/litellm/issues/48755)) - Fixed 400 on system-first models like Qwen3.5 when a later `developer` message appears ([#48897](https://github.com/BerriAI/litellm/issues/48897)) - Fixed streamed chat completions with `n>1` rebuilding each choice ([#49002](https://github.com/BerriAI/litellm/issues/49002)) - Fixed streaming clients missing tool execution when OpenAI-compatible providers end tool calls with `stop` ([#49388](https://github.com/BerriAI/litellm/issues/49388)) - Changed the `include_usage` stream chunk to send an empty `choices` array like OpenAI ([#47109](https://github.com/BerriAI/litellm/issues/47109)) - Changed stalled stream reads to raise a 408 `litellm.Timeout` on chat, messages and responses ([#47198](https://github.com/BerriAI/litellm/issues/47198)) - Changed streams that end without a finish reason to fail as errors instead of reporting a normal completion ([#47665](https://github.com/BerriAI/litellm/issues/47665)) </details> <details><summary>Responses API (34)</summary> - Added `max_agentic_loops` support to raise the streamed MCP auto-execute round cap above 5 ([#47688](https://github.com/BerriAI/litellm/issues/47688)) - Added resuming background response streams with `stream` and `starting_after` on retrieve ([#48819](https://github.com/BerriAI/litellm/issues/48819)) - Added Codex client `tool_search` support on Anthropic and other bridged providers, cutting input tokens ([#48972](https://github.com/BerriAI/litellm/issues/48972)) - Fixed Responses API logging tasks being dropped by garbage collection and cut off at shutdown ([#31074](https://github.com/BerriAI/litellm/issues/31074)) - Fixed non-stream MCP auto-execute on `/v1/responses` logging and billing only one of its two model calls ([#43667](https://github.com/BerriAI/litellm/issues/43667)) - Fixed 400 from malformed `prompt_cache_breakpoint` on `/v1/responses` with `drop_params: true` ([#45330](https://github.com/BerriAI/litellm/issues/45330)) - Fixed later turns on an open Responses WebSocket skipping deployment checks and not counting toward budgets ([#45727](https://github.com/BerriAI/litellm/issues/45727)) - Fixed streamed `/v1/responses` events showing the backend deployment name in nested `response.model` ([#46662](https://github.com/BerriAI/litellm/issues/46662)) - Fixed `/v1/responses` on Claude dropping returned and replayed `compaction` items ([#46754](https://github.com/BerriAI/litellm/issues/46754)) - Fixed Responses WebSocket running `pre_call` guardrails on turn one only and rebilling that cost every turn ([#47013](https://github.com/BerriAI/litellm/issues/47013)) - Fixed OpenAI pass-through `GET /responses/{id}` being billed again on every call ([#47171](https://github.com/BerriAI/litellm/issues/47171)) - Fixed background spend not counting toward the key's project or per-model budgets ([#47504](https://github.com/BerriAI/litellm/issues/47504)) - Fixed the billed cost of background responses not reaching key and team loggers ([#47506](https://github.com/BerriAI/litellm/issues/47506)) - Fixed failed Responses streams billing 0 tokens and bridged `/v1/messages` stream failures not being reported ([#47537](https://github.com/BerriAI/litellm/issues/47537)) - Fixed `clear_tool_uses_20250919` being silently dropped for `openai/` models on the Responses API ([#47861](https://github.com/BerriAI/litellm/issues/47861)) - Fixed per-call `input_cost_per_token` and `output_cost_per_token` being ignored by `responses()` ([#48119](https://github.com/BerriAI/litellm/issues/48119)) - Fixed 500 when replaying `web_search_call` input items that have no `action` ([#48417](https://github.com/BerriAI/litellm/issues/48417)) - Fixed audio, PDF and image tool results and history media being dropped in Responses and Gemini conversions ([#48484](https://github.com/BerriAI/litellm/issues/48484)) - Fixed retries from `previous_response_id` returning 400 on Bedrock after a failed turn ([#48647](https://github.com/BerriAI/litellm/issues/48647)) - Fixed Responses params the chat bridge cannot translate causing 400s like `max_tool_calls` on Bedrock Claude ([#48655](https://github.com/BerriAI/litellm/issues/48655)) - Fixed typed streaming text access breaking on `/v1/responses` when `logprobs` is omitted ([#48686](https://github.com/BerriAI/litellm/issues/48686)) - Fixed `moderation` on `/v1/responses` being dropped before the OpenAI call ([#48708](https://github.com/BerriAI/litellm/issues/48708)) - Fixed later citation batches and pending citations being dropped when streaming Responses ([#48747](https://github.com/BerriAI/litellm/issues/48747)) - Fixed replayed Responses output items being rejected by OpenAI because of a null `created_by` ([#48783](https://github.com/BerriAI/litellm/issues/48783)) - Fixed an empty `function_call_output` reaching models as serialized JSON instead of an empty tool result ([#48845](https://github.com/BerriAI/litellm/issues/48845)) - Fixed Responses compaction by dropping params it rejects and sending Azure compaction to the v1 route ([#48876](https://github.com/BerriAI/litellm/issues/48876)) - Fixed Responses item order, phase and boundaries not being replayed on the next chat turn ([#48912](https://github.com/BerriAI/litellm/issues/48912)) - Fixed bridged Responses requests losing the requested text verbosity when sent to chat completions ([#49049](https://github.com/BerriAI/litellm/issues/49049)) - Fixed deployment `itpm` and `otpm` limits not being checked or counted on later Responses WebSocket turns ([#49069](https://github.com/BerriAI/litellm/issues/49069)) - Fixed `litellm.responses()` through `litellm_proxy/` leaving `e.body`, `e.code` and `e.type` empty on errors ([#49073](https://github.com/BerriAI/litellm/issues/49073)) - Fixed dict `reasoning_effort` losing `mode` and `context` on Azure, ChatGPT, Perplexity, xAI, Fireworks and Copilot ([#49082](https://github.com/BerriAI/litellm/issues/49082)) - Fixed repeated Codex tools being merged for all callers, now only merged when `drop_params` is on ([#49112](https://github.com/BerriAI/litellm/issues/49112)) - Fixed native `/v1/responses` WebSocket turns not counting toward key and tag `tpm_limit` ([#49257](https://github.com/BerriAI/litellm/issues/49257)) - Changed a lone Responses built-in tool on Claude to return 400 before token counting ([#48798](https://github.com/BerriAI/litellm/issues/48798)) </details> <details><summary>Messages API (14)</summary> - Fixed web-search-only requests coming back without native search blocks on streamed replies ([#40967](https://github.com/BerriAI/litellm/issues/40967)) - Fixed retried `/v1/messages` attempts mixing up usage and spend logs, and exhausted retries billing only the last ([#45194](https://github.com/BerriAI/litellm/issues/45194)) - Fixed `safeguard_results` and `stop_details` missing from streamed `/v1/messages` `message_delta` events ([#46096](https://github.com/BerriAI/litellm/issues/46096)) - Fixed the `clear_tool_uses` polyfill ignoring `clear_at_least`, `exclude_tools` and `clear_tool_inputs` ([#47123](https://github.com/BerriAI/litellm/issues/47123)) - Fixed `/v1/messages` PDF documents being sent to Azure OpenAI models as images instead of file parts ([#47166](https://github.com/BerriAI/litellm/issues/47166)) - Fixed 500 on `/v1/messages` and count_tokens for null or numeric `tool_result` text, now a 400 ([#47347](https://github.com/BerriAI/litellm/issues/47347)) - Fixed 400 `Tool reference not found` for tool names over 64 chars when a guardrail rewrites `/v1/messages` ([#47434](https://github.com/BerriAI/litellm/issues/47434)) - Fixed canceled `/v1/messages` streams not closing the upstream request ([#48220](https://github.com/BerriAI/litellm/issues/48220)) - Fixed `stop_sequences` being ignored for OpenAI models on `/v1/messages` ([#48534](https://github.com/BerriAI/litellm/issues/48534)) - Fixed the client's `anthropic-beta` replacing the deployment's on `/v1/messages` ([#48594](https://github.com/BerriAI/litellm/issues/48594)) - Fixed `/v1/messages` streams reporting zero `input_tokens` in `message_start` when bridged to Chat Completions ([#48618](https://github.com/BerriAI/litellm/issues/48618)) - Fixed Anthropic-format `tool_choice` crashing input token counting and causing false budget refusals ([#48624](https://github.com/BerriAI/litellm/issues/48624)) - Fixed 400 on `/v1/messages` caused by a non-spec `custom` object in tool definitions ([#48893](https://github.com/BerriAI/litellm/issues/48893)) - Fixed Mistral thinking text being lost on streaming `/v1/messages` ([#49059](https://github.com/BerriAI/litellm/issues/49059)) </details> <details><summary>Realtime (4)</summary> - Added Soniox realtime speech-to-text on `/v1/realtime` with `stt-rt-v5` ([#44822](https://github.com/BerriAI/litellm/issues/44822)) - Fixed Gemini Live ignoring `session.update` instructions and tools, and answering text clients with audio ([#39864](https://github.com/BerriAI/litellm/issues/39864)) - Fixed `/health` reporting OpenAI realtime deployments with a missing or invalid key as healthy ([#41233](https://github.com/BerriAI/litellm/issues/41233)) - Fixed sessions going silent when a transcript guardrail sent `turn_detection` with no `type` to OpenAI ([#44939](https://github.com/BerriAI/litellm/issues/44939)) </details> <details><summary>Embeddings, images and audio (8)</summary> - Added a per-request input batch limit on `/v1/embeddings` to cap how many inputs one call can send ([#44574](https://github.com/BerriAI/litellm/issues/44574)) - Fixed video status and content polling failing with per-model keys by keeping `model_id` in video ids ([#44614](https://github.com/BerriAI/litellm/issues/44614)) - Fixed `aws_polly/standard`, `long-form` and `generative` being sent with the `neural` engine ([#47963](https://github.com/BerriAI/litellm/issues/47963)) - Fixed image generation ignoring `max_retries` and Router retries multiplying upstream attempts ([#48705](https://github.com/BerriAI/litellm/issues/48705)) - Fixed async image edits with no image returning HTTP 500 instead of 400 ([#48872](https://github.com/BerriAI/litellm/issues/48872)) - Fixed key and team `router_settings` being ignored on `/v1/images/generations` ([#49286](https://github.com/BerriAI/litellm/issues/49286)) - Changed unsupported transcription providers to return 400 naming the provider and model instead of 500 ([#47971](https://github.com/BerriAI/litellm/issues/47971)) - Removed `dall-e-2` and `dall-e-3` support, and image generation without a model now returns 400 ([#48703](https://github.com/BerriAI/litellm/issues/48703)) </details> <details><summary>Batches and files (8)</summary> - Added routing of `POST /v1/batches` to batch deployments and `supported_endpoints` on `/model_group/info` ([#47376](https://github.com/BerriAI/litellm/issues/47376)) - Fixed 400 on `GET /files/{id}/content` with raw `gs://` file ids from the unmanaged Vertex batch flow ([#40621](https://github.com/BerriAI/litellm/issues/40621)) - Fixed fine-tuning jobs failing with 400 for files uploaded with `target_model_names` ([#46552](https://github.com/BerriAI/litellm/issues/46552)) - Fixed completed batches skipping success callbacks and losing usage under redaction ([#47192](https://github.com/BerriAI/litellm/issues/47192)) - Fixed file lists showing every file on the shared key and ignoring `limit` and `after` with `require_managed_files` ([#47682](https://github.com/BerriAI/litellm/issues/47682)) - Fixed batch retrieve showing zero prompt and completion tokens ([#47740](https://github.com/BerriAI/litellm/issues/47740)) - Fixed unbillable batches blocking file deletes and new-batch polling ([#48752](https://github.com/BerriAI/litellm/issues/48752)) - Fixed unmanaged `gs://` Vertex batches being costed through the wrong deployment and reading zero spend ([#49301](https://github.com/BerriAI/litellm/issues/49301)) </details> <details><summary>Vector stores (5)</summary> - Added OpenSearch k-NN search as a vector store provider ([#47759](https://github.com/BerriAI/litellm/issues/47759)) - Fixed `/rag/ingest` and `/rag/query` ignoring team provider credentials and denying viewers and team JWTs ([#45497](https://github.com/BerriAI/litellm/issues/45497)) - Fixed team key vector store search failing when the team's first model is from a different provider ([#46894](https://github.com/BerriAI/litellm/issues/46894)) - Fixed team and key Allowed Vector Stores being ignored on `/v1/responses` `file_search` without a registry ([#48915](https://github.com/BerriAI/litellm/issues/48915)) - Improved Vertex AI Search vector store searches by reusing HTTP/2 connections and OAuth tokens ([#40631](https://github.com/BerriAI/litellm/issues/40631)) </details> <details><summary>Token counting (10)</summary> - Added one log line per token count call with its id, model and count ([#47657](https://github.com/BerriAI/litellm/issues/47657)) - Fixed `/v1/messages/count_tokens` and `/utils/token_counter` failing on histories with a compaction block ([#44784](https://github.com/BerriAI/litellm/issues/44784)) - Fixed DeepSeek V4 token counts missing Azure's billed tokens by using DeepSeek's own tokenizer ([#47173](https://github.com/BerriAI/litellm/issues/47173), [#47353](https://github.com/BerriAI/litellm/issues/47353)) - Fixed `count_tokens` overcounting PDFs about 12x for OpenAI and Azure models ([#47554](https://github.com/BerriAI/litellm/issues/47554)) - Fixed token counts retrying a failed HuggingFace tokenizer download and adding about 23s per request ([#47586](https://github.com/BerriAI/litellm/issues/47586)) - Fixed HTTP 500 when counting tokens for Anthropic search results and unsupported blocks ([#48699](https://github.com/BerriAI/litellm/issues/48699)) - Fixed token counts for OpenAI models newer than the tiktoken table by using `o200k_base` ([#48728](https://github.com/BerriAI/litellm/issues/48728)) - Fixed a loaded Hugging Face tokenizer being hidden when a later download fails ([#48916](https://github.com/BerriAI/litellm/issues/48916)) - Changed Responses input token counts to match inference, fixing Claude counts with tools and `text.format` ([#47363](https://github.com/BerriAI/litellm/issues/47363)) - Improved `token_counter` for inline `auto` images, including images under 768 pixels, with no HTTP requests ([#48526](https://github.com/BerriAI/litellm/issues/48526)) </details> <details><summary>MCP (27)</summary> - Added upstream status and response body to the log when an MCP health check fails ([#44125](https://github.com/BerriAI/litellm/issues/44125)) - Added support for redeeming an ID-JAG as an RFC 7523 `jwt-bearer` grant on the gateway token endpoint ([#46748](https://github.com/BerriAI/litellm/issues/46748)) - Added Authorization URL, Token URL and Scopes fields to OAuth Delegate and True Passthrough server forms ([#46941](https://github.com/BerriAI/litellm/issues/46941)) - Added tool search and multi-select to the MCP toolset editor in the Admin UI ([#48601](https://github.com/BerriAI/litellm/issues/48601)) - Added failure logging for MCP SDK request rejections, shown as `MCP: request_rejected` ([#49184](https://github.com/BerriAI/litellm/issues/49184)) - Fixed `LITELLM_MCP_TOOL_LISTING_TIMEOUT` not applying to paginated `tools/list` on `/mcp` ([#46513](https://github.com/BerriAI/litellm/issues/46513)) - Fixed `tools/list` returning an empty list when servers answer 429, now retried with an error if all fail ([#46649](https://github.com/BerriAI/litellm/issues/46649)) - Fixed 412 on MCP calls with `id_jag_subject_source: refresh_token` after an Okta OIDC login ([#46745](https://github.com/BerriAI/litellm/issues/46745), [#46747](https://github.com/BerriAI/litellm/issues/46747), [#47004](https://github.com/BerriAI/litellm/issues/47004)) - Fixed the caller's LiteLLM key being forwarded to upstream MCP servers when `Authorization` is in `extra_headers` ([#46932](https://github.com/BerriAI/litellm/issues/46932)) - Fixed VS Code sign-in to DCR bridge MCP servers when the upstream rejects loopback redirect URIs ([#46940](https://github.com/BerriAI/litellm/issues/46940)) - Fixed 403 when `proxy_admin_viewer` stores its own MCP credentials ([#46994](https://github.com/BerriAI/litellm/issues/46994)) - Fixed large MCP SSE tool results failing, and invalid size limits breaking MCP discovery and tool calls ([#47034](https://github.com/BerriAI/litellm/issues/47034)) - Fixed streamed MCP auto-execute billing only the first of its two model calls ([#47422](https://github.com/BerriAI/litellm/issues/47422), [#47445](https://github.com/BerriAI/litellm/issues/47445), [#48899](https://github.com/BerriAI/litellm/issues/48899)) - Fixed an upstream MCP 429 on session handshake being reported as an internal error ([#47569](https://github.com/BerriAI/litellm/issues/47569)) - Fixed MCP OAuth sign-in callback returning 403 behind AWS WAF by sealing the `litellm_cp_return_to` cookie ([#47595](https://github.com/BerriAI/litellm/issues/47595)) - Fixed per-user auth MCP servers missing from `/mcp-rest/tools/list` and `/mcp` `tools/list` ([#47677](https://github.com/BerriAI/litellm/issues/47677)) - Fixed streaming MCP follow-ups on `/v1/responses` dropping all but the latest tool round ([#47698](https://github.com/BerriAI/litellm/issues/47698)) - Fixed slow MCP tool calls delaying response headers until the SSE ping ([#47897](https://github.com/BerriAI/litellm/issues/47897)) - Fixed MCP management routes to follow the gateway's existing access rules ([#48080](https://github.com/BerriAI/litellm/issues/48080)) - Fixed a readonly MCP toolset URL returning every tool instead of only the toolset's tools ([#48095](https://github.com/BerriAI/litellm/issues/48095)) - Fixed oversized SSE tool catalogs returning a generic 500, now 502 with the size limit and pagination advice ([#48424](https://github.com/BerriAI/litellm/issues/48424)) - Fixed MCP toolset OAuth login advertising a resource URL that clients reject ([#48474](https://github.com/BerriAI/litellm/issues/48474)) - Fixed revoked or expired browser sessions still being accepted during gateway authorization ([#48494](https://github.com/BerriAI/litellm/issues/48494)) - Fixed MCP OpenAPI loading to accept valid YAML specs ([#48622](https://github.com/BerriAI/litellm/issues/48622)) - Fixed MCP tools in both `allowed_tools` and `disallowed_tools` still being callable ([#48625](https://github.com/BerriAI/litellm/issues/48625)) - Fixed MCP tools hidden by discovery guardrails still being callable by name ([#48669](https://github.com/BerriAI/litellm/issues/48669)) - Changed admin-owned team keys to reach only their team's MCP servers and toolsets ([#49161](https://github.com/BerriAI/litellm/issues/49161)) </details> <details><summary>Agents (10)</summary> - Added the pi harness to `litellm.agent()` and fixed harness turn `cost` ignoring prompt cache prices ([#45467](https://github.com/BerriAI/litellm/issues/45467)) - Added request log entries for rejected A2A calls to forbidden or unknown agents ([#49492](https://github.com/BerriAI/litellm/issues/49492)) - Fixed OpenCode via `litellm.agent()` using the script's folder instead of the sandbox, and hanging on resume ([#47530](https://github.com/BerriAI/litellm/issues/47530)) - Fixed Gemini Managed Agents create returning 400 for `name` and interactions dropping inline `agent_config` ([#48750](https://github.com/BerriAI/litellm/issues/48750)) - Fixed `/a2a` calls to completion-bridge agents logging $0 on send and no spend row on stream ([#49150](https://github.com/BerriAI/litellm/issues/49150)) - Fixed 404 on `/a2a/{agent_id}/message/stream` and `/v1/a2a/{agent_id}/message/stream` ([#49471](https://github.com/BerriAI/litellm/issues/49471)) - Fixed agent `litellm_params.api_key` not being sent as bearer auth on A2A calls ([#49474](https://github.com/BerriAI/litellm/issues/49474)) - Fixed agent list and detail responses showing non-admins the backend URL instead of the proxy URL ([#49483](https://github.com/BerriAI/litellm/issues/49483)) - Fixed a blank search embedding model returning 503 instead of the `*_not_configured` 400 on agent and skill search ([#49485](https://github.com/BerriAI/litellm/issues/49485)) - Changed proxy refusals on `/a2a` POST routes to return JSON-RPC errors ([#49476](https://github.com/BerriAI/litellm/issues/49476)) </details> <details><summary>Guardrails (21)</summary> - Added opt-in `response_masked_data` on PANW Prisma AIRS `post_call` blocks ([#46219](https://github.com/BerriAI/litellm/issues/46219)) - Added `num_retries` to `generic_guardrail_api` for checks lost after connecting ([#46965](https://github.com/BerriAI/litellm/issues/46965)) - Added a native Reco guardrail that blocks requests on Reco errors unless `fail_on_error: false` ([#48243](https://github.com/BerriAI/litellm/issues/48243)) - Added Ovalix guardrail scanning of files, tool calls and tool results ([#49309](https://github.com/BerriAI/litellm/issues/49309)) - Added `litellm.guardrails.apply` SDK helper to invoke configured guardrails and get native findings ([#49460](https://github.com/BerriAI/litellm/issues/49460), [#49469](https://github.com/BerriAI/litellm/issues/49469), [#49470](https://github.com/BerriAI/litellm/issues/49470)) - Fixed guardrail masks on messages and tools not reaching Bedrock Converse requests ([#45874](https://github.com/BerriAI/litellm/issues/45874)) - Fixed `generic_guardrail_api` rejecting Gemini native tools that have no `type` key ([#46338](https://github.com/BerriAI/litellm/issues/46338)) - Fixed corrupted numbered PII tokens in Presidio when one value matches several entities with `output_parse_pii` ([#47024](https://github.com/BerriAI/litellm/issues/47024)) - Fixed `hide-secrets` missing secrets in tool results, tool inputs, `tool_calls` arguments, and system prompts ([#47122](https://github.com/BerriAI/litellm/issues/47122)) - Fixed generic guardrails with `mode: pre_mcp_call` being rejected at startup so MCP calls went unchecked ([#48060](https://github.com/BerriAI/litellm/issues/48060)) - Fixed pass-through guardrails scanning `litellm_metadata` fields added by LiteLLM, which caused false matches ([#48276](https://github.com/BerriAI/litellm/issues/48276)) - Fixed `hide-secrets` ignoring `skip_tool_message_in_guardrail` ([#48689](https://github.com/BerriAI/litellm/issues/48689)) - Fixed guardrails chosen in a Responses WebSocket frame not running, and a 500 with other guardrail settings ([#48725](https://github.com/BerriAI/litellm/issues/48725)) - Fixed Responses tool results not reaching `texts` guardrails ([#48827](https://github.com/BerriAI/litellm/issues/48827)) - Fixed `google_text_moderation` scanning embeddings input ([#48934](https://github.com/BerriAI/litellm/issues/48934)) - Fixed guardrail blocks on streamed `/v1/messages` not returning Anthropic events ([#49010](https://github.com/BerriAI/litellm/issues/49010)) - Fixed streamed `/v1/completions` skipping `post_call` guardrails for masking and blocking ([#49035](https://github.com/BerriAI/litellm/issues/49035)) - Fixed mid-stream upstream errors on `/v1/messages` being turned into a guardrail 400 ([#49088](https://github.com/BerriAI/litellm/issues/49088)) - Changed guardrail blocks mid-stream to end chat completions streams with an error event, not a cut connection ([#46953](https://github.com/BerriAI/litellm/issues/46953)) - Changed `lakera_v2` `on_flagged: block` to reject PII-only detections and added `on_flagged: mask` ([#48944](https://github.com/BerriAI/litellm/issues/48944)) - Improved streaming by running sampled guardrail scans in the background so chunks no longer pause ([#48952](https://github.com/BerriAI/litellm/issues/48952)) </details> <details><summary>Caching (4)</summary> - Fixed streamed cache hits dropping reasoning, citations and audio and reordering tool calls ([#32096](https://github.com/BerriAI/litellm/issues/32096)) - Fixed `cache_control` being stripped from caller messages and tools, which broke caching on Claude fallbacks ([#35602](https://github.com/BerriAI/litellm/issues/35602)) - Fixed the cache key's model group being read from caller metadata instead of the router's metadata ([#48917](https://github.com/BerriAI/litellm/issues/48917)) - Removed the disk cache backend, so `cache_params.type: disk` now stops proxy startup ([#48114](https://github.com/BerriAI/litellm/issues/48114)) </details> <details><summary>Routing and fallbacks (44)</summary> - Added mid-stream fallback continuation so a streaming chat cut off mid-answer continues on the configured fallback ([#41127](https://github.com/BerriAI/litellm/issues/41127)) - Added Kubernetes pod discovery to spread load across pods behind a headless service `api_base` ([#44342](https://github.com/BerriAI/litellm/issues/44342), [#44393](https://github.com/BerriAI/litellm/issues/44393), [#44666](https://github.com/BerriAI/litellm/issues/44666)) - Added opt-in filtering that keeps chat requests off deployments flagged as non-chat ([#46257](https://github.com/BerriAI/litellm/issues/46257)) - Added the `x-litellm-priority` header to set request priority ([#47618](https://github.com/BerriAI/litellm/issues/47618)) - Added opt-in session repinning to the replacement deployment after a successful failover ([#48206](https://github.com/BerriAI/litellm/issues/48206)) - Added a `clamp` mode for complexity routing that keeps the caller's smaller output token limit ([#48632](https://github.com/BerriAI/litellm/issues/48632)) - Added deployment `rpd` and `tpd` daily limits to `enforce_model_rate_limits` ([#48875](https://github.com/BerriAI/litellm/issues/48875), [#48957](https://github.com/BerriAI/litellm/issues/48957)) - Added inferring custom classifier tiers from model mappings, and fixed a crash on plan-mode requests ([#48987](https://github.com/BerriAI/litellm/issues/48987)) - Fixed weighted failover skipping healthy same-group deployments under affinity ([#32309](https://github.com/BerriAI/litellm/issues/32309)) - Fixed `content_policy_fallbacks` not applying after `POST /fallback` or `/config/update` changes ([#45428](https://github.com/BerriAI/litellm/issues/45428)) - Fixed an explicit `allowed_fails: 3` not counting failures and fallbacks cooling down healthy deployments ([#46183](https://github.com/BerriAI/litellm/issues/46183)) - Fixed a `bedrock_mantle` deployment's `mode` switching sibling deployments between Chat Completions and Responses ([#46263](https://github.com/BerriAI/litellm/issues/46263)) - Fixed prompt caching affinity ignoring tools and the `/v1/messages` system prompt when choosing a deployment ([#46411](https://github.com/BerriAI/litellm/issues/46411)) - Fixed all-cooled router 429 rows in SpendLogs not recording which deployments were cooling ([#47120](https://github.com/BerriAI/litellm/issues/47120)) - Fixed `stream_timeout` cutting off slow answers when Bedrock serves a `stream=True` call without streaming ([#47162](https://github.com/BerriAI/litellm/issues/47162)) - Fixed key and team `router_settings` reaching the provider and dropping retries, fallbacks and timeout ([#47257](https://github.com/BerriAI/litellm/issues/47257)) - Fixed 500s from `router_settings.default_priority` or an integer `priority`, and now reject a malformed `priority` ([#47498](https://github.com/BerriAI/litellm/issues/47498)) - Fixed fallback retries logging a negative completion time and dropping latency-routed groups ([#47541](https://github.com/BerriAI/litellm/issues/47541)) - Fixed prompt caching affinity raising on non-dict `tools` entries instead of returning the provider's 400 ([#47565](https://github.com/BerriAI/litellm/issues/47565)) - Fixed cut Responses streams returning `server_error` instead of continuing on a fallback without prefill ([#47672](https://github.com/BerriAI/litellm/issues/47672)) - Fixed `IndexError` and wrong deployment lookups when deployments are deleted or added concurrently ([#47680](https://github.com/BerriAI/litellm/issues/47680)) - Fixed native Responses latency using different units than bridged responses in latency-based routing ([#47727](https://github.com/BerriAI/litellm/issues/47727)) - Fixed the router sending image requests to `supports_vision: false` deployments in mixed model groups ([#48149](https://github.com/BerriAI/litellm/issues/48149), [#48152](https://github.com/BerriAI/litellm/issues/48152), [#48241](https://github.com/BerriAI/litellm/issues/48241)) - Fixed Decisions calls routing to deployments whose provider does not support them ([#48175](https://github.com/BerriAI/litellm/issues/48175)) - Fixed 500s on every request to a model group after one negative or NaN latency sample under latency routing ([#48216](https://github.com/BerriAI/litellm/issues/48216)) - Fixed tag fail-open groups rejecting unmatched tags after affinity or cooldown filtering ([#48470](https://github.com/BerriAI/litellm/issues/48470)) - Fixed one caller's exhausted forwarded Claude subscription putting the shared model into cooldown for others ([#48485](https://github.com/BerriAI/litellm/issues/48485)) - Fixed proxy-wide wildcard deployments shadowing a team's own wildcard deployment for that team's keys ([#48490](https://github.com/BerriAI/litellm/issues/48490)) - Fixed complexity router replacing the held `user_turn` route on Codex compaction requests ([#48627](https://github.com/BerriAI/litellm/issues/48627)) - Fixed session tier pins diverging between proxy workers when using Redis ([#48645](https://github.com/BerriAI/litellm/issues/48645)) - Fixed nested environment references in router classifier connections being sent literally ([#48717](https://github.com/BerriAI/litellm/issues/48717)) - Fixed lifecycle events not passing through on background OpenAI Responses streams that cannot fall back ([#48817](https://github.com/BerriAI/litellm/issues/48817)) - Fixed new fallback rules erasing older ones when Redis refuses `DEL` ([#48999](https://github.com/BerriAI/litellm/issues/48999)) - Fixed a deployment's own `model_id` being overridden by the request body's ([#49045](https://github.com/BerriAI/litellm/issues/49045)) - Fixed 429s on requests without `max_tokens` and token counters not dropping to real usage after calls ([#49054](https://github.com/BerriAI/litellm/issues/49054)) - Fixed streamed context window errors skipping fallbacks when they occur before any output ([#49293](https://github.com/BerriAI/litellm/issues/49293)) - Fixed large requests silently falling back to heuristic routing when they exceed the classifier's input limit ([#49306](https://github.com/BerriAI/litellm/issues/49306)) - Fixed queued requests waiting until a 408 when a dead replica left expired scheduler queue entries ([#49387](https://github.com/BerriAI/litellm/issues/49387)) - Changed per-deployment `reasoning_effort_levels` to apply on every provider and route, including `azure_ai` ([#46601](https://github.com/BerriAI/litellm/issues/46601)) - Changed tag routing to use only prefixed caller tags once `tag_routing_prefix` is set ([#47428](https://github.com/BerriAI/litellm/issues/47428)) - Changed router `allowed_fails` to keep a lone deployment with no fallback in rotation instead of returning 429 ([#48644](https://github.com/BerriAI/litellm/issues/48644)) - Changed `user_turn` routing so child agent requests keep their own tier pin ([#48701](https://github.com/BerriAI/litellm/issues/48701)) - Changed Router init to reject string fallback targets that never routed ([#48943](https://github.com/BerriAI/litellm/issues/48943)) - Removed the deployment `rpm` filter from `enable_pre_call_checks` ([#48873](https://github.com/BerriAI/litellm/issues/48873)) </details> <details><summary>Cost tracking and pricing (36)</summary> - Added `s3://` URI support for `LITELLM_MODEL_COST_MAP_URL` ([#46972](https://github.com/BerriAI/litellm/issues/46972)) - Added `model_info.pricing_provider` to price a deployment from another provider's catalog rates instead of $0 ([#47620](https://github.com/BerriAI/litellm/issues/47620)) - Added `x-litellm-upstream-model` header so pass-through upstreams can report the model name for spend ([#48275](https://github.com/BerriAI/litellm/issues/48275)) - Added a per-guardrail spend breakdown to user, team and organization daily activity ([#48702](https://github.com/BerriAI/litellm/issues/48702)) - Fixed Responses-style bodies sent to `/cursor/chat/completions` not being logged or billed ([#30277](https://github.com/BerriAI/litellm/issues/30277)) - Fixed cached and reasoning token counts dropped from streamed usage, pricing streams at the full input rate ([#36678](https://github.com/BerriAI/litellm/issues/36678)) - Fixed per-second deployment rates billing per token on native `/v1/messages` ([#42417](https://github.com/BerriAI/litellm/issues/42417)) - Fixed streams with no backend usage being recorded at 0 tokens and $0 instead of the token estimate ([#44230](https://github.com/BerriAI/litellm/issues/44230)) - Fixed background interactions going unbilled after a crash, a late restart or on models with their own `api_key` ([#45743](https://github.com/BerriAI/litellm/issues/45743)) - Fixed daily tag spend being lost when the Redis tag buffer write failed ([#47307](https://github.com/BerriAI/litellm/issues/47307)) - Fixed deployments with `model_info.base_model` such as `azure/eu/o3` billing $0 on every request ([#47497](https://github.com/BerriAI/litellm/issues/47497)) - Fixed unpriced Decisions model versions billing $0 and malformed replies failing with an unclear error ([#47508](https://github.com/BerriAI/litellm/issues/47508)) - Fixed non-streamed requests adding double their recorded cost to key spend ([#47510](https://github.com/BerriAI/litellm/issues/47510)) - Fixed $0 billing for undated `azure/eu/*` and `azure/us/*` models and added Volcengine `doubao-seed-2-1` rows ([#47561](https://github.com/BerriAI/litellm/issues/47561)) - Fixed live budget counters missing the partial cost of streams that fail mid-flight ([#47741](https://github.com/BerriAI/litellm/issues/47741)) - Fixed key, team, user and daily spend rows being dropped when the proxy stops right after a request ([#48131](https://github.com/BerriAI/litellm/issues/48131)) - Fixed Azure AI Sonnet 5.5 cache read pricing, Azure web search cost and retired model dates in the cost map ([#48188](https://github.com/BerriAI/litellm/issues/48188)) - Fixed batch costs billing 12-hour cache writes at the plain batch write rate ([#48259](https://github.com/BerriAI/litellm/issues/48259)) - Fixed Price Data Reload wiping custom pricing of DB models, which logged $0 spend on `/v1/messages` ([#48486](https://github.com/BerriAI/litellm/issues/48486)) - Fixed `extra_spend_tag_headers` not matching mixed-case header names ([#48620](https://github.com/BerriAI/litellm/issues/48620)) - Fixed `gpt-5-nano` priority pricing using an input price OpenAI does not sell ([#48763](https://github.com/BerriAI/litellm/issues/48763)) - Fixed a NaN or infinite cost blocking daily spend writes and returning 500 on non-streaming calls ([#48829](https://github.com/BerriAI/litellm/issues/48829)) - Fixed SDK calls sharing one `metadata` dict being billed for earlier calls' Bedrock guardrail traces ([#48923](https://github.com/BerriAI/litellm/issues/48923)) - Fixed a gateway's `usage.cost` being billed as 0 on chat, Responses and streamed calls ([#48953](https://github.com/BerriAI/litellm/issues/48953)) - Fixed streamed `usage.cost` disagreeing with the spend row the proxy writes ([#48955](https://github.com/BerriAI/litellm/issues/48955)) - Fixed response cost differing between the header, spend log and budget counters ([#48993](https://github.com/BerriAI/litellm/issues/48993)) - Fixed wrong rates for long-prompt Gemini Pro batch and long-context Azure GPT-6 Astra Flex requests ([#49114](https://github.com/BerriAI/litellm/issues/49114)) - Fixed explicit zero prices being discarded in batch cache, speech and Perplexity search costs ([#49294](https://github.com/BerriAI/litellm/issues/49294)) - Fixed rerank token usage missing from spend logs ([#49308](https://github.com/BerriAI/litellm/issues/49308)) - Fixed provider-reported totals booking all spend on output, now split by the cost map rate ratio ([#49357](https://github.com/BerriAI/litellm/issues/49357)) - Fixed streamed bridged `/v1/messages` spend booked as input 0 and discounts applied twice ([#49365](https://github.com/BerriAI/litellm/issues/49365)) - Fixed `/openai` pass-through `/v1/decisions` calls logged at $0 with no tokens ([#49392](https://github.com/BerriAI/litellm/issues/49392)) - Changed streamed `/v1/completions` without `stream_options` to bill from provider usage like chat ([#45366](https://github.com/BerriAI/litellm/issues/45366)) - Changed daily activity reports and exports to show missing model and key values under fallback buckets ([#48925](https://github.com/BerriAI/litellm/issues/48925)) - Removed the legacy `max_tokens` cost map field in favor of `max_input_tokens` and `max_output_tokens` ([#48471](https://github.com/BerriAI/litellm/issues/48471)) - Updated cost map prices for models on smaller providers (17 PRs) ([#47411](https://github.com/BerriAI/litellm/issues/47411), [#47623](https://github.com/BerriAI/litellm/issues/47623), [#48694](https://github.com/BerriAI/litellm/issues/48694), [#48825](https://github.com/BerriAI/litellm/issues/48825), [#48844](https://github.com/BerriAI/litellm/issues/48844), [#48849](https://github.com/BerriAI/litellm/issues/48849), [#48858](https://github.com/BerriAI/litellm/issues/48858), [#48879](https://github.com/BerriAI/litellm/issues/48879), [#48884](https://github.com/BerriAI/litellm/issues/48884), [#48891](https://github.com/BerriAI/litellm/issues/48891), [#48900](https://github.com/BerriAI/litellm/issues/48900), [#48978](https://github.com/BerriAI/litellm/issues/48978), [#49123](https://github.com/BerriAI/litellm/issues/49123), [#49124](https://github.com/BerriAI/litellm/issues/49124), [#49125](https://github.com/BerriAI/litellm/issues/49125), [#49146](https://github.com/BerriAI/litellm/issues/49146), [#49148](https://github.com/BerriAI/litellm/issues/49148)) </details> <details><summary>Logging and observability (39)</summary> - Added key alias, user and end user to Langfuse generation metadata on OTEL v2 ([#42697](https://github.com/BerriAI/litellm/issues/42697)) - Added `maximum_spend_logs_size` to delete the oldest spend logs first once they exceed a size limit ([#47450](https://github.com/BerriAI/litellm/issues/47450), [#47451](https://github.com/BerriAI/litellm/issues/47451), [#47500](https://github.com/BerriAI/litellm/issues/47500)) - Added a warning log when `drop_params` drops a requested reasoning level ([#47761](https://github.com/BerriAI/litellm/issues/47761), [#48933](https://github.com/BerriAI/litellm/issues/48933)) - Added `disable_pre_routing_error_logs` to skip spend rows for pre-routing rejections ([#48239](https://github.com/BerriAI/litellm/issues/48239)) - Added `is_silent_experiment` marker on mirrored `silent_model` calls in spend logs and callbacks ([#48888](https://github.com/BerriAI/litellm/issues/48888)) - Added diagnostics that identify aiohttp connection pool saturation behind request timeouts ([#49499](https://github.com/BerriAI/litellm/issues/49499)) - Fixed 404 when deleting callbacks configured under `callbacks` or `failure_callback` ([#42740](https://github.com/BerriAI/litellm/issues/42740)) - Fixed Responses `instructions` still reaching request and response logs when message logging is turned off ([#43315](https://github.com/BerriAI/litellm/issues/43315)) - Fixed OTel spans reporting full cost for cache hits and added a cache-hit flag ([#43604](https://github.com/BerriAI/litellm/issues/43604)) - Fixed failed OTel spans showing an empty status message instead of the error message ([#44185](https://github.com/BerriAI/litellm/issues/44185)) - Fixed spend logs ignoring the caller's `litellm_session_id` when `request_correlation_in_logs: true` ([#44199](https://github.com/BerriAI/litellm/issues/44199)) - Fixed Langfuse showing no time to first token for `langfuse_otel` calls ([#45597](https://github.com/BerriAI/litellm/issues/45597)) - Fixed provider-side 401s missing from `litellm_deployment_failure_responses` and `litellm_deployment_state` ([#46668](https://github.com/BerriAI/litellm/issues/46668)) - Fixed Prometheus `/metrics` not reporting a value for the in-memory spend queue size ([#46956](https://github.com/BerriAI/litellm/issues/46956)) - Fixed the proxy printing plaintext at startup with `json_logs` on, so the banner and model list are now JSON ([#47035](https://github.com/BerriAI/litellm/issues/47035)) - Fixed mid-stream failure alerts reporting `https://api.openai.com` and dropping deployment, key and team ([#47045](https://github.com/BerriAI/litellm/issues/47045)) - Fixed `CustomLogger` receiving both sync and async failure hooks on async call failures ([#47124](https://github.com/BerriAI/litellm/issues/47124)) - Fixed a Prometheus-enabled backend returning 404 on `/metrics` during startup ([#47129](https://github.com/BerriAI/litellm/issues/47129)) - Fixed legacy `s3` success callback writing no object for streamed `/v1/messages` calls ([#47156](https://github.com/BerriAI/litellm/issues/47156)) - Fixed root logger INFO lines being dropped when `json_logs` is on and `LITELLM_LOG` is set ([#47175](https://github.com/BerriAI/litellm/issues/47175)) - Fixed Prometheus deployment failure metrics exporting `api_provider="None"` on many failure paths ([#47384](https://github.com/BerriAI/litellm/issues/47384)) - Fixed spend log size-based cleanup over-deleting rows after a Postgres statistics reset ([#47449](https://github.com/BerriAI/litellm/issues/47449), [#47501](https://github.com/BerriAI/litellm/issues/47501)) - Fixed per-team Arize sampling to decide by trace id so every pod keeps or drops the same traces ([#47468](https://github.com/BerriAI/litellm/issues/47468)) - Fixed dropped streams being logged with 0 prompt and completion tokens instead of the usage already sent ([#47838](https://github.com/BerriAI/litellm/issues/47838)) - Fixed `spend_logs_metadata` being `null` on `/v1beta/interactions` spend log rows ([#47848](https://github.com/BerriAI/litellm/issues/47848)) - Fixed key and team Langfuse destinations losing user, tags and session on OTel v2 without global Langfuse keys ([#48718](https://github.com/BerriAI/litellm/issues/48718)) - Fixed excluded requests like `/model/info` still exporting auth and database spans on OTEL v2 ([#48720](https://github.com/BerriAI/litellm/issues/48720)) - Fixed proxy logs staying red after a failed request under `--detailed_debug` ([#48966](https://github.com/BerriAI/litellm/issues/48966)) - Fixed `/v1/models` requests missing from the caller's OTEL trace ([#49141](https://github.com/BerriAI/litellm/issues/49141)) - Fixed `llm_exceptions` alerts firing on every `BudgetExceededError` rejection ([#49290](https://github.com/BerriAI/litellm/issues/49290)) - Fixed image generation requests refused by the router missing from failure logging ([#49349](https://github.com/BerriAI/litellm/issues/49349)) - Fixed failure metrics losing deployment ids for blocked requests and Ollama metadata lookups calling the wrong server ([#49727](https://github.com/BerriAI/litellm/issues/49727)) - Changed `langfuse_otel` to set trace-level name, input, output and tags on traces LiteLLM starts ([#42705](https://github.com/BerriAI/litellm/issues/42705)) - Changed OTel spans to omit cost when the price lookup failed instead of reporting 0 ([#44187](https://github.com/BerriAI/litellm/issues/44187)) - Changed Prometheus to count successful fallbacks under the originally requested model ([#45555](https://github.com/BerriAI/litellm/issues/45555)) - Changed proxy 401 auth rejections to log one WARNING line without a traceback ([#48613](https://github.com/BerriAI/litellm/issues/48613)) - Changed pass-through routes to trace as `litellm.passthrough` instead of `chat` with OTel v2 or GenAI semconv ([#48675](https://github.com/BerriAI/litellm/issues/48675)) - Changed OTel v2 to stop printing span JSON to stdout unless the operator asks ([#48723](https://github.com/BerriAI/litellm/issues/48723)) - Improved `--detailed_debug` logs by printing the request body only where it enters and leaves the proxy ([#48973](https://github.com/BerriAI/litellm/issues/48973)) </details> <details><summary>Keys, teams and auth (33)</summary> - Added assigning a project to a key that has none through `/key/update`, without regenerating the key ([#41700](https://github.com/BerriAI/litellm/issues/41700)) - Added an opt-in setting for key owners to edit `max_budget` and `budget_limits` on `/key/update` ([#44763](https://github.com/BerriAI/litellm/issues/44763)) - Added mapping of propagated `baggage` values to budget tags ([#47358](https://github.com/BerriAI/litellm/issues/47358)) - Added per-provider and per-model rpm/tpm requirements for `enforce_project_model_quota` ([#47693](https://github.com/BerriAI/litellm/issues/47693)) - Added per-request resolution of `oidc/google/<audience>` `api_key` references ([#47703](https://github.com/BerriAI/litellm/issues/47703)) - Added `general_settings.claude_code_marketplace_auth_required` to put the Claude Code marketplace behind a key ([#48980](https://github.com/BerriAI/litellm/issues/48980)) - Added choosing the session team for Claude Code gateway users ([#49001](https://github.com/BerriAI/litellm/issues/49001)) - Fixed `/team/block` and `/team/unblock` not applying to cached teams until the cache expired ([#35643](https://github.com/BerriAI/litellm/issues/35643)) - Fixed `default_key_generate_params.allowed_routes` being ignored on key generation ([#41236](https://github.com/BerriAI/litellm/issues/41236)) - Fixed SCIM PATCH dropping enterprise extension attributes, now stored in `metadata.scim_enterprise` ([#46738](https://github.com/BerriAI/litellm/issues/46738)) - Fixed 401 when an admin-configured `api_key` is an `oidc/file/` or `oidc/env/` reference ([#47341](https://github.com/BerriAI/litellm/issues/47341)) - Fixed SCIM group delete leaving memberships and members' team keys active, and removing members one at a time ([#47544](https://github.com/BerriAI/litellm/issues/47544)) - Fixed deleted keys staying usable on other workers until the auth cache TTL expired ([#47609](https://github.com/BerriAI/litellm/issues/47609)) - Fixed 403 on `/typesafe` Jev calls with `typesafe/*` allowlists and added Jev to Admin UI model pickers ([#47697](https://github.com/BerriAI/litellm/issues/47697)) - Fixed model family and access group entries in `model_max_budget` never being enforced ([#47722](https://github.com/BerriAI/litellm/issues/47722)) - Fixed team member `total_spend` showing lower than current cycle spend ([#48260](https://github.com/BerriAI/litellm/issues/48260)) - Fixed negative RPM and TPM limits being stored, they now return 422 ([#48444](https://github.com/BerriAI/litellm/issues/48444)) - Fixed concurrency slots staying occupied after finished streams, so keys got rejected with SSE keepalive on ([#48452](https://github.com/BerriAI/litellm/issues/48452)) - Fixed team members' spend never resetting after a temporary budget increase on a member using the team default ([#48495](https://github.com/BerriAI/litellm/issues/48495)) - Fixed 500 on `/key/generate` and `/user/new` with `get_spend_routes` and no license, now 403 ([#48623](https://github.com/BerriAI/litellm/issues/48623)) - Fixed key edit and regenerate sending `duration: ""` when the expiry field is blank ([#48634](https://github.com/BerriAI/litellm/issues/48634)) - Fixed `max_end_user_budget_id` not applying to new end users on cached keys ([#48704](https://github.com/BerriAI/litellm/issues/48704)) - Fixed per-model end user budgets not being enforced for master-key calls that send `user` ([#48815](https://github.com/BerriAI/litellm/issues/48815)) - Fixed TPM reservations ignoring chat media and overcounting Responses tool images as base64 text ([#48865](https://github.com/BerriAI/litellm/issues/48865)) - Fixed credential `PATCH` overwriting the stored secret with its masked read-back value ([#48950](https://github.com/BerriAI/litellm/issues/48950)) - Fixed CLI SSO and Claude Code gateway team lists changing order after team writes ([#49033](https://github.com/BerriAI/litellm/issues/49033)) - Fixed `/health` probing models a key or team is not allowed to use, including wildcard key models ([#49155](https://github.com/BerriAI/litellm/issues/49155)) - Fixed `team_id_upsert` not creating the `team_id_jwt_field` team on UI SSO login ([#49256](https://github.com/BerriAI/litellm/issues/49256)) - Fixed exhausted key or member budgets blocking model discovery and key and user info calls ([#49463](https://github.com/BerriAI/litellm/issues/49463)) - Changed `/key/info` to return a generic 500 for unexpected database errors ([#46190](https://github.com/BerriAI/litellm/issues/46190)) - Changed personal keys to inherit MCP and agent access from owner teams under `personal_key_model_access_from_teams` ([#46293](https://github.com/BerriAI/litellm/issues/46293)) - Changed key `budget_duration` and `budget_id` updates to need admin rights, like `max_budget` ([#47681](https://github.com/BerriAI/litellm/issues/47681)) - Changed free installs to refuse new SSO accounts past five users without locking out existing SSO users ([#48097](https://github.com/BerriAI/litellm/issues/48097)) </details> <details><summary>Admin UI (57)</summary> - Added shareable URL state for Usage views and links from Internal Users to Usage ([#43425](https://github.com/BerriAI/litellm/issues/43425)) - Added a general settings option to enable wildcard model listings from the dashboard ([#43624](https://github.com/BerriAI/litellm/issues/43624), [#47979](https://github.com/BerriAI/litellm/issues/47979)) - Added separate display of subscription-covered providers and account fees on the Usage page ([#44772](https://github.com/BerriAI/litellm/issues/44772)) - Added your own member spend and budget per team to the Your Teams page ([#45674](https://github.com/BerriAI/litellm/issues/45674)) - Added Delete to the guardrail detail page with a typed-name confirmation shared with the table row ([#45893](https://github.com/BerriAI/litellm/issues/45893)) - Added admin-defined custom tags and metadata from `model_info` on the Model Hub ([#46064](https://github.com/BerriAI/litellm/issues/46064)) - Added a 1 hour cache write cost field to the add model and model edit forms ([#46830](https://github.com/BerriAI/litellm/issues/46830)) - Added `LITELLM_UI_USE_SI_TERMINOLOGY` to show SI instead of AI in the Admin UI ([#47440](https://github.com/BerriAI/litellm/issues/47440)) - Added a failed state with Retry on the Logs table, Usage totals and Caching chart when fetches fail ([#47671](https://github.com/BerriAI/litellm/issues/47671)) - Added client usage inspection for users and team members ([#47974](https://github.com/BerriAI/litellm/issues/47974)) - Added sorting of virtual keys by expiry, activity, spend and limits ([#48074](https://github.com/BerriAI/litellm/issues/48074), [#48105](https://github.com/BerriAI/litellm/issues/48105), [#48106](https://github.com/BerriAI/litellm/issues/48106), [#48107](https://github.com/BerriAI/litellm/issues/48107), [#48109](https://github.com/BerriAI/litellm/issues/48109)) - Added usage charts and PDF export to Aria answers ([#48094](https://github.com/BerriAI/litellm/issues/48094)) - Added a Chat UI with a model hub, image generation and usage metrics ([#48108](https://github.com/BerriAI/litellm/issues/48108)) - Added PDF, text, `.docx` and `.xlsx` document uploads to Chat ([#48137](https://github.com/BerriAI/litellm/issues/48137)) - Added an admin setting for the default Usage time range ([#48207](https://github.com/BerriAI/litellm/issues/48207)) - Added budget period tooltips to key, team, user and organization spend tiles ([#48487](https://github.com/BerriAI/litellm/issues/48487), [#48488](https://github.com/BerriAI/litellm/issues/48488)) - Added sorting and persistent display ordering to the Model Group Alias table ([#48500](https://github.com/BerriAI/litellm/issues/48500)) - Added the `authorization` MCP auth type to the create and edit server forms ([#48629](https://github.com/BerriAI/litellm/issues/48629)) - Added Chat replies that keep running after the browser closes, with Stop to cancel ([#49461](https://github.com/BerriAI/litellm/issues/49461)) - Added an optional deep research mode in Chat with persistent source citations ([#49508](https://github.com/BerriAI/litellm/issues/49508), [#49531](https://github.com/BerriAI/litellm/issues/49531)) - Fixed the key edit form not prefilling Organization from the key's `org_id` ([#39535](https://github.com/BerriAI/litellm/issues/39535)) - Fixed UI request logs not showing the internal user's email ([#41150](https://github.com/BerriAI/litellm/issues/41150)) - Fixed the upgrade notice undercounting new features and fixes for patch releases ([#46942](https://github.com/BerriAI/litellm/issues/46942)) - Fixed negative prices being saved on the model edit form ([#46978](https://github.com/BerriAI/litellm/issues/46978)) - Fixed Model Hub showing stale Unhealthy status instead of the newest health check ([#47552](https://github.com/BerriAI/litellm/issues/47552)) - Fixed Admin UI snippets still showing the retired example master key ([#47612](https://github.com/BerriAI/litellm/issues/47612)) - Fixed team per-model rate limit and budget pickers missing access-group models ([#47630](https://github.com/BerriAI/litellm/issues/47630)) - Fixed Tool Policies page and `/v1/tool/list` staying empty with `use_redis_transaction_buffer` enabled ([#47653](https://github.com/BerriAI/litellm/issues/47653)) - Fixed empty headers blocking pass-through endpoint creation and authentication showing off for free users ([#48081](https://github.com/BerriAI/litellm/issues/48081)) - Fixed MCP results being dropped on Chat follow-up messages ([#48124](https://github.com/BerriAI/litellm/issues/48124)) - Fixed the Download CSV Template button in bulk invite doing nothing ([#48199](https://github.com/BerriAI/litellm/issues/48199)) - Fixed copy icons in key and log details silently failing over HTTP ([#48425](https://github.com/BerriAI/litellm/issues/48425)) - Fixed empty value cells for String settings in general settings ([#48431](https://github.com/BerriAI/litellm/issues/48431)) - Fixed the Aria button blocking Logs pagination by moving it into the dashboard header ([#48454](https://github.com/BerriAI/litellm/issues/48454)) - Fixed deleted model parameters returning after saving and refreshing ([#48456](https://github.com/BerriAI/litellm/issues/48456)) - Fixed denied or failed search tool calls showing `No results found` instead of the error ([#48464](https://github.com/BerriAI/litellm/issues/48464), [#48478](https://github.com/BerriAI/litellm/issues/48478)) - Fixed Decisions Playground hiding chat models that also declare decision endpoints ([#48504](https://github.com/BerriAI/litellm/issues/48504), [#48505](https://github.com/BerriAI/litellm/issues/48505)) - Fixed Connect showing the light logo in dark mode and Chat omitting configured branding ([#48532](https://github.com/BerriAI/litellm/issues/48532)) - Fixed the spend log detail drawer breaking for request IDs containing `#`, `?` or `%` ([#48630](https://github.com/BerriAI/litellm/issues/48630)) - Fixed guardrail wizard navigation buttons being hidden when topics are selected ([#48688](https://github.com/BerriAI/litellm/issues/48688)) - Fixed the Playground not showing reasoning for `/v1/responses` ([#48850](https://github.com/BerriAI/litellm/issues/48850)) - Fixed Simple auto router setup hiding Responses models that Advanced setup offers ([#48851](https://github.com/BerriAI/litellm/issues/48851)) - Fixed the Aria Agent avatar showing as a broken image ([#48890](https://github.com/BerriAI/litellm/issues/48890)) - Fixed budget-only team saves silently revoking an unrestricted team's direct model access ([#49258](https://github.com/BerriAI/litellm/issues/49258)) - Fixed Z.AI credential fields missing from the Add Model provider form ([#49314](https://github.com/BerriAI/litellm/issues/49314), [#49325](https://github.com/BerriAI/litellm/issues/49325)) - Fixed auto router templates not resolving when a `*` catch-all lists models over a partial wildcard ([#49331](https://github.com/BerriAI/litellm/issues/49331)) - Changed the Chat page to be on by default, and `enable_chat_ui: false` still hides it ([#46918](https://github.com/BerriAI/litellm/issues/46918)) - Changed the Playground chat to a single composer with a Settings popover and a reply inspector panel ([#46934](https://github.com/BerriAI/litellm/issues/46934)) - Changed the unset Budget Reset Period label from `n/a` to `Never resets` ([#47065](https://github.com/BerriAI/litellm/issues/47065)) - Changed budget dialogs to show spend limit and rate limits without a collapsed Optional Settings section ([#47662](https://github.com/BerriAI/litellm/issues/47662)) - Changed the upgrade notice to follow the changelog instead of the latest GitHub release ([#47998](https://github.com/BerriAI/litellm/issues/47998)) - Changed gateway and usage assistance into a single Aria Agent ([#48082](https://github.com/BerriAI/litellm/issues/48082)) - Changed the Aria empty model state to show in the model picker instead of a banner ([#48223](https://github.com/BerriAI/litellm/issues/48223)) - Changed auto router setup to start with two models and move classifier options under Advanced ([#48451](https://github.com/BerriAI/litellm/issues/48451)) - Changed keyboard hint chips across the Admin UI to one shared style ([#48885](https://github.com/BerriAI/litellm/issues/48885)) - Removed the separate hosted telemetry request sent on Chat submissions ([#48225](https://github.com/BerriAI/litellm/issues/48225)) - Removed the deprecated Old Usage page at `/ui/old-usage`, as `/ui/usage` shows everything it did ([#49284](https://github.com/BerriAI/litellm/issues/49284)) </details> <details><summary>Deployment (18)</summary> - Added a boot warning when config `environment_variables` replaces a key already set in the process environment ([#47165](https://github.com/BerriAI/litellm/issues/47165)) - Added `suppress_startup_output` to turn off startup prints and stop repeated Search Tools lists on reload ([#47708](https://github.com/BerriAI/litellm/issues/47708)) - Added a `litellm-setup` agent skill for self-hosting setup ([#47731](https://github.com/BerriAI/litellm/issues/47731)) - Added `LITELLM_DISABLE_LATEST_RELEASE_CHECK` to turn off the outbound latest release check ([#48065](https://github.com/BerriAI/litellm/issues/48065)) - Added `LITELLM_OFFLINE_MODE` to turn off every fetch LiteLLM makes on its own with one switch ([#48096](https://github.com/BerriAI/litellm/issues/48096)) - Added a pre-upgrade check for old spend-log rows that would fail the `(api_key, startTime)` index build ([#48493](https://github.com/BerriAI/litellm/issues/48493)) - Added `serviceName` and `servicePort` to Helm `ingress.extraPaths` to route paths to custom Services ([#48674](https://github.com/BerriAI/litellm/issues/48674)) - Fixed workers staying stale on configuration changes after a Redis reconnect ([#43313](https://github.com/BerriAI/litellm/issues/43313)) - Fixed `litellm --num_workers 2` crash-looping on slow hosts ([#45039](https://github.com/BerriAI/litellm/issues/45039)) - Fixed `--reload` ignoring `--num_workers`, now honored on uvicorn 0.50.0 or newer ([#45909](https://github.com/BerriAI/litellm/issues/45909)) - Fixed environment variables removed from the database staying active on running workers until restart ([#46709](https://github.com/BerriAI/litellm/issues/46709)) - Fixed rare lock hangs by requiring `anyio>=4.14.0` ([#47378](https://github.com/BerriAI/litellm/issues/47378)) - Fixed proxy startup being blocked by compressible `api_key` values that fit in the Postgres index ([#48911](https://github.com/BerriAI/litellm/issues/48911)) - Fixed tag spend and queued success callbacks being lost on proxy shutdown ([#48988](https://github.com/BerriAI/litellm/issues/48988)) - Changed the Terraform module to default its component images to the module release tag ([#46960](https://github.com/BerriAI/litellm/issues/46960)) - Changed `litellm-helm` to stop installing placeholder models into every install ([#47056](https://github.com/BerriAI/litellm/issues/47056)) - Changed the pre-migration check to a rolled-back dry run of pending migrations ([#48961](https://github.com/BerriAI/litellm/issues/48961)) - Removed the bundled Lens subchart from both Helm charts, which now connect to a separately deployed Lens ([#49169](https://github.com/BerriAI/litellm/issues/49169)) </details> <details><summary>SDK (8)</summary> - Added support for Python 3.15 ([#48359](https://github.com/BerriAI/litellm/issues/48359)) - Fixed tool calls on `completion()` and `responses()` failing with `No module named 'mcp'` on base SDK installs ([#45961](https://github.com/BerriAI/litellm/issues/45961)) Also in [1.108.4](https://github.com/BerriAI/litellm/releases/tag/v1.108.4) - Fixed `mock_response=""` sending a real provider request instead of returning an empty mock ([#48266](https://github.com/BerriAI/litellm/issues/48266)) - Fixed deprecated MCP helper imports requiring optional MCP dependencies ([#48615](https://github.com/BerriAI/litellm/issues/48615)) - Fixed `Unclosed client session` warnings from evicted aiohttp SDK clients ([#48650](https://github.com/BerriAI/litellm/issues/48650)) - Fixed `httpx.ReadError: Connection closed` on streams using the aiohttp transport's own session ([#48920](https://github.com/BerriAI/litellm/issues/48920)) - Changed `import litellm` to raise `ImportError` when the `_native` extension is missing ([#48852](https://github.com/BerriAI/litellm/issues/48852)) - Improved provider timeout errors to name the expired phase and limit instead of `after None seconds` ([#48250](https://github.com/BerriAI/litellm/issues/48250)) </details> <details><summary>Performance (12)</summary> - Fixed memory growth on proxies without Redis, which queued a budget spend increment per request ([#46831](https://github.com/BerriAI/litellm/issues/46831)) - Fixed the first `/openapi.json` request blocking a worker for about 6 seconds ([#47352](https://github.com/BerriAI/litellm/issues/47352)) - Fixed `LiteLLM_ToolTable` writes scaling with workers, and `call_count` now counts real tool calls ([#48089](https://github.com/BerriAI/litellm/issues/48089)) - Improved `lite` command startup time by no longer importing the full SDK ([#46803](https://github.com/BerriAI/litellm/issues/46803)) - Improved streaming so long buffered streams no longer stall other requests on the same worker ([#46925](https://github.com/BerriAI/litellm/issues/46925)) - Improved proxy responsiveness on very large OpenAI chat completions requests ([#46967](https://github.com/BerriAI/litellm/issues/46967)) - Improved streaming speed on OpenAI-compatible providers by cutting per-chunk overhead ([#47030](https://github.com/BerriAI/litellm/issues/47030)) - Improved proxy latency by freezing the startup heap so gen-2 GC passes no longer pause requests for 0.6 to 0.8 s ([#47137](https://github.com/BerriAI/litellm/issues/47137)) - Improved routing by resolving deployment usage limits once instead of on every request ([#47393](https://github.com/BerriAI/litellm/issues/47393)) - Improved savings rollup speed by reading deployment pricing from the router's cached model info ([#47400](https://github.com/BerriAI/litellm/issues/47400)) - Improved proxy CPU use on streamed OpenAI and Azure responses by removing caught exceptions on every chunk ([#49166](https://github.com/BerriAI/litellm/issues/49166)) - Improved request logging speed by skipping the base64 image scan on plain text ([#49228](https://github.com/BerriAI/litellm/issues/49228)) </details> <details><summary>Other (33)</summary> - Added `lite claude --subscription` to keep the claude.ai login while routing through the proxy ([#47385](https://github.com/BerriAI/litellm/issues/47385)) - Added `lite codex --subscription` to keep the ChatGPT login ([#48741](https://github.com/BerriAI/litellm/issues/48741)) - Added each model group's reasoning efforts to pi, OpenCode and Codex when launched through `lite` ([#48949](https://github.com/BerriAI/litellm/issues/48949)) - Fixed agentic web search follow-ups losing the provider prefix for Bedrock Mantle, OpenRouter and Bedrock models ([#38885](https://github.com/BerriAI/litellm/issues/38885)) - Fixed proxy errors returning the string `"None"` for `type` and `param` instead of `null` ([#39540](https://github.com/BerriAI/litellm/issues/39540)) - Fixed Claude Code web searches sending the instructional prefix to the search provider ([#43458](https://github.com/BerriAI/litellm/issues/43458)) - Fixed partial wildcards like `cloudflare/@cf/*` listing no models on `/v1/models` and `/model/info` ([#46390](https://github.com/BerriAI/litellm/issues/46390)) - Fixed `POST /model/update` dropping the `model_info` fields sent in the request ([#46902](https://github.com/BerriAI/litellm/issues/46902)) - Fixed 400 `Invalid JSON` on request bodies sent with `Content-Encoding: zstd` or `gzip` ([#47418](https://github.com/BerriAI/litellm/issues/47418)) - Fixed an `api_base` ending in `/` adding `//` to the provider path and causing 404 errors ([#47535](https://github.com/BerriAI/litellm/issues/47535)) - Fixed `lite pi` sending no session id, so sessions now group and `session_affinity` can pin them ([#47700](https://github.com/BerriAI/litellm/issues/47700)) - Fixed `lite login` tokens being treated as expired after a day when the proxy still accepts them ([#47712](https://github.com/BerriAI/litellm/issues/47712)) - Fixed WebSocket connections ignoring SSL verification, custom CA bundles, client certificates and proxies ([#47968](https://github.com/BerriAI/litellm/issues/47968)) - Fixed 404 on model discovery when `/claude_code_gateway` is the client base URL ([#48221](https://github.com/BerriAI/litellm/issues/48221)) - Fixed missing usage on native decisions calls being reported as zero tokens and $0 cost ([#48261](https://github.com/BerriAI/litellm/issues/48261)) - Fixed non-object JSON request bodies crashing parsing and auth errors, they now return 400 ([#48438](https://github.com/BerriAI/litellm/issues/48438)) - Fixed Decisions responses to keep missing probabilities absent while preserving explicit `0.0` ([#48524](https://github.com/BerriAI/litellm/issues/48524)) - Fixed `/config/field/update` storing boolean settings as text instead of JSON booleans ([#48595](https://github.com/BerriAI/litellm/issues/48595)) - Fixed concurrency slots not being released when synthetic streams complete ([#48628](https://github.com/BerriAI/litellm/issues/48628)) - Fixed client disconnects during body upload returning misleading errors, they now abort with 499 ([#48640](https://github.com/BerriAI/litellm/issues/48640)) - Fixed Decisions providers returning empty or non-JSON 200 bodies raising an unhandled validation error ([#48662](https://github.com/BerriAI/litellm/issues/48662)) - Fixed `lite codex` aliases of Codex-known models missing that model's catalog entry ([#48894](https://github.com/BerriAI/litellm/issues/48894)) - Fixed new model rows not applying on every worker as soon as they are written ([#49034](https://github.com/BerriAI/litellm/issues/49034)) - Fixed partial wildcard health checks probing models outside the wildcard ([#49060](https://github.com/BerriAI/litellm/issues/49060)) - Fixed model listing for wildcard aliases over a provider org showing models that cannot be routed ([#49152](https://github.com/BerriAI/litellm/issues/49152)) - Fixed Headroom compression and retrieval reaching different pods by forwarding tenant-scoped session affinity ([#49255](https://github.com/BerriAI/litellm/issues/49255)) - Fixed HTTP 500 on unrelated model edits when a model has saved access windows ([#49316](https://github.com/BerriAI/litellm/issues/49316)) - Changed shared-handler errors to report the right `type` instead of always `internal_server_error` ([#39555](https://github.com/BerriAI/litellm/issues/39555)) - Changed customer list routes to paginate, defaulting to 50 customers with page sizes from 1 to 100 ([#48700](https://github.com/BerriAI/litellm/issues/48700)) - Changed management and health route errors to stop reporting `type: auth_error` and to use fitting status codes ([#48739](https://github.com/BerriAI/litellm/issues/48739)) - Changed `litellm_settings.drop_params` to apply to Claude Code and Codex requests ([#49058](https://github.com/BerriAI/litellm/issues/49058)) - Changed `/v1/decisions` to honor a deployment's own `supports_audio_input` and `supports_video_input` flags ([#49207](https://github.com/BerriAI/litellm/issues/49207)) - Removed the System One question count cap in Decisions ([#47985](https://github.com/BerriAI/litellm/issues/47985)) </details> *** [Full changelog](https://github.com/BerriAI/litellm/compare/v1.108.0...v1.109.0) ### [`v1.108.8`](https://github.com/BerriAI/litellm/releases/tag/v1.108.8): 1.108.8 (LTS) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.108.7...v1.108.8) > \[!TIP] > The `lts` image tag always points to the latest LTS release. Pull `ghcr.io/berriai/litellm:lts` to stay on the newest LTS patch #### Verify this release ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \ ghcr.io/berriai/litellm:v1.108.8 ``` Same command for `litellm-database`, `litellm-non_root` and the `docker.io/litellm/*` images #### What's new in 1.108.8 (LTS) 4 PRs. 3 fixed. No action needed to upgrade ##### Changes - **Azure:** Fixed 400 on `/v1/decisions` for `azure/` deployments ([#51460](https://github.com/BerriAI/litellm/issues/51460)) - **Batches and files:** Fixed file ownership checks missing other spellings of a Gemini file id, such as `files/<id>` or its URL ([#51503](https://github.com/BerriAI/litellm/issues/51503)) - **MCP:** Fixed sign-in loop ending in `Authentication incomplete` on servers that answer `mcp_all` with detailed scopes, broken since v1.107.0 ([#51578](https://github.com/BerriAI/litellm/issues/51578)) *** Backport [#51640](https://github.com/BerriAI/litellm/issues/51640) · [Full changelog](https://github.com/BerriAI/litellm/compare/v1.108.7...v1.108.8) ### [`v1.108.7`](https://github.com/BerriAI/litellm/releases/tag/v1.108.7): 1.108.7 (LTS) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.108.6...v1.108.7) > \[!TIP] > The `lts` image tag always points to the latest LTS release. Pull `ghcr.io/berriai/litellm:lts` to stay on the newest LTS patch #### Verify this release ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \ ghcr.io/berriai/litellm:v1.108.7 ``` Same command for `litellm-database`, `litellm-non_root` and the `docker.io/litellm/*` images #### What's new in 1.108.7 (LTS) 14 PRs. 3 added, 8 fixed, 1 changed. 1 needs action ##### Action required - **Responses API:** Set `drop_params: true` or stop sending Responses-only fields such as `max_tool_calls` to providers bridged through chat completions ([#48655](https://github.com/BerriAI/litellm/issues/48655)) Also in [1.109.0](https://github.com/BerriAI/litellm/releases/tag/v1.109.0) ##### Changes - **Azure:** Added gpt-image edits on Azure AI Foundry deployments, including deployments under a custom name ([#49650](https://github.com/BerriAI/litellm/issues/49650)) - **AWS Bedrock:** Added image search on Bedrock knowledge bases with `query_image`, including image hits in the results ([#51062](https://github.com/BerriAI/litellm/issues/51062)) - **Vertex AI and Gemini:** Fixed Gemini image calls billing text, thinking and Vertex Priority tokens at the wrong rates ([#51017](https://github.com/BerriAI/litellm/issues/51017)) - **Vertex AI and Gemini:** Fixed `/gemini` `batchGenerateContent` batches logging $0 with model `unknown` instead of billing the calling key ([#51279](https://github.com/BerriAI/litellm/issues/51279)) - **Other providers:** Fixed `reasoning_effort` being dropped or rejected for Baseten models ([#40152](https://github.com/BerriAI/litellm/issues/40152)) Also in [1.109.0](https://github.com/BerriAI/litellm/releases/tag/v1.109.0) - **Other providers:** Fixed `baseten/*` listing retired deployment IDs instead of the current Baseten Model APIs ([#46945](https://github.com/BerriAI/litellm/issues/46945)) Also in [1.109.0](https://github.com/BerriAI/litellm/releases/tag/v1.109.0) - **Responses API:** Fixed missing or repeated `sequence_number` on bridged `/v1/responses` streaming events ([#44944](https://github.com/BerriAI/litellm/issues/44944)) - **Responses API:** Fixed streamed MCP follow-ups keeping only the latest tool round, so models repeated tool calls ([#47698](https://github.com/BerriAI/litellm/issues/47698)) Also in [1.109.0](https://github.com/BerriAI/litellm/releases/tag/v1.109.0) - **Responses API:** Fixed 400 on Bedrock when a `previous_response_id` chain contains a failed turn ([#48647](https://github.com/BerriAI/litellm/issues/48647)) Also in [1.109.0](https://github.com/BerriAI/litellm/releases/tag/v1.109.0) - **Responses API:** Changed Responses-only fields such as `max_tool_calls` to answer 400 on bridged providers unless `drop_params` is on ([#48655](https://github.com/BerriAI/litellm/issues/48655)) Also in [1.109.0](https://github.com/BerriAI/litellm/releases/tag/v1.109.0) - **Batches and files:** Added Gemini batches on `/v1/batches`, with output download through `/v1/files` and spend tracking ([#51032](https://github.com/BerriAI/litellm/issues/51032)) - **Routing and fallbacks:** Fixed Decisions calls on a mixed model group hitting a Bedrock deployment and answering 400 ([#48175](https://github.com/BerriAI/litellm/issues/48175)) Also in [1.109.0](https://github.com/BerriAI/litellm/releases/tag/v1.109.0) *** Backport [#51262](https://github.com/BerriAI/litellm/issues/51262) · Backport [#51266](https://github.com/BerriAI/litellm/issues/51266) · Backport [#51271](https://github.com/BerriAI/litellm/issues/51271) · Backport [#51340](https://github.com/BerriAI/litellm/issues/51340) · [Full changelog](https://github.com/BerriAI/litellm/compare/v1.108.6...v1.108.7) ### [`v1.108.6`](https://github.com/BerriAI/litellm/releases/tag/v1.108.6): 1.108.6 (LTS) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.108.5...v1.108.6) > \[!TIP] > The `lts` image tag always points to the latest LTS release. Pull `ghcr.io/berriai/litellm:lts` to stay on the newest LTS patch #### Verify this release ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \ ghcr.io/berriai/litellm:v1.108.6 ``` Same command for `litellm-database`, `litellm-non_root` and the `docker.io/litellm/*` images #### What's new in 1.108.6 (LTS) 16 PRs. 7 fixed, 2 changed. 1 needs action ##### Action required - **Keys, teams and auth:** Move `tenant_id`, `client_id`, `client_secret`, `azure_username`, `azure_password` and `azure_scope` request tags into `metadata` ([#51254](https://github.com/BerriAI/litellm/issues/51254)) Also in [1.106.9](https://github.com/BerriAI/litellm/releases/tag/v1.106.9) ##### Changes - **OpenAI:** Fixed gpt-5.4 and gpt-5.5 function-tool calls without `reasoning_effort` being bridged to `/v1/responses` ([#50968](https://github.com/BerriAI/litellm/issues/50968)) - **Vertex AI and Gemini:** Fixed Gemini image edits being billed at a flat per-image price instead of by tokens ([#51038](https://github.com/BerriAI/litellm/issues/51038)) - **Chat completions:** Fixed 400 on chat and bridged `/v1/responses` calls that send an `allowed_tools` tool choice ([#50726](https://github.com/BerriAI/litellm/issues/50726)) - **MCP:** Fixed Slack and GitHub MCP reconnects looping between the provider and the Connect page, broken since v1.107.0 ([#51045](https://github.com/BerriAI/litellm/issues/51045)) - **Guardrails:** Changed Zscaler AI Guard to scan only the latest user message on requests, not the whole conversation ([#50139](https://github.com/BerriAI/litellm/issues/50139)) - **Cost tracking and pricing:** Fixed 500 on usage routes when the database drops during key, user or team detail lookups ([#50253](https://github.com/BerriAI/litellm/issues/50253)) - **Keys, teams and auth:** Fixed key, user and team spend going negative after a scheduled budget reset on multi-pod deployments, broken since v1.103.0 ([#51120](https://github.com/BerriAI/litellm/issues/51120)) - **Keys, teams and auth:** Changed model and pass-through routes to refuse body `client_id`, `client_secret`, `tenant_id` and Azure login fields ([#51254](https://github.com/BerriAI/litellm/issues/51254)) Also in [1.106.9](https://github.com/BerriAI/litellm/releases/tag/v1.106.9) - **Admin UI:** Fixed the empty Spend Per User card and missing per-user data on `/user/daily/activity/aggregated` ([#51043](https://github.com/BerriAI/litellm/issues/51043)) *** Backport [#50435](https://github.com/BerriAI/litellm/issues/50435) · Backport [#51064](https://github.com/BerriAI/litellm/issues/51064) · Backport [#51122](https://github.com/BerriAI/litellm/issues/51122) · Backport [#51229](https://github.com/BerriAI/litellm/issues/51229) · Backport [#51240](https://github.com/BerriAI/litellm/issues/51240) · Backport [#51260](https://github.com/BerriAI/litellm/issues/51260) · [Full changelog](https://github.com/BerriAI/litellm/compare/v1.108.5...v1.108.6) ### [`v1.108.5`](https://github.com/BerriAI/litellm/releases/tag/v1.108.5): 1.108.5 (LTS) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.108.4...v1.108.5) > \[!TIP] > The `lts` image tag always points to the latest LTS release. Pull `ghcr.io/berriai/litellm:lts` to stay on the newest LTS patch #### Verify this release ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \ ghcr.io/berriai/litellm:v1.108.5 ``` Same command for `litellm-database`, `litellm-non_root` and the `docker.io/litellm/*` images #### What's new in 1.108.5 (LTS) 5 PRs. 4 fixed. No action needed to upgrade ##### Changes - **OpenAI:** Fixed keyless workload identity deployments answering 401 on image edits, audio, files, batches and other routes ([#50962](https://github.com/BerriAI/litellm/issues/50962)) - **Other providers:** Fixed `ollama_chat/` streams giving every parallel tool call index 0, so clients ran only the first call ([#50718](https://github.com/BerriAI/litellm/issues/50718)) - **Messages API:** Fixed `/v1/messages` answering 400 for a param listed in the deployment's `additional_drop_params` ([#50755](https://github.com/BerriAI/litellm/issues/50755)) - **Cost tracking and pricing:** Fixed incorrect transcription cost for some FLAC files, which could push a key over its budget ([#50860](https://github.com/BerriAI/litellm/issues/50860)) *** [Full changelog](https://github.com/BerriAI/litellm/compare/v1.108.4...v1.108.5) ### [`v1.108.4`](https://github.com/BerriAI/litellm/releases/tag/v1.108.4): 1.108.4 (LTS) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.108.3...v1.108.4) > \[!TIP] > The `lts` image tag always points to the latest LTS release. Pull `ghcr.io/berriai/litellm:lts` to stay on the newest LTS patch #### Verify this release ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \ ghcr.io/berriai/litellm:v1.108.4 ``` Same command for `litellm-database`, `litellm-non_root` and the `docker.io/litellm/*` images #### What's new in 1.108.4 (LTS) 10 PRs. 7 fixed, 1 changed. No action needed to upgrade ##### Changes - **AWS Bedrock:** Fixed 400s from GLM, GPT-6 Luna and Nova 2 on Bedrock when the conversation carries Claude's signed thinking ([#50399](https://github.com/BerriAI/litellm/issues/50399)) - **AWS Bedrock:** Changed `/bedrock` pass-through routes to enforce the model's `guardrailConfig` over any guardrail the caller sends ([#47429](https://github.com/BerriAI/litellm/issues/47429)) - **Vertex AI and Gemini:** Fixed Vertex AI Claude batches failing with 404 at create ([#50365](https://github.com/BerriAI/litellm/issues/50365)) - **Messages API:** Fixed `chatgpt/` models on `/v1/messages` dropping encrypted reasoning between turns ([#50456](https://github.com/BerriAI/litellm/issues/50456)) - **Messages API:** Fixed `/v1/messages` to OpenAI sending `background: true` upstream, which billed full runs that came back empty ([#50462](https://github.com/BerriAI/litellm/issues/50462)) - **Embeddings, images and audio:** Fixed `/v1/videos` polls failing with per-model keys or hitting the wrong provider account with a 404 ([#44614](https://github.com/BerriAI/litellm/issues/44614)) - **Routing and fallbacks:** Fixed team keys being served by a proxy-wide wildcard instead of the team's own BYOK wildcard deployment ([#48490](https://github.com/BerriAI/litellm/issues/48490)) - **SDK:** Fixed `completion()`, `responses()` and `messages` with tools failing with `No module named 'mcp'` on a base install, broken since v1.106.0 ([#45961](https://github.com/BerriAI/litellm/issues/45961)) *** Backport [#50568](https://github.com/BerriAI/litellm/issues/50568) · Backport [#50605](https://github.com/BerriAI/litellm/issues/50605) · Backport [#50693](https://github.com/BerriAI/litellm/issues/50693) · [Full changelog](https://github.com/BerriAI/litellm/compare/v1.108.3...v1.108.4) ### [`v1.108.3`](https://github.com/BerriAI/litellm/releases/tag/v1.108.3): 1.108.3 (LTS) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.108.2...v1.108.3) > \[!TIP] > The `lts` image tag always points to the latest LTS release. Pull `ghcr.io/berriai/litellm:lts` to stay on the newest LTS patch #### Verify this release ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \ ghcr.io/berriai/litellm:v1.108.3 ``` Same command for `litellm-database`, `litellm-non_root` and the `docker.io/litellm/*` images #### What's new in 1.108.3 (LTS) 11 PRs. 3 fixed. No action needed to upgrade ##### Changes - **AWS Bedrock:** Fixed every request failing on a Bedrock deployment with `api_base: ""`, which answered 500 or 400 ([#50449](https://github.com/BerriAI/litellm/issues/50449)) - **MCP:** Fixed tool results over 1 MiB failing with `SSE stream ended without a response`, broken since v1.103.0 ([#47034](https://github.com/BerriAI/litellm/issues/47034)) - **Guardrails:** Fixed model-level `post_call` guardrails checking each response twice and calling the moderation provider twice ([#50463](https://github.com/BerriAI/litellm/issues/50463)) *** Backport [#50484](https://github.com/BerriAI/litellm/issues/50484) · Backport [#50487](https://github.com/BerriAI/litellm/issues/50487) · [Full changelog](https://github.com/BerriAI/litellm/compare/v1.108.2...v1.108.3) </details> --- ### Configuration 📅 **Schedule**: (in timezone Europe/Rome) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).

renovate-bot added the

renovate/container

type/patch

labels

2026-11-08 00:08:12 +01:00

renovate-bot added 1 commit

2026-11-08 00:08:21 +01:00

fix(container): update image ghcr.io/berriai/litellm (v1.108.2 ➔ v1.108.3)

Konflate 1 resource changed

Details

d4191179b1

lab-assistant-bot commented

2026-11-08 00:24:32 +01:00

Collaborator

Routine: only container-image and chart-version changes; 1 resource across 1 app

image	from	to	upstream
`ghcr.io/berriai/litellm`	`v1.108.2@sha256:d8b4ac…`	`v1.109.1@sha256:111759…`	❔

_{konflate 0.6.6 · rendered 57cddb7 · full diff →}

**Routine**: only container-image and chart-version changes; 1 resource across 1 app | image | from | to | upstream | |---|---|---|---| | `ghcr.io/berriai/litellm` | `v1.108.2@sha256:d8b4ac…` | `v1.109.1@sha256:111759…` | ❔ | konflate 0.6.6 · rendered `57cddb7` · [full diff →](https://konflate.aresu.eu/#/pr/2608)

renovate-bot force-pushed renovate/ghcr.io-berriai-litellm-1.x from d4191179b1

Konflate 1 resource changed

Details

to 958fbf2df4

Labeler / Labeler (pull_request) Successful in 3s

Details

Lint / actionlint (pull_request) Successful in 19s

Details

Lint / yamllint (pull_request) Successful in 20s

Details

AI PR Review / AI PR Review (pull_request) Successful in 3m9s

Details

Konflate 1 resource changed

Details

2026-11-10 00:01:37 +01:00

Compare

forgejo-actions added the

area/kubernetes

label

2026-11-10 00:01:42 +01:00

renovate-bot changed title from ~~fix(container): update image ghcr.io/berriai/litellm (v1.108.2 ➔ v1.108.3)~~ to fix(container): update image ghcr.io/berriai/litellm (v1.108.2 ➔ v1.108.4)

2026-11-10 00:02:06 +01:00

forgejo-actions commented

2026-11-10 00:04:47 +01:00

AI Automated Review

Analysis engine: (model unavailable) · pr-reviewer-action v3.8.0

AI review could not run

The configured model endpoint(s) did not return a usable review for this run (reason: Fallback model failed).

This is an automated notice, not a substantive review. Re-run the workflow once the endpoint is reachable; see the action logs for the underlying error.

# AI Automated Review _Analysis engine: (model unavailable) · pr-reviewer-action v3.8.0_ ## AI review could not run The configured model endpoint(s) did not return a usable review for this run (reason: Fallback model failed). This is an automated notice, not a substantive review. Re-run the workflow once the endpoint is reachable; see the action logs for the underlying error.

renovate-bot force-pushed renovate/ghcr.io-berriai-litellm-1.x from 958fbf2df4

Labeler / Labeler (pull_request) Successful in 3s

Details

Lint / actionlint (pull_request) Successful in 19s

Details

Lint / yamllint (pull_request) Successful in 20s

Details

AI PR Review / AI PR Review (pull_request) Successful in 3m9s

Details

Konflate 1 resource changed

Details

to 85daf6a05b

Konflate 1 resource changed

Details

Labeler / Labeler (pull_request) Successful in 5s

Details

Lint / actionlint (pull_request) Successful in 27s

Details

Lint / yamllint (pull_request) Successful in 28s

Details

AI PR Review / AI PR Review (pull_request) Failing after 4m40s

Details

2026-11-11 01:02:07 +01:00

Compare

renovate-bot changed title from ~~fix(container): update image ghcr.io/berriai/litellm (v1.108.2 ➔ v1.108.4)~~ to fix(container): update image ghcr.io/berriai/litellm (v1.108.2 ➔ v1.108.5)

2026-11-11 01:02:36 +01:00

renovate-bot force-pushed renovate/ghcr.io-berriai-litellm-1.x from 85daf6a05b

Konflate 1 resource changed

Details

Labeler / Labeler (pull_request) Successful in 5s

Details

Lint / actionlint (pull_request) Successful in 27s

Details

Lint / yamllint (pull_request) Successful in 28s

Details

AI PR Review / AI PR Review (pull_request) Failing after 4m40s

Details

to 4d73225016

Labeler / Labeler (pull_request) Successful in 7s

Details

Lint / actionlint (pull_request) Successful in 37s

Details

Lint / yamllint (pull_request) Successful in 38s

Details

AI PR Review / AI PR Review (pull_request) Successful in 3m15s

Details

Konflate 1 resource changed

Details

2026-11-11 04:08:08 +01:00

Compare

renovate-bot changed title from ~~fix(container): update image ghcr.io/berriai/litellm (v1.108.2 ➔ v1.108.5)~~ to feat(container): update image ghcr.io/berriai/litellm (v1.108.2 ➔ v1.109.0)

2026-11-11 04:08:34 +01:00

renovate-bot force-pushed renovate/ghcr.io-berriai-litellm-1.x from 4d73225016

Labeler / Labeler (pull_request) Successful in 7s

Details

Lint / actionlint (pull_request) Successful in 37s

Details

Lint / yamllint (pull_request) Successful in 38s

Details

AI PR Review / AI PR Review (pull_request) Successful in 3m15s

Details

Konflate 1 resource changed

Details

to 57cddb7862

Labeler / Labeler (pull_request) Successful in 9s

Details

Lint / actionlint (pull_request) Successful in 57s

Details

Lint / yamllint (pull_request) Successful in 58s

Details

AI PR Review / AI PR Review (pull_request) Successful in 3m35s

Details

Konflate 1 resource changed

Details

2026-11-12 04:05:37 +01:00

Compare

renovate-bot changed title from ~~feat(container): update image ghcr.io/berriai/litellm (v1.108.2 ➔ v1.109.0)~~ to feat(container): update image ghcr.io/berriai/litellm (v1.108.2 ➔ v1.109.1)

2026-11-12 04:06:05 +01:00

Labeler / Labeler (pull_request) Successful in 9s

Details

Lint / actionlint (pull_request) Successful in 57s

Details

Lint / yamllint (pull_request) Successful in 58s

Details

AI PR Review / AI PR Review (pull_request) Successful in 3m35s

Details

Konflate 1 resource changed

Details

This pull request can be merged automatically.

This branch is out-of-date with the base branch

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin "renovate/ghcr.io-berriai-litellm-1.x:renovate/ghcr.io-berriai-litellm-1.x"

git switch "renovate/ghcr.io-berriai-litellm-1.x"

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch "main"

git merge --no-ff "renovate/ghcr.io-berriai-litellm-1.x"

git switch "renovate/ghcr.io-berriai-litellm-1.x"

git rebase "main"

git switch "main"

git merge --ff-only "renovate/ghcr.io-berriai-litellm-1.x"

git switch "renovate/ghcr.io-berriai-litellm-1.x"

git rebase "main"

git switch "main"

git merge --no-ff "renovate/ghcr.io-berriai-litellm-1.x"

git switch "main"

git merge --squash "renovate/ghcr.io-berriai-litellm-1.x"

git switch "main"

git merge --ff-only "renovate/ghcr.io-berriai-litellm-1.x"

git switch "main"

git merge "renovate/ghcr.io-berriai-litellm-1.x"

git push origin "main"

No reviewers

No milestone

No project

No assignees

3 participants

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

GiorgioAresu/home-ops!2608

No description provided.

Rows
Columns

feat(container): update image ghcr.io/berriai/litellm (v1.108.2 ➔ v1.109.1) #2608

Release Notes

v1.109.1

Verify Docker Image Signature

What's Changed

v1.109.0

Verify this release

What's new in 1.109.0

Action required

Highlights

All changes

v1.108.8: 1.108.8 (LTS)

Verify this release

What's new in 1.108.8 (LTS)

Changes

v1.108.7: 1.108.7 (LTS)

Verify this release

What's new in 1.108.7 (LTS)

Action required

Changes

v1.108.6: 1.108.6 (LTS)

Verify this release

What's new in 1.108.6 (LTS)

Action required

Changes

v1.108.5: 1.108.5 (LTS)

Verify this release

What's new in 1.108.5 (LTS)

Changes

v1.108.4: 1.108.4 (LTS)

Verify this release

What's new in 1.108.4 (LTS)

Changes

v1.108.3: 1.108.3 (LTS)

Verify this release

What's new in 1.108.3 (LTS)

Changes

Configuration

AI Automated Review

AI review could not run

Checkout

Merge

`v1.109.1`

`v1.109.0`

`v1.108.8`: 1.108.8 (LTS)

`v1.108.7`: 1.108.7 (LTS)

`v1.108.6`: 1.108.6 (LTS)

`v1.108.5`: 1.108.5 (LTS)

`v1.108.4`: 1.108.4 (LTS)

`v1.108.3`: 1.108.3 (LTS)