Skip to main content

v1.75.8-stable - Team Member Rate Limits

Krrish Dholakia
Krrish Dholakia
CEO, LiteLLM
Ishaan Jaffer
Ishaan Jaffer
CTO, LiteLLM

Deploy this version​

docker run litellm
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:v1.75.8-stable

Key Highlights​

  • Team Member Rate Limits - Individual rate limiting for team members with JWT authentication support.
  • Performance Improvements - New experimental HTTP handler flag for 100+ RPS improvement on OpenAI calls.
  • GPT-5 Model Family Support - Full support for OpenAI's GPT-5 models with reasoning_effort parameter and Azure OpenAI integration.
  • Azure AI Flux Image Generation - Support for Azure AI's Flux image generation models.

Team Member Rate Limits​

LiteLLM MCP Architecture: Use MCP tools with all LiteLLM supported models

This release adds support for setting rate limits on individual members (including machine users) within a team. Teams can now give each agent its own rate limits—so that heavy-traffic agents don’t impact other agents or human users.

Agents can authenticate with LiteLLM using JWT and the same team role as human users, while still enforcing per-agent rate limits.

New Models / Updated Models​

New Model Support​

ProviderModelContext WindowInput ($/1M tokens)Output ($/1M tokens)Features
Azure AIazure_ai/FLUX-1.1-pro--$40/imageImage generation
Azure AIazure_ai/FLUX.1-Kontext-pro--$40/imageImage generation
Vertex AIvertex_ai/deepseek-ai/deepseek-r1-0528-maas65k$1.35$5.4Chat completions + reasoning
OpenRouteropenrouter/deepseek/deepseek-chat-v3-032465k$0.14$0.28Chat completions

Features​

Bugs​

LLM API Endpoints​

Features​

Bugs​

MCP Gateway​

Features​

  • Access Control & Configuration
    • Enhanced MCPServerManager with access groups and description support - PR #13549

Bugs​

  • Authentication
    • Fixed MCP gateway key authentication - PR #13630

Read More

Management Endpoints / UI​

Features​

  • Team Management
    • Team Member Rate Limits implementation - PR #13601
    • JWT authentication support for team member rate limits - PR #13601
    • Show team member TPM/RPM limits in UI - PR #13662
    • Allow editing team member RPM/TPM limits - PR #13669
    • Allow unsetting TPM and RPM in Teams Settings - PR #13430
    • Team Member Permissions Page access column changes - PR #13145
  • Key Management
    • Display errors from backend on the UI Keys page - PR #13435
    • Added confirmation modal before deleting keys - PR #13655
    • Support for user parameter in LiteLLM SDK to Proxy communication - PR #13555
  • UI Improvements
    • Fixed internal users table overflow - PR #12736
    • Enhanced chart readability with short-form notation for large numbers - PR #12370
    • Fixed image overflow in LiteLLM model display - PR #13639
    • Removed ambiguous network response errors - PR #13582
  • Credentials
    • Added CredentialDeleteModal component and integration with CredentialsPanel - PR #13550
  • Admin & Permissions

Bugs​

  • SCIM Integration
    • Fixed SCIM Team Memberships metadata handling - PR #13553
  • Authentication
    • Fixed incorrect key info endpoint - PR #13633

Logging / Guardrail Integrations​

Features​

Bugs​

  • Security
    • Hide sensitive data in /model/info - azure entra client_secret - PR #13577
    • Fixed trivy/secrets false positives - PR #13631

Performance / Loadbalancing / Reliability improvements​

Features​

  • HTTP Performance
    • New 'EXPERIMENTAL_OPENAI_BASE_LLM_HTTP_HANDLER' flag for +100 RPS improvement on OpenAI calls - PR #13625
  • Database Monitoring
  • Error Handling
    • Added safe divide by 0 protection to prevent crashes - PR #13624

Bugs​

  • Dependencies
    • Updated boto3 to 1.36.0 and aioboto3 to 13.4.0 - PR #13665

General Proxy Improvements​

Features​

  • Database
    • Removed redundant use_prisma_migrate flag - now default - PR #13555
  • LLM Translation
    • Added model ID check - PR #13507
    • Refactored Anthropic configurations and added support for anthropic_beta headers - PR #13590

New Contributors​

  • @TensorNull made their first contribution in PR #13458
  • @MajorD00m made their first contribution in PR #13577
  • @VerunicaM made their first contribution in PR #13584
  • @huangyafei made their first contribution in PR #13607
  • @TomeHirata made their first contribution in PR #13561
  • @willfinnigan made their first contribution in PR #13659
  • @dcbark01 made their first contribution in PR #13633
  • @javacruft made their first contribution in PR #13631

Full Changelog​