Berri
FreemiumOpenAI-compatible gateway for managing access to 100+ large language models
Informações da ferramenta
O que é Berri?
LiteLLM is an AI gateway that provides centralized access to multiple large language models through an OpenAI-compatible interface. It supports model fallbacks, load balancing, authentication, observability, budgets, rate limits, and spend tracking.
Recursos principais
OpenAI-compatible gateway
Access more than 100 large language models through the OpenAI format.
Model fallbacks and load balancing
Use provider fallbacks, retries, cooldowns, and load balancing across model deployments.
Spend tracking
Track usage costs by key, user, team, organization, model, and custom tags, with logging to supported storage services.
Budgets and rate limits
Set budgets and RPM or TPM limits by key, user, and team.
Authentication and access control
Manage virtual keys, custom authentication, OIDC/JWT authentication, SSO, model access restrictions, and key rotation.
LLM observability and logging
Connect LLM observability and logging with tools and services including Datadog, S3, GCS, Azure Data Lake, OpenTelemetry, and Prometheus metrics.
Guardrails
Apply request-level, default-on, key-based, and team-based guardrails.
User, team, and organization management
Manage users, teams, organizations, and administrators through CRUD endpoints and UI capabilities.
Pass-through endpoints
Pass requests to supported provider endpoints and return responses without translation for supported migration scenarios.
Casos de uso
- #1
Centralized LLM access
Give developers access to multiple LLM providers through a common OpenAI-compatible interface.
- #2
Provider outage handling
Maintain model availability by falling back across providers and deployments during outages or rate-limit errors.
- #3
AI usage governance
Manage authentication, model access, budgets, rate limits, and spend attribution across users, teams, and organizations.
- #4
Enterprise and self-hosted deployment
Deploy the open-source gateway in self-hosted, on-premises, or air-gapped environments.
Planos de preços
The official source lists self-hosted and air-gapped availability.
Alternativas
Perguntas frequentes
What is LiteLLM used for?
How many language models does LiteLLM support?
Does LiteLLM support model fallbacks?
Does LiteLLM offer spend tracking and budgets?
Can LiteLLM be self-hosted?
What is the price of LiteLLM?
Berri Contato e mídias sociais
- EmpresaLiteLLM
- Suporte
- Site:
- Twitter
- LinkedIn
- GitHub
Comentários
Compartilhe sua experiência com esta ferramenta...