LiteLLM is an open-source large language model (LLM) gateway developed by BerriAI. It provides a unified OpenAI-compatible application programming interface (API) for accessing and routing requests to different model providers. LiteLLM can be deployed as a self-hosted proxy server or used as a Python library within applications.
Functionality LiteLLM provides a common interface for models and services from providers including OpenAI, Anthropic, Google Gemini, Amazon Bedrock, Microsoft Azure, and Hugging Face. It translates requests and responses between provider APIs into a consistent OpenAI-compatible format, allowing applications to switch between models without separately implementing each provider's interface.
The software also provides model routing, load balancing, fallbacks, authentication, rate limiting, usage monitoring, and cost controls. Its proxy server can act as a centralized API gateway through which multiple applications or users access AI models.
LiteLLM also supports the Model Context Protocol (MCP), an open protocol developed by Anthropic for connecting AI applications to external tools and data sources.
Security In March 2026, malicious versions of LiteLLM were distributed through the Python Package Index (PyPI) as part of a software supply-chain attack. TechCrunch reported that AI recruiting company Mercor was among the organizations affected by the compromise.
Compatibility Supported models LiteLLM supports more than 100 large language models through a common OpenAI-compatible interface. Notable supported model families include:
Claude DeepSeek Gemini Gemma GPT-4 GPT-4o GPT-5 Grok Llama Mistral Nemotron OpenAI o1 OpenAI o3 Phi Qwen
Compatible AI coding software LiteLLM can be used as an inference gateway for AI-assisted software development tools, routing their model requests through the LiteLLM proxy. Documented integrations include:
Claude Code Cline Codex CLI OpenCode Pi
See also List of AI-assisted software development tools OpenRouter – proprietary LLM gateway
References
External links
LiteLLM on GitHub