LiteLLM

Headlines · 7

1. 3d
   Retail LLM platform cut agent hallucinations from 15% to 1.5% in six months

   An InfoQ China engineering playbook describes a large retailer's LLM-based inventory recommendation system, where about 15% of agent responses were hallucinations at the end of the first production month; six months later the rate had fallen to 1.5%. The authors say the drop came not from a stronger base model but from treating LLM capabilities as platform infrastructure: shared retry and failure classification, a prompt registry with versioning and rollback, schema enforcement at the platform boundary, per-request token cost attribution, behavioral observability and tool routing. The platform now serves dozens of application teams across inventory audit, replenishment review and discrepancy triage, and is built on Google's ADK orchestrator, LiteLLM for routing to models such as Gemini and GPT, and MCP-based tool services, with a companion codebase that runs locally on llama3.1 via Ollama to avoid API costs.

   InfoQ 中文 AI&大模型 · 🔥 0
2. 3d
   Security roundup: PoeLLM malware hits 3,400+ servers running LiteLLM and Ollama

   iThome Taiwan's weekly security roundup for October 12–16 reports that the PoeLLM malware is targeting publicly exposed AI and open-source services including LiteLLM and Ollama, with more than 3,400 servers compromised and used for mining as well as further scanning and attacks. Citing Lumen's Black Lotus Labs, the report says samples contained a test endpoint linked to the LiteLLM command-injection flaw CVE-2026-42271, a possible infection route, and that the attackers hid command-and-control addresses in an English poem on GitHub that can be edited to switch victims to a new server. The same roundup notes Taiwan's Financial Supervisory Commission issued a consultation document on agentic AI security for the financial sector covering governance, risk tiering and controls, and that Singapore's CSA warned the Carbonato botnet deploys AI agents on compromised Docker hosts to steal credentials.

   iThome 台湾 · 🔥 0
3. 3d
   Claude Fable 5.5 leaks in Azure catalog ahead of an expected launch

   36Kr and TestingCatalog report that Claude Fable 5.5 appeared in Microsoft Azure's East US 2 model catalog, with a LiteLLM entry listing a 1M-token context window, up to 128K output tokens and the same $10/$50 per-million-token pricing as Fable 5.1. TestingCatalog cautions that the entry was removed a day later when Azure queries came back empty and that the values may have been carried over from Fable 5.1. A leaker said partners had been given access to the model and expected a release next week, while a "claude-onigiri-eap" codename briefly surfaced in Artificial Analysis's frontend; Anthropic has confirmed nothing.

   36氪 人工智能 · 🔥 3
4. 5d
   LiteLLM gateway hit by two flaws; upgrade to 1.83.9 urged

   Check Point Research found two vulnerabilities in the open source AI gateway LiteLLM. The first, an access-control bug tracked as CVE-2026-35029 with a severity score of 8.7/10, let low-privileged users reach admin functions controlling the gateway's core configuration, allowing them to read server files, overwrite admin credentials and download and run malware; the second, a server-side request forgery bug tracked as CVE-2026-59823 at 5.3/10, could make the gateway send API requests carrying AI provider keys to third-party servers, handing credentials to attackers. Both are fixed in LiteLLM 1.83.9, and all earlier versions are exposed, so the researchers advise upgrading, rotating potentially exposed keys and verifying that limited roles are actually limited. The report says LiteLLM supports some 140 AI providers and has more than 90 million monthly GitHub downloads and over 61,000 stars, and is often embedded indirectly in other platforms and internal tools.

   TechRadar · 🔥 0
5. Oct 11
   Decepticon open-sources autonomous red-team agent, claims 98% on XBOW benchmark

   Decepticon is an open-source autonomous red-team agent from PurpleAILAB, published on GitHub under Apache-2.0 and available either self-hosted via Docker or through the project's cloud demo. Its documentation says it coordinates 16 specialist agents by kill-chain phase — reconnaissance, exploitation, privilege escalation, lateral movement and C2 — running commands in a Kali Linux sandbox on a separate operational network, and it can be registered as an MCP server for Claude Code or Codex. The README self-reports a 102/104 (98.08%) pass rate on the XBOW validation-benchmarks and lists support for Anthropic, OpenAI, Google Gemini, MiniMax, DeepSeek, xAI, Mistral, OpenRouter, Nvidia NIM and local Ollama models, plus subscription logins such as Claude Max and ChatGPT Pro.

   GitHub Trending · Python · 🔥 0
6. Oct 10
   LiteLLM open-source AI gateway trends on GitHub

   GitHub Trending's daily list features BerriAI's LiteLLM, an open-source AI gateway that offers a single interface for calling 100+ LLM providers — including OpenAI, Anthropic, Gemini, Bedrock, Azure, Vertex AI, vLLM and Nvidia NIM — in OpenAI format. It runs either as a Python SDK or as a self-hosted proxy server with virtual keys, spend tracking, guardrails, load balancing and logging; the repo cites an 8ms P95 latency benchmark at 1k RPS and lists Netflix as an open-source adopter. The README also covers an MCP gateway, A2A agent invocation, and running harnesses such as Claude Code, Codex and OpenCode on any model.

   GitHub Trending（每日） · 🔥 0
7. Oct 9
   PoeLLM cryptomining malware hid C2 addresses in a GitHub poem, hitting 3,400+ servers

   Lumen's Black Lotus Labs detailed a campaign it calls Canto Incognito, in which cryptomining malware named PoeLLM has infected more than 3,400 servers since at least April. Many victims appear to be internet-exposed open-source AI/LLM services such as LiteLLM and Ollama, along with the PDF-conversion API Gotenberg and self-hosted Git platform Gitea; the enterprise gateway Ivanti Sentry may also have been targeted. The malware decoded command-and-control addresses from four words in a two-stanza GitHub poem titled "On the Nature of Connection," revised 11 times so far, and dropped XMRig-based miners that turn infected hosts into scanners and exploit servers. Researchers stressed the poem is only C2 obfuscation, not prompt injection or a jailbreak style of "adversarial poetry."

   iThome 台湾 · 🔥 0

Experience and discussion from the community

Share my LiteLLM experienceAsk about LiteLLM