Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

LMDeploy

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 1

  1. Olla v0.1.0 adds TEI support and reliability improvements

    The Olla project released v0.1.0, its 25th release. The update adds Hugging Face Text Embeddings Inference support for embeddings, reranking and classification; configurable retry and failover behavior; a reworked dashboard; improved model-capability detection; and fixes for concurrency, request statistics, rate limiting, routing, backend discovery and configuration validation. Olla is a Go-based open-source proxy and load balancer that sits between applications and local inference servers. It supports multiple runtimes, including Ollama, vLLM, SGLang, llama.cpp and LM Studio, and provides OpenAI- and Anthropic-compatible interfaces with translation where required.

    r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my LMDeploy experienceAsk about LMDeploy

Nobody has shared their experience with LMDeploy yet.