Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

TabbyAPI

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 2

  1. Qwen 3.6 MoE offloading hits 2,606 t/s prefill on RTX 9070 XT

    Reddit user /u/willeyh reported running Qwen 3.6 35B-A3B in Q4 on an RTX 9070 XT 16GB, with offloaded experts placed in DDR4-3200 RAM. Increasing the chunk size to 5120 reportedly pushed prefill above 2,000 t/s at all tested context lengths and peaked at 2,606 t/s at 32K, while larger chunks produced no further gains. ExLlamaV3's cpu_threads 6 setting slightly improved decoding, but several other flags regressed and cpu_moe_pinned_arena caused core dumps.

    r/LocalLLaMA · 🔥 0
  2. Local LLM inference tools multiply as users weigh speed against quality

    A Reddit user says the local LLM inference ecosystem has expanded rapidly since March 2025, from established tools such as llama.cpp, Ollama, vLLM and TabbyAPI to numerous newer projects. The author reports that llama.cpp continues to improve quickly, while newer engines may prioritize speed or lower resource use at the expense of output quality; their specific model and hardware claims are not independently verified here.

    r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my TabbyAPI experienceAsk about TabbyAPI

Nobody has shared their experience with TabbyAPI yet.