Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

exl3

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 4

  1. DeepSeek-V4.1-Flash benchmarked on dual DGX Spark hardware

    Nick Mermiges of PLI Labs benchmarked DeepSeek-V4.1-Flash-1017 at 2.75 bpw with EXL3 and vLLM across two NVIDIA DGX Spark systems. An EPYC system with two RTX 3090s handled Engram tables and speculative drafting. In the author's tests, reported as medians of three runs, prefill reached 2,618 tokens per second with a 256K prompt, while structured JSON output reached 322 tokens per second across four streams. The report published the code, launch scripts, benchmark tool and raw results, while noting lower prose throughput and slower decoding than some cited deployments. Planned 1M-token context, llama-benchy, BFCL and head-to-head KLD tests were not yet available, and the prefix-cache code was still pending publication.

    r/LocalLLaMA · 🔥 3
  2. yeasah releases 6bpw EXL3 quantization of Qwen3.8-27B

    yeasah released a 6bpw EXL3 quantization of Qwen3.8-27B, according to a Reddit post and the linked model listing. The reported comparison gives it a mean KLD of 0.0002, a 23 GB file size, and 18.8 GiB of VRAM for weights, making it the lowest-KLD result listed in that table.

    r/LocalLLaMA · 🔥 0
  3. Reddit analysis compares Gyro and EXL3 quantization

    A Reddit author’s code analysis says Gyro-S and Gyro-M belong to the same QTIP/Trellis-coded quantization family as EXL3, but use 15-bit states, 128-weight block encoding, and an explicit lookup-table codebook. The author found no published evidence that Gyro outperforms EXL3 in decode speed, perplexity, KLD, or top-1% performance. Direct interoperability between the formats has not been demonstrated, and Gyro’s quantization recipe is currently closed source.

    r/LocalLLaMA · 🔥 0
  4. Reddit user: 1.66bpw exl3 quantization of Qwen3.8-Flash-Next runs under 36GB on one RTX 5090

    A poster on r/LocalLLaMA (u/Unstable_Llama) reports that a 1.66 bit-per-weight exl3 quantization of Qwen3.8-Flash-Next fits entirely in a 36GB footprint on a single RTX 5090 with no offloaded experts. The poster describes inference as stable with no reasoning loops at roughly 200–300 tokens per second, but notes heavy compaction is needed because a 32GB GPU leaves little room for context even with cache quantization. The weights are said to be hosted on Hugging Face under turboderp/Qwen3.8-Flash-Next-exl3; the performance claims are the user's own and have not been independently verified.

    r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my exl3 experienceAsk about exl3

Nobody has shared their experience with exl3 yet.