ISTA DASLab
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Community post reports 262K-context Qwen-Flash-Next Coder on 8 GB VRAM
A Reddit community post reports running a heavily quantized Qwen-Flash-Next Coder through llama.cpp with 262K context, 8 GB of VRAM and 24 GB of RAM. The setup uses a 3.5-bit-per-weight Coder variant with selective mixture-of-experts offloading, keeping most experts in RAM, smaller experts in VRAM and n-gram data on the fastest SSD. The post reports 70 tokens per second for prompt processing and about 8 tokens per second for generation, with generation limited by PCIe. It also notes that the first run may be slower because of memory mapping.
r/LocalLLaMA · 🔥 0
Experience and discussion from the community
Share my ISTA DASLab experienceAsk about ISTA DASLab
Nobody has shared their experience with ISTA DASLab yet.