Strata 0.2.10 targets long-context inference on 8GB GPUs
Open-source local AI tool Strata 0.2.10 adds memory-aware allocation that can reduce the GPU-resident KV cache and offload some cache data to system memory or disk. The developers say the release can launch Qwen3.8IQ3_S with a 128K context on an 8GB device and make DFlash2 the default speculative decoder for single-request workloads. The update also introduces the All-cards Split Engine for coordinated multi-GPU processing, the ISTA-DAS Coder programming model, and fixes for agent and interface functions. Developer tests reported 63–73 tokens per second under an 8GB memory limit and 12.7–14.8 tokens per second on an Intel Arc A750 8GB with a 128K context; the source also notes that offloading shifts pressure to system memory and recommends at least 32GB of RAM.
The update directly targets the memory constraints involved in running long-context local models by combining KV-cache offloading, a lower-memory speculative decoder and coordinated multi-GPU computation.
开源本地 AI 工具Strata近日正式迎来了0.2.10版本更新。作为一款专为消费级显卡设计的本地大模型运行神器,本次更新在降低显存门槛、提升推理速度以及优化多显卡协同方面带来了突破性进展,让普通电脑运行千亿参数大模型变得更加轻松。
显存大幅瘦身:8GB 显卡如何玩转128K 超长上下文?
在以往的本地运行场景中,如果用户尝试在8GB 显存的设备上开启64K 甚至更高的上下文,往往会因为专家缓存空间耗尽而直接导致启动失败,甚至不得不妥协降至48K 左右。
而在 Strata0.2.10中,官方优化了底层资源分配机制:当显存不足时,系统会优先动态调整显存与内存之间的数据分配,尽最大可能保留用户设置的上下文长度。这意味着,用户现在完全可以在8GB 显存的设备上成功启动 Qwen3.8IQ3_S 等大模型,并维持高达128K(即131,072Token)的超长上下文。
其核心原理在于,Strata 会根据实时显存余量自动缩减驻留在 GPU 中的 KV 缓存窗口,必要时还会将部分缓存平摊至系统内存或硬盘。官方测试数据显示,在8GB 显存限制下开启 DFlash2模式时,生成速度可达63至73Token/s;即便是使用英特尔 Arc A7508GB 真实显卡,也能顺利启动128K 上下文并保持12.7至14.8Token/s 的生成速度。
#Strata#DFlash2#MTP#All-cards Split Engine#ISTA-DAS Coder#DeepSeek-V4.1-Flash#Qwen3.8IQ3_S#Claude Code#Intel Arc A750#NVIDIA RTX A6000