Forge 0.2.1为M6增加192K上下文Qwen3.8-27B支持
据开发者报告，ANEMLL Forge 0.2.1 可在配备 32GB 内存的 M6 Mac mini 神经网络引擎上运行 Qwen3.8-27B，上下文窗口最高达 192K token。该版本加入 INT8 K/V 缓存，并让较小的 drafter 共享目标模型的输出头；开发者称可在速度不变的情况下节省 0.86GB ANE 内存。其报告的解码速度随内容在每秒 10–62 token 之间，96K 配置总内存约 21.4GB，首次完整编译 192K 配置约需两小时。
ANEMLL Forge 0.2.1: Qwen3.8-27B on the M6 Neural Engine now runs up to 192K context (M6 32GB).
New: INT8 K/V cache (half of FP16) and a smaller drafter sharing the target's output head — 0.86 GB less ANE memory, same speed.
Decode depends on content: JSON 35–62 tok/s, code 18–32, prose 10–17. Prefill 260 tok/s at short context, 62 at 192K. 128K–192K use a smaller prefill batch (32 vs 64 rows) to stay within memory budget.
Heads up: full 192K first compile takes ~2h on M6 (one time, cached). `--ctx` exports only the entries you need, so compile is much faster: 16K is ~10 min from download to running server. IMO 96K is the sweet spot for size/speed on 32GB: 123 tok/s prefill, ~27 tok/s decode, ~21.4 GB total memory.
Release notes: https://github.com/Anemll/anemll-forge/blob/main/docs/RELEASE\_NOTES\_v0.2.1.md
Video (Tetris generated over ~150K tokens, no compaction): https://x.com/anemll/status/2111941341819408866
Paper: Three Bit MLP Vector Quantization in Apple Neural Engine — https://github.com/Anemll/anemll-forge/blob/main/docs/papers/anemll-forge-v02.pdf
Cheers, Anemll 🤖
#ANEMLL Forge 0.2.1#Qwen3.8-27B#Apple Neural Engine#M6 Mac mini