Nemotron 3.5 Bolt 实测：长上下文省显存
AI 总结
r/LocalLLaMA 用户 /u/1ncehost 发帖称，Nemotron 3.5 Bolt 在其 128 GB 显存配置（4 张 MI100）上可用于小规模多用户场景。该用户称，Bolt 的 decoder 采用 2 bit 量化，其余较敏感部分在原始模型中为 NVFP4；每 5 层线性注意力（Mamba2）配 1 层全注意力（GQA），注意力宽度比 Qwen3 更窄，因此 30k 上下文仅占约 200 MB，256k 上下文约 1 GB，而 Flash Next 与 27B 分别为约 3.4 GB 和 8.5 GB。帖主称该模型基本是经过 REAP 处理并追加后训练的 Nemotron 3.5 Super，智力水平大致与 Qwen3.6 27B 相当，在其硬件上速度约为 27B 的两倍，相对突出的是非幻觉率与长上下文、大前缀缓存的内存占用。
深度见解
按该用户的说法，Bolt 能把数百万上下文 token 放进内存，因此适合小型企业等多用户场景，单张 RTX 5090 或 128 GB 小主机可能是它的理想硬件。
| https://preview.redd.it/0tljsfsl7cwh1.png?width=349&format=png&auto=webp&s=4428c2ab4743ce471d2862eef502ffa38381c8d3 I have been shopping for a faster model than 27B for a workload that Nemotron Ultra was doing well at. Ultra was pretty underwhelming since it came out, as there have been smaller models that are better, but its been free on openrouter, so why not use it if it works? Bolt is at first glance another model that should be passed over as well. It does have some impressive quantization, but its nothing to write home about when 27B is around, much less Flash Next. However, it was just the right size for my 128 GB setup (4x MI100), as it was designed for Sparks, so I took a look at it. A few interesting facts about Bolt: Its intelligence isn't great by today's standards, but it is about the same as Qwen3.6 27B, so not bad. One place it does shine relatively is non-hallucination rate, which is better than some top models. In my testing, this means it tends to wander in thought over many turns, running lots of diagnostics and checks before trying solutions and testing them thoroughly. https://preview.redd.it/1od8gwpw1cwh1.png?width=573&format=png&auto=webp&s=6ca23fce924f8e89bb2c53b4f7fafe4cceb1405b Its reasonably fast so far; about double 27B on my hardware. Where most of the model is just adequate, the place it stands out is it can fit a lot of context into memory, and also a large prefix cache too. This means Bolt is good for small multiuser scenarios like at small businesses. You can realistically fit millions of context tokens into memory, and a single 5090 (with native NVFP4 and 32 GB of memory) or one of the 128 GB SFF is probably the ideal hardware for this model. Its an interesting model, and I hope more like it are released. It certainly has been a long time waiting for a 70B model to pop up again. I'll be testing it some more, but it seems decent so far and I recommend it. submitted by /u/1ncehost [link] [comments] |
#Nemotron 3.5 Bolt#Nemotron 3.5 Super#NVIDIA#Qwen3.6 27B#DeepSeek V4.1 Flash#OpenRouter#RTX 5090