Hands-on: Nemotron 3.5 Bolt suits small multiuser setups with long context
A Reddit user posting in r/LocalLLaMA as /u/1ncehost says Nemotron 3.5 Bolt was the right size for their 128 GB setup (4x MI100) and works for small multiuser workloads. The user reports that Bolt's decoders are 2-bit quantized with other sensitive parts in NVFP4 in the stock model, and that it pairs one full-attention (GQA) layer with every five linear-attention (Mamba2) layers, giving a narrower attention footprint than Qwen3: about 200 MB at 30k context and 1 GB at 256k, versus roughly 3.4 GB for Flash Next and 8.5 GB for 27B. The poster describes Bolt as essentially a REAP-ed Nemotron 3.5 Super with additional post-training, rates its intelligence as roughly on par with Qwen3.6 27B, says it runs about twice as fast as 27B on their hardware, and singles out its non-hallucination rate and memory-efficient long context and prefix cache.
Per the poster, Bolt can hold millions of context tokens in memory, which makes it a fit for small multiuser settings such as small businesses, with a single RTX 5090 or a 128 GB small-form-factor machine as likely ideal hardware.
| https://preview.redd.it/0tljsfsl7cwh1.png?width=349&format=png&auto=webp&s=4428c2ab4743ce471d2862eef502ffa38381c8d3 I have been shopping for a faster model than 27B for a workload that Nemotron Ultra was doing well at. Ultra was pretty underwhelming since it came out, as there have been smaller models that are better, but its been free on openrouter, so why not use it if it works? Bolt is at first glance another model that should be passed over as well. It does have some impressive quantization, but its nothing to write home about when 27B is around, much less Flash Next. However, it was just the right size for my 128 GB setup (4x MI100), as it was designed for Sparks, so I took a look at it. A few interesting facts about Bolt: Its intelligence isn't great by today's standards, but it is about the same as Qwen3.6 27B, so not bad. One place it does shine relatively is non-hallucination rate, which is better than some top models. In my testing, this means it tends to wander in thought over many turns, running lots of diagnostics and checks before trying solutions and testing them thoroughly. https://preview.redd.it/1od8gwpw1cwh1.png?width=573&format=png&auto=webp&s=6ca23fce924f8e89bb2c53b4f7fafe4cceb1405b Its reasonably fast so far; about double 27B on my hardware. Where most of the model is just adequate, the place it stands out is it can fit a lot of context into memory, and also a large prefix cache too. This means Bolt is good for small multiuser scenarios like at small businesses. You can realistically fit millions of context tokens into memory, and a single 5090 (with native NVFP4 and 32 GB of memory) or one of the 128 GB SFF is probably the ideal hardware for this model. Its an interesting model, and I hope more like it are released. It certainly has been a long time waiting for a 70B model to pop up again. I'll be testing it some more, but it seems decent so far and I recommend it. submitted by /u/1ncehost [link] [comments] |
#Nemotron 3.5 Bolt#Nemotron 3.5 Super#NVIDIA#Qwen3.6 27B#DeepSeek V4.1 Flash#OpenRouter#RTX 5090