boro64
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Ternary Nemotron 30B-A3B GGUF model repository shared
A Hugging Face repository identified in an r/LocalLLaMA post presents a GGUF model called Ternary Nemotron 30B-A3B. The submitter describes it as a ternary, 1.58-bit mixture-of-experts model that runs on a 12 GB GPU, supports a 368K-token context, and reaches 172 tokens per second decoding and more than 7,000 tokens per second prefill. The post says one file works with stock llama.cpp and the PrismML/Bonsai fork, listing specific builds where each was validated.
r/LocalLLaMA · 🔥 0
Experience and discussion from the community
Share my boro64 experienceAsk about boro64
Nobody has shared their experience with boro64 yet.