Community user ports Comet 60B A12B to GGUF for llama.cpp
AI summary
Reddit user /u/Felladrin converted Multiverse Computing’s Comet 60B A12B BF16 model into GGUF formats that reportedly run on stock llama.cpp. The Q4_K_M and Q8_0 files are reported as 42.6 GB and 64.2 GB, respectively, and retain the MTP head for speculative decoding through llama-server with the --spec-type draft-mtp option. The conversions are text-only because the vision tower was not converted.
In-depth insight
The reported conversions make a 60B sparse model available for local llama.cpp deployment while preserving its MTP head for speculative decoding, though the claim has not been independently verified or benchmarked.
I wanted to try MultiverseComputingCAI/Comet_60B_A12B_BF16 (Multiverse Computing's compressed version of Nemotron 3.5 Super), so I converted it to GGUF, and it runs on stock llama.cpp:
They keep the MTP head, so you can pass --spec-type draft-mtp to llama-server for speculative decoding.
Just sharing, so you can try it too. Note that these GGUFs are text only (the vision tower wasn't converted). submitted by /u/Felladrin
[link] [comments] |
#Multiverse Computing#Comet 60B A12B#Nemotron 3.5 Super#llama.cpp#llama-server