Reddit user proposes architectural priorities for Gemma 5
A Reddit user shared suggestions for future Gemma models, including a looped transformer, DeepSeek’s Engram, continued 4-bit quantization and more efficient attention mechanisms. The post also proposes a roughly 120B mixture-of-experts model, improved drafter models, linear projections for vision and audio, and abandoning 2-bit quantization for small models. These are community proposals, not a Google announcement; the post contains no Gemma 5 implementation or benchmark results.
The proposal identifies an individual user’s priorities for Gemma’s architecture, quantization and deployment, while showing that the ideas have not been presented as confirmed Google development plans.
| Recently Omar Sanseviero asked for suggestions on Twitter for future Gemma models. Here is my take on this: What do you all think? Feel free to comment your opinion down below. submitted by /u/Paramecium_caudatum_ [link] [comments] |
#Gemma 5#DeepSeek Engram#Compressed Sparse Attention 2#Qwen 3.5#DSpark#DFlash 2#EAGLE-4