Gemma 3 270M

Headlines · 2

1. Oct 21
   Paper: frozen 16 MB embedding table can match learned embeddings in small models

   A paper posted to r/LocalLLaMA, titled "Embeddings Are Overpriced?", proposes replacing a small language model's trainable embedding table with a frozen 16 MB table of 512-bit codes that has no trainable parameters, derived once from a pretrained embedding via principal-component projection and sign binarisation. The authors trained the same Gemma-shaped body from scratch on 2.05B tokens with and without the table; they report the learned table buys 0.20 nats of validation loss for 2.6 times the trainable parameters, while the frozen-table model is better or tied on LAMBADA, HellaSwag and PIQA, with only ARC-Easy favouring the learned table. They also report that with a ternary BitNet b1.58 body the whole model is a 46 MB file that ties the float model on LAMBADA and PIQA and generates 615 to 740 tokens per second on a laptop CPU in 62 MB of memory, and that weights, data and evaluation answers are released.

   r/LocalLLaMA · 🔥 0
2. Oct 21
   Synaptics unveils Astra SR200 edge AI chips with Google Coral NPU

   Synaptics announced the Astra SR200 series of edge AI "xMCU" processors on Oct. 20, built around Google Research's open-source Coral NPU and Synaptics' Torq AI engine, with three pin-compatible models rated at 200, 300 and 400 GOPS. The chips pair Arm Cortex-M85 and Cortex-M52 cores with up to 8MB of SRAM and custom DRAM scalable to hundreds of megabytes, targeting wearables, connected cameras, industrial vision and voice systems; Synaptics says the family remains in sampling with no public pricing or firm production schedule. The Coralboard SL developer kit, based on the separate Astra SL2619 processor and a 1-TOPS Coral NPU implementation, reached general availability through DigiKey the same day, while a Coralboard SR board based on the SR250 was announced without an order date or price.

   cnBeta · 🔥 0

Experience and discussion from the community

Share my Gemma 3 270M experienceAsk about Gemma 3 270M