Author Reports EmbeddingGemma-2 PCA Head Tops Audio-Only MAEB
In two Reddit posts by /u/JeanMichelRanu, the author reports adding a 2.37 MB safetensors file of frozen PCA matrices to Google’s EmbeddingGemma-2 (577M). The adapter compresses acoustic features from Conformer layers 2 and 6 and injects them into the model’s unused 256-dimensional null space without training model weights. According to the author, the MAEB macro score rose from 49.3% to 54.4%, moving the model from 26th to first on audio-only MAEB and from 16th to first on full MAEB. The reported audio-to-audio results include gender clustering rising from 0.05% to 63.1%, artist retrieval from 55.2% to 93.8%, vocal emotion classification from 19.7% to 40.2%, and environmental-sound performance from 38.8% to 56.3%. The author says text-to-audio retrieval across more than 100 languages remained within 0.05 points, while text embeddings were unchanged. Reported trade-offs include the loss of Matryoshka truncation options for audio embeddings and weaker spoken-intent results, including MInDS14 falling from 97.6% to 88.3%. The supplied reports contain the author’s results but no independent rerun or verification.
The author reports that a small, training-free acoustic adapter brought a 577M-parameter model’s MAEB performance above 8B–9B multimodal systems while preserving multilingual text-to-audio retrieval. This could make the approach relevant to local and edge audio-embedding experiments, although the benchmark results have not been independently verified in the supplied reports.
Model & weights: BrunoHays/embeddinggemma-2-nullspace-audio
MAEB (Massive Audio Embedding Benchmark) is the main MTEB leaderboard benchmark for audio and text-to-audio embeddings, currently dominated by 8B-9B multimodal models.
I took Google's edge-sized google/embeddinggemma-2 (577M params, 768d vectors) and boosted its MAEB score from 49.3% to 54.4% (+5.1 points), jumping from Rank #26 to #1 on MAEB audio-only (and #16 to #1 on full MAEB) without training a single weight. The whole addition is a 2.37 MB safetensors file of frozen PCA matrices.
Why this matters for local / edge setups
At 577M parameters and 768d vectors (15x smaller than 9B Omni models), it runs easily on a laptop or edge device.
Standard text-aligned audio models are great at matching spoken words to text, but terrible at grouping audio by voice, emotion, or sound similarity because the final layers keep what words were said and drop how the sound was made.
#EmbeddingGemma-2#BrunoHays/embeddinggemma-2-nullspace-audio#Google