Transformers v5.19.0 adds EmbeddingGemma 2 support
Hugging Face Transformers v5.19.0 adds support for EmbeddingGemma 2, Google’s multimodal embedding model for text, images, audio, and video in a shared 768-dimensional vector space. The release also changes MoE router-logit outputs, expands expert-parallel token dispatch, updates continuous-batching attention behavior, and fixes quantized and per-layer cache handling.
The release makes a new multimodal embedding model available in a widely used open-source framework while updating distributed inference and caching APIs.
Release v5.19.0
New Model additions
EmbeddingGemma2
EmbeddingGemma 2 is a multimodal embedding model from Google built on the Gemma 4 architecture. It encodes text, images, audio, and video, individually or combined in one input, into a shared 768-dimensional vector space for cross-modal retrieval, semantic similarity, clustering, and classification. It uses Matryoshka Representation Learning, so embeddings can be truncated to 512, 256, or 128 dimensions. It also offers configurable visual and video token budgets, and unused vision or audio towers can be disabled at load time to save memory.
Links: Documentation
#Hugging Face Transformers#Hugging Face#Google#EmbeddingGemma 2#Gemma 4