Paper

EmbeddingGemma: Powerful and Lightweight Text Representations

Chen, Yichang, Sreepathihalli, Divyashree, Ye Xia, Sung, Yunhsuan, Goenka, Sonam, Ballantyne, Ian, Cameron, Glenn, Baumgartner, Simon, Visin, Francesco, Sindhu Raghuram Panyam, Co, A J, Kumar, Ravin, Yi-Ting, Chen, Joulin, Armand, Zhang, Biao, Elarabawy, Adham, Hoffmann, Raphael, Ding, Zhongli, Liu, Gaël, Boratko, Michael, Gleicher, Zach, Dua, Sahil, Roberts, Adam, Alfonseca, Enrique, Choi, Min, Wang, Weiyi, Mullins, Ryan, Ariafar, Setareh, Yin, Qin, Gonzalez, Lucas, Lee, Jinhyuk, Sai Meher Karthik Duddu, Sanseviero, Omar, Chen, Ke, Khawaja, Waleed, Zhang, Shijie, Chen, Koert, Potetz, Brian, Martins, Gus, Hui, Hui, Cer, Daniel, Duerig, Tom, Dong, Zhe, Salz, Daniel, Zhang, Jiageng, Hora, Ben, Lisak, Alice, Zou, Joe, Qiu, Steve, Warkentin, Tris, Kim, Dahun, Samari, Babak, Chen, Kaifeng, Suganthan, Paul, Frank Palma Gomez, Gill, Karan, Kenealy, Kathleen, Zhou, Wenlei, Seyedhosseini, Mojtaba, Zhang, Hesen, Sherwood, Mark, Doumanoglou, Andreas, Smoot, Sara, Li, Zhe, Gustavo Hernández Ábrego, Dabral, Tanmaya, Singh, Jyotinder, Lacombe, Olivier, Zheng, Jingxiao, Sharma, Abheesht, Zhang, Shanfeng, Rao, Vikram, Brick, Cormac, Ji, Juyeong

2025

Overview

We introduce EmbeddingGemma, a new lightweight, open text embedding model based on the Gemma 3 language model family. Our innovative training recipe strategically captures knowledge from larger models via encoder-decoder initialization and geometric embedding distillation. We improve model robustness and expressiveness with a spread-out regularizer, and ensure generalizability by merging checkpoints from varied, optimized mixtures. Evaluated on the Massive Text Embedding Benchmark (MTEB) across multilingual, English, and code domains, EmbeddingGemma (300M) achieves state-of-the-art results. Notably, it outperforms prior top models, both proprietary and open, with fewer than 500M parameters, and provides performance comparable to models double its size, offering an exceptional performance-to-cost ratio. Remarkably, this lead persists when quantizing model weights or truncating embedding outputs. This makes EmbeddingGemma particularly well-suited for low-latency and high-throughput use cases such as on-device applications. We provide ablation studies exploring our key design choices. We release EmbeddingGemma to the community to promote further research.

• Publisher: Cornell University Library, arXiv.org
• Subject: Ablation / Embedding / Encoders-Decoders