mmBERT-base

> TL;DR: A state-of-the-art multilingual encoder trained on 3T+ tokens across 1800+ languages, introducing novel techniques for learning low-resource languages during the decay phase. mmBERT is a modern multilingual encoder that significantly outperforms previous generation models like XLM-R on classification, embedding, and retrieval tasks. Built on the ModernBERT architecture with novel multilingual training innovations, mmBERT demonstrates that low-resource languages can be effectively learned during the decay phase of training. It is also significantly faster than any previous multilingual encoder. - Highlights - Quick Start - Model Description - Novel Training Innovations - Model Family - Training Data - Usage Examples - Fine-tuning Examples - Model Architecture - Citation

번역 준비 중입니다. 원문을 표시합니다.