AIが動く。未来が更新される。modernbert-ja-310m
This repository provides Japanese ModernBERT trained by SB Intuitions. Our ModernBERT-Ja-310M is trained on a high-quality corpus of Japanese and English text comprising **4.09T tokens**, featuring a vocabulary size of 102,400 and a sequence length of **8,192** tokens. Additionally, if your GPUs support Flash Attention 2, we recommend using our models with Flash Attention 2.
翻訳準備中のため原文を表示しています
This repository provides Japanese ModernBERT trained by SB Intuitions. Our ModernBERT-Ja-310M is trained on a high-quality corpus of Japanese and English text comprising **4.09T tokens**, featuring a vocabulary size of 102,400 and a sequence length of **8,192** tokens. Additionally, if your GPUs support Flash Attention 2, we recommend using our models with Flash Attention 2.