AI is happening. The future is updating.ModernBERT-base-msmarco
Reproducibility report can be found here: https://chaochunhsu.github.io/patterns/blogs/modernbert_dpr/ This is a sentence-transformers model finetuned from answerdotai/ModernBERT-base on the msmarco-co-condenser-margin-mse-sym-mnrl-mean-v1 dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. I finetune ModernBERT-base using script from offical repo train_st.py on a RTX 4090 GPU with the only change of setting mini-batch size of `CachedMultipleNegativesRankingLoss` to 128. Training for 1 epoch takes less than an hour.
Reproducibility report can be found here: https://chaochunhsu.github.io/patterns/blogs/modernbert_dpr/ This is a sentence-transformers model finetuned from answerdotai/ModernBERT-base on the msmarco-co-condenser-margin-mse-sym-mnrl-mean-v1 dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. I finetune ModernBERT-base using script from offical repo train_st.py on a RTX 4090 GPU with the only change of setting mini-batch size of `CachedMultipleNegativesRankingLoss` to 128. Training for 1 epoch takes less than an hour.