AIが動く。未来が更新される。stablelm-3b-4e1t
`StableLM-3B-4E1T` is a 3 billion parameter decoder-only language model pre-trained on 1 trillion tokens of diverse English and code datasets for 4 epochs. The model is a decoder-only transformer similar to the LLaMA (Touvron et al., 2023) architecture with the following modifications: The dataset is comprised of a filtered mixture of open-source large-scale datasets available on the HuggingFace Hub: Falcon RefinedWeb extract (Penedo et al., 2023), RedPajama-Data (Together Computer., 2023) and The Pile (Gao et al., 2020) both without the *Books3* subset, and StarCoder (Li et al., 2023).
翻訳準備中のため原文を表示しています
`StableLM-3B-4E1T` is a 3 billion parameter decoder-only language model pre-trained on 1 trillion tokens of diverse English and code datasets for 4 epochs. The model is a decoder-only transformer similar to the LLaMA (Touvron et al., 2023) architecture with the following modifications: The dataset is comprised of a filtered mixture of open-source large-scale datasets available on the HuggingFace Hub: Falcon RefinedWeb extract (Penedo et al., 2023), RedPajama-Data (Together Computer., 2023) and The Pile (Gao et al., 2020) both without the *Books3* subset, and StarCoder (Li et al., 2023).