AI正在发生 未来正在更新stablelm-3b-4e1t
`StableLM-3B-4E1T` is a 3 billion parameter decoder-only language model pre-trained on 1 trillion tokens of diverse English and code datasets for 4 epochs. The model is a decoder-only transformer similar to the LLaMA (Touvron et al., 2023) architecture with the following modifications: The dataset is comprised of a filtered mixture of open-source large-scale datasets available on the HuggingFace Hub: Falcon RefinedWeb extract (Penedo et al., 2023), RedPajama-Data (Together Computer., 2023) and The Pile (Gao et al., 2020) both without the *Books3* subset, and StarCoder (Li et al., 2023).
译文尚未就绪，显示来源原文
`StableLM-3B-4E1T` is a 3 billion parameter decoder-only language model pre-trained on 1 trillion tokens of diverse English and code datasets for 4 epochs. The model is a decoder-only transformer similar to the LLaMA (Touvron et al., 2023) architecture with the following modifications: The dataset is comprised of a filtered mixture of open-source large-scale datasets available on the HuggingFace Hub: Falcon RefinedWeb extract (Penedo et al., 2023), RedPajama-Data (Together Computer., 2023) and The Pile (Gao et al., 2020) both without the *Books3* subset, and StarCoder (Li et al., 2023).