normistral-7b-warm

NorMistral-7b-warm is a large Norwegian language model initialized from Mistral-7b-v0.1 and continuously pretrained on a total of 260 billion subword tokens (using six repetitions of open Norwegian texts). This model is a part of the NORA.LLM family developed in collaboration between the Language Technology Group at the University of Oslo, the High Performance Language Technologies (HPLT) project, the National Library of Norway, and the University of Turku. All the models are pre-trained on the same dataset and with the same tokenizer. NorMistral-7b-warm has over 7 billion parameters and is based on the Mistral architecture. The NORA.LLM language model family includes (as of now): - **NorMistral-7b-warm** -- an LLM initialized from Mistral-7b-v0.1 and continuously pretrained on Norwegian data; - **NorMistral-7b-scratch** -- a Mistral-based LLM pretrained from scratch on Norwegian data; - **NorBLOOM-7b-scratch** -- a BLOOM-based LLM pretrained from scratch on Norwegian data.

번역 준비 중입니다. 원문을 표시합니다.