Latxa-Llama-3.1-8B-Instruct

We introduce Latxa 3.1 8B Instruct, an instructed version of Latxa. This new Latxa is based on Llama-3.1 (Instruct), which we trained on our Basque corpus (Etxaniz et al., 2024) comprising 4.3M documents and 4.2B tokens using language adaptation techniques (paper in preparation). > [!WARNING] > DISCLAIMER > > This model is still under development. > Further training details will be released with the corresponding research paper in the near future. Our preliminary experimentation shows that Latxa 3.1 8B Instruct outperforms Llama-3.1-Instruct by a large margin on Basque standard benchmarks, and particularly, on chat conversations. In addition, we organized a public arena-based evaluation, on which Latxat competed against other baselines and proprietary models such as GPT-4o and Claude Sonnet. The results showed that Latxa ranked 3rd, just behind Claude and GPT-4 and above all the other same-size competitors. The official paper is coming soon. Latxa is a family of Large Language Models (LLM) based on Meta’s LLaMA models. Current LLMs exhibit incredible performance for high-resource languages such as English, but, in the case of Basque and other low-resource languages, their performance is close to a random guesser. These limitations widen the gap between high- and low-resource languages when it comes to digital development. We present Latxa to overcome these limitations and promote the development of LLM-based technology and research for the Basque language. Latxa models follow the same architecture as their original counterparts and were further trained in Latxa Corpus v1.1, a high-quality Basque corpora.

번역 준비 중입니다. 원문을 표시합니다.