AI가 움직이고 미래가 업데이트됩니다.LLaMmlein_1B
LLäMmlein 1B is a German LLaMa model trained from scratch using our adapted Tinyllama codebase on the German portion of RedPajama V2. To enhance data quality, we additionally deduplicated the dataset on paragraph level and filtered it using a token-to-word ratio filter. The resulting dataset can be found here. You can use LLäMmlein with the `transformers` library. (Optional: install `flash-attn` to achieve highest efficiency.) In addition to the final model checkpoint, we publish intermediate checkpoints throughout the full training process as unique branches in this repository. A specific checkpoint can be loaded like this:
번역 준비 중입니다. 원문을 표시합니다.
LLäMmlein 1B is a German LLaMa model trained from scratch using our adapted Tinyllama codebase on the German portion of RedPajama V2. To enhance data quality, we additionally deduplicated the dataset on paragraph level and filtered it using a token-to-word ratio filter. The resulting dataset can be found here. You can use LLäMmlein with the `transformers` library. (Optional: install `flash-attn` to achieve highest efficiency.) In addition to the final model checkpoint, we publish intermediate checkpoints throughout the full training process as unique branches in this repository. A specific checkpoint can be loaded like this: