ALMA-13B-Pretrain

模型介绍

**ALMA** (**A**dvanced **L**anguage **M**odel-based tr**A**nslator) is an LLM-based translation model, which adopts a new translation model paradigm: it begins with fine-tuning on monolingual data and is further optimized using high-quality parallel data. This two-step fine-tuning process ensures strong translation performance. Please find more details in our paper. **ALMA-R (NEW!) is released now!** ALMA-R builds upon ALMA models, with further LoRA fine-tuning with our proposed **Contrastive Preference Optimization (CPO)** as opposed to the Supervised Fine-tuning used in ALMA. CPO fine-tuning requires our triplet preference data for preference learning. ALMA-R now can matches or even exceeds GPT-4 or WMT winners! We release six translation models presented in the paper: - **ALMA-7B**: Full-weight Fine-tune LLaMA-2-7B on 20B monolingual tokens and then **Full-weight** fine-tune on human-written parallel data - **ALMA-7B-LoRA**: Full-weight Fine-tune LLaMA-2-7B on 20B monolingual tokens and then **LoRA** fine-tune on human-written parallel data - **ALMA-7B-R (NEW!)**: Further LoRA fine-tuning upon ALMA-7B-LoRA with contrastive preference optimization. - **ALMA-13B**: Full-weight Fine-tune LLaMA-2-7B on 12B monolingual tokens and then **Full-weight** fine-tune on human-written parallel data - **ALMA-13B-LoRA** (Our best system): Full-weight Fine-tune LLaMA-2-7B on 12B monolingual tokens and then **LoRA** fine-tune on human-written parallel data - **ALMA-13B-R (NEW!)**: Further LoRA fine-tuning upon ALMA-13B-LoRA with contrastive preference optimization.

主要特点

特点摘要待依据来源补齐，不凭模型名称推断能力。

局限与注意事项

暂无足够来源支持具体缺点结论。请重点验证实际任务表现、时延、数据政策、许可与版本生命周期。

来源报价与费用分析

同一模型在不同渠道、地区和配置下可能采用不同价格。下表独立保留各渠道报价，不将渠道价格视为厂商官方统一价格。

公开评测与运行配置

暂无已采集的可核验公开评分。模型未上榜不代表能力较低。

常见问题

资料来源与可追溯记录

许可证：mit。模型介绍与第三方评测均以原来源的具体版本和方法为准。