LLaMA3-iterative-DPO-final-ExPO

The extrapolated (ExPO) model based on `RLHFlow/LLaMA3-iterative-DPO-final` and `RLHFlow/LLaMA3-SFT`, as in the "Weak-to-Strong Extrapolation Expedites Alignment" paper. Specifically, we obtain this model by extrapolating **(alpha = 0.3)** from the weights of the SFT and DPO/RLHF checkpoints, achieving superior alignment with human preference. Evaluation results on the **AlpacaEval 2.0** benchmark (you can find the evaluation outputs on the official GitHub repo):

译文尚未就绪，显示来源原文

The extrapolated (ExPO) model based on `RLHFlow/LLaMA3-iterative-DPO-final` and `RLHFlow/LLaMA3-SFT`, as in the "Weak-to-Strong Extrapolation Expedites Alignment" paper. Specifically, we obtain this model by extrapolating **(alpha = 0.3)** from the weights of the SFT and DPO/RLHF checkpoints, achieving superior alignment with human preference. Evaluation results on the **AlpacaEval 2.0** benchmark (you can find the evaluation outputs on the official GitHub repo):

资料来源

huggingface.co ↗