Llama-3.1-Tulu-3-8B-DPO

Tülu3 is a leading instruction following model family, offering fully open-source data, code, and recipes designed to serve as a comprehensive guide for modern post-training techniques. Tülu3 is designed for state-of-the-art performance on a diversity of tasks in addition to chat, such as MATH, GSM8K, and IFEval. The Tülu3 models have limited safety training, but are not deployed automatically with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). It is also unknown what the size and composition of the corpus was used to train the base Llama 3.1 models, however it is likely to have included a mix of Web data and technical sources like books and code. See the Falcon 180B model card for an example of this. DPO: - **Learning Rate**: 5 × 10⁻⁷ (8B), 2.0e-7 (70B, 405B) - **Learning Rate Schedule**: Linear - **Batch Size (effective)**: 128 (8B), 128 (70B), 256(405B) - **KL Penalty Coefficient**: 5 - **Warm-up Ratio**: 0.1 - **Max Sequence Length**: 2,048 - **Epochs**: 1