Skywork-Reward-Gemma-2-27B-v0.2

> IMPORTANT: > This model was trained using the decontaminated version of the original Skywork Reward Preference dataset, now referred to as **v0.2**. The updated dataset, Skywork-Reward-Preference-80K-v0.2, removes 4,957 contaminated pairs from the magpie-ultra-v0.1 subset, which had significant n-gram overlap with the evaluation prompts in RewardBench. You can find the set of removed pairs here. For more detailed information, please refer to this GitHub gist. > > **If your task involves evaluation on RewardBench, we strongly recommend using v0.2 of both the dataset and the models instead of v0.1, to ensure proper decontamination and avoid any contamination issues.** **Skywork-Reward-Gemma-2-27B-v0.2** and **Skywork-Reward-Llama-3.1-8B-v0.2** are two advanced reward models built on the gemma-2-27b-it and Llama-3.1-8B-Instruct architectures, respectively. Both models were trained using the Skywork Reward Data Collection containing only 80K high-quality preference pairs sourced from publicly available data. We include only public data in an attempt to demonstrate that high-performance reward models can be achieved with a relatively small dataset and straightforward data curation techniques, without further algorithmic or architectural modifications. The sources of data used in the Skywork Reward Data Collection are detailed in the Data Mixture section below.