AI is happening. The future is updating.DeepScaleR-1.5B-Preview
DeepScaleR-1.5B-Preview is a language model fine-tuned from DeepSeek-R1-Distilled-Qwen-1.5B using distributed reinforcement learning (RL) to scale up to long context lengths. The model achieves 43.1% Pass@1 accuracy on AIME 2024, representing a 15% improvement over the base model (28.8%) and surpassing OpenAI's O1-Preview performance with just 1.5B parameters. Our training dataset consists of approximately 40,000 unique problem-answer pairs compiled from: - AIME problems (1984-2023) - AMC problems (prior to 2023) - Omni-MATH dataset - Still dataset We employ Deepseek's Group Relative Policy Optimization (GRPO), a simplified RL algorithm that extends PPO by: - Normalizing advantage function over all samples generated from the same prompt. - Applying KL divergence regularization on top of PPO's surrogate loss to prevent significant policy drift.
DeepScaleR-1.5B-Preview is a language model fine-tuned from DeepSeek-R1-Distilled-Qwen-1.5B using distributed reinforcement learning (RL) to scale up to long context lengths. The model achieves 43.1% Pass@1 accuracy on AIME 2024, representing a 15% improvement over the base model (28.8%) and surpassing OpenAI's O1-Preview performance with just 1.5B parameters. Our training dataset consists of approximately 40,000 unique problem-answer pairs compiled from: - AIME problems (1984-2023) - AMC problems (prior to 2023) - Omni-MATH dataset - Still dataset We employ Deepseek's Group Relative Policy Optimization (GRPO), a simplified RL algorithm that extends PPO by: - Normalizing advantage function over all samples generated from the same prompt. - Applying KL divergence regularization on top of PPO's surrogate loss to prevent significant policy drift.