arXiv:2610.37680v1 Announce Type: new Abstract: Closing the gap between frontier and smaller language models is usually approached by distillation: fine-tuning the small model (student) on a frontier model's (teacher) outputs, which binds transfer to training compute and leaves the transferred knowledge largely opaque. We propose Artifact Distillation: a teacher solves training instances and externalizes its problem-solving procedure into an artifact bank that a student queries at inference time, with no gradient updates. Across four LiveBench tasks, a bank written by Claude Opus 4.5 lets Claude Haiku 4.5 outperform unaided Claude Sonnet 4.5 on spatial reasoning (0.707 vs. 0.647) at 60% of its cost, and reach Sonnet 4.5's level on table joining at 89% of its cost. The same raw bank raises
arXiv:2610.37680v1 发布类型：新
摘要：缩小前沿模型与小型语言模型之间的性能差距，通常采用知识蒸馏的方法：在前沿模型（教师）的输出上对小型模型（学生）进行微调，但这使得迁移过程高度依赖于训练计算资源，且被迁移的知识往往难以解释。我们提出了一种名为“Artifact Distillation”的新方法：由教师模型解答一系列训练实例，并将其求解过程以“工件库”的形式外部化；学生模型在推理阶段直接查询该工件库，无需任何梯度更新。在四项LiveBench任务上，由Claude Opus 4.5构建的工件库使Claude Haiku 4.5在空间推理任务上的表现优于未使用辅助的Claude Sonnet 4.5（0.707对比0.647），且仅需其成本的60%；在表格连接任务上，也达到了Sonnet 4.5的水平，仅需其成本的89%。同样一份原始工件库还将Qwen3.5-27B的综合得分从0.730提升至0.800，这表明迁移并不局限于教师模型所属的同一模型家族。
