跳到正文
AI NEWS JOURNAL
ONE EVENT. MANY PERSPECTIVES.

llama.cpp · b11922

tests : fix multi-seq split replay and partial-state cases for trained models (#31216) * tests : fix multi-seq split replay against trained models The rollback snapshot is captured by the graph that decoded the rolled-back tail, while the reference context decodes the same prefix with different batch shapes. The two states differ in the last bits and trained models amplify the difference with every replayed token. Compare only the first replayed token of each seq, then restore the rollback state into the reference context before the seq-1-only tail. Assisted-by: dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF (MacBook, M5 Max, 128GB) * tests : require bit-exact seq-1 isolation in split replay The seq state save is not always bit-lossless - SWA-masked cells are not saved - so reload the rest

1 个独立来源 · 1 篇报道切换最早在前 ↕
按发布时间排序

b11922

测试:修复针对已训练模型的多序列拆分重放及部分状态情形(#31216)* 测试:修复针对已训练模型的多序列拆分重放 回滚快照由解码回滚尾部的图谱捕获,而参考上下文则以不同的批次形状解码同一前缀。这两种状态在最后几位上存在差异,且已训练模型会在每次重放的每个标记中进一步放大这种差异。仅比较每个序列的第一个重放标记,然后在序列-1-only尾部之前,将回滚状态恢复到参考上下文中。 协助者:dsh:llama.cpp/ggml-org/Qwen3.8-Flash-Next-GGUF(MacBook,M5 Max,128GB)* 测试:要求在拆分重放中实现逐位精确的序列-1隔离 序列状态保存并不总是位完全无损——SWA掩码单元不会被保存——因此需重新加载其余部分

事件进展与其他报道