跳到正文
AI NEWS JOURNAL
ONE EVENT. MANY PERSPECTIVES.

llama.cpp · b11906

ggml-cpu : rewrite concat as one row kernel for all types (#31179) Every dst row comes from src0 or src1, or for dim 0 from the row of src0 followed by the row of src1, so a single kernel covers every type: a dense source row is one memcpy, quantized types included, and a strided source is copied element by element with the element size known at compile time. Rows are split across threads instead of i2 alone, which is often 1, and are copied in tiles of 32: for a transposed source the elements are walked in the outer loop, so the rows of a tile read neighboring addresses instead of jumping a full row stride per element. Add transposed b cases to test_concat, the layout of the delta-net conv input, and a perf case for it. **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Sili

1 个独立来源 · 1 篇报道切换最早在前 ↕
按发布时间排序

b11906

ggml-cpu:为所有类型重写 concat 为单行内核(#31179) 每一行目标数据均来自源0或源1;对于维度0,则由源0的一行后接源1的一行构成,因此只需一个内核即可覆盖所有类型:稠密的源行只需一次 memcpy,包括量化类型在内;而对于步长型源,则按元素逐个复制,且元素大小在编译时已知。行数据不再仅通过 i2 分配线程,而是跨多个线程进行划分,i2 的值通常为1;同时以32个元素为一组进行复制:对于转置后的源,在外层循环中遍历元素,因此每个分块的各行读取的是相邻地址,而非每处理一个元素就跳过整行步长。在 test_concat 中新增转置 b 的测试用例、delta-net 卷积输入的布局测试用例,以及相应的性能测试用例。**网站:** - **证明文件:** - **macOS/iOS:** - [macOS Apple Sili

事件进展与其他报道