Skip to content
AI NEWS JOURNAL
ONE EVENT. MANY PERSPECTIVES.

llama.cpp · b11933

vulkan: fix CI llvmpipe by allowing dim 3 broadcast in MUL_MAT (#31239) * vulkan: allow dim 3 broadcast in MUL_MAT supports_op rejected every MUL_MAT whose src0 and src1 differ in ne3, although mul_mm, mul_mmq and mul_mat_vec already broadcast src0 over dim 3 through broadcast3. Such a matmul then ran on the CPU with a copy of the weight, and the split layout changed with the number of streams in the ubatch. Keep the strict check for MUL_MAT_ID only, and route the broadcast cases away from mul_mat_vec_nc, the one path that requires equal ne3. * tests : harden MUL_MAT with permuted dim 3 broadcast cases Harden the MUL_MAT coverage with a weight broadcast over dim 3 against a permuted per-stream activation, the shape of an MLA output projection with a non unified KV cache, for a single token

1 independent sources · 1 reportsOldest first ↕
Newest first

b11933

vulkan: fix CI llvmpipe by allowing dim 3 broadcast in MUL_MAT (#31239) * vulkan: allow dim 3 broadcast in MUL_MAT supports_op rejected every MUL_MAT whose src0 and src1 differ in ne3, although mul_mm, mul_mmq and mul_mat_vec already broadcast src0 over dim 3 through broadcast3. Such a matmul then ran on the CPU with a copy of the weight, and the split layout changed with the number of streams in the ubatch. Keep the strict check for MUL_MAT_ID only, and route the broadcast cases away from mul_mat_vec_nc, the one path that requires equal ne3. * tests : harden MUL_MAT with permuted dim 3 broadcast cases Harden the MUL_MAT coverage with a weight broadcast over dim 3 against a permuted per-stream activation, the shape of an MLA output projection with a non unified KV cache, for a single token

Timeline and related coverage