Skip to content
AI NEWS JOURNAL
AI NEWS / ONE EVENT. MANY PERSPECTIVES.

llama.cpp · b12034

webgpu : fix CI by keeping scratch alloc sizes monotonic in batch length (#31422) * webgpu: keep scratch alloc sizes monotonic in the batch length The flash attention vec path and the MMVQ path append a scratch area after dst, but get_alloc_size only accounted for it when the current batch takes that path. The size was therefore not monotonic in the batch length: a graph reserved at n_ubatch (tiled path, no scratch) or at one token did not cover batches of 2 to 19 query rows, and GGML_SCHED_NO_REALLOC builds aborted on an unexpected reallocation. Pass the row count to both path predicates and size the scratch with the row count clamped to the longest batch the short path serves. * nit: align the mmvq pipeline key with clang-format 22 * webgpu: address review @yomaytk, name the mat-vec row

1 reports

The full article is not ready. Please read the original source.