llama.cpp · b11985

metal : faster MXFP4 mat-vec, full threadgroups for unary/binary/GLU (#31075) * metal : faster MXFP4 mat-vec kernel Decode the fp4 values with the fp16 bit trick instead of a threadgroup lookup table, and load the packed nibbles as aligned 32-bit words. The kernel no longer uses threadgroup memory. * metal : launch contiguous unary and binary ops with full threadgroups The contiguous unary path and the row-broadcast binary path used one thread per threadgroup. Use up to 256 threads per threadgroup and bound-check the tail. * metal : split GLU rows across threadgroups A GLU row ran on one threadgroup, so single-row decode used a single threadgroup for the whole op. Split each row over threadgroups of up to 256 threads. **Website:** - **Attestations:** - **macOS/iOS:** - [macOS Apple Silicon