b12170
The llama.cpp v1.21.70 release includes enhancements to the moe-cache system, introducing an LRFU/MQ-inspired hybrid cache with improved token handling, key computation, and decay calculations. Changes include adjusting the half-life parameter, increasing the number of classes to 8, refining score computation based on age, removing redundant branches, and improving documentation around unused slots and sorting. The release also features updated build binaries for multiple platforms including macOS, Linux, Windows, Android, and openEuler, with support for CUDA, Vulkan, ROCm, OpenVINO, SYCL, and other backends.