Search FoxAI
AI Models 85012
- spoomplesmaxx-whiskeyjack-12B-GGUF↗
mradermacher
- TerraSense-GGUF↗
mradermacher
- Tinyllama-1.1B-Chat-v1.0-OBLITERATED-i1-GGUF↗
mradermacher
- Trida2.0-4B-GGUF↗
mradermacher
- UnslopNemo-12B-v4.1-GGUF↗
mradermacher
- VibeThinker-3B-heretic_decensored-i1-GGUF↗
mradermacher
- voice-realtime-1b-10-GGUF↗
mradermacher
- spanish-gpt2↗
mrm8488
- Nanbeige4.2-3B-GGUF↗
NANI-Nithin
- GLM-5.3-Flash-GSQ-RCO-3.0bit-Q4Kattn-GGUF↗
neuralll
> **Big thanks to @csantiago78**: the expert cache > here builds on their implementation in llama.cpp PR > #27861 ("GPU-resident LRU cache > for host-offloaded MoE expert weights"), the first to get a working hot-expert > cache into llama.cpp. This fork takes it further: filling all free VRAM, smarter > eviction, CPU/GPU overlap and fused kernels. A variant of pfeifferj/GLM-5.3-Flash-GSQ-RCO-GGUF (3.0-bit) made for faster single-stream decoding of this 117 GB model on consumer GPUs with an expert cache. **Only the non-expert Q8_0 weights changed** (attention, shared experts, dense FFN, output head: Q8_0 to Q4_K, ~8.3 GB of the file). All 126 routed-expert tensors, `token_embd` and every F32/BF16 tensor are copied bit-for-bit from the original. Those Q8_0 weights are streamed by the GPU for every token, so shrinking them speeds up decoding; the experts are unchanged.
