foxAIAI is happening. The future is updating.
English
中文English日本語한국어

GLM-5.3-Flash-ewaste-edition-GGUF

This is a quantization of GLM-5.3-flash that is optimized to work better on older hardware. This quant was built using i-matrix GGUF quantizations with the same principle as the GLM-5.2 and GLM-5.3 e-waste editions: **K-quants for the routed experts, no codebook i-quants**, because on pre-AVX-512 CPUs and previous-generation datacenter GPUs (MI100 / gfx908) expert dequantization, not bandwidth, is the decode bottleneck. **Format: mainline llama.cpp `glm5-next`** (merged 2026-09-30, ggml-org/llama.cpp#27773). These files do **not** load in builds that only know the earlier `glm5next` arch from the unmerged PR #27754 (unsloth's September files are that format). Any llama.cpp from 2026-10 on loads them. The NextN/MTP head (blk.45) is in every file for when mainline's MTP support (#27917) lands. All are imatrix-quantized with unsloth's GLM-5.3-Flash importance matrix (88 chunks × 9216 tokens) from an **F32 intermediate** of the FP8 checkpoint (`fp8 × block scale` in F32; a BF16 intermediate re-rounds most weights because the scales are arbitrary floats). Tensors that mainline's quantizer keeps unquantized for this arch (hyper-connection mixers, DSA indexer, KDA gates/betas, `attn_kv_a_mqa`) are F32 in every file; `attn_q_a/q_b` and `nextn.eh_proj` are Q8_0 (mainline's floor). The MTP head's experts are Q4_K in every variant.

This is a quantization of GLM-5.3-flash that is optimized to work better on older hardware. This quant was built using i-matrix GGUF quantizations with the same principle as the GLM-5.2 and GLM-5.3 e-waste editions: **K-quants for the routed experts, no codebook i-quants**, because on pre-AVX-512 CPUs and previous-generation datacenter GPUs (MI100 / gfx908) expert dequantization, not bandwidth, is the decode bottleneck. **Format: mainline llama.cpp `glm5-next`** (merged 2026-09-30, ggml-org/llama.cpp#27773). These files do **not** load in builds that only know the earlier `glm5next` arch from the unmerged PR #27754 (unsloth's September files are that format). Any llama.cpp from 2026-10 on loads them. The NextN/MTP head (blk.45) is in every file for when mainline's MTP support (#27917) lands. All are imatrix-quantized with unsloth's GLM-5.3-Flash importance matrix (88 chunks × 9216 tokens) from an **F32 intermediate** of the FP8 checkpoint (`fp8 × block scale` in F32; a BF16 intermediate re-rounds most weights because the scales are arbitrary floats). Tensors that mainline's quantizer keeps unquantized for this arch (hyper-connection mixers, DSA indexer, KDA gates/betas, `attn_kv_a_mqa`) are F32 in every file; `attn_q_a/q_b` and `nextn.eh_proj` are Q8_0 (mainline's floor). The MTP head's experts are Q4_K in every variant.

Sources

huggingface.co ↗