AI is happening. The future is updating.Qwen3.8-Flash-Next-Rulith-EXL3-GGUF
EXL3 (trellis-coded) weights of Qwen/Qwen3.8-Flash-Next in a GGUF container, for Rulith Inference on AMD Strix Halo (Windows). **The quantization is turboderp's.** The EXL3 tensors are taken bit for bit from turboderp/Qwen3.8-Flash-Next-exl3 (ExLlamaV3 1.4.4, codebook `mul1`, calibration 250 rows x 2048) and only re-packed; all credit for the quantization goes to turboderp and the ExLlamaV3 project. The non-EXL3 tensors (hyper-connection mixers, router, norms, embeddings, the n-gram table codec) follow the layout and types Rulith Inference is tuned for. 4.05 bpw is the recommended one; 3.05 bpw is smaller and generates 7-12% faster, at twice the KL divergence (see the comparison below). In both, the first shard holds only the metadata (11 MB), so a client can read the model's header before the weights; the third is one tensor, the per-layer token embedding table (39.0 / 32.6 GB), which cannot be split.
EXL3 (trellis-coded) weights of Qwen/Qwen3.8-Flash-Next in a GGUF container, for Rulith Inference on AMD Strix Halo (Windows). **The quantization is turboderp's.** The EXL3 tensors are taken bit for bit from turboderp/Qwen3.8-Flash-Next-exl3 (ExLlamaV3 1.4.4, codebook `mul1`, calibration 250 rows x 2048) and only re-packed; all credit for the quantization goes to turboderp and the ExLlamaV3 project. The non-EXL3 tensors (hyper-connection mixers, router, norms, embeddings, the n-gram table codec) follow the layout and types Rulith Inference is tuned for. 4.05 bpw is the recommended one; 3.05 bpw is smaller and generates 7-12% faster, at twice the KL divergence (see the comparison below). In both, the first shard holds only the metadata (11 MB), so a client can read the model's header before the weights; the third is one tensor, the per-layer token embedding table (39.0 / 32.6 GB), which cannot be split.