AI is happening. The future is updating.Muse-Glimmer-30B-Heretic-Uncensored-GGUF
This repository disseminates a comprehensive suite of 27 GGUF (GPT-Generated Unified Format) quantizations of **Muse-Glimmer-30B-Heretic-Uncensored**, an abliterated derivative of Meta's Muse-Glimmer-30B produced with Heretic v1.4.0 by darkc0de. The release spans a memory footprint of 6.53 GB (IQ1_S) to 55.73 GB (F16/BF16), thereby accommodating deployment across heterogeneous hardware, from CPU-only systems to 32 GB VRAM accelerators. All sub-4-bit quantizations were calibrated with an importance matrix (imatrix) to mitigate quality degradation. The fidelity of the Q4_K_S variant relative to the F16 reference is quantified via token-level perplexity on a fixed evaluation set. The model exhibits **zero refusals on adult/NSFW content**; this property was empirically verified. The quantization selection is derived from the model's architectural properties and the memory constraints of representative target hardware. The model employs **grouped-query attention with 2 KV heads**, which renders the KV cache exceptionally compact: - 52 layers × 2 KV heads × 128 head_dim = **52 KiB/token at FP16** - Q8_0 KV cache: **26 KiB/token** → 64K context ≈ 1.7 GB - Q4_0 KV cache: **13 KiB/token** → 64K context ≈ 0.85 GB
This repository disseminates a comprehensive suite of 27 GGUF (GPT-Generated Unified Format) quantizations of **Muse-Glimmer-30B-Heretic-Uncensored**, an abliterated derivative of Meta's Muse-Glimmer-30B produced with Heretic v1.4.0 by darkc0de. The release spans a memory footprint of 6.53 GB (IQ1_S) to 55.73 GB (F16/BF16), thereby accommodating deployment across heterogeneous hardware, from CPU-only systems to 32 GB VRAM accelerators. All sub-4-bit quantizations were calibrated with an importance matrix (imatrix) to mitigate quality degradation. The fidelity of the Q4_K_S variant relative to the F16 reference is quantified via token-level perplexity on a fixed evaluation set. The model exhibits **zero refusals on adult/NSFW content**; this property was empirically verified. The quantization selection is derived from the model's architectural properties and the memory constraints of representative target hardware. The model employs **grouped-query attention with 2 KV heads**, which renders the KV cache exceptionally compact: - 52 layers × 2 KV heads × 128 head_dim = **52 KiB/token at FP16** - Q8_0 KV cache: **26 KiB/token** → 64K context ≈ 1.7 GB - Q4_0 KV cache: **13 KiB/token** → 64K context ≈ 0.85 GB