AIが動く。未来が更新される。Nemotron-3.5-Bolt-E4B-GSQ-RCO-GGUF
Mixed-precision **GSQ-RCO** GGUF quantizations of nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16, optimized for efficient, high-fidelity local inference in `llama.cpp`. The model supports a context length of up to 65,536 tokens (64K). Standard quantization severely degrades hybrid sliding-attention models with shared Key-Value caches, multimodal projections, and speculative draft targets. **GSQ-RCO** solves this through two targeted techniques: - **RCO:** Preserves all 54 shared `k_proj`, `v_proj`, and `norm` tensors across layers 24–41 in high precision (`Q5_K`/`Q6_K`), maintaining structural attention geometry, visual projection alignment, and DFlash draft compatibility. - **GSQ:** Optimizes per-layer precision guided by Hessian sensitivity calibration.
翻訳準備中のため原文を表示しています
Mixed-precision **GSQ-RCO** GGUF quantizations of nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16, optimized for efficient, high-fidelity local inference in `llama.cpp`. The model supports a context length of up to 65,536 tokens (64K). Standard quantization severely degrades hybrid sliding-attention models with shared Key-Value caches, multimodal projections, and speculative draft targets. **GSQ-RCO** solves this through two targeted techniques: - **RCO:** Preserves all 54 shared `k_proj`, `v_proj`, and `norm` tensors across layers 24–41 in high precision (`Q5_K`/`Q6_K`), maintaining structural attention geometry, visual projection alignment, and DFlash draft compatibility. - **GSQ:** Optimizes per-layer precision guided by Hessian sensitivity calibration.