全站搜索
AI 模型 85012
- gemma-3-1b-it-kk↗
Xkev
- sub38-2↗
yopmail12917
- Nemotron-3.5-Bolt-E4B-GSQ-RCO-GGUF↗
abenzerps
Mixed-precision **GSQ-RCO** GGUF quantizations of nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16, optimized for efficient, high-fidelity local inference in `llama.cpp`. The model supports a context length of up to 65,536 tokens (64K). Standard quantization severely degrades hybrid sliding-attention models with shared Key-Value caches, multimodal projections, and speculative draft targets. **GSQ-RCO** solves this through two targeted techniques: - **RCO:** Preserves all 54 shared `k_proj`, `v_proj`, and `norm` tensors across layers 24–41 in high precision (`Q5_K`/`Q6_K`), maintaining structural attention geometry, visual projection alignment, and DFlash draft compatibility. - **GSQ:** Optimizes per-layer precision guided by Hessian sensitivity calibration.
- GLM-5.3-GGUF↗
AesSedai
- manaca-1b-instruct↗
AIInstituteLNCC
- Qwen3.5-397B-A17B-MXFP4↗
amd
The model was quantized from Qwen/Qwen3.5-397B-A17B-FP8 using AMD-Quark. The weights are quantized to MXFP4 and activations are quantized to MXFP4. For further details or issues, please refer to the AMD-Quark documentation or contact the respective developers. The GSM8K results were obtained using the vLLM framework, based on the Docker image `rocm/vllm-dev:nightly_main_20260211`, and vLLM is installed inside the container.
- Qwen3.8-27B-TURBO-NInfer↗
amynporb
- sortformer-finetuned↗
arda-argmax
- internlm_OREAL-DeepSeek-R1-Distill-Qwen-7B-GGUF↗
bartowski
- katanemo_Arch-Function-Chat-1.5B-GGUF↗
bartowski
