gemma-4-31B-it-qat-NVFP4-Blackwell

> [!TIP] > **There is a newer version of this model:** > gemma-4-31B-it-qat-NVFP4-mse-Blackwell. > Same base model and calibration set, but with MSE-based weight scaling > instead of absmax, which keeps outliers from crushing the rest of each block. > This repo stays as it is so existing deployments don't change under their feet. Gemma 4 31B IT quantized to NVFP4 for native FP4 inference on NVIDIA Blackwell GPUs. Quantized from Google's unquantized QAT weights (`google/gemma-4-31B-it-qat-q4_0-unquantized`) with NVIDIA Model Optimizer (`modelopt` 0.44.0), default absmax scaling. - **Base Model:** google/gemma-4-31B-it-qat-q4_0-unquantized - **Quantization Format:** `NVFP4` (native FP4 on Blackwell) - **Group Size:** 16 (per-group scaling) - **Excluded Modules:** `lm_head`, `model.embed_vision*`, `model.vision_tower*` (kept in high precision) - **KV cache scales:** none in the checkpoint; use `--kv-cache-dtype fp8` and let vLLM scale dynamically - **Producer:** NVIDIA Model Optimizer (`modelopt` 0.44.0)

翻訳準備中のため原文を表示しています