AI is happening. The future is updating.NVIDIA-Nemotron-3.5-Lightning-30B-A3B-W4A16
This is a calibrated W4A16 quantization of NVIDIA Nemotron 3.5 Lightning 30B-A3B, intended for high-throughput NVIDIA GPU serving with vLLM. It has been validated on a single 24 GB RTX 3090 at a 640K production context window. - **Model developer:** NVIDIA - **Quantization / packaging:** useful-quants - **Source checkpoint:** NVIDIA's BF16 release at revision `63a200063804e06fdb41d6717e43bc92f67859d2` - **Governing license:** OpenMDW 1.1; the unmodified upstream license is included as `LICENSE`. This is a third-party, post-training quantization. useful-quants did not train, fine-tune, align, or otherwise develop Nemotron. The conversion does not change NVIDIA's intended-use, safety, or license terms; see the upstream model card for those terms, language coverage, architecture, reasoning behavior, training/data disclosure, and NVIDIA's source-model benchmark suite.
This is a calibrated W4A16 quantization of NVIDIA Nemotron 3.5 Lightning 30B-A3B, intended for high-throughput NVIDIA GPU serving with vLLM. It has been validated on a single 24 GB RTX 3090 at a 640K production context window. - **Model developer:** NVIDIA - **Quantization / packaging:** useful-quants - **Source checkpoint:** NVIDIA's BF16 release at revision `63a200063804e06fdb41d6717e43bc92f67859d2` - **Governing license:** OpenMDW 1.1; the unmodified upstream license is included as `LICENSE`. This is a third-party, post-training quantization. useful-quants did not train, fine-tune, align, or otherwise develop Nemotron. The conversion does not change NVIDIA's intended-use, safety, or license terms; see the upstream model card for those terms, language coverage, architecture, reasoning behavior, training/data disclosure, and NVIDIA's source-model benchmark suite.