NVIDIA-Nemotron-Nano-9B-v2-NVFP4

モデル紹介

NVIDIA-Nemotron-Nano-9B-v2-NVFP4 is a quantized version of NVIDIA-Nemotron-Nano-9B-v2 and is a large language model (LLM) trained from scratch by NVIDIA, and designed as a unified model for both reasoning and non-reasoning tasks. It responds to user queries and tasks by first generating a reasoning trace and then concluding with a final response. The model's reasoning capabilities can be controlled via a system prompt. If the user prefers the model to provide its final answer without intermediate reasoning traces, it can be configured to do so, albeit with a slight decrease in accuracy for harder prompts that require reasoning. Conversely, allowing the model to generate reasoning traces first generally results in higher-quality final solutions to queries and tasks. The model uses a hybrid architecture consisting primarily of Mamba-2 and MLP layers combined with just four Attention layers. For the architecture, please refer to the Nemotron-H tech report. Nano 9B v2 has 56 layers in total: 27 Mamba layers, 25 MLP layers, and 4 attention layers. In the FP8/NVFP4 quantized version, the Mamba and MLP layers are quantized to FP8/NVFP4, while all 4 attention layers and the Conv1d components within the Mamba layers are kept in BF16. We used ModelOpt Post-training Quantization (PTQ) for both FP8 and NVFP4 quantization. For NVFP4, the first and last 2 layers are kept in BF16 to better preserve accuracy. On top of PTQ, Quantization-Aware Distillation (QAD) was also applied to NVFP4 for additional accuracy recovery.

主な特徴

特点摘要待依据来源补齐，不凭模型名称推断能力。

制約と注意点

具体的な制約を判断する出典が不足しています。実際の性能、遅延、データ方針、ライセンス、提供期間を確認してください。

出典別料金と費用分析

料金はチャネル・地域・設定によって異なります。各チャネルの料金は個別に表示し、提供元の統一公式料金とは見なしません。

公開評価と実行設定

暂无已采集的可核验公开评分。模型未上榜不代表能力较低。

よくある質問

出典と追跡記録

许可证：other。模型介绍与第三方评测均以原来源的具体版本和方法为准。