Qwen3.8-Flash-Next-FP4-Local

**IMPORTANT**: the NVFP4 scale factors in this checkpoint are in a new lossless compression format, but upstream vLLM support for this format has not yet landed. **You must use one of the containers provided below.** The NVIDIA Qwen3.8-Flash-Next-FP4-Local model is the quantized version of Alibaba's Qwen3.8-Flash-Next model, which is an auto-regressive language model that uses an optimized transformer architecture. Qwen3.8-Flash-Next is a causal language model with a vision encoder, hybrid attention (Gated DeltaNet and Qwen Sparse Attention), Mixture-of-Experts, gated residual streams, and n-gram embeddings. For more information, please check here. The NVIDIA Qwen3.8-Flash-Next-FP4-Local model is quantized with Model Optimizer using quantization-aware distillation (QAD). The checkpoint occupies approximately **96 GiB on disk** and is best suited for local AI hardware. It has been validated with vLLM on the following hardware: * **NVIDIA RTX PRO 6000 Blackwell** — single GPU, with PLE (n-gram embedding table) offload. 850K tokens of FP8 KV Cache * **NVIDIA DGX Spark** — Can run with or without PLE offload. 1M tokens of FP8 KV Cache with PLE offload

翻訳準備中のため原文を表示しています