AI is happening. The future is updating.Qwen3.8-27B-GPTQ-Int4
4-bit GPTQ quantization of Qwen/Qwen3.8-27B, produced with GPTQModel. The `mtp.*` tensors (the model's Multi-Token-Prediction / speculative-decoding draft head, 15 tensors) are **kept in bf16, not quantized** — GPTQModel's `qwen3_5.py` model definition preserves them via its `out_of_model_tensors` mechanism rather than passing them through the quantization loop at all (upstream `transformers` doesn't load MTP weights into the normal inference graph in the first place, so there is nothing to quantize there). This means the MTP speculative-decoding path below works out of the box. Vision tower weights are similarly untouched — only the `language_model.*` weights are quantized (this checkpoint is intended for **text-only serving**).
4-bit GPTQ quantization of Qwen/Qwen3.8-27B, produced with GPTQModel. The `mtp.*` tensors (the model's Multi-Token-Prediction / speculative-decoding draft head, 15 tensors) are **kept in bf16, not quantized** — GPTQModel's `qwen3_5.py` model definition preserves them via its `out_of_model_tensors` mechanism rather than passing them through the quantization loop at all (upstream `transformers` doesn't load MTP weights into the normal inference graph in the first place, so there is nothing to quantize there). This means the MTP speculative-decoding path below works out of the box. Vision tower weights are similarly untouched — only the `language_model.*` weights are quantized (this checkpoint is intended for **text-only serving**).