Qwen3.8-27B-AWQ

4-bit AWQ quantization of Qwen/Qwen3.8-27B, calibrated in-house on 2×RTX 3090 (Ampere, sm_86) for **single-user 256K-context serving under SGLang**. Thinking, image, and **video** are all preserved and probe-verified. Weights are ~18.7 GB, so the model fits two 24 GB cards at TP=2 with room for a deep KV pool (697K tokens measured at `--max-running 1`). Single-user decode throughput (M=1, 100 output tokens, depth server-verified via `actual_input_tokens`):

번역 준비 중입니다. 원문을 표시합니다.