サイト内検索
AI モデル 85005
- Qwen3.8-27B-W4A16-AWQ↗
philbert440
INT4 weight-only quantization of Qwen/Qwen3.8-27B — **the whole model, nothing stripped**: vision tower intact, MTP speculative-decoding head intact, thinking mode calibrated with real reasoning traces. 55 GB BF16 → **19.5 GB**, which puts a current-generation 27B VLM (with headroom for 32K+ context) on a pair of 32 GB V100s — and it serves on modern GPUs with stock vLLM, no ceremony. Validated end-to-end on **2×V100-32GB under 1Cat-vLLM 1.2.2** (SM70 TurboMind dequant path): coherence, think-tag discipline, image understanding, math, long-form generation, and MTP speculative decoding in both draft modes. **Methodology:** warm serve (2 discarded warmup generations), fixed-length generations via `ignore_eos` so every run produces exactly the stated token count, official Qwen3.8 sampling per mode (thinking `1.0/0.95/20`, instruct `0.7/0.80/20 + presence 1.5`), varied prompts. Reported as mean ± sd tokens/s. Rig: 2×V100-32GB, 1Cat-vLLM 1.2.2, TP2, `fp8_e5m2` KV, `max_num_seqs 4`, MTP K=2.
- FFNet-78S-LowRes↗
qualcomm
- Llama-3.1-Nemotron-Nano-8B-v1-GGUF↗
QuantFactory
- Qwen2-VL-7B↗
阿里巴巴 · Qwen
- Qwen3-0.6B-MLX-6bit↗
阿里巴巴 · Qwen
- mt5-tiny-random↗
stas
- TableGPT2-7B↗
tablegpt
- TAIDE-LX-7B-Chat↗
taide
- d38a8↗
tao1000
- timeout_prediction_model↗
taroyz
