AI正在发生 未来正在更新Apertus-v1.5-70B-FP8
Full-precision FP8 quantization of swiss-ai/Apertus-v1.5-70B using the FP8_DYNAMIC scheme on all linear projections (excluding lm_head, embeddings, and vision/audio tokenizers which remain in BF16). Use this checkpoint when maximum quality is critical and GPU memory allows -- 71 GiB leaves approximately 14 GiB for KV cache, enough for roughly 60k context with fp8 KV cache on a 96 GiB card. **Thanks to:** Swiss AI, blancsw, Anunay-Yadav, Oleg, Cyrilvallez, AryanAhadinia, robmsmt, and Neural Magic. See Attribution for details.
译文尚未就绪，显示来源原文
Full-precision FP8 quantization of swiss-ai/Apertus-v1.5-70B using the FP8_DYNAMIC scheme on all linear projections (excluding lm_head, embeddings, and vision/audio tokenizers which remain in BF16). Use this checkpoint when maximum quality is critical and GPU memory allows -- 71 GiB leaves approximately 14 GiB for KV cache, enough for roughly 60k context with fp8 KV cache on a 96 GiB card. **Thanks to:** Swiss AI, blancsw, Anunay-Yadav, Oleg, Cyrilvallez, AryanAhadinia, robmsmt, and Neural Magic. See Attribution for details.