Kimi-K3-DSpark

A long-context DSpark speculator for Kimi K3. It supports context lengths of up to 1 million tokens. A DSpark speculator for the Kimi K3 target, enabling faster inference through speculative decoding. DSpark extends the DFlash parallel-draft backbone with a Markov logit-bias head and a per-position confidence head. This checkpoint was trained with SpecForge using hidden states from a live SGLang target engine. - **Base model:** `moonshotai/Kimi-K3` - **Format:** Safetensors (single-file BF16, 2,249,289,601 parameters) - **Draft:** 5 full-attention Qwen3-style GQA layers, hidden size 7168, 64 query heads / 16 KV heads, and `block_size=7` - **Verification width:** 1 current token + 7 draft tokens - **Auxiliary target layers:** `[7, 23, 51, 67, 83]` - **Trained context:** 65,536 tokens - **Target weights:** embedding and unembedding weights are not included

번역 준비 중입니다. 원문을 표시합니다.