Inkling-Small-DSpark-Preview

A DSpark speculator for the Inkling-Small NVFP4 target, enabling faster inference through speculative decoding. DSpark extends the DFlash parallel-draft backbone with a Markov logit-bias head and a per-position confidence head. This checkpoint was trained with SpecForge using hidden states from a live SGLang target engine. **Preview:** This is a research preview. Official training and evaluation are in progress. - **Base model:** `thinkingmachines/Inkling-Small-NVFP4`; the exact matching target checkpoint and tokenizer are required. - **Format:** Safetensors (single-file BF16, 0.90B trainable parameters; draft weights only). - **Draft:** 5 layers (Qwen3-style GQA), hidden 4096, 64 heads / 16 KV heads, head_dim 64, FFN 8192, rope_theta 8000000, `block_size=15`. - **RoPE:** YaRN, factor 128, `original_max_position_embeddings=8192`, `max_position_embeddings=1048576` (matches the target addressable range); config carries both `rope_scaling` and `rope_parameters` schemas. - **Vocabulary:** 200,058 tokenizer entries and 201,024 padded weight rows; `mask_token_id=200064`. - **DSpark heads:** Markov rank 256 (vanilla) and confidence head (with-Markov). - **Aux hidden-state layers:** `[5, 11, 23, 29, 35]`. - **Trained context:** 8,192 native; long-context adaptation with sequences up to 65,536. - **Target weights:** target embedding and unembedding weights are not included in this checkpoint.

译文尚未就绪，显示来源原文