AI is happening. The future is updating.DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw
**TL;DR** — EXL3 quantization of deepseek-ai/DeepSeek-V4.1-Flash-1017 at **2.90 bpw** (196.3 GiB). On held-out text its perplexity is **+1.8% (code) / +0.4% (prose)** above the unquantized model. Tensor names, dtypes and per-tensor bitrates match Mia-AiLab/DeepSeek-V4.1-Flash-EXL3-2.9bpw tensor for tensor, so the MiaAI-Lab/DeepSeek-v4.1-Flash-EXL3-2x-DGX-Sparks serving kit's bundled K map describes it unchanged. The two Engram tables are **not** in this repo: take them from the 1017 base repo, because they changed in this release. > **Chat template:** this repo ships `chat_template.jinja`, a Jinja port of DeepSeek's 1017 `encoding/encoding.py` (from jayleaton's MIT template via a community PR by dpolist, amended for the 1017 encoder). It renders byte-identically to the reference on our 8,455-case test corpus. Runtimes that require a template file at startup need it (verified to render identically under Python Jinja and TensorFold's Zig engine); vLLM-based kits use their built-in DeepSeek encoder. > **Agentic use / reasoning effort:** with tools in the conversation, each turn's reasoning stays in the prompt and is re-read by every later turn, so reasoning cost compounds. On our benchmark of the original 1017 weights (3,738 jobs: agentic coding, tau-bench, LiveCodeBench, AIME, chat), `reasoning_effort` **high** (75, the default) matched **max** (100) on quality at 0.55–0.8× the cost, and **low** (50) matched too at 0.4–0.6× (results in discussion #4). Avoid max for agent loops unless you need the headroom.
**TL;DR** — EXL3 quantization of deepseek-ai/DeepSeek-V4.1-Flash-1017 at **2.90 bpw** (196.3 GiB). On held-out text its perplexity is **+1.8% (code) / +0.4% (prose)** above the unquantized model. Tensor names, dtypes and per-tensor bitrates match Mia-AiLab/DeepSeek-V4.1-Flash-EXL3-2.9bpw tensor for tensor, so the MiaAI-Lab/DeepSeek-v4.1-Flash-EXL3-2x-DGX-Sparks serving kit's bundled K map describes it unchanged. The two Engram tables are **not** in this repo: take them from the 1017 base repo, because they changed in this release. > **Chat template:** this repo ships `chat_template.jinja`, a Jinja port of DeepSeek's 1017 `encoding/encoding.py` (from jayleaton's MIT template via a community PR by dpolist, amended for the 1017 encoder). It renders byte-identically to the reference on our 8,455-case test corpus. Runtimes that require a template file at startup need it (verified to render identically under Python Jinja and TensorFold's Zig engine); vLLM-based kits use their built-in DeepSeek encoder. > **Agentic use / reasoning effort:** with tools in the conversation, each turn's reasoning stays in the prompt and is re-read by every later turn, so reasoning cost compounds. On our benchmark of the original 1017 weights (3,738 jobs: agentic coding, tau-bench, LiveCodeBench, AIME, chat), `reasoning_effort` **high** (75, the default) matched **max** (100) on quality at 0.55–0.8× the cost, and **low** (50) matched too at 0.4–0.6× (results in discussion #4). Avoid max for agent loops unless you need the headroom.