全站搜索
AI 模型 51
- Ling-3.0-flash-oQ5↗
TensorFold
- DeepSeek-V4.1-Flash-1017-TensorFold-e2e3-p4-h8↗
BobClawblaw
- Qwen3.8-Flash-Next-MLX-4bit↗
TensorFold
A native Apple-silicon conversion of Qwen/Qwen3.8-Flash-Next , quantised directly from the official BF16 checkpoint. This repository contains a 4-bit affine MLX conversion of Qwen3.8 Flash Next. It was produced directly from Qwen's BF16 weights using group size 32. The smaller group is intentional: it also covers the model's 160-wide hashed n-gram embedding tables instead of leaving them in BF16. The upstream tokenizer, chat template, vision processor, and generation configuration are preserved. The optional upstream MTP head is not included in this checkpoint.
- DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw↗
LibertAIDAI
**TL;DR** — EXL3 quantization of deepseek-ai/DeepSeek-V4.1-Flash-1017 at **2.90 bpw** (196.3 GiB). On held-out text its perplexity is **+1.8% (code) / +0.4% (prose)** above the unquantized model. Tensor names, dtypes and per-tensor bitrates match Mia-AiLab/DeepSeek-V4.1-Flash-EXL3-2.9bpw tensor for tensor, so the MiaAI-Lab/DeepSeek-v4.1-Flash-EXL3-2x-DGX-Sparks serving kit's bundled K map describes it unchanged. The two Engram tables are **not** in this repo: take them from the 1017 base repo, because they changed in this release. > **Chat template:** this repo ships `chat_template.jinja`, a Jinja port of DeepSeek's 1017 `encoding/encoding.py` (from jayleaton's MIT template via a community PR by dpolist, amended for the 1017 encoder). It renders byte-identically to the reference on our 8,455-case test corpus. Runtimes that require a template file at startup need it (verified to render identically under Python Jinja and TensorFold's Zig engine); vLLM-based kits use their built-in DeepSeek encoder. > **Agentic use / reasoning effort:** with tools in the conversation, each turn's reasoning stays in the prompt and is re-read by every later turn, so reasoning cost compounds. On our benchmark of the original 1017 weights (3,738 jobs: agentic coding, tau-bench, LiveCodeBench, AIME, chat), `reasoning_effort` **high** (75, the default) matched **max** (100) on quality at 0.55–0.8× the cost, and **low** (50) matched too at 0.4–0.6× (results in discussion #4). Avoid max for agent loops unless you need the headroom.
- Qwen3.8-Flash-Next-MLX-oQ4-MTP↗
TensorFold
- Ling-3.0-flash-oQ2↗
TensorFold
- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-oQ4↗
TensorFold
- Qwen3.8-Flash-Next-MLX-oQ8-MTP↗
TensorFold
- DeepSeek-V4.1-Flash-MLX-2bit-MTP↗
TensorFold
- Ling-3.0-flash-oQ4↗
TensorFold