Ornith-1.5-35B-A3B-GGUF

GGUF quantizations of ornith-ai/Ornith-1.5-35B-A3B, made with an importance matrix, plus a KL-divergence comparison with the other published GGUFs of this model. > [!NOTE] > These files were made for a quantization experiment and are evaluated only by the KLD measurements below, not by task benchmarks, long-context use or speed tests. Check them on your own workload before relying on them. The four "comparison only" files are there so that the effect of `-hq` in the tables below can be checked; at each size the `-hq` file is better. All files are Q4_K_M with up to two kinds of changes. The `-hq` part raises the tensors outside the routed experts to Q8_0: attention, the linear-attention (SSM) projections, the shared expert, and the MTP projection. The `-exps` part changes the routed expert tensors (`ffn_{gate,up,down}_exps`) in layers 0-39. The token embedding and the output keep the Q4_K_M types (Q4_K and Q6_K). All files keep the built-in MTP (NextN) layer, so they work with `--spec-type draft-mtp`. For image input, use the vision projector from the official repo (`mmproj-Ornith-1.5-35B-BF16.gguf`).

번역 준비 중입니다. 원문을 표시합니다.