전체 검색

AI 모델 2119

• Qwen3.8-27B-AWQ-INT4-FP8KV↗

  abhishekchohan

  The scales were measured on the quantized model itself, not on the BF16 base, because activation-aware smoothing and int4 rounding change the K/V activations the served model actually writes to cache. Calibration replayed the exact same sequences that were used to calibrate the AWQ model: **112 packed sequences × 262,144 tokens (~29.4M tokens)** from the same Nemotron Post-Training v3 SFT blend (instruction-following chat, math, science, agentic code, multilingual), same shuffle seed, rendered with thinking enabled, so the measured KV distribution matches the deployment regime the weights were tuned for, at the exact context length the model serves at. While the model processed those sequences, we tracked the running maximum absolute value of the K states (after QK-norm) and V states each of the 16 global attention layers writes to the cache. Each scale is `amax / 448` (the fp8-e4m3 representable maximum), symmetric, per tensor, so no cached value can clip. The result is 32 per-layer scalars (`k_scale` / `v_scale`). Only the 16 global attention layers accumulate KV; the 48 DeltaNet linear-attention layers carry recurrent state and have no KV scales.

• Nanbeige4.1-3B↗

  Nanbeige

  Nanbeige4.1-3B is built upon Nanbeige4-3B-Base and represents an enhanced iteration of our previous reasoning model, Nanbeige4-3B-Thinking-2511, achieved through further post-training optimization with supervised fine-tuning (SFT) and reinforcement learning (RL). As a highly competitive open-source model at a small parameter scale, Nanbeige4.1-3B illustrates that compact models can simultaneously achieve robust **reasoning**, **preference alignment**, and **effective agentic behaviors**. We evaluate Nanbeige4.1-3B across a broad and diverse set of benchmarks covering **general reasoning**, and **deep-search capabilities**. On general reasoning tasks including **code**, **math**, **science**, **alignment**, and **tool-use** benchmarks, Nanbeige4.1-3B not only significantly outperforms same-scale models such as **Qwen3-4B**, but also demonstrates overall superior performance compared to larger models including **Qwen3-30B-A3B-2507** and **Qwen3-32B**.

• MiniMax-M3-EAGLE3↗

  Inferact

  **Inferact/MiniMax-M3-EAGLE3** is an EAGLE3 draft model for accelerating inference of MiniMax-M3. It is served end-to-end with **vLLM** and was trained using **TorchSpec** — a torch-native online speculative-decoding training framework that runs FSDP training and vLLM-based target inference concurrently, learning from **MiniMax-M3-regenerated responses and live vLLM-generated hidden states** to match the base model's exact token distribution. The draft is a **1-layer** dense Llama (`LlamaForCausalLMEagle3`, ~3.3 B params) operating on MiniMax-M3's `hidden_size=6144` / `vocab_size=200064`; at serve time it shares the target's embedding and LM head (EAGLE3). See `config.json` for the full architecture. All numbers are measured end-to-end against `MiniMaxAI/MiniMax-M3-MXFP8` served with vLLM at `tensor-parallel-size=4`, `num_speculative_tokens=3`, and `--enforce-eager`. Greedy draft sampling (`topk=1`).

• Qwen3-0.6B-Gensyn-Swarm-burrowing_subtle_parrot↗

  SamsBuk

• evf-sam2-multitask↗

  YxZhang

  This is the checkpoint holder of EVF-SAM. Please refer to `"inference.py"` in the source code for detailed usage. We haven't supported `"AutoModel.from_pretrained(...)"` yet, please import the model script from source code.

• Qwythos-27B-v1-GGUF↗

  empero-ai

  GGUF quantizations of **empero-ai/Qwythos-27B-v1** for llama.cpp, Ollama, LM Studio, jan, KoboldCpp, and other GGUF runtimes. Qwythos-27B-v1 is a dense, hybrid Qwen3.5-27B—the larger continuation of Qwythos-9B, trained on the same curriculum at a larger scale. Nothing was removed to make this conversion fit: its native MTP head, vision tower, and 1,048,576-token (1M) context capability are intact. This is the pre-RL v1 checkpoint, following SFT, DPO, and ESFT; an RL-trained v2 will follow, while v1 is a complete checkpoint in its own right. For the full capability, training, and evaluation writeup, see the **base model card**. This card is intentionally focused on choosing a file and running it. The repository is named `Qwythos-27B-v1-GGUF`, but the files on disk deliberately use the shorter `Qwythos-27B-*` names below. Use those exact filenames when downloading or passing `-m`.

• Samantha-V2.0-i1-GGUF↗

  mradermacher

• Ternary-Bonsai-2-2B-mlx-2bit↗

  prism-ml

  > **PrismML-Eng/Bonsai-demo is the source of truth for running these models.** Where anything here disagrees with it, it is right. For an OpenAI-compatible server, run `python -m mlx_vlm.server --model bonsai2-2b-mlx` and send the sampling settings in each request. For CUDA, CPU, and llama.cpp on Metal, use the GGUF packs of the same weights: **Ternary-Bonsai-2-2B-gguf**.

• Qwen3-Omni-30B-A3B-Instruct-FP8-Dynamic↗

  sammysun0711

• samvit_base_patch16.sa1b↗

  timm