전체 검색
AI 모델 2119
- Qwythos-27B-v1-GGUF↗
empero-ai
GGUF quantizations of **empero-ai/Qwythos-27B-v1** for llama.cpp, Ollama, LM Studio, jan, KoboldCpp, and other GGUF runtimes. Qwythos-27B-v1 is a dense, hybrid Qwen3.5-27B—the larger continuation of Qwythos-9B, trained on the same curriculum at a larger scale. Nothing was removed to make this conversion fit: its native MTP head, vision tower, and 1,048,576-token (1M) context capability are intact. This is the pre-RL v1 checkpoint, following SFT, DPO, and ESFT; an RL-trained v2 will follow, while v1 is a complete checkpoint in its own right. For the full capability, training, and evaluation writeup, see the **base model card**. This card is intentionally focused on choosing a file and running it. The repository is named `Qwythos-27B-v1-GGUF`, but the files on disk deliberately use the shorter `Qwythos-27B-*` names below. Use those exact filenames when downloading or passing `-m`.
- Samantha-V2.0-i1-GGUF↗
mradermacher
- Ternary-Bonsai-2-2B-mlx-2bit↗
prism-ml
> **PrismML-Eng/Bonsai-demo is the source of truth for running these models.** Where anything here disagrees with it, it is right. For an OpenAI-compatible server, run `python -m mlx_vlm.server --model bonsai2-2b-mlx` and send the sampling settings in each request. For CUDA, CPU, and llama.cpp on Metal, use the GGUF packs of the same weights: **Ternary-Bonsai-2-2B-gguf**.
- Qwen3-Omni-30B-A3B-Instruct-FP8-Dynamic↗
sammysun0711
- samvit_base_patch16.sa1b↗
timm
- Qwen3.8-Flash-Next-Gyro-abliterated-Huihui-GGUF↗
agentionai
> **Updated 2026-10-13: both files.** Same file names, sizes and tensor types, better weights from an improved encode. This is huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated encoded the same way as our Gyro files. Abliteration is huihui-ai's work: it removes the refusal direction from the model's weights. We only quantized it. **Which setup fits your hardware, NVIDIA and AMD instructions, context limits, vision and speculative decoding are all on the main Gyro card and apply unchanged here; only the file names differ.**
- gemma-4-31B-it-qat-w4a16-ct↗
Google
Hugging Face | GitHub | Launch Blog | Documentation | Technical Report > [!Note] > This model card is for the new versions of the Gemma 4 family optimized with Quantization-Aware Training (QAT), which allows preserving similar quality to bfloat16 while dramatically reducing the memory requirements to load the model. > Four versions of the QAT checkpoints are available: > * **Unquantized QAT checkpoints** (Q4_0): Half-precision weights extracted from the QAT pipeline, ideal for custom downstream compilation and research. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B, and their drafter models. > * **GGUF** (Q4_0): Ready-to-deploy formats for broad ecosystem compatibility. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B. > * **Mobile-optimized** (wNa8o8): A custom schema engineered explicitly for mobile hardware efficiency. It features targeted 2-bit decoding layers, optimized KV caches, and static activations to maximize VRAM savings. Available for Gemma 4 E2B and E4B. > * **Compressed Tensors** (w4a16): QAT checkpoints serialized in the compressed-tensors format for native, optimized inference with vLLM. Available for Gemma 4 E2B, E4B, 12B, and 31B. > * **Assistant Compatibility**: When using multi-token prediction (speculative decoding) with an assistant model alongside a QAT target model, the assistant model must also be a QAT checkpoint with the same precision to ensure compatibility. Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages.
- Fun-ASR-MLT-Nano-2512-gguf↗
handy-computer
GGUF conversions of FunAudioLLM/Fun-ASR-MLT-Nano-2512 for use with transcribe.cpp. Ported from upstream commit cf67a93, pinned 2026-05-06. Validated against the FunASR reference at transcribe.cpp commit f094d28 on 2026-05-06. Offline speech-to-text covering 31 languages, with focused optimization on East and Southeast Asian languages: Chinese, English, Cantonese, Japanese, Korean, Vietnamese, Indonesian, Thai, Malay, Filipino, plus Arabic, Hindi, and 19 European languages (Bulgarian, Croatian, Czech, Danish, Dutch, Estonian, Finnish, Greek, Hungarian, Irish, Latvian, Lithuanian, Maltese, Polish, Portuguese, Romanian, Slovak, Slovenian, Swedish). Same architecture as Fun-ASR-Nano-2512 (~800M trainable parameters: frozen SenseVoiceEncoderSmall + 2-layer audio adaptor + bundled Qwen3-0.6B LLM); trained on a smaller multilingual corpus ("hundreds of thousands of hours" per the model card, vs Nano's "tens of millions"). Takes a 16 kHz mono WAV and emits text. Not streaming, no translation, no timestamps. ITN (inverse text normalization) is on by default, matching upstream, for readable casing, punctuation, and digits; pass `--no-itn` for spoken-form output.
- slimsam-50-uniform↗
nielsr
- Qwen3.8-27B-MTPLX-Optimized-Speed↗
Youssofal
**MTPLX: the fastest way to run Qwen 3.8 on a Mac. Native multi-token-prediction speculative decoding on Apple Silicon, two to three times the speed of plain decoding, exact at any temperature.** This is the model MTPLX recommends for coding on a modern Mac with 32 GB or more. It is Qwen3.8-27B with its native multi-token-prediction head kept, so MTPLX can draft several tokens ahead and verify them in one pass. Same model, same output distribution, a lot more tokens per second. This is a Qwen3.8-27B MLX pack for MTPLX, the fastest way to run Qwen 3.8 27B on a Mac. All numbers on a MacBook Pro M5 Max, fans verified at maximum, sampled at the model's own settings. Conditions and sources: mtplx.com/benchmarks.