전체 검색
AI 모델 2119
- GLM-5.3-Flash-UD-Q4_K_XL-layers↗
meshllm
This package is derived from unsloth/GLM-5.3-Flash-GGUF and keeps the original GGUF distribution split into per-layer artifacts for distributed inference. For upstream architecture details, chat template guidance, sampling recommendations, license terms, and benchmark notes, see the source model card: unsloth/GLM-5.3-Flash-GGUF. Generated by the Mesh LLM HF Jobs splitter from `mesh-llm` ref `f932c4d1dc12b3e3a670d5f470cedd5cdcc5db39`. Each artifact is checksummed as it is written, uploaded to this repository, and removed from the job workspace before the next artifact is produced.
- Nethena-13B-GGUF↗
TheBloke
GGUF is a new format introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp. The creator of the source model has listed its license as `cc-by-nc-4.0`, and this quantization has therefore used that same license. As this model is based on Llama 2, it is also subject to the Meta Llama 2 license terms, and the license files for that are additionally included. It should therefore be considered as being claimed to be licensed under both licenses. I contacted Hugging Face for clarification on dual licensing but they do not yet have an official position. Should this change, or should Meta provide any feedback on this situation, I will update this section accordingly.
- gemma-4-26B-A4B-it-NVFP4↗
cklaus
NVFP4-quantized version of google/gemma-4-26B-A4B-it in **modelopt format**, optimized for vLLM on NVIDIA Blackwell (SM120) GPUs. Other available NVFP4 quantizations of this model quantize attention layers too, which causes: 1. Quality degradation from aggressive attention quantization 2. vLLM's QKV fusion scale bug (fused projections take `max()` of global scales, causing underflow) 1. Started from RedHatAI/gemma-4-26B-A4B-it-NVFP4 (compressed-tensors format) 2. Dequantized all 115 self-attention projections (q/k/v/o across 30 layers) back to BF16 using FP4-E2M1 lookup table 3. Converted from compressed-tensors to modelopt format: - `weight_packed` -> `weight` (same data) - `weight_global_scale` (divisor) -> `weight_scale_2` (1/divisor) - `input_global_scale` (divisor) -> `input_scale` (1/divisor) 4. Updated quantization config to match NVIDIA's modelopt pattern with proper `exclude_modules`
- dolphin-2.6-mixtral-8x7b↗
dphn
This Dolphin is *really good* at coding, I trained with a lot of coding data. It is *very* obedient but it is not DPO tuned - so you still might need to encourage it in the system prompt as I show in the below examples. New in 2.6 - Fixed a training configuration issue that improved the quality a lot - Due to popular demand, added back samantha-based empathy data - Replaced synthia and pure-dove with Capybara This model is uncensored. I have filtered the dataset to remove alignment and bias. This makes the model more compliant. You are advised to implement your own alignment layer before exposing the model as a service. It will be highly compliant to any requests, even unethical ones. Please read my blog post about uncensored models. https://erichartford.com/uncensored-models You are responsible for any content you create using this model. Enjoy responsibly.
- SAM-GGUF↗
mradermacher
- GLM-5.3-BF16↗
zai-org
GLM-5.3 uses the same base model as GLM-5.2 — every gain comes from post-training. Compared with GLM-5.2, it is much better at complex coding and long-horizon tasks: + Stronger Coding: GLM-5.3 is the most capable open-weights model for coding, with a 50% improvement over GLM-5.2 on our in-house Z.ai Code Bench. It also achieve open-source SOTA on public benchmarks including Terminal Bench 3.0 and Agents' Last Exam. + Emergent Cyber Capability: As we scaled post-training, cyber capability developed faster than we expected. GLM-5.3 is state of the art on CyberGym for vulnerability discovery, and its gains are largest further up the exploitation chain, where it more than doubles GLM-5.2 on exploitation benchmarks. GLM-5.3 supports deployment with the following frameworks. Feel free to try them out:
- punct_cap_seg_47_language↗
1-800-BAD-CODE
This model accepts as input lower-cased, unpunctuated, unsegmented text in 47 languages and performs punctuation restoration, true-casing (capitalization), and sentence boundary detection (segmentation). All languages are processed with the same algorithm with no need for language tags or language-specific branches in the graph. This includes continuous-script and non-continuous script languages, predicting language-specific punctuation, etc. This model is fun to play with, but the results could be better. I would recommend these newer, better models: * Better English model * Better Romance languages model * Better 47-language
- samantha-1.1-westlake-7b-laser-GGUF↗
dagbs
- gemma-4-12B-it-qat-w4a16-ct↗
Google
Hugging Face | GitHub | Launch Blog | Documentation | Technical Report > [!Note] > This model card is for the new versions of the Gemma 4 family optimized with Quantization-Aware Training (QAT), which allows preserving similar quality to bfloat16 while dramatically reducing the memory requirements to load the model. > Four versions of the QAT checkpoints are available: > * **Unquantized QAT checkpoints** (Q4_0): Half-precision weights extracted from the QAT pipeline, ideal for custom downstream compilation and research. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B, and their drafter models. > * **GGUF** (Q4_0): Ready-to-deploy formats for broad ecosystem compatibility. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B. > * **Mobile-optimized** (wNa8o8): A custom schema engineered explicitly for mobile hardware efficiency. It features targeted 2-bit decoding layers, optimized KV caches, and static activations to maximize VRAM savings. Available for Gemma 4 E2B and E4B. > * **Compressed Tensors** (w4a16): QAT checkpoints serialized in the compressed-tensors format for native, optimized inference with vLLM. Available for Gemma 4 E2B, E4B, 12B, and 31B. > * **Assistant Compatibility**: When using multi-token prediction (speculative decoding) with an assistant model alongside a QAT target model, the assistant model must also be a QAT checkpoint with the same precision to ensure compatibility. Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages.
- Kataguru-Sceptic-35B-A3B-v2.6-GGUF↗
kataguru
**Kataguru-Sceptic-35B-A3B-v2.6-GGUF** is the `llama.cpp` / LM Studio / Ollama GGUF release of **Kataguru-Sceptic-35B-A3B-v2.6**, a bilingual (English & Finnish) 35-billion parameter (3B active parameters per token, 254 routed experts per layer) multimodal Mixture-of-Experts (MoE) model. Unlike the 25.44 GB `NVFP4` and 27.85 GB `AWQ` server packages, these imatrix-calibrated 3-bit GGUF quantizations (`12.0 GiB` to `15.3 GiB`) are specifically engineered to fit **wholly inside the VRAM of a single 16 GB or 24 GB GPU**. 1. **Dual-Stage 2D Zero-Swap Epistemic Triage (System 1 Reflex)**: - Evaluates candidate training samples in ** 83.46 % | 81.72 % | **+1.74 pp** | 80.10 % | **+3.36 pp** | | 2 | **WinoGrande** | Contextual Reasoning (`acc`) | 75.45 % | 71.82 % | **+3.63 pp** | 71.10 % | **+4.35 pp** | | 3 | **MedQA (USMLE)** | Clinical Medicine 4-opts (`acc`) | 85.31 % | 84.10 % | **+1.21 pp** | 78.40 % | **+6.91 pp** | | 4 | **TruthfulQA MC2** | Epistemic Truthfulness (`acc`) | 52.47 % | 52.89 % | -0.42 pp | 49.80 % | **+2.67 pp** | | 5 | **TruthfulQA MC1** | Single-True Factuality (`acc`) | 36.23 % | 34.39 % | **+1.84 pp** | 31.40 % | **+4.83 pp** | | 6 | **ARC-Easy** | Standard STEM (`acc_norm`) | 82.95 % | 74.83 % | **+8.12 pp** | 72.98 % | **+9.97 pp** | | 7 | **HellaSwag** | Linguistic Coherence (`acc_norm`) | 82.42 % | 83.30 % | -0.88 pp | 74.60 % | **+7.82 pp** | | 8 | **OpenBookQA** | Scientific Application (`acc_norm`) | 45.00 % | 46.00 % | -1.00 pp | 44.80 % | **+0.20 pp** | | 9 | **ARC-Challenge** | Advanced STEM (`acc_norm`) | 62.54 % | 57.94 % | **+4.60 pp** | 58.62 % | **+3.92 pp** | | 10 | **BoolQ** | Natural Language Logic (`acc`) | 90.37 % | 85.78 % | **+4.59 pp** | 89.60 % | **+0.77 pp** | | 11 | **GSM8K Math** (CoT) | Exact Match (`strict` / `flex`) | 89.39 % / 90.07 % | 1.06 % / 12.36 %\* | **+88.33 pp** | 76.50 % | **+12.89 pp** | | **$\Sigma_{10}$** | **10-Task Mean (1–10)** | **Mean Excluding GSM8K Format Gap** | 69.62 % | **67.28 %** | **+2.34 pp** | **65.14 %** | **+4.48 pp** | | **$\Sigma_{11}$** | **11-Task Mean (1–11)** | **Official Overall Cognitive Mean** | 71.42 % | **61.26 % (62.28 % flex)** | **+10.16 pp** | **66.17 %** | **+5.25 p