전체 검색
AI 모델 84768
- Ornith-1.5-35B-A3B-Abliterated-MLX-4bit↗
PocketAiHub
- ATPO-G-Qwen3-VL-8B-GGUF↗
prithivMLmods
- coreOCR-7B-050325-preview↗
prithivMLmods
- VisionGuardrail-9B-GGUF↗
prithivMLmods
- Qwen3.8-27B-Uncensored-oQ4e-mtp↗
pyros-vault
- Qwen3-4B-GGUF↗
QuantFactory
- Qwen3-Coder-16B-A3B-KREAP-MLX↗
RAS-LAB
- TDD↗
RED-AIGC
- Llama-3.2-3B-Instruct-FP8↗
RedHatAI
Quantized version of Llama-3.2-3B-Instruct. It achieves scores within 0.7% of the scores of the unquantized model for MMLU, ARC-Challenge, GSM-8k, Hellaswag, Winogrande and TruthfulQA. This model was obtained by quantizing the weights of Llama-3.2-3B-Instruct to FP8 data type. This optimization reduces the number of bits used to represent weights and activations from 16 to 8, reducing GPU memory requirements (by approximately 50%) and increasing matrix-multiply compute throughput (by approximately 2x). Weight quantization also reduces disk size requirements by approximately 50%. Only weights and activations of the linear operators within transformers blocks are quantized. Weights are quantized with a symmetric static per-channel scheme, where a fixed linear scaling factor is applied between FP8 and floating point representations for each output channel dimension. Activations are quantized with a symmetric per-tensor scheme, where a fixed linear scaling factor is applied between FP8 and floating point representations for the entire activation tensor. Weights are quantized by rounding to nearest FP8 representation. The llm-compressor library was applied to quantize the model, usin 512 sequences sequences taken from Neural Magic's LLM compression calibration dataset.
- NousResearch_-_Hermes-3-Llama-3.1-70B-gguf↗
RichardErkhov
