Ternary-Bonsai-8B-gguf

Prism ML Website  |  White Paper  |  Demo & Examples  |  Discord - **White Paper** - **Demo repo** — examples for serving, benchmarking, and integrating Bonsai - **Discord** — community support and updates - **Kernels**: Q2_0 is not yet in mainline `llama.cpp`. Use our fork at PrismML-Eng/llama.cpp (`prism` branch, default) which adds Q2_0 support for CPU (NEON/generic) and Metal. Upstream PR coming soon. Each weight takes a value from {-1, 0, +1}, with one shared FP16 scale per group of 128 weights:

번역 준비 중입니다. 원문을 표시합니다.