Yue2-3B-GGUF

This repository contains audio.cpp-native GGUF weights for Yue2-3B. The model has been merged into the main branch. The demos below were generated with `yue2-3b-bf16.gguf` and `yue2-vae-f32.gguf`. The `_q8.wav` comparison files were generated with `yue2-3b-q8_0.gguf` and `yue2-vae-f16.gguf` using the same prompts and seeds. The `_q4_0.wav` comparison files were generated with `yue2-3b-q4_0.gguf` and `yue2-vae-f16.gguf`. Replace ` ` with your audio.cpp checkout and ` ` with this downloaded GGUF repo directory. To generate the Q4_0 versions, use the same commands and replace `--session-option yue2.model_gguf=yue2-3b-bf16.gguf` with `--session-option yue2.model_gguf=yue2-3b-q4_0.gguf`, and use `--session-option yue2.vae_gguf=yue2-vae-f16.gguf`. Measured with audio.cpp server mode on an RTX 5090, using the official `tonight-awake` longform example. Each run restarted the server and used one short warmup request before the measured longform request. Peak VRAM was sampled continuously during the measured request.