AI is happening. The future is updating.diffusiongemma-26B-A4B-it-GGUF
Jun 12 Update: You can now run DiffusionGemma in Unsloth Studio. See our Guide . You can also fine-tune DiffusionGemma in Unsloth . Inference settings are auto set. Example of DiffusionGemma (4-bit GGUF) running in Unsloth Studio with executable code: These GGUFs run with the DiffusionGemma build of `llama.cpp` (the DiffusionGemma PR ggml-org/llama.cpp#24423). DiffusionGemma is a block-diffusion architecture, so it needs that branch plus the dedicated `llama-diffusion-cli` runner - the standard `llama-cli` / `llama-server` cannot generate from it yet. `-n` / `--n-predict` is the only length knob you need: it auto-derives the diffusion block count and grows the batch/context to fit. On a single GPU the prompt prefix KV cache turns on automatically.
Jun 12 Update: You can now run DiffusionGemma in Unsloth Studio. See our Guide . You can also fine-tune DiffusionGemma in Unsloth . Inference settings are auto set. Example of DiffusionGemma (4-bit GGUF) running in Unsloth Studio with executable code: These GGUFs run with the DiffusionGemma build of `llama.cpp` (the DiffusionGemma PR ggml-org/llama.cpp#24423). DiffusionGemma is a block-diffusion architecture, so it needs that branch plus the dedicated `llama-diffusion-cli` runner - the standard `llama-cli` / `llama-server` cannot generate from it yet. `-n` / `--n-predict` is the only length knob you need: it auto-derives the diffusion block count and grows the batch/context to fit. On a single GPU the prompt prefix KV cache turns on automatically.