diffusiongemma-26B-A4B-it

DiffusionGemma is a generative model built by Google DeepMind. Based on the 26B A4B Mixture-of-Experts (MoE) Gemma 4 architecture, DiffusionGemma generates tokens using discrete diffusion. This open-weights model is multimodal, handling text, image, and video inputs to generate text output. Built on a MoE foundation, DiffusionGemma is designed to improve generation speed (tokens per second) while remaining deployable across various hardware environments. DiffusionGemma builds upon the architectural and capability advancements of Gemma 4, introducing several core features: * **Discrete Text Diffusion** – Shifts from token-by-token autoregression to block-autoregressive multi-canvas sampling. It generates text by iteratively denoising blocks of tokens (a 'canvas') in parallel, significantly increasing decoding speed. * **Multimodal Input Processing** – Processes interleaved text, image (with variable aspect ratio and resolution support), and video inputs to generate text outputs. * **Encoder-Decoder Architecture** – Utilizes an autoregressive encoder to process and cache the prompt context, paired with a decoder that applies bidirectional attention over the generation canvas. * **Mixture-of-Experts (MoE) Efficiency** – Leverages a sparse MoE design (8 active experts out of 128 total) to provide strong reasoning capabilities while maintaining a low memory footprint suitable for local execution. * **Thinking Mode (Reasoning)** – Designed as a highly capable reasoner, with configurable thinking modes. * **Optimized for Small Batch Size Inference –** Specifically engineered for low-latency, high-speed generation on a single capable accelerator. * **Native System Prompt Support** – As with Gemma 4, it supports updating the `system` role, enabling more structured and controllable conversations.