Google has unveiled DiffusionGemma, a new experimental AI model that generates text using diffusion rather than the autoregressive approach used by most large language models today. The company says the model can deliver up to four times faster text generation on dedicated GPUs while running on consumer hardware.

The model builds on Google’s Gemma 4 family and Gemini Diffusion research. Unlike traditional language models that generate text one token at a time from left to right, DiffusionGemma creates and refines blocks of text in parallel.

According to Google, the approach enables output speeds exceeding 1,000 tokens per second on an NVIDIA H100 GPU and more than 700 tokens per second on an NVIDIA GeForce RTX 5090.

The company says DiffusionGemma is aimed at developers working on speed-sensitive applications such as interactive editing, rapid content iteration, code infilling, and other workflows where low latency is more important than maximum output quality.

Breaking text bottlenecks

Most large language models generate text sequentially, predicting one token after another. While effective, this process can leave local hardware underutilized when serving a single user.

DiffusionGemma takes a different approach. Instead of generating text word by word, it creates a 256-token block at once and then repeatedly refines it through multiple passes.

Google compares the difference to moving from a typewriter to a printing press. Rather than waiting for each token to be generated before producing the next one, the model processes an entire section of text simultaneously.

The company says this shifts the bottleneck from memory bandwidth to compute performance, allowing modern GPUs to operate more efficiently during local inference.

Another key feature is bi-directional attention. Since the model generates text in parallel, every token can attend to every other token during generation. This makes it better suited for tasks where future context matters, such as code completion, in-line editing, mathematical structures, and biological sequences.

Google highlighted a demonstration in which DiffusionGemma was fine-tuned to solve Sudoku puzzles, a task that can be challenging for conventional autoregressive models because later tokens influence earlier decisions.

Designed for local AI

The model uses a 26-billion-parameter mixture-of-experts architecture but activates only 3.8 billion parameters during inference. According to Google, this allows the model to fit within roughly 18 GB of VRAM when quantized, making it accessible on high-end consumer GPUs.

DiffusionGemma also includes an iterative self-correction mechanism. Because it evaluates an entire text block during refinement, it can identify and fix mistakes as generation progresses.

However, Google acknowledged that the model prioritizes speed over quality. The company said standard Gemma 4 models remain the preferred choice for production environments where output quality is the primary concern.

The speed advantage is also most apparent in local deployments and low-concurrency environments. In cloud settings serving large numbers of users simultaneously, conventional autoregressive models can often utilize hardware efficiently through batching, reducing the benefits of diffusion-based generation.

Google has released DiffusionGemma under an Apache 2.0 license through Hugging Face and is supporting deployment through tools including MLX, vLLM, Hugging Face Transformers, NVIDIA NeMo, and Unsloth.

Recommended Articles