\uselogo # DiffusionGemma Technical Report ###### Abstract We introduce DiffusionGemma, an experimental open-weight language model that uses discrete diffusion to generate text at exceptionally high speed. Rather than decoding one token at a time, DiffusionGemma iteratively refines blocks of 256 tokens in parallel, avoiding the sequential decoding bottleneck of conventional autoregressive (AR) large language models. Instead of training from scratch, we obtain DiffusionGemma by fine-tuning the mixture-of-experts Gemma 4 model with 3.8B activated and 25.2B total parameters. Our compute-efficient two-stage training pipeline uses fewer than 10% of the starting AR model’s total training token budget. The first stage uses supervised fine-tuning to teach bidirectional denoising, while the second stage combines reinforcement learning with sampler distillation to jointly improve generation quality and inference efficiency. DiffusionGemma establishes a new Pareto frontier for the trade-off between generation speed and model capability. Averaged across our full evaluation suite, it generates around 20 tokens per forward pass and achieves roughly 1,500 output tokens per second on a single NVIDIA H100 GPU, which is substantially faster than AR models even with state-of-the-art speculative decoding. DiffusionGemma also retains the starting model’s support for thinking mode, multimodal inputs, and long contexts. Despite diffusion fine-tuning, it remains capable of AR generation with only minor performance degradation, suggesting a path toward hybrid diffusion-AR decoding. ## 1 Introduction Autoregressive (AR) models dominate the current Large Language Model (LLM) landscape, but their strict left-to-right, token-by-token generation creates a problematic memory bottleneck. When serving many requests simultaneously, one can achieve acceptable throughput by batching, but serving single or low-concurrency requests is fundamentally memory-bound: time spent transferring model weights and context KV cache from memory to the accelerator far exceeds time spent on actual computation. This leaves the accelerator’s compute units under-utilized and limits per-user generation speed. Speculative decoding can improve utilization by drafting a candidate sequence, typically 8 tokens, from a small “drafter” model and feeding the draft to the LLM for verification ( Leviathan et al., 2023 ; Chen et al., 2023a ; Xia et al., 2024 ) . With a draft length of 8 this can produce 3-6 tokens per forward pass (TPF) ( Li et al., 2026 ) . However, the draft-then-verify paradigm is still limited: on one hand, AR drafters are bottlenecked by sequential generation; on the other hand, parallel drafters—building on early blockwise decoding ( Stern et al., 2018 ) and lookahead strategies ( Zhao et al., 2024 ) —exhibit declining acceptance rates at later draft positions ( Cheng et al., 2026 ) . Text diffusion circumvents this bottleneck by predicting entire blocks of tokens simultaneously (Figure 22 ), effectively shifting execution from a memory-bound regime toward a compute-bound one. Recently, there has been a surge of interest in text diffusion ( Deschenaux and Gulcehre, 2024 ) , with models like Gemini Diffusion ( Google DeepMind, 2025 ) , Mercury ( Inception Labs et al., 2025 ) , LLaDA ( Nie et al., 2026 ; Bie et al., 2025 ) , Seed Diffusion ( Song et al., 2025 ) , and Nemotron-Labs-Diffusion ( Fu et al., 2026 ) . However, the current landscape forces a stark compromise between speed, intelligence, and accessibility. Some models, e.g., Gemini Diffusion and Mercury, are locked behind proprietary APIs. On the other hand, existing open-weights alternatives either exhibit limited reasoning capabilities and multimodal understanding, or fail to deliver on the extreme latency benefits promised by the diffusion technology, or both. Until now, there has been no text diffusion model that is highly intelligent, exceptionally fast, and openly accessible. We introduce DiffusionGemma to bridge this gap. A finetuned text diffusion variant of the Gemma 4 26B A4B mixture-of-experts (MoE) model ( Gemma Team et al., 2026 ) , the model establishes a new Pareto frontier in the intelligence-to-speed trade-off for generative text modeling (Figure 1 ). Notably, it extends the speed frontier beyond both prior text diffusion models and the Gemma 4 AR family across all parameter scales (from E2B up to 31B), even when the AR models are equipped with multi-token prediction ( Google DeepMind, 2026 ; DeepSeek-AI, 2024 , MTP,) , a state-of-the-art speculative decoding technique. To unlock this new speed-to-intelligence frontier, DiffusionGemma maximizes algorithmic efficiency by outputting blocks of 256 tokens simultaneously in around 12 forward passes. Put differently, DiffusionGemma generates on average 20 TPF—a step-change improvement over the ∼ \sim 3-6 TPF that can be achieved by state-of-the-art speculative decoding methods. This massive reduction in total forward passes directly offsets the computational cost of the diffusion model’s heavier individual forward passes and as a result, DiffusionGemma yields generation speeds of around 1,500 tokens per second (TPS) on a single NVIDIA H100 GPU. Furthermore, early benchmarking by third-party inference providers such as Unsloth (2026) demonstrates these speeds can scale up to 2,000 TPS on the NVIDIA RTX 6000. To avoid the prohibitive computational cost of pretraining it is common practice to initialize text diffusion models from pretrained AR models ( Han et al., 2024 ; Gong et al., 2025 ) . We warm-start DiffusionGemma from the final post-trained, publicly released weights of the Gemma 4 26B A4B MoE model ( Gemma Team et al., 2026 ) . By adopting the same transformer backbone, we efficiently repurpose the AR weights to support both causal attention for context encoding and bidirectional attention for the diffusion process while inheriting their full capabilities. We employ a two-stage training pipeline (see Figure 2 ) that utilizes less than 10% of the AR model’s total training tokens: topsep=0pt Supervised fine-tuning (SFT) : We first run an SFT phase to adapt the model to attend to a context of clean tokens as well as denoising a block of 256 noisy tokens (as dictated by the diffusion process) with bidirectional attention across the block. topsep=0pt Sampler distillation and reinforcement learning ( SD ⋅ \cdot RL ) : We then apply an online learning phase that simultaneously improves generation quality (by maximizing rewards) and unlocks ultra-low latency (by substantially reducing the number of forward passes). By preserving features of its AR starting point, DiffusionGemma exhibits strong multimodal understanding, long-context capabilities, and thinking mode, enabling it to generate reasoning traces prior to responding. These are hallmarks of current frontier systems ( Gemini Team, 2025 ) . Although adapting the model to the diffusion regime introduces a performance penalty compared to the original AR baseline, the massive gains in inference speed offer an entirely new operating point for latency-critical applications. DiffusionGemma retains the ability to generate text autoregressively. This dual mode opens up the possibility to route requests dynamically based on latency constraints and task complexity, as well as to employ hybrid decoding approaches. ##### Open-weights release. We release DiffusionGemma with open weights under a permissive Apache 2.0 license, aiming to democratize access to state-of-the-art text diffusion technology. By providing full, unrestricted access to the model parameters, we hope to empower a diverse ecosystem of researchers, developers, and practitioners. For the research community, this transparent release provides a robust, white-box baseline to deeply probe the underlying mechanics of discrete diffusion, accelerating foundational research and pushing the theoretical frontier of text generation. For developers, the liberal licensing removes friction, ensuring seamless integration into both experimental prototypes and commercial applications without restrictive usage barriers. Crucially, DiffusionGemma’s highly efficient architecture makes it an ideal foundation for rapid, domain-specific adaptation. By dramatically lowering the computational barrier to entry, the community can easily finetune ultra-fast, task-specific models tailored to their own unique use cases, even in environments with constrained compute budgets. This capacity for lightweight, rapid iteration has catalyzed immediate community adoption. Despite the model being available for only a few weeks at the time of writing, it is already serving as the engine for specialized downstream applications. These rapidly emerging use cases showcase the model’s versatility across highly diverse and demanding domains, spanning from multilingual automatic speech recognition ( Interfaze, 2026 ) to interactive radiology report drafting in healthcare ( Van Puyvelde et al., 2026 ) . ##### Outline. The remainder of this technical report is organized as follows. Section 2 formalizes the discrete diffusion framework and the theoretical foundations of our approach. Section 3 details the DiffusionGemma architecture, including the bidirectional decoding mechanism and sampling algorithm. We then describe the two stages of our training pipeline: the SFT phase in Section 4 , followed by our online learning phase combining sampler distillation and reinforcement learning in Section 5 . Section 6 breaks down our low-level inference optimizations. Section 7 presents experimental results. Section 8 provides instructions and a practical example of finetuning DiffusionGemma for downstream applications. In Section 9 , we showcase some practical advantages of text diffusion. Finally, we discuss limitations and known issues in Section 10 , before concluding in Section 11 . ## 2 Generative Text Modeling with Discrete Diffusion Historically, LLMs have treated text generation as a strictly sequential process ( Bengio et al., 2003 ; Mikolov et al., 2010 ; Graves, 2013 ; Sutskever et al., 2014 ; Vaswani et al., 2017 ; Gu et al., 2018 ) . Classical AR language modeling factorizes the joint probability of a sequence x x of length L L into an exact product of conditional probabilities: p ⁡ ( x ) = ∏ i = 1 L p ⁡ ( x i ∣ x < i ) p(x)=\prod_{i=1}^{L}p(x^{i}\mid x^{ 0 \tau_{t}>0 can sharpen the model’s predictions before we calculate the final transition probabilities. A transition mapping Step \operatorname{Step} , as described in Equation ( 2 ), computes the categorical distribution used to sample the updated canvas x t − Δ ​ t ∈ 𝒱 C x_{t-\Delta t}\in\mathcal{V}^{C} , which, along with the new self-conditioning signal z t − Δ ​ t z_{t-\Delta t} , is then fed into the next denoising step. Section 3.3 details our specific choices for transition mapping Step \operatorname{Step} and the temperature τ t \tau_{t} . ##### Multinomial diffusion. Our denoising iteration (Equation 6 ) uses multinomial (or uniform) diffusion rather than masked diffusion ( Hoogeboom et al., 2021 ; Austin et al., 2021 ) . Because all tokens can transition between one another, the model can continuously correct its own errors: tokens accepted during earlier denoising steps ( t ′ > t t^{\prime}>t ) within the current canvas can still be revised. However, we note that tokens from previously generated canvases are permanently frozen. ##### Remark on interpretability. Recent interpretability analyses demonstrate that while the self-conditioning signal z t z_{t} introduces a continuous latent space injection into the sequence, these intermediate self-conditioning vectors map robustly to an interpretable token bottleneck, preserving the model’s algorithmic transparency ( Engels et al., 2026 ; Asaria et al., 2026 ) . ### 3.3 The DiffusionGemma Sampler In contrast to AR generation, which relies on rigid heuristics like temperature, top- p p , and top- k k , diffusion modeling introduces a vastly richer sampling design space that includes discrete predictor-corrector mechanisms and multi-step solvers ( Lezama et al., 2023 ; Deschenaux et al., 2026 ; Liu et al., 2026a ; Yao et al., 2026 ; Ren et al., 2025 ) . Framing text generation as an iterative temporal process allows us to decouple the inference-time algorithm from the underlying architecture, granting fine-grained control over the trade-off between computational cost and generation quality. While we described the overall approach of canvas denoising in Section 3.2 , this subsection details a specific instantiation: the entropy-bounded sampler ( Ben-Hamu et al., 2026 ) with temperature annealing and adaptive stopping (Algorithm 1 ). While this is our default and recommended sampler, DiffusionGemma is modular and is not strictly bound to this exact sampling configuration for high-quality inference. ##### Entropy-bounded token refinement. After producing the marginal distributions p θ ​ ( x 0 ∣ x t , z t , H ) p_{\theta}(x_{0}\mid x_{t},z_{t},H) with the denoiser and applying temperature scaling (detailed below), tokens are sampled from the resulting probability distribution. We employ an entropy-bounded sampler ( Ben-Hamu et al., 2026 ) , whereby tokens are accepted in rank order from lowest to highest entropy (similar to the MaskGIT decoding scheme by Chang et al. (2022) ), ensuring that their mutual information bound remains strictly below a predefined error tolerance threshold ( b = 0.1 b=0.1 ). Once the threshold is attained, all other tokens are renoised uniformly at random, maintaining these uncommitted positions as a uniform prior to force local exploration during the next forward pass. ##### Temperature annealing. To balance rate of convergence and linguistic diversity, token probabilities are artificially sharpened via tempering. A temperature τ t < 1 \tau_{t}<1 is annealed linearly from an initial value of τ max = 0.8 \tau_{\max}=0.8 down to τ min = 0.4 \tau_{\min}=0.4 across the fractional denoising timescale t ∈ [ 0 , 1 ] t\in[0,1] . While static temperature adjustments are traditionally used to truncate the unreliable tail of token distributions and prevent text degeneration, this dynamic annealing ensures that the model explores diverse token possibilities in early, highly-noised states and aggressively commits to high-confidence sequences as the semantic structure crystallizes ( Zhang et al., 2024 ) . ##### Adaptive stopping heuristic. To optimize inference efficiency, the sampler dynamically halts the denoising process based on the model’s step-wise uncertainty, strictly capping iterations at N N . This early termination is triggered when two conditions are simultaneously satisfied: topsep=0pt Confident predictions: The mean predictive entropy across the entire canvas falls below a predefined threshold ( e stop = 0.005 e_{\mathrm{stop}}=0.005 ). topsep=0pt Stable predictions: The deterministic sequence predictions (i.e., the most likely tokens) from two consecutive denoising steps are identical. By successfully bypassing redundant refinement steps, this mechanism allows the model to dynamically scale its inference-time compute to the complexity of the prompt. As illustrated in Figure 5 , instead of exhausting the maximum N = 48 N=48 budget, the model averages approximately 12 12 effective denoising steps (defined in Section 3.4 ) across the downstream evaluations shown in the figure. This yields a 4 × 4\times reduction in overall latency without sacrificing generation quality. Furthermore, the boxplots reveal distinct convergence profiles across domains: the model tends to deploy fewer steps for structured tasks like code and more for natural language. Moreover, it is not only the domain, but also the task complexity that dictates this behavior; for instance, harder code problems (e.g., LiveCodeBench) naturally require more steps to converge than easier ones (e.g., HumanEval). ### 3.4 Inference Efficiency Metrics Because adaptive stopping turns the number of denoising steps into a random variable, we introduce a set of metrics to quantify denoising inference efficiency. Let K K be the total number of canvases generated, N k ≤ N N_{k}\leq N be the number of denoising steps executed for the k k -th canvas, and C k ≤ C C_{k}\leq C be the number of valid tokens produced in that canvas (i.e., all C C tokens, or up to the first end-of-sequence token if one is generated). First, we define the Total Tokens as the sum of all valid tokens generated across the entire sequence: Total Tokens ≜ ∑ k = 1 K C k . \text{Total Tokens}\triangleq\sum_{k=1}^{K}C_{k}. (7) We define the Total Denoising Steps as the absolute number of denoising steps executed across the entire generation: Total Denoising Steps ≜ ∑ k = 1 K N k . \text{Total Denoising Steps}\triangleq\sum_{k=1}^{K}N_{k}. (8) Since each denoising step incurs a fixed computational cost, the Total Denoising Steps is the primary driver of end-to-end generation latency. However, because this absolute metric inherently scales with the total sequence length, we also evaluate a normalized measure of model efficiency. We define the Effective Denoising Steps as the token-weighted average of steps across all canvases: Effective Denoising Steps ≜ 1 Total Tokens ​ ∑ k = 1 K N k ​ C k . \text{Effective Denoising Steps}\triangleq\frac{1}{\text{Total Tokens}}\sum_{k=1}^{K}N_{k}C_{k}. (9) We weight by the number of valid tokens to prevent the metric from being downwardly biased by the final canvas, which tends to require fewer denoising steps when it is only partially filled. Note that without adaptive stopping, the Effective Denoising Steps metric always reduces to the denoising budget N N . Finally, we calculate the Tokens Per Forward (TPF) : TPF ≜ Total Tokens Total Denoising Steps + K − 1 , \text{TPF}\triangleq\frac{\text{Total Tokens}}{\text{Total Denoising Steps}+K-1}, (10) where the K − 1 K-1 term accounts for the single additional forward pass required between canvases to encode the newly generated clean tokens and append their key-value pairs to the KV cache. ### 3.5 Retained Autoregressive Capability Because DiffusionGemma shares the exact same transformer architecture as Gemma 4, the final DiffusionGemma weights can be seamlessly loaded back into the original architecture to perform standard AR generation using causal attention, exactly as the base model does. As demonstrated in Section 7 , Table 3 , the model maintains robust capabilities in this setting; its performance scores in AR mode land squarely between those of DiffusionGemma’s primary text diffusion mode and the baseline Gemma 4 checkpoint from which DiffusionGemma was initialized. ## 4 Supervised Finetuning We start from the publicly released Gemma 4 26B A4B ( Gemma Team et al., 2026 ) checkpoint and run an extended finetuning phase where the model adapts to predicting blocks of 256 tokens from noisy inputs. We use a block-diagonal attention mask, enabling bidirectional attention within each block without allowing the model to condition on other denoising blocks. For a given canvas, the model conditions on the prompt and previous (uncorrupted) tokens via the encoder KV cache. We use discrete multinomial diffusion as our corruption process and uniformly sample noisy tokens from the vocabulary ( Hoogeboom et al., 2021 ; Austin et al., 2021 ) . For a given canvas, we sample a noise level t ∼ Unif ⁡ [ 0 , 1 ] t\sim\operatorname{Unif}[0,1] and noise each token in the canvas with probability t t . Given a clean context of prompt and previous canvases H H (encoded through the KV cache), a self-conditioning signal z t z_{t} , and a noisy canvas x t x_{t} , the model is trained to minimize the cross-entropy loss between its predictions and the ground-truth canvas extracted from the training data ( Gat et al., 2024 ) : L ( θ ) = − ∑ i = 1 C log p θ ( x 0 i ∣ x t , z t , H ) , L(\theta)=-\sum_{i=1}^{C}\log p_{\theta}(x_{0}^{i}\mid x_{t},z_{t},H), (11) where superscript i i denotes indexing along the canvas dimension; p θ p_{\theta} is parameterized via a softmax transformation over the neural network’s output logits. As shown in Figures 6 and 7 , denoising performance improves rapidly within the initial steps of training, after which it settles into a log-linear performance improvement trend. Thinking performance benefits from extended SFT, as the model initially struggles with maintaining coherent reasoning traces, often collapsing into stuttering or cycles—a common challenge for internalizing reasoning in language models ( Zelikman et al., 2024 ) . ## 5 Sampler Distillation & Reinforcement Learning Following the SFT stage, the model achieves strong generation quality when using a high number of denoising steps. However, its performance on advanced reasoning and coding tasks is somewhat poorer than the baseline AR model. More critically, when operating in the few-step regime required for ultra-low latency inference, generation quality collapses. To address this, we target a dual improvement: pushing the model’s intelligence while simultaneously compressing its denoising trajectory. Traditionally, achieving these two goals requires a decoupled, multi-stage pipeline that treats reward-driven alignment and sampler distillation as distinct phases. We bypass this with a unified online learning stage, coined sampler distillation & reinforcement learning ( SD ⋅ \cdot RL ), which optimizes both axes concurrently. Relying on a joint objective, a single gradient update drives: topsep=0pt Reward maximization : Elevating absolute generation quality and alignment, analogous to standard RL for AR and diffusion models ( Black et al., 2024 ; Liu et al., 2025 ; Zheng et al., 2026 ; Zhao et al., 2026 ; Ma et al., 2026 ; Wallace et al., 2024 ; Fan et al., 2023 ; Clark et al., 2024 ; Dong et al., 2023 ; Lee et al., 2023 ) . topsep=0pt Sampler distillation : Mapping this high-quality generation to the few-step regime, addressing a compression challenge unique to iterative diffusion frameworks ( Salimans and Ho, 2022 ; Song et al., 2023 ; Deschenaux and Gulcehre, 2025 ; Fu et al., 2025 ; Luo et al., 2023 ; Sauer et al., 2024 ; Liu et al., 2024 ; Yin et al., 2024 ; Hoogeboom et al., 2026 ) . ##### Training setup. We adapt the data distribution used by the Gemma 4 RL recipe ( Gemma Team et al., 2026 ) for our use case. Encompassing both thinking and non-thinking modes, our setup targets improvements across a variety of capabilities such as helpfulness, mathematical reasoning, coding and instruction-following. Initialized from the SFT weights, the model acts as an online teacher that generates denoising trajectories (using a sampler configured with high maximum denoising steps and mild temperature annealing) to establish a high-quality reference. The SD ⋅ \cdot RL joint objective uses these trajectories to simultaneously maximize reward and drive sampler distillation in order to compress the model’s highest-quality output into the few-step regime. ##### Training dynamics & implicit curriculum effect. Over the course of SD ⋅ \cdot RL training, two synergistic dynamics emerge, as shown in Figure 8 . First, the online teacher’s average reward steadily increases, reflecting improved fundamental capabilities. Second, facilitated by the adaptive stopping mechanism, the online teacher progressively requires fewer effective denoising steps to achieve these high rewards. This acceleration occurs because the SD ⋅ \cdot RL objective systematically reduces the predictive entropy of the model. Crucially, the interplay between the reward objective and adaptive stopping induces a curriculum learning effect. Early in training, high predictive entropy delays the adaptive stopping trigger. As the model’s confidence improves and entropy drops, adaptive stopping triggers earlier. This seamlessly shifts the training distribution toward ever shorter denoising trajectories, allowing the algorithm to dynamically pace its own sampler distillation. Consequently, and in contrast to RL for AR models, prolonging the SD ⋅ \cdot RL phase remains highly beneficial even after the reward metric plateaus; continued entropy reduction translates directly into further inference speedups. ##### Improved speed-to-intelligence Pareto frontier. As illustrated in Figure 9 , SD ⋅ \cdot RL training significantly expands the Pareto frontier established by the SFT checkpoint. On the quality axis, it yields a 10-point improvement on the combined GPQA-Diamond and LiveCodeBench-v6 score. On the efficiency axis, it quadruples the TPF from 5 to nearly 20, unlocking ultra-low latency inference. As shown in Figure 9 , evaluating the SFT checkpoint with the default DiffusionGemma sampler (maximum of N = 48 N=48 denoising steps) results in poor downstream accuracies and artificially low effective denoising steps. This counterintuitive behavior stems from the SFT model frequently degenerating into repetitive token loops in this restricted-step regime. Once it falls into a loop, its predictive entropy collapses, which prematurely triggers the adaptive stopping mechanism. Figures 16 and 17 (Appendix B ) provide generated samples on the GPQA-Diamond benchmark that illustrate how these degeneracies manifest: the SFT model begins with a valid, logical reasoning trace but suddenly collapses into a token repetition loop. In contrast, the samples post- SD ⋅ \cdot RL avoid these degenerative traps to sustain coherent reasoning at minimal latency. ##### SD ⋅ \cdot RL specialization in the few-step regime. For the SFT checkpoint, downstream performance scales consistently with the maximum number of denoising steps N N up to 192, at which point denoising is roughly any-order autoregressive. Following SD ⋅ \cdot RL , the scaling behavior improves faster in N N and plateaus earlier: performance improves steadily up to N = 48 N=48 , but exhibits diminishing returns thereafter (Figure 10 ). This early saturation emerges because the SD ⋅ \cdot RL objective explicitly specializes the model for the few-step regime by aggressively minimizing predictive entropy. ##### Emergent conciseness. An emergent property of our SD ⋅ \cdot RL optimization is that it encourages the model to produce concise, token-efficient outputs. This contrasts with RL for AR models, which often maximizes reward by inducing longer reasoning traces. Our final checkpoint produces generations nearly 2 × 2\times shorter than the SFT checkpoint (Figure 9 ). While this means the model forgoes some capability gains typically associated with extended reasoning, it acts as a multiplier on inference speed gains. Compounding fewer total tokens with fewer effective denoising steps per canvas directly drives exceptionally low end-to-end latency: DiffusionGemma requires less than 5 % 5\% of the total forward passes used by the Gemma 4 AR baseline across our eval suite. ## 6 Inference Optimizations The preceding sections focused on maximizing the number of tokens produced per forward pass through our SFT and SD ⋅ \cdot RL pipeline. We now turn to the complementary axis: minimizing the wall-clock cost of each forward pass through targeted GPU-level optimizations. Fundamentally, the throughput advantage of text diffusion over AR decoding is dictated by two competing quantities: on one hand, each denoising step decodes TPF \mathrm{TPF} tokens, and thus requires TPF \mathrm{TPF} times fewer forward passes than an AR baseline. On the other hand, because each forward pass processes a canvas of 256 tokens, each such step is r r times slower, and so the overall throughput relative to the AR baseline becomes TPF / r \mathrm{TPF}/r . The forward pass overhead can be substantially reduced by hardware optimizations. ##### Low batch size serving. While text diffusion inference requires more floating-point operations (FLOPs) per generated token than AR models, it relies on significantly fewer forward passes. Because LLM serving on modern hardware accelerators is typically memory-bound—largely dictated by KV cache capacity and memory bandwidth ( Kwon et al., 2023 ; Pope et al., 2023 ) —this reduction in memory transfers creates a latency advantage that outweighs the higher computational cost. As a result, text diffusion is highly effective in low-batch-size scenarios, leveraging available compute capacity to minimize per-request latency. For simplicity, our analysis focuses on the single-request inference throughput (a batch size of 1) of DiffusionGemma, comparing it directly to its AR counterpart, Gemma 4 26B A4B. Reference inference implementations are available in HuggingFace Transformers ( Google DeepMind Team, 2026 ) and vLLM ( The vLLM Team and Google DeepMind Team, 2026 ) . ##### GPU time breakdown. Figure 11 presents the per-step GPU kernel time breakdown for both models; we focus on GPU kernel time rather than end-to-end latency to isolate model-specific bottlenecks. For each model individually, end-to-end latency exceeds GPU kernel time by approximately 1 ms due to detokenization and other CPU-side serving overhead, with this gap being similar for both the Gemma 4 AR and DiffusionGemma models. Each DiffusionGemma step processes 256 × 256\times more tokens, yet is only 3.2 × 3.2\times slower than the single-token AR step. The time difference can be mainly attributed to three operations: mixture-of-experts (MoE), sampling, and attention. Other operations (e.g., shared expert, attention output projection, etc.) are at most 2 × 2\times slower. topsep=0pt MoE. When serving a single request, the MoE layer computation is memory-bound, with its running time dominated by the transfer of expert weights from high-bandwidth memory, a well-documented bottleneck when serving sparse MoE models ( Rajbhandari et al., 2022 ; Huang et al., 2024 ) . For the Gemma 4 AR model, only 8 unique experts are activated per token per MoE layer. For the DiffusionGemma model, on average, approximately 84 unique experts are activated per canvas of 256 tokens per MoE layer, as measured on the PG-19 benchmark ( Rae et al., 2020 ) . 4 4 4 Dataset available at https://github.com/google-deepmind/pg19 . Activating more experts per forward pass results in a 4.3 × 4.3\times slower MoE kernel. It is the same type of penalty incurred by verification in speculative decoding, but scaled to larger token parallelism. For a dense architecture, this overhead would be eliminated, reducing the per-step feed-forward network slowdown to less than 2 × 2\times . topsep=0pt Sampling. Beyond AR’s single-token softmax-and-sample, text diffusion sampling requires additional operations, most notably a self-conditioning embedding matmul and softmax over the full canvas of 256 tokens (see Algorithm 1 ). These operations are performed in full precision with a vocabulary dimension of 262k. Consequently, text diffusion sampling takes 3.06ms while AR sampling takes only 0.56ms. We implement sampling using standard PyTorch primitives optimized with torch.compile , rather than hand-written GPU kernels, to facilitate easier extensibility by the community. topsep=0pt Attention. Unlike AR, DiffusionGemma uses bidirectional attention over a canvas of 256 tokens. As such, we cannot utilize the fast single-token decoding attention available to AR models, but we can take advantage of the highly-optimized FlashAttention-4 kernel ( Dao et al., 2022 ; Zadouri et al., 2026 ) . Under these optimizations, the attention operation is 4 × 4\times slower for the DiffusionGemma model than the Gemma 4 AR model. ##### Eliminating CPU-GPU synchronization. There is additional complexity when serving DiffusionGemma: denoising steps depend on adaptive stopping and there are two types of forward passes (the denoising step and the KV cache update) that require different attention masks and that can occur within the same batch. To minimize request latency, it is important that this complexity is handled solely by GPU operations without triggering any additional CPU-GPU synchronization. This is achieved by extending asynchronous scheduling to the text diffusion models and introducing a per-sequence causal attention flag (see The vLLM Team and Google DeepMind Team, 2026 for more details). ##### Throughput for batch size of 1 serving. The decoding throughput for text diffusion models is calculated by the following formula: TPS ≜ TPF t fwd , \text{TPS}\triangleq\dfrac{\text{TPF}}{t_{\text{fwd}}}, (12) where TPS is Tokens Per Second, TPF is Tokens Per Forward (see Equation 10 ), and t fwd t_{\text{fwd}} is the time needed for a single denoising step, which varies with context length (due to attention). On an H100 GPU (FP8 precision), a single denoising step of DiffusionGemma takes t fwd = 13.56 ​ ms t_{\text{fwd}}=13.56\text{ms} on average (end-to-end; the per-step GPU time is 12.63ms, see Figure 11 ) when serving a single request with 4096 input tokens and 1024 output tokens. TPF varies depending on the task. Assuming a TPF of 19.74 (the average TPF across the 7 benchmarks reported in Table 3 ), the average decoding throughput of the model is 1456 TPS—a 7.1 × 7.1\times improvement over the Gemma 4 AR model (204 TPS) and a 4.8 × 4.8\times improvement over the AR model with MTP (303 TPS), on the same device setup. ##### Multi-user throughput. Figure 12 shows the trade-off between total and per-user throughput of the Gemma 4 AR model (with MTP) and DiffusionGemma depending on the number of concurrent users (i.e., batch size). Importantly, these results are without targeted optimization for batch sizes larger than 1: current kernel selection is suboptimal, and sampling has not been tuned to scale with batch size; for example, applying top- k k truncation to the sampling step is expected to yield significant throughput gains at higher batch sizes with negligible impact on output quality. Despite this, DiffusionGemma offers substantially higher TPS per user and higher total throughput than the Gemma 4 AR (MTP) model in the low batch size regime, with AR models beginning to gain a throughput advantage only at moderate batch sizes (around 32 concurrent requests). ##### Toward real traffic throughput. Compared to the AR equivalent, DiffusionGemma changes the compute characteristics of two transformer layers: for the attention layer, it performs TPF × \times fewer transfers of KV cache; for the MoE layer, it performs proportionally more FLOPs (scaling with the number of effective denoising steps). This has the potential to address one of the challenges in modern LLM serving: memory-bound attention limiting compute utilization in FFW/MoE layers ( Zhu et al., 2025a ; Tang et al., 2024 ; Zhu et al., 2025b ), which is particularly relevant for agentic workflows with long context. DiffusionGemma effectively trades data movement for compute, which is favorable on modern GPU hardware where compute-to-bandwidth ratios continue to grow; a thorough empirical analysis under realistic traffic conditions is beyond the scope of this work. ## 7 Experimental Results We evaluate DiffusionGemma’s capabilities and inference efficiency across four operational modes: text diffusion (TD) versus autoregressive (AR) generation, each evaluated with and without thinking enabled; unless otherwise specified, TD with thinking is enabled. We benchmark the model against its AR initialization (Gemma 4 26B A4B), contemporary open-weight text diffusion models (LLaDA 2.1 Flash 100B and Nemotron Diffusion 14B), and the proprietary Mercury 2 API. We consider a diverse suite of benchmarks spanning core domains such as mathematical reasoning, code generation, general knowledge, multimodal understanding, and instruction following alongside agentic capabilities. Specifically, for mathematical reasoning, we utilize AIME ( Dekoninck et al., 2026 ) , GSM8K ( Cobbe et al., 2021 ) , MGSM ( Shi et al., 2023 ) , Putnam ( Tsoukalas et al., 2024 ) , and HiddenMath (internal). Coding performance is measured against LiveCodeBench-v6 ( Jain et al., 2025 ) , Codeforces ( Quan et al., 2025 ) , HumanEval ( Chen et al., 2021 ) , BigCodeBench ( Zhuo et al., 2025 ) , LBPP (v2) ( Matton et al., 2024 ) , and Natural2Code (internal). Broad and expert-level general knowledge is measured against GPQA-Diamond ( Rein et al., 2024 ) , BIG-Bench ( Suzgun et al., 2023 ) , MMMLU ( OpenAI, 2024 ) , and MMLU-Pro ( Wang et al., 2024 ) , while multimodal reasoning is evaluated on MMMU-Pro ( Yue et al., 2025 ) . Finally, strict instruction following is assessed via IFEval ( Zhou et al., 2023 ) , and agentic task completion is evaluated through the Tau-bench suite, encompassing the Retail, Airline, and Telecom environments ( Yao et al., 2024 ) . The complete per-benchmark results for all models and inference configurations are reported in Table 3 . Table 4 provides a complementary analysis of DiffusionGemma’s decoding efficiency, reporting Tokens Per Forward (TPF; Equation 10 ), Tokens Per Second (TPS; Equation 12 ), effective denoising steps (Equation 9 ), total generated tokens (Equation 7 ), and end-to-end generation latency, excluding prefill time. For a higher-level comparison, Figure 13 groups the benchmarks into three capability areas—reasoning and knowledge, coding, and instruction following and agentic behaviour—and reports the unweighted mean of the 0–100 benchmark scores within each area, together with output throughput. A model is shown for a capability area only if it completed every constituent benchmark; missing bars therefore indicate incomplete benchmark coverage rather than a score of zero. Hatched bars denote the no-think variants. DiffusionGemma sets a new performance frontier for text diffusion models, substantially outperforming existing open-weight diffusion baselines while increasing TPF by approximately an order of magnitude. In terms of quality it is highly competitive with Mercury 2, a closed-weight text-diffusion model, while reaching roughly 1,500 output tokens per second on a single previous-generation H100 GPU, a roughly 2.5 × 2.5\times speedup over Mercury 2. Relative to the AR model used for initialization, DiffusionGemma in text-diffusion (TD) mode trades some absolute benchmark performance for substantially greater decoding speed. Although the conversion reduces performance across the three capability areas, TD mode delivers nearly 5 × 5\times the output throughput of the original Gemma 4 AR baseline under heavily optimized MTP serving: 1,479 tokens per second compared with 303 tokens per second. The two-stage training pipeline nevertheless retains support for autoregressive decoding. When run in standard left-to-right AR mode, DiffusionGemma recovers part of the performance gap observed in TD mode and narrows the capability gap to the original baseline, albeit at lower throughput. This dual-mode capability could enable requests to be routed dynamically according to latency requirements and task complexity. * Benchmarks are grouped into: reasoning and knowledge ( AIME 2026 , GPQA Diamond , BigBench EH , GSM8K , MGSM , MMMLU , Putnam , MMLU-Pro , and HiddenMath ); coding ( LiveCodeBench V6 , HumanEval , BigCodeBench , LBPP , and Natural2Code ); and instruction following and agentic behaviour ( IFEval , Tau2 Retail , Tau2 Airline , and Tau2 Telecom ). ## 8 Open-Source Downstream SFT Alongside DiffusionGemma, we release an open-source finetuning toolkit that allows practitioners to adapt the model to their own domain-specific datasets. We build on top of Hackable Diffusion ( Crepy et al., 2026 ) , a modular open-sourced research toolbox for generative modeling. We provide Low-Rank Adaptation (LoRA) recipes ( Hu et al., 2022 ) to enable finetuning on consumer hardware. ##### Finetuning procedure. Our open-source SFT toolkit includes both a causal encoder and a diffusion decoder objective. Training sequences are of length P + K ​ C P+KC where P P is the number of prompt tokens, K K is the number of canvases and C C is the canvas size. To compute the total loss, the encoder first processes all tokens in the entire sequence, populates a KV cache H H , and provides next token predictions that are fed into a standard cross-entropy loss for the encoder. The decoder loss is computed by uniformly sampling a canvas k k from the K K canvases available. For canvas k k , we evaluate a denoising cross-entropy loss using the decoder’s predicted logits given the current noisy state, x t x_{t} , the KV cache H H for the prompt and any prior canvases in the sequence (i.e., canvas 1 1 , 2 2 , …, k − 1 k-1 ); for 50 % 50\% of the data-points in a batch, the decoder is also conditioned on a self-conditioning state z t z_{t} computed from a previous forward pass. The other 50 % 50\% have z t = 0 z_{t}=\textbf{0} . Formally, we write the losses as L encoder ( θ ) = − 1 P + K ​ C ∑ j = 1 P + K ​ C log p θ ( x j ∣ x 1 : j − 1 ) , L decoder ( θ ) = − 1 C ∑ i = 1 C log p θ ( x 0 i ∣ x t , z t , H ) . L_{\text{encoder}}(\theta)=-\frac{1}{P+KC}\sum_{j=1}^{P+KC}\log p_{\theta}(x^{j}\mid x^{1:j-1}),\quad\quad L_{\text{decoder}}(\theta)=-\frac{1}{C}\sum_{i=1}^{C}\log p_{\theta}(x_{0}^{i}\mid x_{t},z_{t},H). (13) For the encoder loss, j j indexes along all tokens in the sequence (prompt and canvases); for the decoder loss, i i indexes along tokens in the given canvas. The final loss is the sum of the two losses. ##### LoRA for parameter-efficient finetuning. LoRA is applied to all linear operations (attention projections, MLP gates, MoE routers, and the self-conditioning feedforward block). This allows us to achieve strong downstream performance while training only a small fraction of the model’s parameters, using 2 × \times A100 80GB GPUs. All of our training details are reported in Appendix C . ##### Case study: Sudoku puzzle solving. Solving Sudoku puzzles is a compelling testbed for discrete diffusion due to the non-autoregressive nature of the task. We finetune DiffusionGemma on an open-source dataset of Sudoku puzzles. 5 5 5 https://www.kaggle.com/datasets/rohanrao/sudoku With full finetuning, the sampler (Algorithm 1 ) achieves >85% puzzle-level accuracy evaluated on a held out set of 4096 4096 puzzles (Figure 14 ). Decreasing the LoRA rank trades-off accuracy and compute. In Table 5 , we report performance of the original model as well as the finetuned model. See Appendix C for additional results on PubMedQA ( Jin et al., 2019 ) . ## 9 Practical Advantages of Text Diffusion In this manuscript, we have primarily emphasized low latency as the principal advantage of text diffusion over standard AR language modeling ( Ghazvininejad et al., 2019 ; Gong et al., 2023 ; Li et al., 2022 ; Dieleman et al., 2022 ) . However, the architectural paradigm of text diffusion offers several benefits that extend beyond computational efficiency. Here, we demonstrate the practical advantages through concrete examples and qualitative analysis of samples generated by the model. To isolate the intrinsic capabilities of the architecture, we disable explicit thinking modes in both DiffusionGemma and the baseline Gemma AR model. This allows us to observe and evaluate the underlying generative process. ### 9.1 Bidirectional Reasoning and Self-Correction AR next-token prediction is inherently causal; during the generation of a given token, the model can only attend to the preceding context. It fundamentally lacks the capacity to condition upon tokens that have yet to be generated. By contrast, text diffusion operates with full bidirectional attention across the canvas. This non-causal property allows tokens at arbitrary positions across the canvas to attend simultaneously to both past and future representations—a critical capability for complex planning and reasoning tasks where early decisions depend on eventual outcomes ( Papadopoulos et al., 2024 ; Kitouni et al., 2024 ; Zhang-Li et al., 2024 ; Nagarajan et al., 2025 , e.g.,) . Consequently, locally within a canvas, future tokens can directly influence the formation of earlier tokens. Because text diffusion employs an iterative refinement process, it possesses a built-in mechanism for self-correction. Any premature commitments generated in earlier denoising steps can be revised during subsequent denoising iterations ( Reid et al., 2022 ; Han et al., 2023 ) . While we have previously demonstrated the efficacy of bidirectional reasoning and self-correction in structured logic puzzles like Sudoku, we demonstrate these mechanics in two reasoning examples. Consider the multi-step arithmetic problem detailed in Figure 15 . Constrained by causal generation, the AR Gemma model must commit to the first token of the final answer before articulating the intermediate calculations. This forces an incorrect initial prediction ( − 1 -1 ) that the model must later correct. DiffusionGemma, conversely, leverages its bidirectional attention canvas to simultaneously evolve the final answer and its underlying logic. As visualized in the accompanying denoising trace, the parallel diffusion process explores incorrect intermediate states but uses the emerging reasoning tokens to course-correct, seamlessly converging on the correct answer ( − 25 -25 ). See Appendix G.1 for the second example of logical reasoning and correction via bidirectional attention and denoising. ### 9.2 Dynamic and Adaptive Computation Unlike AR models, which inherently allocate a fixed amount of computation per generated token, text diffusion facilitates dynamic test-time compute. Text diffusion models can autonomously calibrate the computational effort expended on a given prompt, trading increased inference time for enhanced generative performance based on the inherent difficulty of the task ( Ye et al., 2024 ) . DiffusionGemma automatically adapts to task difficult via adaptive stopping. As illustrated in Figure 5 , tasks of varying complexity organically elicit different numbers of effective denoising steps. This allows the model to conserve compute on easier queries while expending more compute on complex reasoning. In Appendix G.2 we concretely illustrate the adaptive behavior of text diffusion by contrasting a structurally “hard” generation task against a structurally “easy” task and show that the easy task requires fewer denoising steps, as well as showing how the information propagates differently across the canvas of tokens in these two cases. Furthermore, the maximum number of denoising steps serves as an explicit configuration parameter to manage the latency-quality tradeoff. A lower step count forces the model to traverse the reverse process more coarsely, yielding faster results with a corresponding reduction in output precision. Expanding the step count allows for a finer resolution during generation, maximizing quality while extending the required compute time. Figure 10 plots this continuous relationship, demonstrating how the model can be dynamically calibrated to suit varied operational constraints. ### 9.3 Structured and Constrained Outputs In many practical applications, the desired output adheres to a rigid, highly structured format (e.g., JSON schemas) or exhibits strong lexical dependence on the input prompt, such as in optical character recognition (OCR), code editing tasks, or fine-grained syntactic control ( Chen et al., 2023b ; Li et al., 2022 ) . Because text diffusion generates and refines all tokens in parallel, it seamlessly exploits these structural priors to accelerate convergence. AR models lack this capability; constrained by strict sequential decoding, they must incur the same O ⁡ ( N ) O(N) computational cost to generate N N tokens, even when those tokens consist of rigid boilerplate or verbatim copies of the input context. By contrast, DiffusionGemma can identify and lock in predictable syntactic structures across the entire sequence simultaneously. We highlight this efficiency through two real-world examples: strict JSON extraction (Appendix G.3 , Figures 27 and 27 ) and Python code debugging (Appendix G.3 , Figures 29 and 29 ). Both cases clearly illustrate that the highly constrained output allows the diffusion process to converge in merely two to three steps, drastically reducing latency compared to sequential decoding. ## 10 Limitations & Known Issues In the previous section, we discussed new emergent properties of text diffusion and advantages of our approach relative to AR language modeling. While DiffusionGemma establishes a new Pareto frontier for generative text modeling and inference efficiency, the current experimental release has these known limitations: topsep=0pt Performance gap relative to the AR baseline: Lower absolute performance than its AR initialization (Gemma 4 26B A4B) stems from several practical constraints: bypassing native diffusion pretraining to warm-start from AR weights; relying on a comparatively short SFT phase due to compute budget constraints; using an online learning algorithm ( SD ⋅ \cdot RL ) that explicitly targets ultra-low latency, intrinsically trading off asymptotic performance; and inheriting architectural, optimization, and data-mixture decisions from the AR baseline that may be suboptimal for the discrete diffusion paradigm. topsep=0pt Generation length and conciseness: As mentioned in Section 5 , our final checkpoint produces highly concise outputs. While this emergent brevity acts as a multiplier for inference speed, it precludes the model from leveraging the quality improvements typically unlocked by longer, more elaborate reasoning traces. topsep=0pt Occasional token stuttering: In rare instances, the model’s output degenerates into repetitive loops or localized stuttering (e.g., endlessly repeating a common token like “the the the”). While our SD ⋅ \cdot RL training successfully mitigates the vast majority of such instances, this uncommon artifact remains a direct consequence of operating in an ultra-low latency regime, where the aggressively reduced number of denoising steps can occasionally compromise the robustness of the generation process. topsep=0pt Occasional omission of closing thought tags in multimodal tasks: When processing multimodal prompts, the model does not always reliably generate a closing thought tag (even when the reasoning is correct). This can artificially drag down performance in thinking mode on specific benchmarks; for example, on MMMU-Pro, the thinking score drops below the non-thinking score (54.3 vs. 66.0). This was identified too late in the pipeline to apply a fix for this release. topsep=0pt Throughput limits at high batch sizes: DiffusionGemma excels at low batch sizes by trading memory-bandwidth costs for compute, outperforming Gemma 4 AR (with MTP) in both per-user and total throughput for up to ∼ \sim 32 concurrent users (Figure 12 ). Beyond this point, the higher per-token compute cost causes AR models to gain a throughput advantage. As noted in Section 6 , these results are obtained without targeted batch-size optimization; a thorough empirical analysis under realistic traffic conditions remains future work. ## 11 Conclusion DiffusionGemma demonstrates a practical, compute-efficient path to ultra-fast text generation. By finetuning the existing Gemma 4 26B A4B AR model to perform text diffusion through our two-stage training pipeline (SFT and SD ⋅ \cdot RL ), it establishes a new Pareto frontier for the speed-to-intelligence tradeoff—achieving around 1,500 tokens per second on a single H100 while retaining highly competitive reasoning and multimodal capabilities. By releasing DiffusionGemma as an experimental open-weight model—alongside reference implementations in HuggingFace Transformers and vLLM—we aim to empower the open-source community to push the boundaries of text diffusion. We hope researchers and practitioners will build upon this approach, whether by finetuning for specialized tasks, exploring novel sampling algorithms, or further optimizing inference efficiency. ## Appendix A Related work ### A.1 Continuous Diffusion for Text While discrete diffusion remains highly effective, recent advances indicate a resurgence of continuous and hybrid approaches. Flow-matching frameworks like Embedded Language Flows have demonstrated that by abandoning per-step token supervision and remaining in an unrestricted continuous embedding space until a final discretization step, continuous models can substantially outperform discrete baselines ( Hu et al., 2026 ) . Building on this continuous paradigm, recent advances in flow maps for discrete data have shown that by aligning training dynamics with the geometry of the probability simplex, these generative trajectories can be compressed into single-step mappings, achieving high-quality parallel language generation in one or a few steps ( Lee et al., 2026 ; Potaptchik et al., 2026 ) . Similarly, models utilizing pretrained contextual autoencoders (e.g., Cosmos, TEncDM) map text into compressed, smooth latent spaces where Gaussian diffusion operates efficiently without rounding errors, providing a continuous manifold that lends itself well to advanced guidance techniques ( He et al., 2024 ; Meshchaninov et al., 2025 ; Shabalin et al., 2025 ) , while recent approaches like hyperspherical flows avoid Gaussian corruption by rotating token embeddings on a hypersphere to better match the geometric structure of language ( Deschenaux and Gulcehre, 2026b ) . Other hybrid frameworks, such as CANDI ( Pynadath et al., 2026 ) , address the “temporal dissonance” of applying Gaussian noise to discrete data by decoupling discrete and continuous corruption, allowing the model to simultaneously learn conditional structure and continuous geometry. Concurrently, Score Entropy Discrete Diffusion has bridged these domains by applying continuous-time Markov chains to learn the probability ratios of discrete data distributions ( Lou et al., 2024 ) . These breakthroughs suggest that the initial failures of continuous text diffusion may have been artifacts of sub-optimal spatial geometries and restrictive training objectives rather than inherent modality limitations. ### A.2 Speculative Decoding Speculative decoding ( Leviathan et al., 2023 ; Chen et al., 2023a ; Xia et al., 2024 ) reduces serving latency by utilizing a smaller draft model to propose token sequences that are verified in parallel by a larger target model. The overall latency depends on the generation time of both the draft and target models. AR drafters ( Li et al., 2026 ) are fundamentally constrained by sequential generation: increasing draft quality requires more parameters, which increases per-token-latency and diminishes end-to-end gains. Parallel drafters such as Medusa ( Cai et al., 2024 ) circumvent this by generating many tokens at once, but do so independently, yielding suboptimal acceptance rates. Diffusion-based drafters recover inter-token dependencies by instead modeling the joint distribution over draft tokens. TiDAR ( Liu et al., 2026b ) uses a single model for diffusion-based drafting, and AR verification—a configuration that DiffusionGemma also supports—while DFlash ( Chen et al., 2026 ) employs a separate model as the drafter. However, it exhibits suffix decay, with acceptance rates declining at later draft positions ( Cheng et al., 2026 ) . DSpark ( Cheng et al., 2026 ) addresses this by adding a lightweight AR module on top of the parallel backbone. In contrast, DiffusionGemma operates as a standalone diffusion model, eliminating the verification bottleneck and scaling to longer generated canvases than draft-then-verify approaches. ## Appendix B Samples Before and After SD ⋅ \cdot RL Training To illustrate the impact of our SD ⋅ \cdot RL phase, Figures 16 , 17 contrast representative generations from the SFT model and the final checkpoint. They show how SD ⋅ \cdot RL training resolves the severe repetitive looping observed in the SFT baseline, allowing the model to complete complex reasoning traces. ## Appendix C Additional Open-Source Downstream Finetuning Results In Section 8 , we describe our finetuning strategy as well as our main results on Sudoku puzzle solving. We now provide full training recipes as well as additional results for PubMedQA ( Jin et al., 2019 ) . ##### Sudoku solving trace summary. In Figure 18 , we present one trace summary for a successful solving of a Sudoku puzzle. ##### Practical recipe summary. Table 7 summarizes the key hyperparameters used in Sudoku and PubMedQA. Full training and evaluation code is available open-source via the Hackable Diffusion adapter. For our LoRA strategy to update all the linear layers, using rank 8 for Sudoku solving problem, we only finetune 8M parameters. ##### Case study: PubMedQA. To demonstrate applicability beyond structured reasoning, we finetune DiffusionGemma on PubMedQA ( Jin et al., 2019 ) , a biomedical question-answering benchmark where the model must read a medical research abstract and produce both a categorical answer ( yes , no , or maybe ) and a detailed explanatory paragraph. The model is evaluated only on the long task using BLEU score against reference explanations, showing that DiffusionGemma can be effectively adapted to domain-specific natural language generation tasks with minimal data and compute. ## Appendix D Prompt Formatting To facilitate complex, multi-turn interactions, multimodal inputs, and agentic workflows, the model employs a structured Jinja2-based chat template. This template serializes the conversation history, system instructions, and tool schemas into a standardized string format using specialized control tokens. Explicit formatting ensures the model can accurately distinguish between user inputs, internal reasoning, tool invocations, and system-level contexts. What follows is a summary of the main features, but please refer to the implementation 6 6 6 https://huggingface.co/google/diffusiongemma-26B-A4B-it/blob/main/chat_template.jinja for full details. ### D.1 BOS and EOS Special Tokens As in other Gemma models, every conversation must start with a special BOS token, which has no text rendering but corresponds to token integer 2. The BOS special token must either be manually prepended to the tokenized input or usually by the tokenizer via an add_bos=True keyword argument. At the other end, when a block-AR generation completes, it ends in the usual special token followed by padding with the EOS token. Similarly, the EOS token corresponds to the integer 1 but has no text rendering when detokenized. ### D.2 Conversational Structuring The template strictly delineates conversation turns using opening and closing tags. <|turn> and tokens encapsulate individual messages, and the opening token is appended with the specific role (e.g., <|turn>system\n ). The model expects four roles: system , user , model , and tool for tool call responses. The template natively supports multimodal routing by parsing content arrays for specific media types, injecting <|image|> tokens into the context stream where appropriate. In the forward pass of the model, these tokens are replaced by the input image features, as opposed to the corresponding token embedding. ### D.3 Thinking Channels As with the Gemma 4 models, DiffusionGemma supports thinking mode, which can be enabled by adding the thinking token <|think|> to the system instruction. When thinking is enabled, the model will emit an internal reasoning channel (which could technically still be empty) followed by the final answer: ⬇ Importantly, even when the <|think|> token is not present in the system instruction, the model will still emit an empty thought channel as follows: For multi-turn conversations, do not include previous hidden thoughts in the conversation history. Only include the final assistant response before the next user turn. ### D.4 Tool and Function Calling Serialization A significant portion of the template is dedicated to parsing and serializing JSON-like tool schemas into a compact, token-efficient format. All JSON-like schemas, e.g. tool definitions or tool responses, rely on the custom delimiter <|"|> (as opposed to a plain " ) for clarity. topsep=0pt <|tool> and : Used within the system prompt to define available function schemas, including their descriptions, parameters, and required arguments. topsep=0pt <|tool_call> and : Model requests to invoke tools are formatted as <|tool_call>call:function_name{arguments} . topsep=0pt <|tool_response> and : Results returned from external tools are appended to the context window wrapped by these tokens, allowing the model to seamlessly integrate external data into its subsequent turns. ## Appendix E Mercury 2 Speed Estimation ##### Estimation data. As Mercury 2 ( Inception Labs, 2026 ) is a closed source model, we do not have direct access to generate speed metrics such as TPF and TPS. We take a black-box approach and estimate TPS by querying OpenRouter’s API 7 7 7 https://openrouter.ai with the default maximum sequence (input+output) length of 50,000. We ran two independent rounds of queries — the first on July 9–10, 2026 (with a small number of retries on July 11), the second round between July 25–26, 2026 — and obtained consistent results across both measurements. Each API response includes the following metadata: topsep=0pt the number of input tokens (prompt tokenization); topsep=0pt how many prompt tokens were cached (KV-cache hits vs. fresh prefill); topsep=0pt total output tokens (split into thinking tokens and answer tokens); topsep=0pt the wall-clock time for the request (including network latencies and other overheads). Response metadata does not include a breakdown of generation speed. We estimate generate TPS via a least-squares model, detailed below. ##### Estimation model. We estimate per-token generation speed by fitting the following model via non-negative least-squares (NNLS): Total Time = α ⋅ effective_prefill_tokens + β ⋅ total_output_tokens + γ , \text{Total Time}=\alpha\cdot\text{effective\_prefill\_tokens}+\beta\cdot\text{total\_output\_tokens}+\gamma, (14) where effective_prefill_tokens = prompt_tokens − cached_tokens \text{effective\_prefill\_tokens}=\text{prompt\_tokens}-\text{cached\_tokens} is the number of tokens requiring fresh computation, α \alpha is the per-token prefill time, β \beta is the per-token generation time, and γ \gamma captures constant per-request overhead (network latency, etc.). From the fitted β \beta , we obtain the generation speed as 1 / β 1/\beta TPS. The non-negativity constraint reflects that processing tokens, generating tokens, and per-request overhead can only add time. In practice, we disable caching across queries. In order to capture any potential per-task variation in generation speed (e.g. from adaptive computation methods), we fit Equation ( 14 ) independently per benchmark, before taking an arithmetic average. Similarly, for open weight models we compute aggregate TPS measurements by first measuring per-task generative speed and then taking the arithmetic average. ##### Speed estimates. Our speed measurements for Figure 1 are based on the GPQA-Diamond and LiveCodeBench-v6 datasets. For Mercury 2, we estimate the generative speed per task using the above methodology. Figure 19 shows the corresponding analysis; we obtain a TPS of 452.6 for GPA Diamond and 525.5 TPS for LiveCodeBench-v6, which translates into an average speed of 489 TPS. Our speed measurements for Table 3 are based on seven benchmarks where we have measurements for all models: AIME 2026, GPQA Diamond, HumanEval, LBPP, LiveCodeBench-v6, MGSM, and Natural2Code. Again, for Mercury 2 we fit our speed estimation model on each task separately first before taking an arithmetic average. Figures 20 and 21 report the corresponding analysis; we obtain average TPS estimates of 600 TPS for high reasoning effort and 547 TPS for medium reasoning effort. We note that some evals (notably HumanEval and Natural2Code) have outliers that generate 50,000 tokens at extremely high speeds. These give a slightly favorable bias to our generation speed estimates and also explain why high reasoning effort have a higher TPS than medium reasoning effort. These outliers represent corrupted outputs where the model gets stuck in a thinking loop and exhaust the maximum generation length without returning a valid response. Inception reports ∼ \sim 1000 TPS on NVIDIA Blackwell GPUs 8 8 8 https://www.inceptionlabs.ai/blog/introducing-mercury-2 , retrieved July 28, 2026. while Artificial Analysis 9 9 9 https://artificialanalysis.ai/models/mercury-2 , retrieved July 28, 2026. reports a median of ∼ \sim 987 TPS on undisclosed hardware. The difference to our estimate can be due to a number of reasons—in particular different hardware, serving optimizations, and/or different datasets used for measurement (this may have a large impact due to adaptive computation). It is worth noting that Artificial Analysis also reports substantial variance. Our estimate reflects the average speed a user would experience via the OpenRouter API. ## Appendix F Denoising trajectory sampled from DiffusionGemma ## Appendix G More on Practical Advantages of Text Diffusion ### G.1 Bidirectional Reasoning and Self-Correction Consider the frog crossing puzzle in Figure 23 . This example highlights the necessity of self-correction in logical deduction. The prompt introduces a deceptive mathematical trap—an infinite loop—that the AR baseline blindly falls into, generating a contradictory “Yes”. DiffusionGemma’s parallel generation exhibits a distinct self-correction trajectory. While it initially assigns high probability to the intuitive but incorrect “Yes”, the bidirectional propagation of the logical constraints across the sequence enables the model to revise its stance and cleanly output “No” before the final tokens are sampled. ### G.2 More on Dynamic and Adaptive Computation To concretely demonstrate the adaptive behavior of text diffusion, we contrast a structurally “hard” generation task against an “easy” task in Figures 24 and 25 . Both tasks involve generating a sequence of binary digits governed by identical local logical rules. However, in the structurally hard variant (Figure 24 ), each new token strictly depends on the two immediately preceding generated tokens. This causal dependency requires sequential reasoning, forcing the model to resolve the logic in a predominantly left-to-right manner. Consequently, the diffusion process adaptively expends more computational effort, requiring 7 denoising steps to fully resolve the sequence. It is crucial to note, however, that even on this structurally difficult, sequential problem, DiffusionGemma successfully decodes the entire sequence of 40 tokens (20 binary digits and 20 delimiting spaces) in only 7 denoising steps—a substantial acceleration compared to the 40 discrete sequential steps an AR model would require. Conversely, the “easy” convolutional variant (Figure 25 ) asks the model to apply the exact same logic rules over a statically provided input string. Because the causal dependency on the model’s own dynamic output is removed, the task lacks sequential structure. DiffusionGemma immediately leverages its bidirectional attention to independently resolve all local rules in parallel, allowing the entire output sequence to converge simultaneously in merely 4 denoising steps. ### G.3 Structured and Constrained Outputs ## Instructions for reporting errors We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below: Click the "Report Issue" ( ) button, located in the page header. Tip: You can select the relevant text first, to include it in your report. Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all. Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .