DiffusionGemma Technical Report — part 5/6

Cleaned-text code points [62308, 77774). Bibliography omitted here; source.md includes it.

Furthermore, the maximum number of denoising steps serves as an explicit configuration parameter to manage the latency-quality tradeoff. A lower step count forces the model to traverse the reverse process more coarsely, yielding faster results with a corresponding reduction in output precision. Expanding the step count allows for a finer resolution during generation, maximizing quality while extending the required compute time. Figure 10 plots this continuous relationship, demonstrating how the model can be dynamically calibrated to suit varied operational constraints.

### 9.3 Structured and Constrained Outputs

In many practical applications, the desired output adheres to a rigid, highly structured format (e.g., JSON schemas) or exhibits strong lexical dependence on the input prompt, such as in optical character recognition (OCR), code editing tasks, or fine-grained syntactic control ( Chen et al., 2023b ; Li et al., 2022 ) . Because text diffusion generates and refines all tokens in parallel, it seamlessly exploits these structural priors to accelerate convergence. AR models lack this capability; constrained by strict sequential decoding, they must incur the same O ⁡ ( N ) O(N) computational cost to generate N N tokens, even when those tokens consist of rigid boilerplate or verbatim copies of the input context. By contrast, DiffusionGemma can identify and lock in predictable syntactic structures across the entire sequence simultaneously. We highlight this efficiency through two real-world examples: strict JSON extraction (Appendix G.3 , Figures 27 and 27 ) and Python code debugging (Appendix G.3 , Figures 29 and 29 ). Both cases clearly illustrate that the highly constrained output allows the diffusion process to converge in merely two to three steps, drastically reducing latency compared to sequential decoding.

## 10 Limitations & Known Issues

In the previous section, we discussed new emergent properties of text diffusion and advantages of our approach relative to AR language modeling. While DiffusionGemma establishes a new Pareto frontier for generative text modeling and inference efficiency, the current experimental release has these known limitations:

topsep=0pt Performance gap relative to the AR baseline: Lower absolute performance than its AR initialization (Gemma 4 26B A4B) stems from several practical constraints: bypassing native diffusion pretraining to warm-start from AR weights; relying on a comparatively short SFT phase due to compute budget constraints; using an online learning algorithm ( SD ⋅ \cdot RL ) that explicitly targets ultra-low latency, intrinsically trading off asymptotic performance; and inheriting architectural, optimization, and data-mixture decisions from the AR baseline that may be suboptimal for the discrete diffusion paradigm.

topsep=0pt Generation length and conciseness: As mentioned in Section 5 , our final checkpoint produces highly concise outputs. While this emergent brevity acts as a multiplier for inference speed, it precludes the model from leveraging the quality improvements typically unlocked by longer, more elaborate reasoning traces.

topsep=0pt Occasional token stuttering: In rare instances, the model’s output degenerates into repetitive loops or localized stuttering (e.g., endlessly repeating a common token like “the the the”). While our SD ⋅ \cdot RL training successfully mitigates the vast majority of such instances, this uncommon artifact remains a direct consequence of operating in an ultra-low latency regime, where the aggressively reduced number of denoising steps can occasionally compromise the robustness of the generation process.

topsep=0pt Occasional omission of closing thought tags in multimodal tasks: When processing multimodal prompts, the model does not always reliably generate a closing thought tag (even when the reasoning is correct). This can artificially drag down performance in thinking mode on specific benchmarks; for example, on MMMU-Pro, the thinking score drops below the non-thinking score (54.3 vs. 66.0). This was identified too late in the pipeline to apply a fix for this release.

topsep=0pt Throughput limits at high batch sizes: DiffusionGemma excels at low batch sizes by trading memory-bandwidth costs for compute, outperforming Gemma 4 AR (with MTP) in both per-user and total throughput for up to ∼ \sim 32 concurrent users (Figure 12 ). Beyond this point, the higher per-token compute cost causes AR models to gain a throughput advantage. As noted in Section 6 , these results are obtained without targeted batch-size optimization; a thorough empirical analysis under realistic traffic conditions remains future work.

## 11 Conclusion

DiffusionGemma demonstrates a practical, compute-efficient path to ultra-fast text generation. By finetuning the existing Gemma 4 26B A4B AR model to perform text diffusion through our two-stage training pipeline (SFT and SD ⋅ \cdot RL ), it establishes a new Pareto frontier for the speed-to-intelligence tradeoff—achieving around 1,500 tokens per second on a single H100 while retaining highly competitive reasoning and multimodal capabilities. By releasing DiffusionGemma as an experimental open-weight model—alongside reference implementations in HuggingFace Transformers and vLLM—we aim to empower the open-source community to push the boundaries of text diffusion. We hope researchers and practitioners will build upon this approach, whether by finetuning for specialized tasks, exploring novel sampling algorithms, or further optimizing inference efficiency.

## Appendix A Related work

### A.1 Continuous Diffusion for Text

While discrete diffusion remains highly effective, recent advances indicate a resurgence of continuous and hybrid approaches. Flow-matching frameworks like Embedded Language Flows have demonstrated that by abandoning per-step token supervision and remaining in an unrestricted continuous embedding space until a final discretization step, continuous models can substantially outperform discrete baselines ( Hu et al., 2026 ) . Building on this continuous paradigm, recent advances in flow maps for discrete data have shown that by aligning training dynamics with the geometry of the probability simplex, these generative trajectories can be compressed into single-step mappings, achieving high-quality parallel language generation in one or a few steps ( Lee et al., 2026 ; Potaptchik et al., 2026 ) . Similarly, models utilizing pretrained contextual autoencoders (e.g., Cosmos, TEncDM) map text into compressed, smooth latent spaces where Gaussian diffusion operates efficiently without rounding errors, providing a continuous manifold that lends itself well to advanced guidance techniques ( He et al., 2024 ; Meshchaninov et al., 2025 ; Shabalin et al., 2025 ) , while recent approaches like hyperspherical flows avoid Gaussian corruption by rotating token embeddings on a hypersphere to better match the geometric structure of language ( Deschenaux and Gulcehre, 2026b ) . Other hybrid frameworks, such as CANDI ( Pynadath et al., 2026 ) , address the “temporal dissonance” of applying Gaussian noise to discrete data by decoupling discrete and continuous corruption, allowing the model to simultaneously learn conditional structure and continuous geometry. Concurrently, Score Entropy Discrete Diffusion has bridged these domains by applying continuous-time Markov chains to learn the probability ratios of discrete data distributions ( Lou et al., 2024 ) . These breakthroughs suggest that the initial failures of continuous text diffusion may have been artifacts of sub-optimal spatial geometries and restrictive training objectives rather than inherent modality limitations.

### A.2 Speculative Decoding

Speculative decoding ( Leviathan et al., 2023 ; Chen et al., 2023a ; Xia et al., 2024 ) reduces serving latency by utilizing a smaller draft model to propose token sequences that are verified in parallel by a larger target model. The overall latency depends on the generation time of both the draft and target models. AR drafters ( Li et al., 2026 ) are fundamentally constrained by sequential generation: increasing draft quality requires more parameters, which increases per-token-latency and diminishes end-to-end gains. Parallel drafters such as Medusa ( Cai et al., 2024 ) circumvent this by generating many tokens at once, but do so independently, yielding suboptimal acceptance rates. Diffusion-based drafters recover inter-token dependencies by instead modeling the joint distribution over draft tokens. TiDAR ( Liu et al., 2026b ) uses a single model for diffusion-based drafting, and AR verification—a configuration that DiffusionGemma also supports—while DFlash ( Chen et al., 2026 ) employs a separate model as the drafter. However, it exhibits suffix decay, with acceptance rates declining at later draft positions ( Cheng et al., 2026 ) . DSpark ( Cheng et al., 2026 ) addresses this by adding a lightweight AR module on top of the parallel backbone. In contrast, DiffusionGemma operates as a standalone diffusion model, eliminating the verification bottleneck and scaling to longer generated canvases than draft-then-verify approaches.

## Appendix B Samples Before and After SD ⋅ \cdot RL Training

To illustrate the impact of our SD ⋅ \cdot RL phase, Figures 16 , 17 contrast representative generations from the SFT model and the final checkpoint. They show how SD ⋅ \cdot RL training resolves the severe repetitive looping observed in the SFT baseline, allowing the model to complete complex reasoning traces.

## Appendix C Additional Open-Source Downstream Finetuning Results

In Section 8 , we describe our finetuning strategy as well as our main results on Sudoku puzzle solving. We now provide full training recipes as well as additional results for PubMedQA ( Jin et al., 2019 ) .

##### Sudoku solving trace summary.

In Figure 18 , we present one trace summary for a successful solving of a Sudoku puzzle.

##### Practical recipe summary.

Table 7 summarizes the key hyperparameters used in Sudoku and PubMedQA. Full training and evaluation code is available open-source via the Hackable Diffusion adapter. For our LoRA strategy to update all the linear layers, using rank 8 for Sudoku solving problem, we only finetune 8M parameters.

##### Case study: PubMedQA.

To demonstrate applicability beyond structured reasoning, we finetune DiffusionGemma on PubMedQA ( Jin et al., 2019 ) , a biomedical question-answering benchmark where the model must read a medical research abstract and produce both a categorical answer ( yes , no , or maybe ) and a detailed explanatory paragraph. The model is evaluated only on the long task using BLEU score against reference explanations, showing that DiffusionGemma can be effectively adapted to domain-specific natural language generation tasks with minimal data and compute.

## Appendix D Prompt Formatting

To facilitate complex, multi-turn interactions, multimodal inputs, and agentic workflows, the model employs a structured Jinja2-based chat template. This template serializes the conversation history, system instructions, and tool schemas into a standardized string format using specialized control tokens. Explicit formatting ensures the model can accurately distinguish between user inputs, internal reasoning, tool invocations, and system-level contexts. What follows is a summary of the main features, but please refer to the implementation 6 6 6 https://huggingface.co/google/diffusiongemma-26B-A4B-it/blob/main/chat_template.jinja for full details.

### D.1 BOS and EOS Special Tokens

As in other Gemma models, every conversation must start with a special BOS token, which has no text rendering but corresponds to token integer 2. The BOS special token must either be manually prepended to the tokenized input or usually by the tokenizer via an add_bos=True keyword argument. At the other end, when a block-AR generation completes, it ends in the usual special token <turn|> followed by padding with the EOS token. Similarly, the EOS token corresponds to the integer 1 but has no text rendering when detokenized.

### D.2 Conversational Structuring

The template strictly delineates conversation turns using opening and closing tags. <|turn> and <turn|> tokens encapsulate individual messages, and the opening token is appended with the specific role (e.g., <|turn>system\n ). The model expects four roles: system , user , model , and tool for tool call responses.

The template natively supports multimodal routing by parsing content arrays for specific media types, injecting <|image|> tokens into the context stream where appropriate. In the forward pass of the model, these tokens are replaced by the input image features, as opposed to the corresponding token embedding.

### D.3 Thinking Channels

As with the Gemma 4 models, DiffusionGemma supports thinking mode, which can be enabled by adding the thinking token <|think|> to the system instruction. When thinking is enabled, the model will emit an internal reasoning channel (which could technically still be empty) followed by the final answer: ⬇

Importantly, even when the

<|think|> token is not present in the system instruction, the model will still emit an empty thought channel as follows:

For multi-turn conversations, do not include previous hidden thoughts in the conversation history. Only include the final assistant response before the next user turn.

### D.4 Tool and Function Calling Serialization

A significant portion of the template is dedicated to parsing and serializing JSON-like tool schemas into a compact, token-efficient format. All JSON-like schemas, e.g. tool definitions or tool responses, rely on the custom delimiter <|"|> (as opposed to a plain " ) for clarity.

topsep=0pt <|tool> and <tool|> : Used within the system prompt to define available function schemas, including their descriptions, parameters, and required arguments.

topsep=0pt <|tool_call> and <tool_call|> : Model requests to invoke tools are formatted as <|tool_call>call:function_name{arguments}<tool_call|> .

topsep=0pt <|tool_response> and <tool_response|> : Results returned from external tools are appended to the context window wrapped by these tokens, allowing the model to seamlessly integrate external data into its subsequent turns.

## Appendix E Mercury 2 Speed Estimation

##### Estimation data.

As Mercury 2 ( Inception Labs, 2026 ) is a closed source model, we do not have direct access to generate speed metrics such as TPF and TPS. We take a black-box approach and estimate TPS by querying OpenRouter’s API 7 7 7 https://openrouter.ai with the default maximum sequence (input+output) length of 50,000. We ran two independent rounds of queries — the first on July 9–10, 2026 (with a small number of retries on July 11), the second round between July 25–26, 2026 — and obtained consistent results across both measurements. Each API response includes the following metadata: topsep=0pt the number of input tokens (prompt tokenization);

topsep=0pt how many prompt tokens were cached (KV-cache hits vs. fresh prefill);

topsep=0pt total output tokens (split into thinking tokens and answer tokens);

topsep=0pt the wall-clock time for the request (including network latencies and other overheads).

Response metadata does not include a breakdown of generation speed. We estimate generate TPS via a least-squares model, detailed below.

##### Estimation model.