Beyond Scattered Acceptance: Fast and Coherent Inference for DLMs via Longest Stable Prefixes — part 2/3

Cleaned-text code points [15483, 31463). Bibliography omitted here; source.md includes it.

The process begins with a single forward pass of the model p θ p_{\theta} on the current composite state ( X F ( k ) , X A ( k ) ) (X_{F}^{(k)},X_{A}^{(k)}) . This pass yields logits for all N k N_{k} positions in the active suffix. From these logits, we compute a stability diagnostic for each position i ∈ { 1 , … , N k } i\in\{1,\dots,N_{k}\} . We use the logit margin , defined as the difference between the top-two logit values: δ i ≜ z ( 1 ) ​ ( i ) − z ( 2 ) ​ ( i ) . \delta_{i}\triangleq z_{(1)}(i)-z_{(2)}(i). (1) The margin serves as a simple yet effective low-cost proxy for the model’s local decisiveness. A large margin indicates that the model has high confidence in its top prediction for token i i relative to all alternatives, suggesting this token is stable and unlikely to change in subsequent refinement steps. Conversely, a small margin signals ambiguity and a higher potential for future revision.

##### Distinction from Blockwise Autoregressive Decoding.

A crucial distinction must be made between LSP and standard Blockwise Autoregressive (AR) decoding. In Blockwise AR, a block is generated greedily and frozen; the model never observes “future” tokens during that block’s generation. In contrast, LSP preserves the fundamental advantage of DLMs: bidirectional lookahead . During every diffusion step, the model performs a forward pass on the composite state ( X F ( k ) , X A ( k ) ) (X_{F}^{(k)},X_{A}^{(k)}) . The active suffix X A ( k ) X_{A}^{(k)} acts as a noisy, bidirectional lookahead buffer. This allows the model to refine the tokens in the candidate block based on the global context of the future sequence, resolving narrative and logical dependencies before the prefix is committed.

##### Approximate KV Caching for Prefix States.

In a strict bidirectional model, the internal representations (KV pairs) of the prefix tokens theoretically depend on the active suffix tokens. Recomputing the entire sequence at every step, however, defeats the purpose of caching. LSP employs an approximate KV caching strategy, treating the committed prefix as fixed context. This is supported by recent findings in diffusion acceleration ( Wu et al., 2025 ) , which demonstrate that KV activations exhibit high similarity across adjacent inference steps, and reusing cached KVs for stable tokens results in a negligible performance drop while unlocking massive systemic speedups.

##### Targeted Block Sizing via Adaptive Thresholding.

Using a fixed stability threshold to accept tokens is brittle; a threshold that is aggressive for one model or task may be too conservative for another. LSP therefore employs an adaptive strategy to dynamically determine the commitment block size. The goal is to ensure that the active sequence length N k N_{k} decays at a steady, geometric rate, which is the key to achieving a near-quadratic total work complexity.

To achieve this, we define L ′ ​ ( τ ) L^{\prime}(\tau) as the length of the longest consecutive run of positions, starting from the beginning of the active suffix, whose logit margins all exceed a given threshold τ \tau . Instead of fixing τ \tau , LSP efficiently searches for a threshold τ k \tau_{k} such that the resulting block length L ′ ​ ( τ k ) L^{\prime}(\tau_{k}) falls within a target fractional range of the current active sequence length: L ′ ​ ( τ k ) ∈ [ α ​ N k , β ​ N k ] , L^{\prime}(\tau_{k})\in[\alpha N_{k},\;\beta N_{k}], (2) where 0 < α ≤ β ≤ 1 0<\alpha\leq\beta\leq 1 are user-specified fractions (e.g., α = 0.25 , β = 0.50 \alpha=0.25,\beta=0.50 ). The parameter α \alpha prevents overly cautious steps that would slow down convergence, while β \beta prevents overly aggressive commitments that might introduce errors. This search can be implemented efficiently in O ⁡ ( N k ) O(N_{k}) time by first computing the prefix-minimum of the margin scores and then selecting a target length m ∈ [ ⌈ α ​ N k ⌉ , ⌊ β ​ N k ⌋ ] m\in[\lceil\alpha N_{k}\rceil,\lfloor\beta N_{k}\rfloor] that satisfies the condition. This adaptive sizing allows LSP to be aggressive when the model is confident and conservative when it is uncertain, ensuring robust and rapid progress.

##### Structural Coherence via Boundary Snapping and Monotone Progress.

Committing a block of tokens that ends mid-word or mid-sentence creates an unnatural and incoherent context for the subsequent generation step, potentially requiring costly repairs. To enhance global coherence, LSP trims the candidate block of length L ′ ​ ( τ k ) L^{\prime}(\tau_{k}) to a more natural structural boundary. Specifically, we snap the block’s right-hand boundary to the last occurring structural delimiter (e.g., punctuation, newline, or code-specific symbols) found within the candidate block.

Let 𝒟 \mathcal{D} be a set of such delimiters, L min ≥ 1 L_{\min}\geq 1 be a minimum guaranteed block size, and W ≥ 0 W\geq 0 be a lookback window. The final commitment length L L is determined as: L ≜ max ⁡ { L min , max ⁡ { j ≤ L ′ : y ^ j ∈ 𝒟 ∧ L ′ − j ≤ W } } . L\triangleq\max\Big\{L_{\min},\;\max\{j\leq L^{\prime}\;:\;\hat{y}_{j}\in\mathcal{D}\wedge L^{\prime}-j\leq W\}\Big\}. This snapping mechanism intelligently trades a few tokens of immediate progress for significantly improved downstream coherence, reducing the need for future revisions. To guarantee termination, if no suitable delimiters are found and the candidate block is shorter than L min L_{\min} , a fallback rule ensures that at least one token is committed ( L ← 1 L\leftarrow 1 ). This guarantees that the frozen prefix X F X_{F} grows monotonically in every iteration.

The complete, integrated procedure is detailed in Algorithm 1 . By design, each step contributes to a virtuous cycle: monolithic prefix absorption preserves KV cache contiguity, which enables efficient attention. Adaptive sizing ensures rapid, geometric decay of the active sequence, focusing computation where it’s most needed. Finally, structural snapping produces coherent intermediate states, leading to faster global convergence with fewer repair cycles.

## 4 Experiments

### 4.1 Experimental Setup

##### Models and Benchmarks.

Our empirical evaluation is conducted on two prominent open-source Diffusion Language Models, LLaDA-8B ( Nie et al., 2025 ) and Dream-7B ( Ye et al., 2025 ) , to demonstrate the general applicability of our scheduling approach. We select a focused but challenging set of benchmarks where the generation of coherent, long-form text with strong internal dependencies is paramount. For assessing performance on mathematical reasoning , we use GSM8K ( Cobbe et al., 2021 ) , a dataset of grade-school math word problems where correctness depends on a valid chain of thought. Performance is measured by exact match accuracy of the final answer. For code generation , a domain that demands strict syntactic and logical correctness, we employ the widely-used HumanEval ( Chen et al., 2021 ) and MBPP ( Austin et al., 2021b ) benchmarks. Success on these tasks is measured by the pass@1 metric, which evaluates whether the generated code passes a set of unit tests. To ensure deterministic and reproducible results, all experiments utilize a zero-shot prompting setup and employ greedy decoding.

##### Baseline and LSP Configuration.

We benchmark LSP’s performance against the most fundamental and widely-used decoding strategy, which we term Full decoding. This baseline represents the standard iterative refinement process of a DLM, using the complete step budget available ( T max = L T_{\max}=L , where L L is the generation length). The ‘Full‘ baseline serves as the reference for generation quality and provides the 1.0 × 1.0\times anchor for our speedup calculations. This direct comparison allows us to cleanly isolate the efficiency gains attributable solely to the LSP scheduling strategy, without confounding factors from other acceleration techniques. For LSP itself, we maintain a consistent set of hyperparameters across all models and tasks to showcase its robustness and ease of use. The fractional acceptance interval is set to [ α , β ] = [ 0.25 , 0.50 ] [\alpha,\beta]=[0.25,0.50] , encouraging a steady, geometric decay of the active suffix. We use a minimal block length of L min = 1 L_{\min}=1 to guarantee progress and a structural snapping window of W = 16 W=16 tokens, a modest value chosen to balance coherence with aggressive commitment. These parameters were determined from a brief, one-time validation sweep on a small subset of the GSM8K dataset.

### 4.2 Main Results and Analysis

Table 1 presents the main results of our evaluation. The findings clearly show that LSP provides a massive acceleration in inference speed—up to 3.4 × \times —while preserving the high generation quality of the full-budget baseline. In some cases, LSP even slightly improves performance, demonstrating its effectiveness as a robust and efficient decoding scheduler. On the GSM8K mathematical reasoning task, LSP achieves a 1.5 × \times speedup with LLaDA-8B while also delivering a marginal improvement in accuracy (+0.5%). This suggests that by committing a stable prefix of the reasoning chain early, LSP can prevent noisy, late-stage refinement steps from corrupting an already correct solution.

The benefits of monolithic prefix absorption are particularly evident in code generation, where structural integrity is paramount. On HumanEval, LSP accelerates inference by 1.2 × \times with a negligible impact on the success rate. This confirms that the prefix-first topology, augmented by structural snapping, is highly effective at generating coherent, syntactically valid code blocks more efficiently than iterative, full-sequence refinement. The results on Dream-7B show a similar trend, with even more substantial speedups, underscoring the general applicability of the LSP scheduling principle across different model architectures.

##### Global Planning in Creative Writing.

Unlike reasoning tasks that are highly sequential, creative writing stresses a model’s ability to perform global planning. We evaluated LSP on a subset of the WritingPrompts dataset ( N = 500 N=500 ). Using Gemini 2.5 Flash as an impartial judge on a 1-5 scale, LSP achieved a Coherence score of 4.38 and Creativity score of 4.31 (vs. Full Decoding’s 4.42 and 4.35), while being 1.82 × \times faster. The statistically indistinguishable coherence scores validate that LSP’s bidirectional lookahead buffer successfully resolves long-term narrative dependencies before the prefix is committed.

### 4.3 Ablation Studies and Analysis

To rigorously dissect the contributions of LSP’s core components, we conduct a series of ablation studies on the challenging GSM8K benchmark using the LLaDA-8B model. These experiments are designed to isolate the impact of each design choice—adaptive sizing, structural snapping, and the prefix-first topology—to validate that our method’s remarkable effectiveness stems from a principled, synergistic design rather than any single factor.

#### 4.3.1 The Critical Role of Adaptive Sizing

##### Motivation.

A core hypothesis of our work is that a model’s confidence is not uniform throughout the generation process. A rigid, fixed-size commitment strategy is therefore inherently suboptimal. Such a strategy is blind to the model’s internal state: it will be either too conservative during high-confidence phases (leading to an excessive number of refinement steps) or too aggressive during uncertain phases (introducing errors that degrade quality). Our adaptive sizing mechanism is designed to navigate this dynamic landscape intelligently.

##### Analysis.

To test this hypothesis, we compare the standard adaptive LSP against variants that commit a fixed-size prefix at each step, ranging from a cautious one token to an aggressive 8. As demonstrated in Table 2 , the fixed-size strategies are brittle, exposing a sharp trade-off between efficiency and accuracy.

Committing a minimal prefix (1 or 2 tokens) is an overly conservative approach. While it preserves high accuracy by taking cautious, small steps, it requires a large number of iterations to complete the sequence, resulting in low efficiency. Conversely, committing a large fixed block (8 tokens) is overly aggressive. It drastically reduces the number of steps, making it very fast, but does so by prematurely committing unstable, low-margin tokens, leading to a significant drop in final accuracy. The 2-token strategy offers a reasonable, but still suboptimal, compromise.

LSP’s adaptive approach elegantly resolves this dilemma. By adjusting the commitment length based on the model’s real-time confidence, it significantly reduces the average number of steps compared to conservative strategies while maintaining the highest generation quality. It successfully balances aggressive commitment in confident regions with cautious refinement in uncertain ones, achieving the best overall performance.

#### 4.3.2 Enhancing Coherence with Structural Snapping

##### Motivation.

Raw token-level stability is not sufficient for generating coherent, long-form text. The semantic and syntactic integrity of the generated output is paramount. Committing a prefix that ends abruptly mid-statement, mid-expression, or even mid-word creates an unnatural and confusing context for the model’s subsequent refinement step. Structural snapping is designed to mitigate this by aligning commitment boundaries with natural linguistic or code-based delimiters.

##### Analysis.

We evaluate the impact of this mechanism by disabling it, which results in a greedier strategy that always commits the full candidate block L ′ L^{\prime} identified by adaptive sizing. The results in Table 3 are unambiguous. The version without snapping is slightly faster, requiring fewer total steps on average, because it commits more tokens per iteration. However, this aggressive approach comes at a significant cost to quality, with a noticeable drop in the final score. The reason is that committing incoherent prefixes (e.g., ‘the final answer is 3.141‘) pollutes the context for subsequent denoising steps. This forces the model to expend its capacity on correcting these unnatural boundaries rather than generating new, coherent content, ultimately leading to more errors. The small efficiency cost of snapping (a slightly higher step count) is overwhelmingly justified by the substantial gain in generation quality. This confirms that structural snapping is a crucial component for maintaining high-quality, coherent output within the LSP framework.

##### Qualitative Evidence.

To complement the aggregate ablation results, we provide a granular visualization of LSP on GSM8K in Table 4 . Each iteration commits the Longest Stable Prefix as a contiguous block, and the resulting commit boundaries consistently snap to natural reasoning units (e.g., clauses and arithmetic expressions) rather than arbitrary token positions. This behavior illustrates why structural snapping is critical: it prevents the model from freezing an incoherent partial phrase that would otherwise pollute the context for subsequent denoising, thereby reducing downstream repair and improving final correctness.

#### 4.3.3 Prefix-First vs. Scattered: The Power of Topology

##### Motivation.

Finally, we directly test our central thesis: that the topology of commitment is a primary driver of efficiency in DLM inference. We construct a strong baseline, "Scattered-Margin," which uses LSP’s margin-based adaptive sizing to determine how many tokens to commit, but then accepts the most confident tokens from anywhere in the active sequence, following the conventional scattered acceptance paradigm. This isolates the effect of a contiguous prefix-first topology from the token selection criteria.

##### Analysis.

Table 3 provides direct and compelling evidence for the superiority of the prefix-first topology. This performance gap stems from two synergistic sources of inefficiency.