Latent Recurrent Thoughts: Recurrent Refinement of Proposed Latents for Reasoning with Frozen LLMs — part 2/5

Cleaned-text code points [15170, 30178). Bibliography omitted here; source.md includes it.

The proposer g ψ g_{\psi} is a bidirectional Transformer encoder. It encodes the bare question x x – with no task instruction, since a model trained from scratch has no instruction-following prior to invoke. The tokens of x x are embedded with the frozen decoder’s table E E , so the proposer and ℳ \mathcal{M} share an input geometry at no parameter cost; a trainable map P ↓ : ℝ d → ℝ d ′ \mathrm{P}_{\!\downarrow}:\mathbb{R}^{d}\!\to\!\mathbb{R}^{d^{\prime}} (its output scaled by d ′ \sqrt{d^{\prime}} ) reduces these embeddings to the working dimension. We append K K learnable query vectors and process the length- ( | x | + K ) (|x|{+}K) sequence with two bidirectional blocks (self-attention + + SwiGLU), so the queries attend to the whole problem and to one another, then read the K K query positions out and lift them back to the decoder’s embedding space with a trainable P ↑ : ℝ d ′ → ℝ d \mathrm{P}_{\!\uparrow}:\mathbb{R}^{d^{\prime}}\!\to\!\mathbb{R}^{d} , giving L ( 0 ) ∈ ℝ K × d L^{(0)}\in\mathbb{R}^{K\times d} . Producing all K K latents in one bidirectional pass mirrors SoftCoT’s single-pass proposal, isolating the proposer architecture as the variable under study.

#### Cost.

The proposer adds ≈ 4.2 {\approx}\,4.2 M trainable parameters ( P ↓ \mathrm{P}_{\!\downarrow} 1.0 1.0 M, query vectors 8 8 K, two blocks ≈ 2.2 {\approx}\,2.2 M, P ↑ \mathrm{P}_{\!\uparrow} 1.0 1.0 M) – essentially the budget SoftCoT already spends on its assistant-to-decoder projection alone.

### 3.3 Recurrent Refiner

The base latents L ( 0 ) L^{(0)} are produced in a single forward pass and are not themselves the product of multi-step computation. The refiner r ϕ r_{\phi} supplies that computation: it is the recursive reasoning module of TRM ( Jolicoeur-Martineau, 2025 ) , repurposed to iteratively improve a set of latents rather than to solve a puzzle end to end.

#### States and updates.

The refiner maintains two states in the working dimension – a fast scratch state z L z_{L} and a slow integrating state z H z_{H} , both in ℝ K × d ′ \mathbb{R}^{K\times d^{\prime}} – initialized from learned, input-independent buffers. The proposed latents enter as an external signal u = P ↓ ′ ​ ( L ( 0 ) ) ∈ ℝ K × d ′ u=\mathrm{P}^{\prime}_{\!\downarrow}(L^{(0)})\in\mathbb{R}^{K\times d^{\prime}} , where P ↓ ′ \mathrm{P}^{\prime}_{\!\downarrow} is a down-projection separate from the proposer’s. One high-level cycle performs T T fast updates followed by a single slow update (we write T T , not L L , to avoid collision with the latents L L ), z L \displaystyle z_{L} ← f ( z L , z H + u ) ( × T ) , \displaystyle\leftarrow f\big(z_{L},\;z_{H}+u\big)\qquad(\times T), z H \displaystyle z_{H} ← f ⁡ ( z H , z L ) , \displaystyle\leftarrow f\big(z_{H},\;z_{L}\big), where f f is a single transition block – of the same class as the proposer’s, with independent weights – applied recursively, as in TRM. Crucially, u u is re-injected at every fast update – not only at initialization – so refinement stays anchored to the problem while z H z_{H} integrates the scratch state across cycles. After unrolling, the refined latents are formed as a bounded residual correction to the proposed base: Δ = P ↑ ′ ​ ( z H ) ∈ ℝ K × d , L ⋆ = L ( 0 ) + Δ . \Delta=\mathrm{P}^{\prime}_{\!\uparrow}\big(z_{H}\big)\in\mathbb{R}^{K\times d},\qquad L^{\star}=L^{(0)}+\Delta. Predicting the correction Δ \Delta rather than regenerating L ⋆ L^{\star} keeps the refiner tied to the instance-conditioned proposal; a light penalty λ ​ ∥ Δ ∥ 2 \lambda\lVert\Delta\rVert^{2} with λ = 0.01 \lambda{=}0.01 (§ 3.5 ) discourages drift toward a generic, instance-agnostic latent.

#### Truncated-gradient unrolling.

Following TRM, the refiner is unrolled deeply but trained cheaply (Algorithm 1 ). We run S S outer iterations of H H high-level cycles each; every cycle but the last runs under stop-gradient, advancing z L , z H z_{L},z_{H} without building a computation graph, so backpropagation traverses only the final cycle while the preceding unroll merely warms up the latent state. Unlike TRM’s adaptive-computation variant, which supervises every step, we take the cross-entropy once , on the final L ⋆ L^{\star} . Gradients flow from ℳ \mathcal{M} ’s answer logits – through the frozen decoder, whose activations are differentiated but whose weights are untouched – into L ⋆ L^{\star} , Δ \Delta , the refiner and – in Stage 1 – the proposer.

#### Cost.

The refiner adds ≈ 7 {\approx}\,7 M trainable parameters, comparable to EBM-CoT’s energy module. LRT’s two modules together total ≈ 11 {\approx}\,11 M trainable parameters; the 8 8 B-parameter decoder is frozen throughout, so LRT trains fewer than 0.2 % 0.2\% of the parameters it reasons with.

### 3.4 Injection and Decoding

Following SoftCoT, ℳ \mathcal{M} receives the concatenation [ I ; x ; L ⋆ ] [\,I;\,x;\,L^{\star}\,] : a task instruction I I and the question x x mapped to input embeddings through E E , followed by the K K refined latents as additional input embeddings. Because P ↑ ′ \mathrm{P}^{\prime}_{\!\uparrow} and the residual already output in ℝ d \mathbb{R}^{d} , L ⋆ L^{\star} needs no further projection. The instruction supplies the decoder’s instruction-following prior and L ⋆ L^{\star} the instance-specific reasoning; θ \theta is never modified.

### 3.5 Training and Inference

#### Two-stage training.

LRT is trained in two stages, each optimizing a single module against the cross-entropy of the frozen decoder; gradients pass through ℳ \mathcal{M} ’s activations but never update θ \theta . Stage 1 trains the proposer alone: the base latents L ( 0 ) L^{(0)} are injected directly, as [ I ; x ; L ( 0 ) ] [I;x;L^{(0)}] , and g ψ g_{\psi} is optimized so that ℳ \mathcal{M} decodes the correct answer. Stage 2 freezes g ψ g_{\psi} and trains the refiner on ℒ = ℒ CE ​ ( ℳ ⁡ ( I , x , L ⋆ ) , y ) + λ ​ ∥ Δ ∥ 2 , \mathcal{L}=\mathcal{L}_{\text{CE}}\big(\mathcal{M}(I,x,L^{\star}),\,y\big)\;+\;\lambda\,\lVert\Delta\rVert^{2}, the answer cross-entropy under the refined latents plus the residual penalty of § 3.3 , with λ = 0.01 \lambda{=}0.01 . Since g ψ g_{\psi} is frozen, each instance’s base latents L ( 0 ) L^{(0)} are precomputed once and cached, so Stage 2 needs no forward pass through the proposer. Across both stages the only supervision is the final answer – or, for StrategyQA, the answer with its reference rationale – and never a record of the search that produced it.

#### Refiner unrolling.

We unroll the refiner for S = 3 S{=}3 outer iterations of H = 3 H{=}3 high-level cycles; each cycle applies T = 4 T{=}4 fast updates and one slow update – five passes through the transition block – for 3 × 3 × 5 = 45 3{\times}3{\times}5{=}45 passes in total. As described in § 3.3 , only the final cycle (five passes) is differentiated; the preceding 40 40 run under stop-gradient, and the cross-entropy is taken once, on L ⋆ L^{\star} .

#### Inference.

At test time the proposer emits its query latents and we keep the first K infer = 4 K_{\text{infer}}{=}4 before the refiner – a train/inference asymmetry ( K = 32 K{=}32 during training) adopted from EBM-CoT, whose latent-count ablation peaks near four. The refiner then runs the full S × H S{\times}H recursion (no stop-gradient is needed without a backward pass), and ℳ \mathcal{M} greedy-decodes the answer from [ I ; x ; L ⋆ ] [I;x;L^{\star}] . Optimization hyperparameters are reported with the experiments.

## 4 Experiments

### 4.1 Experimental Setup

#### Benchmarks.

We evaluate on two families of reasoning tasks. The symbolic family stresses computation away from the natural-language substrate: Countdown-4 (CD4), in which four integers must be combined with arithmetic operations to reach a target value ( Gandhi et al., 2024 ) , and Sudoku , 9 × 9 9{\times}9 constraint-satisfaction puzzles ( Ye et al., 2025 ) . Both supply answer supervision but no ground-truth search traces. The natural-language family comprises HumanEval ( Chen et al., 2021 ) and MBPP ( Austin et al., 2021 ) – Python function synthesis from docstrings – and StrategyQA ( Geva et al., 2021 ) , yes/no questions requiring implicit multi-step reasoning.

#### Metrics.

We report exact solve rate on Countdown-4 and Sudoku, pass@1 on HumanEval and MBPP, and accuracy on StrategyQA. All values are percentages. Table 1 reports mean ± \pm standard deviation over three random seeds; the ablation tables report single-seed values. Avg. is an unweighted mean, over the five benchmarks unless a caption states otherwise.

#### Backbone and baselines.

Every LLM-based method uses the same frozen Qwen3-8B decoder ( Yang et al., 2025 ) . Table 1 compares two groups under this single controlled setting. The first adds no trainable parameters: Zero-Shot CoT ( Kojima et al., 2022 ) (henceforth zero-shot CoT) and a Direct (no CoT) baseline that answers without intermediate steps. The second is the family of trained frozen-decoder latent modules to which LRT belongs: SoftCoT ( Xu et al., 2025b ) and EBM-CoT ( Chen et al., 2025b ) , 2 2 2 EBM-CoT has no official implementation; the EBM-CoT results reported here are from our own reproduction of the method as described by Chen et al. (2025b) and may deviate from the authors’ intended setup. which – like LRT – keep the decoder frozen and train only a small latent module. All of these share LRT’s interface exactly – same decoder, same prompt, same training data and budget – so the latent module is the only variable. Appendix C adds two parameter-efficient prompt-adaptation controls at a matched ≈ 11 {\approx}11 M trainable budget, Prefix-tuning ( Li and Liang, 2021 ) and P-tuning v2 ( Liu et al., 2022 ) , which test whether task supervision plus trainable soft prompts alone reproduce LRT’s gain; they do not.

#### What is deliberately not a baseline, and why.

Three families are excluded by design rather than oversight. (i) Methods that update decoder weights – Coconut ( Hao et al., 2025 ) , CODI ( Shen et al., 2025 ) , Token-Assorted ( Su et al., 2025 ) , and adapter- or LoRA-style tuning ( Houlsby et al., 2019 ; Hu et al., 2022 ) – leave the frozen-decoder setting that defines our study, and fine-tuning is known to degrade strong instruction-tuned backbones ( Xu et al., 2025b ) . (ii) Test-time scaling such as self-consistency ( Wang et al., 2023 ) and SoftCoT++ ( Xu et al., 2025a ) is orthogonal: it aggregates predictions from whatever base method is used, so it composes with LRT rather than competing. (iii) From-scratch symbolic solvers (MGDM, TRM, EqR) cannot process natural language at all, and appear in Table 2 as reference points. Comparisons crossing these boundaries are informative but uncontrolled, and labeled as such throughout.

#### Reproduction and harness notes.

Three details affect how the baseline numbers should be read; Appendix B states them in full. EBM-CoT has no official implementation, so its rows are our reproduction, validated against the authors’ reported StrategyQA result ( 70.96 70.96 vs. 71.04 71.04 ours). The CoT baselines run Qwen3 in non-thinking mode for budget comparability, which departs substantially from the official technical-report numbers; we validated the harness by reproducing the official MBPP result with thinking mode on. TRM is not natively a sequence-to-sequence model, so Countdown-4 required an adaptation of our own, whose limits we state in § 4.3 .

#### Training.

LRT is trained in the two stages of § 3.5 ; hyperparameters are in Appendix A .

### 4.2 Main Results

Table 1 reports results across all five benchmarks.

#### Symbolic reasoning.

The symbolic benchmarks expose a sharp failure of the existing latent-injection methods. On Countdown-4, SoftCoT and EBM-CoT reach only 5.9 5.9 and 8.4 8.4 – far below both the no-reasoning Direct baseline ( 27.8 27.8 ) and zero-shot CoT ( 30.0 30.0 ) – and the same ordering recurs on Sudoku. Injecting latents from a generic assistant is thus not merely unhelpful off the natural-language substrate but actively destructive, corroborating the diagnosis of § 3.2 . LRT reverses this: with a task-dedicated proposer and a recurrent refiner it reaches 56.7 56.7 on Countdown-4 and 49.2 49.2 on Sudoku, the only frozen-decoder method to improve over zero-shot CoT on either task, and by a wide margin ( + 26.7 +26.7 and + 25.3 +25.3 points). On Countdown-4 it even surpasses MGDM ( 52.0 52.0 ; Table 2 ), a diffusion model trained from scratch specifically for symbolic decoding.

#### Natural-language reasoning.

On HumanEval, MBPP, and StrategyQA – the substrate for which soft-thought methods were designed – latent injection behaves as intended: SoftCoT and EBM-CoT both improve over Direct prompting and zero-shot CoT. LRT is strongest on all three (e.g. 37.8 37.8 vs. 25.0 25.0 for EBM-CoT on HumanEval), showing that a task-dedicated proposer and a recurrent refiner help even where the generic pipeline already works. Averaged over the five benchmarks, LRT reaches 54.1 54.1 , against 33.5 33.5 for EBM-CoT and 35.0 35.0 for zero-shot CoT; notably, SoftCoT’s average ( 29.5 29.5 ) falls below the no-reasoning Direct baseline ( 30.9 30.9 ), as its symbolic collapse cancels its natural-language gains. Empirically, we observed that both SoftCoT and EBM-CoT collapse to predicting a near-uniform latent across instances, which may in part explain the sharp drop in performance. Note that injecting a constant latent is not the same as direct decoding, since those vectors still play a non-trivial part in the model’s attention patterns.

### 4.3 Reference Points Outside the Controlled Setting

Table 1 holds the decoder, prompt, data, and budget fixed. Three comparisons that break one of those controls are nonetheless informative, and we report them separately so that they are not mistaken for baselines.

#### Thinking-mode prompting.

In thinking mode with Qwen3’s recommended sampling and a 32,768 32{,}768 -token budget, the same backbone reaches 85.3 85.3 on Countdown-4 and 92.1 92.1 on HumanEval, well above LRT; LRT leads on Sudoku ( 49.2 49.2 vs. 0.5 0.5 ) and MBPP ( 51.5 51.5 vs. 49.3 49.3 ), and thinking mode edges ahead on StrategyQA ( 76.4 76.4 vs. 75.1 ± 2.3 75.1\pm 2.3 , within one standard deviation) – while spending 8.9 8.9 – 453.9 453.9 TFLOP per example against LRT’s ≈ 1 {\approx}1 – 2 2 . Two things follow. Our claim must be read as scoped: LRT outperforms prompted, non-thinking CoT at a fraction of its cost, not test-time scaling in general. But a weak prompting setup cannot explain the main result either, since SoftCoT and EBM-CoT use the same decoder, prompts, data, and budget as LRT and reach 5.9 5.9 and 8.4 8.4 on Countdown-4. It is also notable that the task where thinking mode spends the most compute, Sudoku, is exactly where it fails ( 0.5 0.5 ) while a 7 7 M recurrent refiner reaches 49.2 49.2 : token-space search is not a substitute for iterative computation on a constraint-satisfaction substrate.

#### From-scratch symbolic solvers.