Latent Recurrent Thoughts: Recurrent Refinement of Proposed Latents for Reasoning with Frozen LLMs — part 5/5

Cleaned-text code points [61695, 66255). Bibliography omitted here; source.md includes it.

The Countdown-4 and Sudoku training and evaluation sets are synthetically generated by us following the public procedures of Gandhi et al. (2024) and Ye et al. (2025) , respectively. They contain no third-party or human-authored content and are not derived from any access-restricted source.

#### Software packages and parameter settings.

All training is implemented in PyTorch ( torch 2.11.0+cu128) and built on the Hugging Face transformers library (4.57.6), with accelerate 1.10.1, einops 0.8.1, and xformers 0.0.35; numerical and data utilities use numpy 2.4.4, scipy 1.16.2, scikit-learn 1.7.2, and pydantic 2.11.7. The Qwen3 and Llama-3.1 decoders are loaded together with their official pretrained tokenizers, and questions are tokenized with each decoder’s native tokenizer using default settings; no additional text normalization or stemming is applied. Optimization uses the AdamW implementation from PyTorch with the hyperparameters reported in Appendix A ( β 1 = 0.9 \beta_{1}{=}0.9 , β 2 = 0.999 \beta_{2}{=}0.999 , ϵ = 10 − 8 \epsilon{=}10^{-8} , weight decay 0.01 0.01 , gradient clipping 1.0 1.0 , cosine schedule with 5% warmup). Inference is served with SGLang ( sglang 0.5.12, flashinfer 0.6.11.post1, flash-attn 4.0.0b14, httpx 0.28.1); all reported results use greedy decoding, so no sampling temperature, top- p p , or top- k k parameters apply. For evaluation, pass@1 on HumanEval and MBPP is computed with the official human-eval execution harness ( Chen et al., 2021 ) with k = 1 k{=}1 , and a problem counts as solved only if the greedily decoded function passes all provided unit tests in the harness’s sandbox. Exact solve rate on Countdown-4 and Sudoku is computed by our own checker, which verifies that the decoded answer satisfies the task constraints (a valid arithmetic expression reaching the target, or a complete and consistent Sudoku grid). StrategyQA accuracy is exact match on the normalized yes/no answer. We do not use ROUGE, BLEU, or other learned or n-gram scoring packages. All package versions are pinned in the requirements files released with our code.

### K.2 Artifacts We Create

We release our LRT implementation under the MIT License at https://github.com/czl-david/latent-recurrent-thoughts . The intended use of these artifacts is non-commercial research on latent reasoning and frozen-decoder methods; this is stated explicitly in the accompanying documentation. This intended use is compatible with the access conditions of every artifact our work builds upon: the permissive licenses of Qwen3, TRM, HumanEval, and the baseline code allow research reuse and redistribution, the Llama 3.1 Community License permits research use subject to its attribution and naming requirements, and the CC BY 4.0 terms of MBPP permit reuse with attribution. Our released artifacts do not include or redistribute the MBPP, HumanEval, or StrategyQA data themselves; they consist only of our own code, our trained module weights, and synthetic data we generated. Consistent with the principle that derivatives of data accessed for research purposes should remain within research contexts, we restrict the distributed artifacts to research use and require any derivative works to retain compatible, research-oriented terms.

## Appendix L Use of AI Assistants

We used an AI-based coding assistant for minor, non-substantive help during implementation, limited to routine tasks such as autocompletion, boilerplate code, and debugging suggestions. We also used an AI writing assistant to check grammar and improve the clarity of phrasing in the manuscript.

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .