Flow Reasoning Models: Turning Flows Into Efficient Recurrent Reasoners — part 3/4

Cleaned-text code points [30587, 45833). Bibliography omitted here; source.md includes it.

discrete flows on their own past predictions creates an iterative refinement process, while training on
recursively generated states with Fixed-Point Forcing dramatically improves performance and makes
additional recurrent depth productive. Together, these results suggest that recurrent reasoning need
not rely on a specialized recurrent architecture: it can emerge from the learned refinement dynamics
of a general-purpose generative model. Scaling this approach to larger models and less structured
tasks is an important direction for future work.
9

References
Michael S. Albergo, Nicholas M. Boffi, and Eric Vanden-Eijnden. Stochastic interpolants: A unifying
framework for flows and diffusions.Journal of Machine Learning Research, 26(209):1–80, 2025.
URLhttps://www.jmlr.org/papers/v26/23-1605.html.
Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured
denoising diffusion models in discrete state-spaces. InAdvances in Neural Information Processing
Systems, volume 34, pp. 17981–17993, 2021. URL https://proceedings.neurips.cc/
paper/2021/hash/958c530554f78bcd8e97125b70e6973d-Abstract.html.
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for se-
quence prediction with recurrent neural networks. InAdvances in Neural Information Processing
Systems, volume 28, 2015. URL https://proceedings.neurips.cc/paper/2015/hash/
e995f98d56967d946471af29d7bf99f1-Abstract.html.
Ting Chen, Ruixiang Zhang, and Geoffrey Hinton. Analog bits: Generating discrete data using
diffusion models with self-conditioning, 2023. URLhttps://arxiv.org/abs/2208.04202.
Yuxin Chen, Chumeng Liang, Hangke Sui, Ruihan Guo, Chaoran Cheng, Jiaxuan You, and Ge Liu.
Langflow: Continuous diffusion rivals discrete in language modeling, 2026. URL https://
arxiv.org/abs/2604.11748.
Itai Gat, Tal Remez, Neta Shaul, Felix Kreuk, Ricky T. Q. Chen, Gabriel Synnaeve,
Yossi Adi, and Yaron Lipman. Discrete flow matching. InAdvances in Neural In-
formation Processing Systems, volume 37, pp. 133345–133385, 2024. doi: 10.52202/
079017-4239. URL https://proceedings.neurips.cc/paper_files/paper/2024/
hash/f0d629a734b56a642701bba7bc8bb3ed-Abstract-Conference.html.
Keya Hu, Linlu Qiu, Yiyang Lu, Hanhong Zhao, Tianhong Li, Yoon Kim, Jacob Andreas, and Kaiming
He. Elf: Embedded language flows, 2026. URLhttps://arxiv.org/abs/2605.10938.
Benhao Huang, Zhengyang Geng, and Zico Kolter. Equilibrium reasoners: Learning attractors
enables scalable reasoning, 2026. URLhttps://arxiv.org/abs/2605.21488.
Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging
the train-test gap in autoregressive video diffusion, 2025. URL https://arxiv.org/abs/2506.
08009.
Alexia Jolicoeur-Martineau. Less is more: Recursive reasoning with tiny networks, 2025. URL
https://arxiv.org/abs/2510.04871.
Jaeyeon Kim, Kulin Shah, Vasilis Kontonis, Sham Kakade, and Sitan Chen. Train for the worst,
plan for the best: Understanding token ordering in masked diffusions, 2025. URL https:
//arxiv.org/abs/2502.06768.
Chanhyuk Lee, Jaehoon Yoo, Manan Agarwal, Sheel Shah, Jerry Huang, Aditi Raghunathan, Se-
unghoon Hong, Nicholas M. Boffi, and Jinwoo Kim. Flow map language models: One-step lan-
guage modeling via continuous denoising, 2026. URL https://arxiv.org/abs/2602.16813.
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching
for generative modeling. InInternational Conference on Learning Representations, 2023. URL
https://openreview.net/forum?id=PqvMRDCJT9t.
Sajad Movahedi, Vera Milovanovi ´c, Shlomo Libo Feigin, Alexander Theus, Thomas Hofmann,
Valentina Boeva, T. Konstantin Rusch, and Antonio Orvieto. Fixed-point reasoners: Stable and
adaptive deep looped transformers, 2026. URLhttps://arxiv.org/abs/2606.18206.
Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured
prediction to no-regret online learning. InProceedings of the Fourteenth International Conference
on Artificial Intelligence and Statistics, volume 15 ofProceedings of Machine Learning Research,
pp. 627–635. PMLR, 2011. URLhttps://proceedings.mlr.press/v15/ross11a.html.
10

Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T
Chiu, Alexander Rush, and V olodymyr Kuleshov. Simple and effective masked diffusion language
models, 2024. URLhttps://arxiv.org/abs/2406.07524.
Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin Chiu, and
V olodymyr Kuleshov. The diffusion duality.arXiv preprint arXiv:2506.10892, 2025. URL
https://arxiv.org/abs/2506.10892.
Amin Sghaier, Ali Parviz, and Alexia Jolicoeur-Martineau. Probabilistic tiny recursive model, 2026.
URLhttps://arxiv.org/abs/2605.19943.
Kulin Shah, Nishanth Dikkala, Xin Wang, and Rina Panigrahy. Causal language modeling can elicit
search and reasoning capabilities on logic puzzles, 2024. URL https://arxiv.org/abs/2409.
10502.
Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis K. Titsias. Simplified and
generalized masked diffusion for discrete data, 2025. URL https://arxiv.org/abs/2406.
04329.
Hannes Stark, Bowen Jing, Chenyu Wang, Gabriele Corso, Bonnie Berger, Regina Barzilay,
and Tommi Jaakkola. Dirichlet flow matching with applications to DNA sequence design.
InProceedings of the 41st International Conference on Machine Learning, volume 235 of
Proceedings of Machine Learning Research, pp. 46495–46513. PMLR, 2024. URL https:
//proceedings.mlr.press/v235/stark24b.html.
Guan Wang, Jin Li, Yuhao Sun, Xing Chen, Changling Liu, Yue Wu, Meng Lu, Sen Song, and
Yasin Abbasi Yadkori. Hierarchical reasoning model, 2025a. URL https://arxiv.org/abs/
2506.21734.
Guanghan Wang, Yair Schiff, Subham Sekhar Sahoo, and V olodymyr Kuleshov. Re-
masking discrete diffusion models with inference-time scaling. InAdvances in Neu-
ral Information Processing Systems, volume 38, pp. 147282–147339, 2025b. doi: 10.
52202/085713-4927. URL https://proceedings.neurips.cc/paper_files/paper/
2025/hash/d877ea0dd78bdbe54830670618c1de09-Abstract-Conference.html.
Guanghan Wang, Yair Schiff, Subham Sekhar Sahoo, and V olodymyr Kuleshov. Remasking discrete
diffusion models with inference-time scaling, 2026. URL https://arxiv.org/abs/2503.
00307.
Jaehoon Yoo, Wonjung Kim, Floor Eijkelboom, Chanhyuk Lee, Nicholas M. Boffi, Seunghoon Hong,
and Jinwoo Kim. Self-conditioned flow map language models via fixed-point flows, 2026. URL
https://arxiv.org/abs/2607.00714.
A Experimental Details
A.1 Datasets
Sudoku-Extreme.We use Sudoku-Extreme to test long-horizon constraint satisfaction on hard
9×9 puzzles that require extended chains of deduction and search. The benchmark was introduced
with the Hierarchical Reasoning Model as a small-data test of recurrent reasoning (Wang et al.,
2025a); we represent each released puzzle as its unique solution together with a mask that clamps the
given clues. For each training seed, we draw a difficulty-balanced subset of 1,000 puzzles from the
official training split and apply random Sudoku symmetries as augmentation. We select checkpoints
on a fixed held-out validation subset and report exact-grid solve rate on a fixed 1,000-puzzle subset
of the official test split.
Zebra.We use Zebra to test whether recurrent refinement transfers from grid-local constraints
to heterogeneous relational reasoning. Introduced as a reasoning benchmark by Shah et al. (2024),
the dataset contains approximately 1.5 million training and 0.1 million test instances spanning three
to six houses and attributes. Following the uniform 3×3 –6×6 mixture introduced by Shah et al.
(2024) and used by Kim et al. (2025), we train and evaluate across all four puzzle sizes rather than
11

Setting Sudoku-Extreme Zebra3×3–6×6Maze-Unique
Architecture
Parameters7.0M25.0M8.0M
Hidden size / blocks / heads256/7/8 384/12/12 256/8/8
Sequence length / vocabulary81/10size-dependent /21 900/6
Optimization
Optimizer / betas AdamW /(0.9,0.999)AdamW /(0.9,0.999)AdamW /(0.9,0.999)
Learning rate / weight decay3×10 −4 /0 3×10 −4 /0 1×10 −4 /1.0
Global batch size per step128 128 64
Warmup steps0 0 500
Gradient clip1.0 1.0 1.0
EMA decay0.9999 0.9999 0.9999
Dropout0.1 0.1 0.1
Learning-rate schedule constant constant constant
Stage-A / Stage-B iterations100K /100K200K /200K300K /100K
Self-conditioning and FPF
Self-conditioning probability0.5 0.5 0.5
FPF rollout probability / depth0.5/16 0.5/16 0.5/16
Supervised states / noise scale4/0 4/0 4/0
Table 3:Per-dataset architecture and training hyperparameters.Global batch size is the number
of puzzles used in each optimizer step.
restricting the benchmark to a fixed 5×5 slice. We retain each puzzle’s original clue serialization but
replace its variable, solver-ordered trace with a canonical attribute-by-house solution grid. We report
exact-grid solve rate on the mixed held-out split.
Maze-Unique.We use Maze-Unique to test structured path finding under an unambiguous exact-
solve criterion. Released with Equilibrium Reasoners (Huang et al., 2026), this 30×30 path-finding
benchmark provides official, revision-pinned splits of 1,000 training and 1,000 test mazes; we encode
walls, start, and goal as clamped cells and generate the remainder of the grid. We prefer it to the
more common Maze-Hard benchmark because every instance has a unique solution path, making
exact-grid accuracy coincide with path validity and optimality instead of penalizing an alternative
valid route. It also permits direct comparison with recent recurrent reasoners evaluated on the same
split.
A.2 Models and Training
All experiments use a non-causal DiT backbone and the categorical clean-prediction objective.
AdamW with β= (0.9,0.999) , gradient clipping at 1.0, EMA decay 0.9999, dropout 0.1, and a
constant learning-rate schedule are shared across datasets. The learning rate, weight decay, batch
size, warmup, training budget, and architecture vary by dataset and are reported in Table 3. In every
case, Stage A trains self-conditioning from scratch; Stage B initializes from the selected Stage-A
weights with a fresh optimizer and EMA state before applying FPF. For each training seed, we select
the checkpoint and inference configuration (including sampler and NFE) with the highest validation
solve rate. Table 2 aggregates the resulting selected operating points across seeds.
A.3 Baseline Provenance and Comparability
We organize baselines into three families because they make different architectural commitments.
The autoregressive baseline is a plain causal decoder-only transformer trained with teacher-forced
next-token cross-entropy over a clue-grid-then-solution-grid sequence: no denoising, no recurrence,
and no mechanism to revise an already-committed token, making it the naive baseline the rest of the
comparison is meant to contextualize (Shah et al., 2024). Diffusion language models and flows use a
general-purpose generative denoising formulation that can share the same non-causal DiT backbone
as FRM. Small recurrent reasoning models instead introduce task-oriented recurrent state, update,
or halting mechanisms. This grouping clarifies the role of each comparison; it does not by itself
imply matched capacity or compute, so Table 1 reports the model size used on each benchmark and
Appendix A.4 reports inference cost.
12

Accuracy provenance.Unless noted here, entries in Table 1 are measurements from our evaluation
harness using checkpoints that we trained or official checkpoints that we evaluated. For Zebra,
the FMLM, FLM, and FRM entries are our measurements. The daggered MDLM and adaptive-
MDLM accuracies are published values from Kim et al. (2025) on Shah’s uniform 3×3 –6×6
mixture; we could not reproduce those reported accuracies in our pipeline. Our weight-backed Zebra
MDLM reproduction peaks at 49.2% under our shorter training budget and supplies the measured
operating points used for efficiency analysis. We omit ReMDM and specialized recurrent reasoners
on Zebra because we did not obtain a reliable result under the same mixed protocol; the dashes
therefore indicate unavailable comparisons, not zero accuracy. For Maze-Unique, EqR is a published
reference value, whereas TRM is our 0.264M-parameter reproduction trained and evaluated on the
Maze-Unique split, reaching 77.9% exact solve rate. The 44.9% result reported by the original TRM
work is for Maze-Hard, not Maze-Unique, and is therefore excluded from this table. The remaining
Maze-Unique entries are measured from our trained reproductions or evaluated checkpoints. Thus,
published accuracies support the headline comparison, whereas every point in an accuracy–compute
frontier is produced from a checkpoint available to our profiler.
A.4 Inference Compute Accounting
Number of function evaluations.We report the realized number of learned-model evaluations
required to solve one puzzle. For our Euler, SDE, and held-time samplers, NFE counts every iterative
update and the final readout evaluation; for methods with adaptive stopping, branching, or breadth,
we count all evaluations actually executed for that instance. NFE exposes the number of sequential
model calls and is easy to reproduce, but it is not a sufficient cross-method compute measure: one
evaluation can differ substantially in cost across architectures, sequence lengths, model sizes, and
internally recurrent networks. The appendix table reportssolved-puzzle mean NFE: for each puzzle
that is solved at some point along its evaluated trajectory, we record the first NFE at which it is solved
and average over solved puzzles only. This is an oracle trajectory diagnostic unless correctness can
be detected without access to the ground truth; it should therefore not be interpreted as deployable
early stopping for every method.
Floating-point operations.We therefore use total FLOPs per puzzle as the primary hardware-
independent compute measure. We profile an actual batch-one forward pass with PyTorch’s operator-
level FLOP counter, disabling fused attention so that its matrix multiplications are visible, and
multiply the measured per-evaluation cost by the realized NFE; methods with nonuniform calls are
profiled and summed by call type. This quantity should be interpreted as a lower bound on the work
required to solve an instance: it counts floating-point arithmetic but not memory traffic, kernel-launch
and framework overhead, control flow, communication, or data movement. We therefore present the
full accuracy–compute frontiers rather than compressing each method to a single operating-point
ratio.
B Additional Accuracy and Ablation Results
This section documents the headline accuracy–compute comparison and training-recipe ablation;
Appendix A gives datasets, checkpoint provenance, and FLOP accounting.
B.1 Accuracy–Compute Evaluation
Each frontier contains measured operating points only. We vary each checkpoint’s native compute
control and charge every learned-model call. Fixed-depth methods use the requested depth; adaptive
methods use realized mean computation. We neither extrapolate unsupported method–dataset pairs
nor include published accuracies lacking a compatible profiled checkpoint, although those values
may still appear in Table 1.
13