Reasoning on the Simplex: Geometric Fixed-Point Models — part 4/4
Cleaned-text code points [47433, 52897). Bibliography omitted here; source.md includes it.
Zero-shot scoring is next-token log-likelihood of each answer choice, with a BOS token, full continuation, and left truncation only on overflow. The state starts uniform; the iteration count K K is fixed (no residual halt). Headline metrics are accuracy on ARC-Easy (test, N = 2376 N{=}2376 ), SciQ (test, N = 1000 N{=}1000 ), and PIQA (validation, N = 1838 N{=}1838 ), and length-normalized accuracy on HellaSwag (validation, N = 10042 N{=}10042 ). Final arithmetic is FP32 states and activations with TF32 matmul.
Table 5 is the official depth sweep for this checkpoint. Most of the gain appears between K = 8 K{=}8 and K = 24 K{=}24 . The step K = 24 → 32 K{=}24\to 32 does not improve all four tasks, which is why the main text uses the training depth K = 24 K{=}24 . These scores do not show that the simplex state has reached a fixed point.
GPT-2 small and medium in Table 2 of the main text use one forward pass, batch 16, bfloat16, in the same task configurations. GPT-2 large (774M) on the same four tasks is 53.0 / 79.7 / 70.1 / 45.3; we omit it from the main table because it is several times larger than the 201M GFPR-LM run.
## Appendix G FPRM diagnostic protocol
The traces in the residual-comparison figure of Section 2 of the main text use FPRM’s public Sudoku checkpoint and its evaluation loop, run in float64 for 80 steps on one test puzzle, with step-size decay 0.997 0.997 and patience 10 10 as in the released configuration. GFPR is run on the same puzzle with β = 0.7 \beta=0.7 from the uniform state. The residual is the RMS ratio ‖ F ( u ) − u ‖ / ‖ u ‖ \|F(u)-u\|/\|u\| at every step.
#### Local amplification probe (power iteration).
At state u u we set J F = ∂ F / ∂ u J_{F}=\partial F/\partial u and access it only through Jacobian–vector products from reverse-mode autodiff on F F . With a unit probe v 0 v_{0} , for t = 1 , … , K pi t=1,\ldots,K_{\mathrm{pi}} we compute w t = J F v t − 1 w_{t}=J_{F}v_{t-1} , normalize v t = w t / ‖ w t ‖ 2 v_{t}=w_{t}/\|w_{t}\|_{2} , and report the probe value α ^ ( u ) = ‖ w K pi ‖ 2 \widehat{\alpha}(u)=\|w_{K_{\mathrm{pi}}}\|_{2} . For a diagonalizable J F J_{F} with a simple dominant eigenvalue, α ^ ( u ) \widehat{\alpha}(u) approaches | λ max ( J F ) | |\lambda_{\max}(J_{F})| as K pi → ∞ K_{\mathrm{pi}}\to\infty ; for a non-normal J F J_{F} , finite K pi K_{\mathrm{pi}} can reflect transient amplification and need not certify ρ ( J F ) \rho(J_{F}) ( Trefethen and Embree, 2005 ) . The right-hand trace in the main-text figure plots α ^ ( u k ) \widehat{\alpha}(u_{k}) at each iterate u k u_{k} with K pi = 20 K_{\mathrm{pi}}{=}20 (float32). At FPRM’s halt (step 36) the residual is 3.2 ⋅ 10 − 2 3.2\cdot 10^{-2} and α ^ ≈ 1.03 \widehat{\alpha}\approx 1.03 ; after the halt the residual stays between 1.3 ⋅ 10 − 2 1.3\cdot 10^{-2} and 3.1 ⋅ 10 − 2 3.1\cdot 10^{-2} . GFPR’s residual falls below 10 − 12 10^{-12} by step 27, with α ^ ≈ 1.4 × 10 − 3 \widehat{\alpha}\approx 1.4\times 10^{-3} at the final step of the trace. On the final states of this one puzzle we cross-check with restarted Arnoldi on the flattened Jacobian (Krylov dimension 35, 3 restarts; exact JVPs for FPRM, finite-difference products for GFPR). The Ritz values are not the full spectrum, but they approximate its outer part. For FPRM near its halt (step 33 in this run) the largest Ritz values are the pair − 0.126 ± 1.031 i -0.126\pm 1.031i ( | λ | ≈ 1.039 |\lambda|\approx 1.039 ), followed by real values near − 0.98 -0.98 ; no Ritz value has real part outside ( − 1 , 1 ) (-1,1) . The pair becomes contracting for the damped map only when s < 2 ( 1 − Re λ ) / | 1 − λ | 2 ≈ 0.966 s<2(1-\operatorname{Re}\lambda)/|1-\lambda|^{2}\approx 0.966 ; the step size at halt is at least 0.997 3 ≈ 0.991 0.997^{3}\approx 0.991 , giving | 1 − s + s λ | ≈ 1.03 |1-s+s\lambda|\approx 1.03 . For GFPR the largest Ritz value is 1.34 ⋅ 10 − 3 1.34\cdot 10^{-3} .
The population audits use held-out validation puzzles. Halting statistics come from 1000 puzzles run with the authors’ loop up to 35000 steps (median halt at step 126, median step size at halt 0.965 0.965 ). The population audit applies the same probe on 128 puzzles after 8000 iterations in float32 with halting disabled, with K pi = 10 K_{\mathrm{pi}}{=}10 at the final state; 75% have α ^ ( u ) > 1 \widehat{\alpha}(u)>1 . We have not run Arnoldi on these states, so this fraction is not a certified statement that ρ ( J F ) > 1 \rho(J_{F})>1 on 75% of puzzles.
## Instructions for reporting errors
We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:
Click the "Report Issue" ( ) button, located in the page header.
Tip: You can select the relevant text first, to include it in your report.
Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.
Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .