Reliable Chain-of-Thought via Prefix Consistency — part 3/8
Cleaned-text code points [31685, 47671). Bibliography omitted here; source.md includes it.
### A.2 Chain-of-Thought
##### Internal answer determination.
Several lines of evidence suggest that LLMs determine their answer internally before the visible CoT concludes. Lanham et al. [2023] found that truncating a CoT mid-reasoning and forcing an early answer often does not change the prediction, especially for larger models. Boppana et al. [2026] confirmed this with attention probes: on easy problems, the internal answer is determined well before the visible reasoning ends. Zhang et al. [2025] showed that linear probes on hidden states can classify the correctness of the future final answer at intermediate CoT stages. These observations are consistent with the r C > r W r_{C}>r_{W} asymmetry we measure.
##### Error propagation.
In CoT reasoning, errors compound across steps: once an error is introduced, subsequent reasoning builds on it [ Gan et al., 2025 ] . Kim et al. [2025] modeled CoT as a metastable Markov process on a reasoning graph, where dense intra-cluster (easy) and sparse inter-cluster (difficult) steps induce timescale separation. Our r C r_{C} and r W r_{W} are analogous in spirit to within-cluster persistence rates, although we measure them at the answer level under regeneration rather than at the reasoning-step level along a single trace.
##### Reasoning robustness.
Several works have studied the robustness of LLM reasoning. Yu et al. [2025] found that reasoning models can be brittle to minor perturbations in the input. von Recum et al. [2026] systematically evaluated seven intervention types on open-weight reasoning LLMs and found that robustness degrades more when interventions occur early in the CoT. Jiang et al. [2025] showed that correct answers persist even when reasoning logic is perturbed, suggesting a decoupling between answer stability and reasoning faithfulness. Our work uses this asymmetry between correct and wrong reasoning traces as a practical signal for answer aggregation.
##### Overthinking and length scaling.
Reasoning models tend to allocate compute disproportionately to problem difficulty, which limits the improvements from extending a single CoT. Chen et al. [2025] reported that o1-like models generate up to 1953% more tokens than non-thinking models on trivial arithmetic, and reach the correct answer in the first generated solution in over 92% of cases while later solutions still account for roughly 40% of tokens. Pu et al. [2025] introduced DUMB500 to quantify overthinking on easy problems and proposed ThoughtTerminator, a training-free decoding-time termination scheme that reduces overthinking tokens by 76% to 98% with minimal accuracy loss. Aggarwal et al. [2026] formalized the trade-off as a joint over- and underthinking benchmark and found that no current model balances the two: even o3 reaches only 71.1% on their unified score. These results motivate aggregating multiple short samples rather than extending a single long trace, which is the setting our method targets.
## Appendix B Limitations and Future Work
The effectiveness of prefix consistency depends on the discrimination gap D = r C − r W D=r_{C}-r_{W} . When incorrect answers are themselves stable under regeneration, r W r_{W} remains high, and the advantage of PC-WMV over Standard MV becomes small (e.g., on very difficult problems). Likewise, when the correct answer is reproduced only weakly from the chosen prefix, r C r_{C} remains low, and the signal becomes less informative. In this sense, the method is most effective in regimes where regeneration behavior meaningfully separates correct from incorrect traces.
The practical advantage of PC-WMV also depends on the available room above Pass@1. When Pass@1 is already close to Standard MV plateau, the aggregated vote on wrong answers is small, so even a strong reliability signal yields only limited accuracy improvements. This explains the small-gap settings in our experiments where PC-WMV does not outperform Standard MV (or simple verbalized-confidence baselines) by a large margin (Appendix D.3 ).
Although prefix consistency does not require token log-probabilities, it is not without cost: it incurs additional inference cost through prefix truncation and regeneration. This trade-off may be less favorable in deployments where log-probability access is readily available and inexpensive. Relatedly, the method introduces hyperparameters such as the truncation fraction τ \tau , the number of regenerations K K , and the weighting function w w , and in this work, we use fixed defaults rather than selecting them adaptively for each model or task.
A further practical limitation is that our method assumes access to an explicit Chain-of-Thought trace that can be truncated and continued from a prefix. This assumption does not hold for many frontier closed models and commercial APIs, including systems where the full internal reasoning trace is hidden or only summarized. In such settings, prefix consistency cannot be applied directly, even if the model can generate correct final answers. Extending the method to settings without visible CoT, for example by using intermediate summaries, structured scratchpads, or other externally exposed reasoning states, is an important direction for future work.
More broadly, our results establish prefix consistency as a useful behavioral signal, but not as a mechanistic explanation of why correct traces are more reproducible than incorrect ones. Understanding the origin of the observed asymmetry r C > r W r_{C}>r_{W} remains future work. Finally, our evaluation is limited to reasoning-oriented models and math/science-style benchmarks, so the extent to which the same phenomenon holds for other domains, model families, or API settings remains to be established.
As shown in Section 4.4 , r C r_{C} rises with problem easiness, r W r_{W} depends on the model and category but only weakly on Pass@1, and D = r C − r W D=r_{C}-r_{W} inherits both. This pattern is consistent with reinforcement learning with verifiable rewards encouraging reasoning that survives verification [ Wen et al., 2026 ] . State-of-the-art LLMs are designed to increase the mass of correct reasoning paths via widening the discrimination gap D D . We do not test this connection directly, and whether the same pattern holds for non-RLVR-trained reasoners is left to future work.
## Appendix C Asymptotic Analysis
We analyze the asymptotic convergence of PC-WMV, prove Theorem 1 along the way, and verify the required assumptions empirically.
### C.1 Notation and Useful Tools
Fix τ ∈ ( 0 , 1 ) \tau\in(0,1) throughout, and omit it from subscripts: e.g., write a ~ i \tilde{a}_{i} for a ~ i ( τ ) \tilde{a}_{i}^{(\tau)} and c i ( a ) c_{i}(a) for c i ( τ ) ( a ) c_{i}^{(\tau)}(a) . The score takes values c i ( a ) ∈ { 0 , 1 2 , 1 } c_{i}(a)\in\{0,\tfrac{1}{2},1\} .
We define the transition probability as follows: T ( b → a ) := Pr [ a ~ i = a | a i = b ] . T(b\rightarrow a)\;:=\;\Pr\!\bigl[\tilde{a}_{i}=a\,\big|\,a_{i}=b\bigr]. (12) The reproduction rates of Section 3.1 can be written in terms of T T : r C = T ( a ⋆ → a ⋆ ) , r W = 𝔼 [ T ( a i → a i ) | a i ≠ a ⋆ ] . r_{C}\;=\;T(a^{\star}\rightarrow a^{\star}),\qquad r_{W}\;=\;\mathbb{E}\!\left[T(a_{i}\rightarrow a_{i})\,\big|\,a_{i}\neq a^{\star}\right]. (13) Let π \pi and π → \pi^{\rightarrow} be the marginal distributions of a i a_{i} and a ~ i \tilde{a}_{i} , respectively. Here π ( a ⋆ ) \pi(a^{\star}) is the same per-problem Pass@1 introduced in Section 2 and used in Theorem 1 . We keep the π \pi notation in this appendix so the population-level expressions in Eq. ( 14 ) below align with the proof. π ( a ) := Pr [ a i = a ] , π → ( a ) := Pr [ a ~ i = a ] = ∑ b ∈ 𝒜 π ( b ) T ( b → a ) . \pi(a)\;:=\;\Pr[a_{i}=a],\qquad\pi^{\rightarrow}(a)\;:=\;\Pr[\tilde{a}_{i}=a]\;=\;\sum_{b\in\mathcal{A}}\pi(b)\,T(b\rightarrow a). (14) All quantities above ( T T , r C r_{C} , r W r_{W} , π \pi , π → \pi^{\rightarrow} ) are defined per problem. The results in this section are statements about a fixed problem q q , and the i.i.d. assumption across i i refers to independent samples within q q .
For completeness, we restate the definitions of the MV and PC-WMV votes. Let w : [ 0 , 1 ] → ℝ ≥ 0 w\colon[0,1]\to\mathbb{R}_{\geq 0} satisfy w ( 0 ) = 0 w(0)=0 . For each a ∈ 𝒜 a\in\mathcal{A} , define the aggregated PC-WMV score over N N groups by V N ( w ) ( a ) := ∑ i = 1 N w ( c i ( a ) ) , V_{N}^{(w)}(a):=\sum_{i=1}^{N}w\bigl(c_{i}(a)\bigr), where each term is the per-group PC-WMV vote given in Eq. ( 8 ). We then define the PC-WMV estimator and MV voting methods by a ^ N PC := arg max a ∈ 𝒜 V N ( w ) ( a ) , a ^ N MV := arg max a ∈ 𝒜 1 N ∑ i = 1 N 𝟏 { a i = a } . \hat{a}^{\mathrm{PC}}_{N}\;:=\;\argmax_{a\in\mathcal{A}}V_{N}^{(w)}(a),\qquad\hat{a}^{\mathrm{MV}}_{N}\;:=\;\argmax_{a\in\mathcal{A}}\,\frac{1}{N}\sum_{i=1}^{N}\mathbf{1}\{a_{i}=a\}. (15)
We next identify the population objective associated with the PC-WMV estimator.
###### Proposition 1 (Population objective) .
Assume that the pairs ( a i , a ~ i ) (a_{i},\tilde{a}_{i}) are i.i.d. across i i , and that, conditional on a i a_{i} , the variable a ~ i \tilde{a}_{i} is drawn from T ( a i , ⋅ ) T(a_{i},\cdot) . Then, for every a ∈ 𝒜 a\in\mathcal{A} , 1 N V N ( w ) ( a ) → a . s . Φ w ( a ) := ∑ b ∈ 𝒜 π ( b ) 𝔼 Z ∼ Bern ( T ( b → a ) ) [ w ( 𝟏 { a = b } + Z 2 ) ] . \frac{1}{N}V_{N}^{(w)}(a)\;\xrightarrow{\mathrm{a.s.}}\;\Phi_{w}(a)\;:=\;\sum_{b\in\mathcal{A}}\pi(b)\,\mathbb{E}_{Z\sim\mathrm{Bern}(T(b\rightarrow a))}\!\left[w\!\left(\frac{\mathbf{1}\{a=b\}+Z}{2}\right)\right]. (16) Moreover, if Φ w \Phi_{w} has a unique maximizer, then a ^ N PC \hat{a}_{N}^{\mathrm{PC}} converges to that maximizer almost surely.
###### Proof.
For each fixed a ∈ 𝒜 a\in\mathcal{A} , the random variables { w ( c i ( a ) ) } i = 1 N \{w(c_{i}(a))\}_{i=1}^{N} are i.i.d., so the strong law of large numbers yields 1 N V N ( w ) ( a ) → a . s . 𝔼 [ w ( c i ( a ) ) ] . \frac{1}{N}V_{N}^{(w)}(a)\;\xrightarrow{\mathrm{a.s.}}\;\mathbb{E}[w(c_{i}(a))]. Conditional on a i = b a_{i}=b , we have 𝟏 { a ~ i = a } ∼ Bern ( T ( b → a ) ) \mathbf{1}\{\tilde{a}_{i}=a\}\sim\mathrm{Bern}(T(b\rightarrow a)) , and 2 c i ( a ) = 𝟏 { a = b } + 𝟏 { a ~ i = a } . 2c_{i}(a)=\mathbf{1}\{a=b\}+\mathbf{1}\{\tilde{a}_{i}=a\}. Hence, 𝔼 [ w ( c i ( a ) ) ∣ a i = b ] = 𝔼 Z ∼ Bern ( T ( b → a ) ) [ w ( 𝟏 { a = b } + Z 2 ) ] . \mathbb{E}[w(c_{i}(a))\mid a_{i}=b]=\mathbb{E}_{Z\sim\mathrm{Bern}(T(b\rightarrow a))}\!\left[w\!\left(\frac{\mathbf{1}\{a=b\}+Z}{2}\right)\right]. Taking expectation with respect to b ∼ π b\sim\pi gives Eq. ( 16 ). For the final claim, 1 N V N ( w ) ( a ) → Φ w ( a ) \frac{1}{N}V_{N}^{(w)}(a)\to\Phi_{w}(a) a.s. for each a a in the finite set 𝒜 \mathcal{A} , so when Φ w \Phi_{w} has a unique maximizer a ⋆ a^{\star} , eventually a ^ N PC = arg max a V N ( w ) ( a ) = a ⋆ \hat{a}_{N}^{\mathrm{PC}}=\argmax_{a}V_{N}^{(w)}(a)=a^{\star} almost surely. ∎
We next derive an explicit decomposition of Φ w \Phi_{w} that exposes its dependence on the marginal π ( a ) \pi(a) and the self-transition T ( a → a ) T(a\rightarrow a) .
###### Proposition 2 (Exact decomposition of Φ w \Phi_{w} ) .
For every a ∈ 𝒜 a\in\mathcal{A} , Φ w ( a ) = w ( 1 2 ) [ π ( a ) + π → ( a ) ] + λ w π ( a ) T ( a → a ) , λ w := w ( 1 ) − 2 w ( 1 2 ) . \Phi_{w}(a)\;=\;w(\tfrac{1}{2})\,\bigl[\pi(a)+\pi^{\rightarrow}(a)\bigr]\;+\;\lambda_{w}\,\pi(a)\,T(a\rightarrow a),\qquad\lambda_{w}\;:=\;w(1)-2\,w(\tfrac{1}{2}). (17)
###### Proof.
Fix a ∈ 𝒜 a\in\mathcal{A} . Since c i ( a ) c_{i}(a) is the average of the two indicators 𝟏 { a i = a } and 𝟏 { a ~ i = a } , \mathbf{1}\{a_{i}=a\}\qquad\text{and}\qquad\mathbf{1}\{\tilde{a}_{i}=a\}, it takes only the three values 0 0 , 1 2 \tfrac{1}{2} , and 1 1 .
More precisely, c i ( a ) = 1 c_{i}(a)=1 if and only if both a i = a a_{i}=a and a ~ i = a \tilde{a}_{i}=a hold. Thus, Pr ( c i ( a ) = 1 ) = π ( a ) T ( a → a ) . \Pr\bigl(c_{i}(a)=1\bigr)=\pi(a)\,T(a\rightarrow a).
Next, c i ( a ) = 1 2 c_{i}(a)=\tfrac{1}{2} if and only if exactly one of the two events { a i = a } \{a_{i}=a\} and { a ~ i = a } \{\tilde{a}_{i}=a\} occurs. Therefore, Pr ( c i ( a ) = 1 2 ) = Pr ( a i = a , a ~ i ≠ a ) + Pr ( a i ≠ a , a ~ i = a ) . \Pr\bigl(c_{i}(a)=\tfrac{1}{2}\bigr)=\Pr(a_{i}=a,\tilde{a}_{i}\neq a)+\Pr(a_{i}\neq a,\tilde{a}_{i}=a). The first term is Pr ( a i = a , a ~ i ≠ a ) = π ( a ) ( 1 − T ( a → a ) ) . \Pr(a_{i}=a,\tilde{a}_{i}\neq a)=\pi(a)\bigl(1-T(a\rightarrow a)\bigr). For the second term, recall that π → ( a ) = Pr ( a ~ i = a ) \pi^{\rightarrow}(a)=\Pr(\tilde{a}_{i}=a) , so Pr ( a i ≠ a , a ~ i = a ) = Pr ( a ~ i = a ) − Pr ( a i = a , a ~ i = a ) = π → ( a ) − π ( a ) T ( a → a ) . \Pr(a_{i}\neq a,\tilde{a}_{i}=a)=\Pr(\tilde{a}_{i}=a)-\Pr(a_{i}=a,\tilde{a}_{i}=a)=\pi^{\rightarrow}(a)-\pi(a)T(a\rightarrow a). Combining the two expressions yields Pr ( c i ( a ) = 1 2 ) = π ( a ) + π → ( a ) − 2 π ( a ) T ( a → a ) . \Pr\bigl(c_{i}(a)=\tfrac{1}{2}\bigr)=\pi(a)+\pi^{\rightarrow}(a)-2\pi(a)T(a\rightarrow a).
Finally, since w ( 0 ) = 0 w(0)=0 and c i ( a ) ∈ { 0 , 1 2 , 1 } c_{i}(a)\in\{0,\tfrac{1}{2},1\} , Φ w ( a ) = 𝔼 [ w ( c i ( a ) ) ] = w ( 1 ) Pr ( c i ( a ) = 1 ) + w ( 1 2 ) Pr ( c i ( a ) = 1 2 ) . \Phi_{w}(a)=\mathbb{E}[w(c_{i}(a))]=w(1)\Pr\bigl(c_{i}(a)=1\bigr)+w(\tfrac{1}{2})\Pr\bigl(c_{i}(a)=\tfrac{1}{2}\bigr). Substituting the above probabilities, we obtain Φ w ( a ) = w ( 1 ) π ( a ) T ( a → a ) + w ( 1 2 ) [ π ( a ) + π → ( a ) − 2 π ( a ) T ( a → a ) ] . \Phi_{w}(a)=w(1)\pi(a)T(a\rightarrow a)+w(\tfrac{1}{2})\bigl[\pi(a)+\pi^{\rightarrow}(a)-2\pi(a)T(a\rightarrow a)\bigr]. Rearranging terms gives Φ w ( a ) = w ( 1 2 ) [ π ( a ) + π → ( a ) ] + ( w ( 1 ) − 2 w ( 1 2 ) ) π ( a ) T ( a → a ) , \Phi_{w}(a)=w(\tfrac{1}{2})\bigl[\pi(a)+\pi^{\rightarrow}(a)\bigr]+\bigl(w(1)-2w(\tfrac{1}{2})\bigr)\pi(a)T(a\rightarrow a), which is exactly Eq. ( 17 ). ∎
### C.2 Asymptotic Convergence of PC-WMV
Throughout this subsection we assume the i.i.d. setup of Proposition 1 .
By Proposition 2 , applied to a ⋆ a^{\star} and to any wrong a a and subtracting, Φ w ( a ⋆ ) − Φ w ( a ) = \displaystyle\Phi_{w}(a^{\star})-\Phi_{w}(a)\;= w ( 1 2 ) [ ( π ( a ⋆ ) + π → ( a ⋆ ) ) − ( π ( a ) + π → ( a ) ) ] ⏟ pooled-mass term \displaystyle\underbrace{w(\tfrac{1}{2})\,\bigl[(\pi(a^{\star})+\pi^{\rightarrow}(a^{\star}))-(\pi(a)+\pi^{\rightarrow}(a))\bigr]}_{\text{pooled-mass term}} (18) + λ w [ π ( a ⋆ ) T ( a ⋆ → a ⋆ ) − π ( a ) T ( a → a ) ] ⏟ self-reproduction term . \displaystyle+\underbrace{\lambda_{w}\,\bigl[\pi(a^{\star})\,T(a^{\star}\rightarrow a^{\star})-\pi(a)\,T(a\rightarrow a)\bigr]}_{\text{self-reproduction term}}. By Proposition 1 , identifying a ⋆ a^{\star} as the population maximizer requires this difference to be positive for every wrong a a . The two assumptions below control the two terms separately.
###### Assumption 1 (Self-reproduction dominance) .
For every a ≠ a ⋆ a\neq a^{\star} , π ( a ⋆ ) T ( a ⋆ → a ⋆ ) > π ( a ) T ( a → a ) . \pi(a^{\star})\,T(a^{\star}\rightarrow a^{\star})>\pi(a)\,T(a\rightarrow a). (19)
##### Interpretation.
π ( a ⋆ ) T ( a ⋆ → a ⋆ ) \pi(a^{\star})\,T(a^{\star}\rightarrow a^{\star}) is the population mass of groups whose initial answer is correct and is reproduced correctly, and π ( a ) T ( a → a ) \pi(a)\,T(a\rightarrow a) is the corresponding mass for the wrong answer a a that is reproduced as itself. Assumption 1 requires that correct self-reproduction dominate every individual wrong self-reproduction. Equivalently, it makes the self-reproduction term in the decomposition strictly positive whenever λ w > 0 \lambda_{w}>0 .