Reliable Chain-of-Thought via Prefix Consistency — part 1/8

Cleaned-text code points [0, 15693). Bibliography omitted here; source.md includes it.
# Reliable Chain-of-Thought via Prefix Consistency

###### Abstract

Large Language Models often improve accuracy on reasoning tasks by sampling multiple Chain-of-Thought (CoT) traces and aggregating them with majority voting (MV), a test-time technique called self-consistency. When we truncate a CoT partway through and regenerate the remainder, we observe that traces with correct answers reproduce their original answer more often than traces with wrong answers. We use this difference as a reliability signal, prefix consistency , that weights each candidate answer by how often it reappears under regeneration. It requires no access to token log-probabilities or self-rating prompts. Across five reasoning models and four math and science benchmarks, prefix consistency is the best correctness predictor in most settings, and reweighting votes by it reaches Standard MV plateau accuracy at up to 21 × 21\times fewer tokens (median 4.6 × 4.6\times ). Our code is available at https://github.com/naoto-iwase/prefix-consistency .

## 1 Introduction

Large Language Models (LLMs) have shown strong reasoning ability when allowed to produce Chain-of-Thought (CoT) reasoning ( Kojima et al., 2022 ; Wei et al., 2022 ) . Generating intermediate reasoning steps substantially improves performance on challenging tasks such as math ( Zhou et al., 2023 ; Fu et al., 2023 ) , scientific reasoning ( Lu et al., 2022 ; Wang et al., 2024 ) , and knowledge-intensive question answering ( Trivedi et al., 2023 ; Wang et al., 2023a ) .

A simple and effective way to further improve the accuracy of the final answer is majority voting (MV, also known as self-consistency), which samples a diverse set of CoTs and returns the most frequent answer ( Wang et al., 2023b ) . A limitation is that Standard MV treats all CoT outputs equally and still fails when the correct answer is in the minority.

To improve MV, the standard approach has been to use weighted majority voting (WMV). WMV refines MV by weighting each generation according to its quality. The more reliable a generation is, the greater the signal it receives. Existing WMV methods derive a per-sample reliability signal from the generated trace, including response probability ( Wang et al., 2023b ) , self-certainty ( Kang et al., 2025 ) , DeepConf ( Fu et al., 2026 ) , verbalized confidence elicited in text ( Lin et al., 2022 ; Taubenfeld et al., 2025 ) , and P(True) ( Kadavath et al., 2022 ) . Previous studies have demonstrated that these reliability-aware aggregation methods outperform Standard MV. However, these signals often fail to separate correct from wrong traces on difficult problems, the regime where Standard MV most needs improvement (Figure 2 ).

We introduce a novel reliability signal, prefix consistency , and incorporate it into WMV. This signal is motivated by the observation that correct reasoning traces tend to be more reproducible under regeneration than incorrect ones. We truncate each sample’s CoT at a specified fraction and regenerate continuations from the prefix (Figure 1 ). Prefix consistency requires no access to token log-probabilities. Since regenerated answers also participate in voting, our method recovers correct answers absent from the initial samples.

Our contributions are: 1. We propose prefix consistency , a reliability signal that truncates each sample’s CoT and regenerates from the prefix, and use it to form prefix-consistency-weighted majority voting (PC-WMV). PC-WMV requires no access to token log-probabilities.

2. Across 4 benchmarks and 5 model scales, prefix consistency outperforms existing WMV baselines (e.g. DeepConf, P(True), Self-certainty) as a correctness predictor (best macro-averaged AUROC on 15 out of 20 (model, benchmark) cells, .63–.80). On many problems with Pass@1 below 50%, where Standard MV fails by default, prefix consistency still discriminates correct from wrong traces (Section 4.4 ), leaving room for PC-WMV to find the correct answer.

3. In cost-equivalent comparison against the primary WMV baselines (DeepConf tail, P(True), Self-certainty) and adaptive-stopping baselines (AC, ESC), PC-WMV is the most cost-efficient on the majority of the 20 (model, benchmark) settings, reaching Standard MV plateau at a median 4.6 × 4.6\times fewer tokens (up to 21 × 21\times vs. Standard MV and 10 × 10\times vs. AC sweep, Figure 1 and Table 4 ).

## 2 Preliminary

We consider a benchmark 𝒬 \mathcal{Q} , a set of problems. For each problem q ∈ 𝒬 q\in\mathcal{Q} , we have an answer space 𝒜 \mathcal{A} and a correct answer a q ⋆ ∈ 𝒜 a^{\star}_{q}\in\mathcal{A} . Given q q , an LLM generates a trace y y , i.e., a sequence of tokens that represents a CoT followed by a final summary, from which we parse the final answer a ∈ 𝒜 a\in\mathcal{A} . We write ( y , a ) ∼ LLM ( ⋅ ∣ q ) (y,a)\sim\mathrm{LLM}(\cdot\mid q) . We write Pass @ 1 q := Pr ( y , a ) ∼ LLM ( ⋅ ∣ q ) [ a = a q ⋆ ] \mathrm{Pass@1}_{q}:=\Pr_{(y,a)\sim\mathrm{LLM}(\cdot\mid q)}[a=a^{\star}_{q}] for the per-problem single-sample success probability, and report the macro-average over 𝒬 \mathcal{Q} as the benchmark-level Pass@1. When the context is clear, we suppress q q in the notation (e.g., a ⋆ a^{\star} instead of a q ⋆ a^{\star}_{q} ).

To improve the accuracy over Pass@1 at test time, we draw N N independent samples { ( y i , a i ) } i = 1 N \{(y_{i},a_{i})\}_{i=1}^{N} and aggregate them into a single output. A standard method that aggregates the N N answers is majority voting (MV, also known as self-consistency), which returns the most frequent answer: a ^ N MV = arg  max a ∈ 𝒜 ∑ i = 1 N 𝟏 [ a i = a ] . \hat{a}^{\mathrm{MV}}_{N}=\argmax_{a\in\mathcal{A}}\sum_{i=1}^{N}\mathbf{1}[a_{i}=a]. (1)

We refer to this unweighted aggregator as Standard MV (Eq. ( 1 )). Standard MV treats all samples equally and fails when the correct answer is not the mode of the answer distribution, typically observed when an LLM faces challenging problems where Pass@1 accuracy is below 50%. A natural extension is weighted majority voting (WMV), where each sample i i contributes a weighted vote v i  ( a ) ≥ 0 v_{i}(a)\geq 0 for answer a a : a ^ N WMV = arg  max a ∈ 𝒜 ∑ i = 1 N v i ( a ) . \hat{a}^{\mathrm{WMV}}_{N}=\argmax_{a\in\mathcal{A}}\sum_{i=1}^{N}v_{i}(a). (2)

For sample i i , let ℓ i \ell_{i} denote the model’s token-level log-probabilities available to the signal. Prior WMV methods extract a confidence signal s ⁡ ( y i , ℓ i ) ≥ 0 s(y_{i},\ell_{i})\geq 0 from the trace and apply a weighting function w : ℝ ≥ 0 → ℝ ≥ 0 w\colon\mathbb{R}_{\geq 0}\to\mathbb{R}_{\geq 0} to it: v i ( a ) = w ( s ( y i , ℓ i ) ) ⋅ 𝟏 [ a i = a ] . v_{i}(a)=w(s(y_{i},\ell_{i}))\cdot\mathbf{1}[a_{i}=a]. (3) Another class of WMV methods adopts verbalized signals that require no log-probability access ( ℓ i = ∅ \ell_{i}=\emptyset ), where s s depends only on the text of y i y_{i} . Other methods (Self-certainty, DeepConf, Response probability) set ℓ i \ell_{i} to the per-token log-probabilities along y i y_{i} . P(True) sets ℓ i \ell_{i} to the log-probability of the “True” token under a self-rating prompt. Appendix E gives the explicit form of s s and w w for each baseline. However, such confidence signals often fail to separate correct traces from wrong traces on difficult problems. We next introduce prefix consistency, a signal that requires no log-probability access ( ℓ i = ∅ \ell_{i}=\emptyset ).

## 3 Prefix Consistency

We propose prefix consistency, a reliability signal: truncate each sample’s CoT at an intermediate point and regenerate its continuation, treating samples whose initial answer reappears as more reliable than those whose answer changes (Figure 1 ).

### 3.1 Prefix Consistency as a Reliability Signal

For each sample ( y i , a i ) (y_{i},a_{i}) , let | y i | |y_{i}| denote the number of tokens in y i y_{i} . We truncate y i y_{i} after its first ⌈ τ  | y i | ⌉ \lceil\tau|y_{i}|\rceil tokens for a fixed fraction τ ∈ ( 0 , 1 ) \tau\in(0,1) , and regenerate K K continuations from this prefix, yielding the multiset: A i ( τ , K ) = { a i , a ~ i , 1 ( τ ) , … , a ~ i , K ( τ ) } . A_{i}^{(\tau,K)}=\{a_{i},\,\tilde{a}_{i,1}^{(\tau)},\ldots,\tilde{a}_{i,K}^{(\tau)}\}. (4) We refer to A i ( τ , K ) A_{i}^{(\tau,K)} as the i i -th group . For the following discussion, we focus on the case when K = 1 K=1 . We will write A i ( τ ) A_{i}^{(\tau)} for A i ( τ , 1 ) A_{i}^{(\tau,1)} and write a ~ i ( τ ) \tilde{a}_{i}^{(\tau)} for a ~ i , 1 ( τ ) \tilde{a}_{i,1}^{(\tau)} . Extending this to arbitrary K K is straightforward.

The key empirical phenomenon is a reproduction-rate asymmetry : a regenerated answer is more likely to match the initial answer when the initial answer is correct. Let r C  ( τ ) r_{C}(\tau) and r W  ( τ ) r_{W}(\tau) denote the probabilities of reproducing the initial answer, conditioned on whether it is correct or wrong: r C  ( τ ) := Pr ⁡ [ a ~ i ( τ ) = a i ∣ a i = a ⋆ ] , r W  ( τ ) := Pr ⁡ [ a ~ i ( τ ) = a i ∣ a i ≠ a ⋆ ] r_{C}(\tau):=\Pr[\tilde{a}_{i}^{(\tau)}=a_{i}\mid a_{i}=a^{\star}],\qquad r_{W}(\tau):=\Pr[\tilde{a}_{i}^{(\tau)}=a_{i}\mid a_{i}\neq a^{\star}] (5) Across models and benchmarks, we consistently observe the following inequality (Table 1 ): r C  ( τ ) > r W  ( τ ) . r_{C}(\tau)>r_{W}(\tau). (6) In other words, when the initial answer is correct, regeneration from its prefix tends to produce the same answer. When the initial answer is incorrect, regeneration more often produces a different incorrect answer than the same incorrect answer. Figure 2 illustrates this asymmetry on FrontierScience-Olympiad with GPT-OSS-120B and contrasts it with two baseline signals (DeepConf tail and P(True)) that fail to distinguish between correct and incorrect traces.

To exploit this observation, we score each candidate a ∈ A i ( τ ) a\in A_{i}^{(\tau)} by its reproducibility within the group: c i ( τ ) ( a ) := | { a ′ ∈ A i ( τ ) : a ′ = a } | 2 . c_{i}^{(\tau)}(a):=\frac{|\{a^{\prime}\in A_{i}^{(\tau)}:a^{\prime}=a\}|}{2}. (7) We denote the prefix consistency score of a a in group i i by c i ( τ )  ( a ) ∈ { 0 , 1 / 2 , 1 } c_{i}^{(\tau)}(a)\in\{0,1/2,1\} . Unlike conventional per-sample reliability signals (e.g., DeepConf tail, P(True)), which assign a single scalar to each sample’s initial answer, c i ( τ )  ( a ) c_{i}^{(\tau)}(a) is defined for every candidate a ∈ A i ( τ ) a\in A_{i}^{(\tau)} , including regenerated answers that did not appear among the initial answers.

### 3.2 Prefix-Consistency-Weighted Majority Voting (Algorithm 1 )

We set the WMV weight in Eq. ( 2 ) using Eq. ( 7 ): v i  ( a ) = w ⁡ ( c i ( τ )  ( a ) ) v_{i}(a)\;=\;w\!\left(c_{i}^{(\tau)}(a)\right) (8) where w : [ 0 , 1 ] → ℝ ≥ 0 w\colon[0,1]\to\mathbb{R}_{\geq 0} with w ⁡ ( 0 ) = 0 w(0)=0 .

We refer to this method as prefix-consistency-weighted majority voting (PC-WMV) (Algorithm 1 ). Since Eq. ( 8 ) weights every distinct a ∈ A i ( τ ) a\in A_{i}^{(\tau)} rather than only the initial answer a i a_{i} , PC-WMV’s aggregated vote ∑ i v i  ( a ) \sum_{i}v_{i}(a) can be positive for regenerated answers absent from the initial N N samples. This is the operational consequence of using a per-candidate signal rather than a per-sample one.

We now demonstrate that this additional flexibility results in a clear advantage over Standard MV in situations where Standard MV is proven to fail. Standard MV fails when the correct answer occurs less frequently than a wrong answer. The following theorem, in the simplest case of a binary answer space, demonstrates how PC-WMV uses the reproduction-rate asymmetry to recover the correct answer in this regime.

The formal proof of Theorem 1 is in Appendix C.3 .

##### Hyperparameters.

Prefix consistency has two hyperparameters: the truncation fraction τ ∈ ( 0 , 1 ) \tau\in(0,1) and the number of regenerations per group K ∈ ℕ K\in\mathbb{N} . PC-WMV adds a third, the weighting function w : [ 0 , 1 ] → ℝ ≥ 0 w\colon[0,1]\to\mathbb{R}_{\geq 0} with w ⁡ ( 0 ) = 0 w(0)=0 .

## 4 Experiments

We conduct experiments on science (FrontierScience-Olympiad ( Wang et al., 2025 ) ) and math (HMMT Feb 2026, AIME 2025, Brumo 2025 ( Balunović et al., 2025 ) ) datasets. We evaluate on five reasoning LLMs: GPT-OSS-120B, GPT-OSS-20B ( OpenAI, 2025 ) , Nemotron3-30B ( NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 ) ( NVIDIA, 2025a ) , Nemotron2-9B ( NVIDIA-Nemotron-Nano-9B-v2 ) ( NVIDIA, 2025b ) , and Ministral3-14B ( Ministral-3-14B-Reasoning-2512 ) ( Mistral AI, 2026 ) .

We compare our proposed methods against Standard MV, three primary WMV baselines (Self-certainty ( Kang et al., 2025 ) , DeepConf tail ( Fu et al., 2026 ) , and P(True) ( Kadavath et al., 2022 ; Taubenfeld et al., 2025 ) ), and two adaptive-stopping rules over MV (Adaptive Consistency, AC ( Aggarwal et al., 2023 ) , and Early-Stopping Self-Consistency, ESC ( Li et al., 2024 ) ). The details of these methods are documented in Appendix E .

##### Hyperparameters of prefix consistency.

Unless otherwise specified, we fix the truncation fraction at τ = 0.75 \tau=0.75 . The results in the main paper use K = 1 K=1 throughout. We accordingly suppress τ \tau in the notation and write c i  ( a ) , r C , r W , D c_{i}(a),r_{C},r_{W},D for c i ( τ )  ( a ) , r C  ( τ ) , r W  ( τ ) , D ⁡ ( τ ) c_{i}^{(\tau)}(a),r_{C}(\tau),r_{W}(\tau),D(\tau) .

### 4.1 Prefix Consistency as a Correctness Predictor

We report AUROC ¯ \overline{\mathrm{AUROC}} , the macro-averaged AUROC over problems with at least one correct and one wrong initial sample (formal definition in Appendix G.2 ).

Note that some previous work ( Xiong et al., 2024 ; Fadeeva et al., 2024 ) adopted AUROC pooled across problems, which differs from AUROC ¯ \overline{\mathrm{AUROC}} . However, as argued in Taubenfeld et al. (2025) , such an AUROC pooled across problems conflates within-problem discrimination with cross-problem score-difficulty correlation, and only the former predicts whether confidence-weighted self-consistency improves over Standard MV. They also report that calibration metrics such as Expected Calibration Error (ECE) and Brier score are similarly unsuitable. In their data, the best-calibrated source (verbalized binary) gave the smallest improvement while the strongest method (P(True) therein) was only moderately calibrated. At vote time, every WMV method (including PC-WMV and all baselines) compares scores only among samples from the same problem, and thus we consider AUROC ¯ \overline{\mathrm{AUROC}} to better measure discriminative ability between correct traces and wrong traces.

Table 1 reports the discrimination gap D = r C − r W D=r_{C}-r_{W} for prefix consistency: D > 0 D>0 on every (model, benchmark) cell, confirming the asymmetry r C > r W r_{C}>r_{W} . Table 2 reports AUROC ¯ \overline{\mathrm{AUROC}} for prefix consistency against the WMV baselines. Prefix consistency has the highest AUROC ¯ \overline{\mathrm{AUROC}} on 15 15 of 20 20 cells (typically around 0.7 0.7 ), separating correct from wrong traces more clearly than the baselines. Baselines’ AUROC ¯ \overline{\mathrm{AUROC}} often hovers near 0.5 0.5 (= random 1 1 1 AUROC ¯ = 0.5 \overline{\mathrm{AUROC}}=0.5 is the value attained by a random signal that is uninformative about correctness. ) on harder cells, where their scores differ little between correct and wrong samples, reaching ∼ 0.7 \sim 0.7 only on some easier cells.

Abbreviations: FSci = FrontierScience-Olympiad, HMMT = HMMT Feb 2026, AIME = AIME 2025, Brumo = Brumo 2025.

### 4.2 Weighted Majority Voting Results

We compare PC-WMV against existing WMV methods under the same computational cost.

##### Weighting variants.