Self-Consistency Improves Chain of Thought Reasoning in Language Models — part 3/3

Cleaned-text code points [31132, 38950). Bibliography omitted here; source.md includes it.

One limitation of self-consistency is that it incurs more computation cost. In practice people can try a small number of paths (e.g., 5 or 10) as a starting point to realize most of the gains while not incurring too much cost, as in most cases the performance saturates quickly (Figure 2 ). As part of future work, one could use self-consistency to generate better supervised data to fine-tune the model, such that the model can give more accurate predictions in a single inference run after fine-tuning. In addition, we observed that language models can sometimes generate incorrect or nonsensical reasoning paths (e.g., the StrategyQA example in Table 4 , the two population numbers are not exactly correct), and further work is needed to better ground models’ rationale generations.

## Reproducibility Statement

In experiments, we included four different language models with varying scales. Two of them are public models: UL2 is a completely open-sourced model with model checkpoints available at https://github.com/google-research/google-research/tree/master/ul2 ; GPT-3 is also a public model with public API available at https://openai.com/api/ . For GPT-3, we have included two public engines (“code-davinci-001” and “code-davinci-002”) to further aid reproducibility, as Codex is currently free so anyone can reproduce the results. In addition, as our results make use of LaMDA-137B and PaLM-540B that are not publicly available, we provide the exact input prompts for all tasks in Appendix A.3 (and note that we do not perform any finetuning and only apply prompting to off-the-shelf language models).

## Ethics Statement

As we stated in the discussion, language models can sometimes generate nonsensical or non-factual reasoning paths, so one should use language models’ outputs with extra caution. We deal with reasoning tasks mostly and the generated rationales are only used for inspecting how a model reaches its answer. One could potentially use the generated rationales to further check why the model makes certain mistakes or whether the model contains any biases when performing a certain task. For language model in real-world use, further work is needed to better ground models’ predictions and improve model’s factuality and safety, to ensure the models do not cause harms to users.

## Appendix A Appendix

### A.1 Additional Experiment Results

#### A.1.1 Robustness to Sampling Strategies and Parameters

In Figure 6 we ablate the results with respect to different sampling strategies and parameters by varying T T in temperature sampling and k k in Top- k k sampling, on LaMDA-137B. We show that self-consistency is robust to various sampling strategies and parameters.

In Figure 7 and Figure 8 , we show the results of self-consistency compared with greedy decoding a single path over LaMDA-137B and PaLM-540B, respectively. Self-consistency improves over greedy decode by a quite significant margin on both models, on top of high accuracy already achieved by scaling up model sizes.

We further show additional sampled reasoning paths from the LaMDA-137B model in Table 12 , and sampled reasoning paths from the PaLM-540B model in Table 13 . We see that the diversity in the additionally sampled reasoning paths indeed helps the model arrive at a more correct final answer after aggregation.

#### A.1.2 Robustness to different sets of prompts

In Table 9 , we further show that self-consistency is quite robust to different sets of input prompts. We manually wrote 3 different sets of chain-of-thought as prompts to the model. Across all sets of prompts, self-consistency yields consistent gains over the original CoT approach.

#### A.1.3 Compared to model ensembles

Additionally, we provide results of directly ensembling the outputs from multiple language models . The results are shown in Table 10 , by greedily decoding sequences from 3 language models and taking the majority vote (averaged over 10 runs). Note this is a typical ensemble approach (averaging over the predictions over multiple models) and it achieves a performance significantly worse than self-consistency (self-consistency over PaLM-540B gets an accuracy of 74.4%), as lower-capacity models drag down the performance of higher-capacity models. In addition, this approach is limited in two ways: 1) It requires multiple models for an ensemble which might not always be available, while self-consistency only requires one single model to “self-ensemble”; 2) If one of the models is much weaker, it can actually hurt the final performance.

#### A.1.4 Combining self-consistency with other ensembling strategies

Self-consistency is completely compatible with other ensemble strategies, although the gains achieved by self-consistency are significantly higher than other ensemble strategies (and can “override” the performance gains achieved by other ensemble strategies). We further performed experiments and include the results in Table 11 (for a fair comparison, we use 40 sets of prompts, or 40 prompt permutations to compare with self-consistency with 40 paths, all experiments are based on PaLM-540B).

### A.2 Details on Resources and Inference

For all four language models we perform prompting-based inference only. For UL2 we use TPU v3 (2x2 configuration, 4 chips, 8 cores). For GPT-3 models the experiments are done though the public API. 10 10 10 https://beta.openai.com/docs/api-reference/making-requests For LaMDA-137B we use TPU v3 (8x8 configuration, 64 chips, 128 cores). For PaLM-540B we use TPU v4 (4x4x12 configuration, 192 chips, 384 cores). Most inference jobs take 1 to 4 hours (over about 1,000 examples) for each task on UL2 and LaMDA-137B, and about 2 to 12 hours on PaLM-540B. Some tasks (e.g., commonsense reasoning) take longer but do not exceed 2 days for each task.

For GPT-3 models, we use 128 max tokens for all methods, without frequency penalty or presence penalty. For all models, we take the generated outputs until the start of the next “Q:" to parse the final answers, consistent with our prompting format.

### A.3 Full Sets of Prompts

We list the full details of the prompts used for two newly-introduced datasets, AQUA-RAT ( Ling et al., 2017 ) and AI2 Reasoning Challenge (ARC) ( Clark et al., 2018 ) , where we manually composed the example chain-of-thought in this paper, in Table 14 and Table 15 , respectively.

As additional information, we also list the exact set of prompts used for all arithmetic reasoning tasks in Table 17 , since there are multiple sets of prompts introduced in Wei et al. (2022) . The prompts for CommonsenseQA and StrategyQA are the same as used in Wei et al. (2022) .

We provide the exact prompts used for common NLP tasks in the following tables as well, including NLI (Table 18 , Table 19 , Table 20 ) and Closed-Book Question-Answering tasks (Table 16 , Table 21 ).

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .