Self-Consistency Improves Chain of Thought Reasoning in Language Models

arxiv:2203.11171

Original source · Paper JSON

Reasoning and the "physics" of language models · 2022-03-21

Cites (1) · Cited by (27)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3

Contents

• Self-Consistency Improves Chain of Thought Reasoning in Language Models
• Abstract
• 1 Introduction
• 2 Self-Consistency over Diverse Reasoning Paths
• 3 Experiments
• 3.1 Experiment setup
• Tasks and datasets.
• Language models and prompts.
• Sampling scheme.
• 3.2 Main Results
• Arithmetic Reasoning
• Commonsense and Symbolic Reasoning
• 3.3 Self-Consistency Helps When Chain-of-Thought Hurts Performance
• 3.4 Compare to other existing approaches
• Comparison to Sample-and-Rank
• Comparison to Beam Search
• Comparison to Ensemble-based Approaches
• 3.5 Additional Studies
• Self-Consistency is Robust to Sampling Strategies and Scaling
• Self-Consistency Improves Robustness to Imperfect Prompts
• Self-Consistency Works for Non-Natural-Language Reasoning Paths and Zero-shot CoT
• 4 Related work
• Reasoning in language models.
• Sampling and re-ranking in language models.
• Extract reasoning paths.
• Consistency in language models.
• 5 Conclusion and Discussion
• Reproducibility Statement
• Ethics Statement
• Appendix A Appendix
• A.1 Additional Experiment Results
• A.1.1 Robustness to Sampling Strategies and Parameters
• A.1.2 Robustness to different sets of prompts
• A.1.3 Compared to model ensembles
• A.1.4 Combining self-consistency with other ensembling strategies
• A.2 Details on Resources and Inference
• A.3 Full Sets of Prompts
• Instructions for reporting errors