Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristics

arxiv:2410.21272

Original source · Paper JSON

Reasoning and the "physics" of language models · 2024-10-28

Cites (3) · Cited by (7)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4
5. Part 5

Contents

• Arithmetic Without Algorithms: Language Models Solve Math with a Bag of Heuristics
• Abstract
• 1 Introduction
• 2 Arithmetic circuit discovery
• 2.1 Circuit discovery and evaluation
• Models and Data
• Method
• Results
• 2.2 Identifying answer-promoting components
• 3 MLP neurons implement arithmetic heuristics
• 3.1 Decomposing circuit MLPs to individual neurons
• 3.2 MLP neurons act as memorized heuristics
• 4 Arithmetic prompts are answered with a bag of heuristics
• 4.1 Classifying neurons to heuristic types
• 4.2 Heuristic types are combined to answer arithmetic prompts
• Knocking out neurons by heuristic type.
• Knocking out neurons by prompt.
• 4.3 Failure modes of the bag of heuristics
• 5 Tracking heuristics development across training steps
• 6 Related work
• Memorization and generalization in LLMs.
• Arithmetic reasoning interpretability.
• 7 Conclusions
• 8 Limitations and discussion
• Acknowledgments
• Appendix A Llama3-8B circuit evaluation results
• Appendix B Llama3-8B circuit additional components
• B.1 MLP0
• B.2 Attention head patterns
• Appendix C Multi-layer perceptron implementation details
• Appendix D Neuron intersection between operators
• Appendix E Heuristic types descriptions
• Appendix F Heuristic Neuron Classification Algorithm
• Appendix G Additional implementation details
• Arithmetic prompts.
• Circuit discovery via activation patching.
• Circuit faithfulness evaluation.
• Linear probing for correct answers.
• Heuristic classification.
• Appendix H Model accuracies on arithmetic prompts
• Appendix I Results on additional models
• Appendix J Additional examples for arithmetic heuristics
• Instructions for reporting errors