Pre-trained Large Language Models Use Fourier Features to Compute Addition

arxiv:2406.03445

Original source · Paper JSON

Reasoning and the "physics" of language models · 2024-06-05

Cites (2) · Cited by (4)

Complete cleaned Markdown · Source Markdown including bibliography

Read in parts

1. Part 1
2. Part 2
3. Part 3
4. Part 4

Contents

• Pre-trained Large Language Models Use Fourier Features to Compute Addition
• 1 Introduction
• 2 Problem Setup
• Task and Dataset.
• Model.
• Transformers.
• 3 Language Models Solve Addition with Fourier Features
• 3.1 Behavioral Analysis
• Extracting intermediate predictions.
• LLMs progressively compute the final answers.
• 3.2 Fourier Features in MLP & Attention Outputs
• Logits for MLP and attention have periodic structures.
• Logits for MLP and attention are approximately sparse in the Fourier space.
• Final logits are superpositions of these outlier Fourier components.
• Why is high-frequency classification helpful?
• 3.3 Fourier Features are Causally Important for Model Predictions
• Filtering out Fourier components.
• Different roles of frequency components in approximation and classification tasks.
• 4 Effects of Pre-training
• 4.1 Fourier features in Token Embedding
• Number embedding exhibits approximate sparsity in the Fourier space.
• 4.2 Contrasting Pre-trained Models with Models Trained from Scratch
• Fourier features are learned during pre-training.
• Pre-trained token embeddings improve model training.
• 4.3 Fourier Features in Prompted Pre-Trained Models
• Pre-trained LLMs use Fourier features to compute addition during in-context learning.
• Closed-source models exhibit similar behavior.
• 5 Related Work
• Learning mathematical tasks.
• Mechanisms of pre-trained LMs.
• Fourier features in Neural Networks.
• 6 Conclusion
• Acknowledgments
• Appendix
• Roadmap.
• Appendix A Formal Definition of Transformer and Logits in Fourier Space
• Definition A.1 (Transformer) .
• Definition A.2 (Intermediate Logits) .
• Definition A.3 (Fourier Basis) .
• Remark A.4 (Discrete Fourier transformer (DFT) and inverse DFT) .
• Definition A.5 (Logits in Fourier Space) .
• Definition A.6 (Loss-pass / High-pass Filter) .
• Remark A.7 .
• Appendix B Fourier Components Separation and Selection of τ \tau
• Definition B.1 (Single-Pass Filter) .
• Remark B.2 .
• Appendix C Does Fourier Features Generalize?
• C.1 Token Embedding for Other LMs
• C.2 Multiplication Task
• C.3 Same Results for other format
• C.4 Fourier Features in Other Pre-trained LM
• Appendix D Supporting Evidence For the Fourier Features
• Appendix E More Experiments on GPT-2-XL Trained from Scratch
• Appendix F Details of Experimental Settings
• Fine-tuned GPT-2-XL
• Train GPT-2-XL from scratch
• Train GPT-2 from scratch
• Create the addition dataset in main content
• Create the addition dataset in Appendix C.3 with different format
• Experiments Compute Resources
• Licenses for Existing Assets & Open Access to Data and Code.
• Instructions for reporting errors