{"id":"arxiv:2406.03445","title":"Pre-trained Large Language Models Use Fourier Features to Compute Addition","date":"2024-06-05","section":"Reasoning and the \"physics\" of language models","quality":0.375032,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/","source_url":"https://arxiv.org/abs/2406.03445","text_available":true,"authors":["Tianyi Zhou","Deqing Fu","Vatsal Sharan","Robin Jia"],"outgoing":[{"source":"arxiv:2406.03445","target":"arxiv:2301.05217","evidence":"id","id":"arxiv:2301.05217","title":"Progress measures for grokking via mechanistic interpretability","url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/f71e3b65a8b3f35beb171d17f6c65878e0914227a0042ccba59b88952554baef/"},{"source":"arxiv:2406.03445","target":"arxiv:2309.14316","evidence":"id","id":"arxiv:2309.14316","title":"Physics of Language Models: Part 3.1, Knowledge Storage and Extraction","url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/b87376d90e99d2bb590e1a02ed50a55c90881591ed7d3e00c3d5be2aaacbb2af/"}],"incoming":[{"source":"arxiv:2410.21272","target":"arxiv:2406.03445","evidence":"id","id":"arxiv:2410.21272","title":"Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristics","url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/397b826ba289bd4f14f457562942cb092c38550a4c49451e95ffcc7b376c8cb0/"},{"source":"arxiv:2502.00873","target":"arxiv:2406.03445","evidence":"title","id":"arxiv:2502.00873","title":"Language Models Use Trigonometry to Do Addition","url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/7766b2640c6b5ab53a295e8d60920cb2419bfbbd812810d9b35d4b7c323922ed/"},{"source":"arxiv:2512.16902","target":"arxiv:2406.03445","evidence":"title","id":"arxiv:2512.16902","title":"In-Context Algebra","url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/5aa7087a88619eafd156aad9217e16f6d1ce284c65534322239be0e02f04e553/"},{"source":"arxiv:2606.03982","target":"arxiv:2406.03445","evidence":"title","id":"arxiv:2606.03982","title":"Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics","url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/48f07d03c9308ca8fb43ecd2ac22978f6dcbd4bc081056f022ca511711942dbf/"}],"text_parts":[{"start":0,"end":15633,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html"},{"start":15633,"end":31346,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html"},{"start":31346,"end":47089,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html"},{"start":47089,"end":51451,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-4.html"}],"contents":[{"title":"Pre-trained Large Language Models Use Fourier Features to Compute Addition","offset":0,"level":1,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"1 Introduction","offset":1195,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"2 Problem Setup","offset":4882,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"Task and Dataset.","offset":4902,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"Model.","offset":5622,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"Transformers.","offset":5997,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"3 Language Models Solve Addition with Fourier Features","offset":6912,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"3.1 Behavioral Analysis","offset":7785,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"Extracting intermediate predictions.","offset":7992,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"LLMs progressively compute the final answers.","offset":8823,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"3.2 Fourier Features in MLP & Attention Outputs","offset":9383,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"Logits for MLP and attention have periodic structures.","offset":9436,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"Logits for MLP and attention are approximately sparse in the Fourier space.","offset":10753,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"Final logits are superpositions of these outlier Fourier components.","offset":12400,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"Why is high-frequency classification helpful?","offset":13301,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"3.3 Fourier Features are Causally Important for Model Predictions","offset":14758,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"Filtering out Fourier components.","offset":15593,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-1.html#text"},{"title":"Different roles of frequency components in approximation and classification tasks.","offset":16815,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"4 Effects of Pre-training","offset":18906,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"4.1 Fourier features in Token Embedding","offset":19559,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"Number embedding exhibits approximate sparsity in the Fourier space.","offset":19604,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"4.2 Contrasting Pre-trained Models with Models Trained from Scratch","offset":21059,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"Fourier features are learned during pre-training.","offset":21444,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"Pre-trained token embeddings improve model training.","offset":21968,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"4.3 Fourier Features in Prompted Pre-Trained Models","offset":22784,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"Pre-trained LLMs use Fourier features to compute addition during in-context learning.","offset":22936,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"Closed-source models exhibit similar behavior.","offset":23754,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"5 Related Work","offset":24483,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"Learning mathematical tasks.","offset":24502,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"Mechanisms of pre-trained LMs.","offset":25219,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"Fourier features in Neural Networks.","offset":26227,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"6 Conclusion","offset":27199,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"Acknowledgments","offset":28461,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"Appendix","offset":28851,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"Roadmap.","offset":28864,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"Appendix A Formal Definition of Transformer and Logits in Fourier Space","offset":29566,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"Definition A.1 (Transformer) .","offset":29741,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-2.html#text"},{"title":"Definition A.2 (Intermediate Logits) .","offset":31873,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"Definition A.3 (Fourier Basis) .","offset":32747,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"Remark A.4 (Discrete Fourier transformer (DFT) and inverse DFT) .","offset":34157,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"Definition A.5 (Logits in Fourier Space) .","offset":34487,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"Definition A.6 (Loss-pass / High-pass Filter) .","offset":36120,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"Remark A.7 .","offset":37345,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"Appendix B Fourier Components Separation and Selection of τ \\tau","offset":37757,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"Definition B.1 (Single-Pass Filter) .","offset":37895,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"Remark B.2 .","offset":38843,"level":6,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"Appendix C Does Fourier Features Generalize?","offset":42646,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"C.1 Token Embedding for Other LMs","offset":42695,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"C.2 Multiplication Task","offset":42864,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"C.3 Same Results for other format","offset":44532,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"C.4 Fourier Features in Other Pre-trained LM","offset":45079,"level":3,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"Appendix D Supporting Evidence For the Fourier Features","offset":45353,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"Appendix E More Experiments on GPT-2-XL Trained from Scratch","offset":46482,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"Appendix F Details of Experimental Settings","offset":47015,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"Fine-tuned GPT-2-XL","offset":47063,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-3.html#text"},{"title":"Train GPT-2-XL from scratch","offset":47427,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-4.html#text"},{"title":"Train GPT-2 from scratch","offset":47811,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-4.html#text"},{"title":"Create the addition dataset in main content","offset":48349,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-4.html#text"},{"title":"Create the addition dataset in Appendix C.3 with different format","offset":48939,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-4.html#text"},{"title":"Experiments Compute Resources","offset":49429,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-4.html#text"},{"title":"Licenses for Existing Assets & Open Access to Data and Code.","offset":49805,"level":4,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-4.html#text"},{"title":"Instructions for reporting errors","offset":50487,"level":2,"url":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/part-4.html#text"}],"text_code_points":51451,"source_markdown":"https://vinnik-dmitry07.github.io/favorite-papers/agents/papers/169a3e64467f1e0862508b86b2198187fc8eeb3ab114a47d2bd4caaea4a9ee2b/source.md"}