Pre-trained Large Language Models Use Fourier Features to Compute Addition — part 1/4

Cleaned-text code points [0, 15633). Bibliography omitted here; source.md includes it.

# Pre-trained Large Language Models Use Fourier Features to Compute Addition

Pre-trained large language models (LLMs) exhibit impressive mathematical reasoning capabilities, yet how they compute basic arithmetic, such as addition, remains unclear. This paper shows that pre-trained LLMs add numbers using Fourier features—dimensions in the hidden state that represent numbers via a set of features sparse in the frequency domain. Within the model, MLP and attention layers use Fourier features in complementary ways: MLP layers primarily approximate the magnitude of the answer using low-frequency features, while attention layers primarily perform modular addition (e.g., computing whether the answer is even or odd) using high-frequency features. Pre-training is crucial for this mechanism: models trained from scratch to add numbers only exploit low-frequency features, leading to lower accuracy. Introducing pre-trained token embeddings to a randomly initialized model rescues its performance. Overall, our analysis demonstrates that appropriate pre-trained representations (e.g., Fourier features) can unlock the ability of Transformers to learn precise mechanisms for algorithmic tasks.

## 1 Introduction

Mathematical problem solving has become a crucial task for evaluating the reasoning capabilities of large language models (LLMs) [ 19 , 7 , 22 , 11 ] . While LLMs exhibit impressive mathematical abilities [ 34 , 17 , 1 , 44 , 39 , 4 , 12 ] , it remains unclear how they perform even basic mathematical tasks. Do LLMs apply mathematical principles when solving math problems, or do they merely reproduce memorized patterns from the training data?

In this work, we unravel how pre-trained language models solve simple mathematical problems such as “Put together 15 15 and 93 93 . Answer: ”. Prior work has studied how Transformers, the underlying architecture of LLMs, perform certain mathematical tasks. Most studies [ 6 , 18 , 41 , 2 , 10 , 29 , 15 , 36 ] focus on Transformers with a limited number of layers or those trained from scratch; [ 20 ] analyzes how the pre-trained GPT-2-small performs the greater-than task. Our work focuses on a different task from prior interpretability work—integer addition—and shows that pre-trained LLMs learn distinct mechanisms from randomly initialized Transformers.

In § 3 , we show that pre-trained language models compute addition with Fourier features—dimensions in the hidden state that represent numbers via a set of features sparse in the frequency domain. First, we analyze the behavior of pre-trained LLMs on the addition task after fine-tuning, which leads to almost perfect accuracy on the task. Rather than merely memorizing answers from the training data, the models progressively compute the final answer layer by layer. Next, we analyze the contributions of individual model components using Logit Lens [ 3 ] . We observe that some components primarily approximate the answer—they promote all numbers close to the correct answer in magnitude—while other components primarily classify the answer modulo m m for various numbers m m . Then, we use Fourier analysis to isolate features in the residual stream responsible for the low-frequency “approximation” and high-frequency “classification” subtasks. Identifying these features allows us to precisely ablate the ability of the model to perform either approximation or classification by applying a low-pass or high-pass filter, respectively, to the outputs of different model components. We find that MLP layers contribute primarily to approximation, whereas attention layers contribute primarily to classification.

In § 4 , we show that pre-training is crucial for learning this mechanism. The same network trained from scratch with random initialization not only shows no signs of Fourier features, but also has lower accuracy. We identify pre-trained token embeddings as a key source of inductive bias that help the pre-trained model learn a more precise mechanism for addition. Across the pre-trained token embeddings of many different pre-trained models, Fourier analysis uncovers large magnitudes of components with periods 2 2 , 5 5 , and 10 10 . Introducing pre-trained token embeddings when training the model from scratch enables the model to achieve perfect test accuracy. Finally, we show that the same Fourier feature mechanism is present not only in models that were pre-trained and then fine-tuned, but also in frozen pre-trained LLMs when prompted with arithmetic problems.

Overall, our work provides a mechanistic perspective on how pre-trained LLMs compute addition through the lens of Fourier analysis. It not only broadens the scope from only investigating few-layer Transformers trained to fit a particular data distribution to understanding LLMs as a whole, but also hints at how pre-training can lead to more precise model capabilities.

## 2 Problem Setup

#### Task and Dataset.

We constructed a synthetic addition dataset for fine-tuning and evaluation purposes. Each example involves adding two numbers ≤ 260 \leq 260 , chosen because the maximum number that can be represented by a single token in the GPT-2-XL tokenizer is 520 520 . For each pair of numbers between 0 0 and 260 260 , we randomly sample one of five natural language question templates and combine it with the two numbers. The dataset is shuffled and then split into training ( 80 % 80\% ), validation ( 10 % 10\% ), and test ( 10 % 10\% ) sets. More details are provided in Appendix F . In Appendix C.3 , we show our that results generalize to a different dataset formatted with reverse Polish notation.

#### Model.

Unless otherwise stated, all experiments focus on the pre-trained GPT-2-XL model that has been fine-tuned on our addition dataset. This model, which consists of 48 48 layers and approximately 1.5 1.5 billion parameters, learns the task almost perfectly, with an accuracy of 99.74 % 99.74\% on the held-out test set. We examine other models in § 4.2 and § 4.3 .

#### Transformers.

We focus on decoder-only Transformer models [ 42 ] , which process text sequentially, token by token, from left to right. Each layer ℓ \ell in the Transformer has an attention module with output Attn ( ℓ ) \mathrm{Attn}^{(\ell)} and an MLP module with output MLP ( ℓ ) \mathrm{MLP}^{(\ell)} . Their outputs are added together to create a continuous residual stream h h [ 9 ] , meaning that the token representation accumulates all additive updates within the residual stream, with the representation h ( ℓ ) h^{(\ell)} in the ℓ \ell -th layer given by: h ( ℓ ) = h ( ℓ − 1 ) + Attn ( ℓ ) + MLP ( ℓ ) . \displaystyle h^{(\ell)}=h^{(\ell-1)}+\mathrm{Attn}^{(\ell)}+\mathrm{MLP}^{(\ell)}. (1) The output embedding W U W^{U} projects the residual stream to the space of the vocabulary; applying the softmax function then yields the model’s prediction. We provide formal definitions in Appendix A .

## 3 Language Models Solve Addition with Fourier Features

In this section, we analyze the internal mechanisms of LLMs when solving addition tasks, employing a Fourier analysis framework. We first show that the model initially approximates the solution before iteratively converging to the correct answer (§ 3.1 ). We then show that the model refines its initial approximation by computing the exact answer modulo 2 2 , 5 5 , and 10 10 , employing Fourier components of those same periods (§ 3.2 ). Finally, we demonstrate through targeted ablations that the identified Fourier components are causally important for the model’s computational processes (§ 3.3 ). Specifically, we show that MLP layers primarily approximate the magnitude of the answer, using low-frequency features, while attention layers primarily perform modular addition using high-frequency components.

### 3.1 Behavioral Analysis

Our first goal is to understand whether the model merely memorizes and recombines pieces of information learned during training, or it performs calculations to add two numbers.

#### Extracting intermediate predictions.

To elucidate how LLMs perform computations and progressively refine their outputs towards the correct answer, we extract model predictions at each layer from the residual stream. Let L L denote the number of layers. Using the Logit Lens method [ 3 ] , instead of generating predictions by computing logits W U ​ h ( L ) W^{U}h^{(L)} , predictions are derived through W U ​ h ( ℓ ) W^{U}h^{(\ell)} where ℓ ∈ [ L ] \ell\in[L] . We compute the accuracy of the prediction using each intermediate state h ( ℓ ) h^{(\ell)} .

If the models merely retrieve and recombine pieces of information learned during training, certain layers will directly map this information to predictions. For instance, [ 25 ] demonstrates that there is a specific MLP module directly maps a country to its capital.

#### LLMs progressively compute the final answers.

Figure 1 a instead shows that the model progressively approaches the correct answer, layer by layer. The model is capable of making predictions that fall within the range of ± 2 \pm 2 and ± 10 \pm 10 relative to the correct answer in the earlier layers, compared to the exact-match accuracy. This observation implies that the Transformer’s layer-wise processing structure is beneficial for gradually refining predictions through a series of transformations and updates applied to the token representations.

### 3.2 Fourier Features in MLP & Attention Outputs

#### Logits for MLP and attention have periodic structures.

We now analyze how each MLP and attention module contributes to the final prediction. We transform the output of the attention and MLP output at layer ℓ \ell into the token space using W U ​ Attn ( ℓ ) W^{U}\mathrm{Attn}^{(\ell)} and W U ​ MLP ( ℓ ) W^{U}\mathrm{MLP}^{(\ell)} at each layer, thereby obtaining the logits ℒ \mathcal{L} for each MLP and attention module. We use the running example “Put together 15 15 and 93 93 . Answer: 108 108 ” to demonstrate how the fine-tuned GPT-2-XL performs the computation. As illustrated in Figure 1 b and Figure 1 c, both the MLP and attention modules exhibit a periodic pattern in their logits across the output number space, e.g., the MLP in layer 33 33 , outlined in green, promotes all numbers that are congruent to 108 ​ mod ​ 2 108\textrm{ mod }2 (in Figure 19 in the appendix, we zoom into such layers to make this clearer). Overall, we observe two distinct types of computation within these components. Some components predominantly assign a high weight to numbers around the correct answer, which we term approximation . Meanwhile, other components predominantly assign a high weight to all numbers congruent to a + b ​ mod ​ c a+b\textrm{ mod }c for some constant c c , which we term classification .

#### Logits for MLP and attention are approximately sparse in the Fourier space.

It is natural to transform the logits into Fourier space to gain a better understanding of their properties such as the periodic pattern. We apply the discrete Fourier transform to represent the logits as the sum of sine and cosine waves of different periods: the k k -th component in Fourier space has period 520 / k 520/k and frequency k / 520 k/520 (see Appendix A for more details). Let ℒ ^ \widehat{\mathcal{L}} denote the logits in Fourier space. Figure 2 shows the Fourier space logits for two layers from Figure 1 b and Figure 1 c that have a clear periodic pattern. We find that the high-frequency components in Fourier space, which we define as components with index greater or equal to 50 50 , are approximately sparse as depicted in Figure 2 . This observation aligns with [ 29 ] , which found that a one-layer Transformer utilizes particular Fourier components within the Fourier space to solve the modular addition task.

In Figure 3 , we show that similar sparsity patterns in Fourier space hold across the entire dataset. We compute the logits in Fourier space for the last 15 15 layers, i.e., ℒ ^ Attn ( ℓ ) \widehat{\mathcal{L}}_{\mathrm{Attn}}^{(\ell)} and ℒ ^ MLP ( ℓ ) \widehat{\mathcal{L}}_{\mathrm{MLP}}^{(\ell)} where ℓ ∈ [ 32 , 47 ] \ell\in[32,47] , for all test examples and average them. We annotate the top- 10 10 outlier high-frequency components based on their magnitude. The MLPs also exhibit some strong low-frequency components; the attention modules do not exhibit strong low-frequency components, only high-frequency components.

#### Final logits are superpositions of these outlier Fourier components.

The final logits, ℒ ( L ) \mathcal{L}^{(L)} , are the sum of all ℒ MLP ( l ) \mathcal{L}^{(l)}_{\mathrm{MLP}} and ℒ Attn ( l ) \mathcal{L}^{(l)}_{\mathrm{Attn}} across all layers l ∈ [ L ] l\in[L] . Figure 4 elucidates how these distinct Fourier components contribute to the final prediction, for the example “Put together 15 15 and 93 93 . Answer: 108 108 ”. We select the top- 5 5 Fourier components of ℒ ^ ( L ) \widehat{\mathcal{L}}^{(L)} based on their magnitudes and transfer them back to logits in number space via the inverse discrete Fourier transform (Figure 4 a). The large-period (low-frequency) components approximate the magnitude while the small-period (high-frequency) components are crucial for modular addition. Figure 4 b shows that aggregating these 5 5 waves is sufficient to predict the correct answer.

#### Why is high-frequency classification helpful?

The Fourier basis comprises both cos \cos and sin \sin waves (see Definition A.3 ). By adjusting the coefficients of cos \cos and sin \sin , the trained model can manipulate the phase of the logits in Fourier space (number shift in number space), aligning the peak of the wave more closely with the correct answer. As shown in Figure 4 a, consider a wave with a period of 2 2 . Here, the peak occurs at every even number in the number space, corresponding to the mod ​ 2 \textrm{ mod }2 task. In contrast, for components with a large period such as 520 520 , the model struggles to accurately position the peak at 108 108 (also see Figure 13 in the appendix for the plot of this component with period 520 520 in the full number space). This scenario can be interpreted as solving a “ mod 520 ” task—a classification task among 520 520 classes—which is challenging for the model to learn accurately. Nevertheless, even though the component with a period of 520 520 does not solve the “ mod 520 ” task precisely, it does succeed in assigning more weight to numbers near 108 108 . The classification results from the high-frequency components can then provide finer-grained resolution to distinguish between all the numbers around 108 108 assigned a large weight by the lower frequencies. Due to this, the low-frequency components need not be perfectly aligned with the answer to make accurate predictions.

### 3.3 Fourier Features are Causally Important for Model Predictions

In the previous section, we demonstrated that there are outlier Fourier components in the logits generated by both the MLP and attention modules, as shown in Figure 3 . We also illustrated that, in one example, the high-frequency components primarily approximate the magnitude, while the low-frequency components are crucial for modular addition tasks, as depicted in Figure 4 . In this section, through an ablation study conducted across the entire test dataset, we show that both types of components are essential for correctly computing sums. Moreover, we reveal that the MLP layers primarily approximate the magnitude of the answer using low-frequency features, whereas the attention layers are responsible for modular addition using high-frequency features.

#### Filtering out Fourier components.