The Lookahead Limitation: Why Multi-Operand Addition is Hard for LLMs — part 1/3
Cleaned-text code points [0, 15820). Bibliography omitted here; source.md includes it.
# The Lookahead Limitation: Why Multi-Operand Addition is Hard for LLMs
###### Abstract
Autoregressive large language models (LLMs) exhibit impressive performance across various tasks but struggle with simple arithmetic, such as additions of two or more operands. We show that this struggle arises from LLMs’ use of a simple one-digit lookahead heuristic , which works fairly well (but not perfect) for two-operand addition but fails in multi-operand cases, where the carry-over logic is more complex. Our probing experiments and digit-wise accuracy evaluation show that LLMs fail precisely where a one-digit lookahead is insufficient to account for cascading carries. We analyze the impact of tokenization strategies on arithmetic performance and show that all investigated models, regardless of tokenization, are inherently limited in the addition of multiple operands due to their reliance on a one-digit lookahead heuristic. Our findings reveal fundamental limitations that prevent LLMs from generalizing to more complex numerical reasoning.
## 1 Introduction
Large language models (LLMs) demonstrate remarkable performance across a wide range of tasks Bai et al. (2023) ; Team et al. (2024) ; Guo et al. (2025) , yet consistently struggle with simple arithmetic tasks, such as the addition of multiple or large numbers McLeish et al. (2024) ; Shen et al. (2023) ; Zhou et al. (2023) ; Zhou et al. (2024) .
Figure 1 shows an example of an addition with 2 operands, 147 147 and 255 255 , each with three digits ( 0 0 to 9 9 ). The length of an operand is the number of digits it contains. Figure 1 provides an example where the LLM fails (even in a two-operand case) to provide a correct output due to its insensitivity to a carry emerging from later computations.
The difficulty LLMs face in such tasks stems from the mismatch between the left-to-right nature of autoregressive language modeling and the right-to-left structure of standard arithmetic algorithms. Conventional addition methods process numbers digit by digit from right to left, propagating carries, while LLMs generate numbers sequentially from left to right without explicit intermediate calculations. This raises the question: What strategy do LLMs use to handle this misalignment in addition?
In this work, we show that in fact LLMs rely on a simple heuristic that enables high (though not perfect) accuracy in adding two operands (e.g., 147 + 291 = 438 147+291=438 , henceforth two-operand addition ). This heuristic attempts to bridge the gap between the left-to-right generation and the resulting need to ’look ahead’ to account for propagating carries from less significant digits. Rather than performing an exhaustive lookahead to fully anticipate carry propagation, LLMs rely on a simple heuristic that involves a lookahead of only a single digit to anticipate the value of carries in addition . We show that while this strategy works fairy well for two-operand addition, due to relevant digit combinatorics, it deteriorates substantially with multiple operands (e.g., in four-operand addition such as 147 + 245 + 312 + 104 = 808 147+245+312+104=808 , henceforth generalized as multi-operand addition for any number of operands > 2 >2 ), where anticipating carries becomes less predictable. The reliance on the heuristic explains the lack of robustness in LLMs’ arithmetic performance.
Figure 1 illustrates this shortcoming of the heuristic: A one-digit lookahead anticipates no carry (because for the sum of the second, i.e. middle, digits in the operands 4 + 5 = 9 4+5=9 ), leading to the inaccurate prediction of the first result digit as 3 3 , unable to accurately anticipate the cascading carry originating from the unit position.
To gather evidence that the heuristic accurately describes the strategy used by LLMs to solve addition from left to right, we present results from three state-of-the-art LLMs with different tokenization strategies (single digit and multiple digit) for numerical outputs. By evaluating prediction accuracy on carefully curated datasets and employing probing techniques, we provide multiple lines of evidence that LLMs struggle specifically with addition tasks where a one-digit lookahead is insufficient to account for cascading carries. For instance, in two-operand addition, we show that this issue occurs when the sum of the digits at the lookahead position is 9 9 , leading to failure in correctly predicting the numerical value at the current position. For example, in 147 + 255 = 147+255= , no carry is predicted for the middle digits, even though a cascading carry from the 10 0 10^{0} position affects the sum of the 10 1 10^{1} digits, and thus the 10 2 10^{2} position.
Our findings show that all investigated LLMs are inherently limited in their performance on multi-operand addition tasks due to this heuristic, regardless of their tokenization strategy.
Our contributions are as follows: • Evaluation of Addition Capabilities : We show that LLMs fail on multi-operand addition (Section 2 ) and then systematically evaluate the capabilities of LLMs on two-operand addition tasks via probing (Section 3 ).
• Heuristic Discovery : Inspired by results of the evaluation, we formalize left-to-right addition in LLMs for multi-operand addition with a simple heuristic that uses a shallow lookahead of one to attempt left-to-right addition ( H1 , Section 4 ).
• Empirical Validation : We demonstrate that H1 is fragile in multi-operand addition and explain the performance decline as a function of the increasing number of operands in large comprehensive addition experiments. We find that model performance aligns precisely with the predicted limitations of H1 (Sections 5 and 6 ). We find that H1 holds independently of tokenization strategies (Section 7 ).
## 2 LLMs Struggle with Multi-Operand Addition
In this section, we define the data and models used in this work and demonstrate that LLMs fail on multi-operand additions by looking at prediction accuracy.
### 2.1 Models and Data
#### Models.
We compare Mistral-7B Jiang et al. (2023) , Gemma-7B Team et al. (2024) and Meta-Llama-3-8B Grattafiori et al. (2024) ; AI@Meta (2024) as they employ different tokenization strategies for numerical outputs: While Mistral and Gemma exclusively employ a single-digit tokenization strategy for their numeric input and generated output (e.g., input = [’1’, ’4’, ’7’, ’+’, ’2’, ’5’, ’5’, ’=’], output = [’4’, ’0’, ’2’]), Llama-3 employs a multi-digit numeric tokenization strategy (e.g., input = [’ 147’, ’ +’, ’ 255’, ’ =’], output = [’ 402’]), typically favoring numeric tokens of length 3.
#### Data.
For all experiments in this paper, we compile a range of datasets containing simple arithmetic task prompts of the form 147 + 255 = . We create a dataset for each addition task ranging from 2-operand to 11-operand addition, where each operand is a triple-digit number between 100 and 899. Each of the 10 datasets contains 5,000 unique arithmetic problems, both in a zero-shot and one-shot setting. In the zero-shot setting, an example for a 2-operand addition prompt is “147 + 255 = ”. An example for a 4-operand addition prompt is “251 + 613 + 392 + 137 = ”. Our one-shot prompt template follows the scheme q1 r1; q2 , e.g. “359 + 276 = 635; 147 + 255 = ”, where q1 is a sample query from the same dataset and r1 is the correct result of the addition task in q1 . q2 is the query containing the addition task to be solved.
In the remainder of the paper, we use s n s_{n} (with n ≥ 0 n\geq 0 ) to denote the result digit generated at digit position 10 n 10^{n} . For example, in “147 + 255 =”, with expected output 402, s 2 = 4 s_{2}=4 , s 1 = 0 s_{1}=0 , and s 0 = 2 s_{0}=2 .
### 2.2 LLM Accuracy on Addition Tasks
Figure 2 illustrates the significant decline in performance of Mistral-7B Jiang et al. (2023) , Gemma-7B Team et al. (2024) and Meta-Llama-3-8B AI@Meta (2024) in multi-operand addition as the number of operands increases. This drastic decrease highlights the inability of these models to generalize effectively to addition tasks involving a higher number of operands, despite their strong overall capabilities.
## 3 Probing LLMs on Digits in Two-Operand Addition Tasks
Solving arithmetic tasks presents a fundamental challenge for LLMs, as they generate text from left to right, while addition requires a right-to-left process due to carry propagation from the least significant to the most significant digit. For instance, predicting the first result digit s 2 = 4 s_{2}=4 in “147 + 255 = ” requires the model to anticipate that a carry originating from s 0 s_{0} cascades through s 1 s_{1} to s 2 s_{2} . Robust left-to-right addition thus requires a lookahead spanning all result digits, raising the question: Do LLMs internally represent future result digits when predicting s 2 s_{2} - and if so, how far can they “look into the future”?
To answer this question, we probe whether models accurately encode future result digits s 1 s_{1} or s 0 s_{0} while generating s 2 s_{2} . Building on Levy and Geva (2024) , who show that, irrespective of a model’s numeric tokenization strategy, LLMs internally represent numbers digit-by-digit in base 10, we analyze digit-wise probing accuracy on the two-operand addition dataset described in Section 2.1 .
### 3.1 Methodology and Experiments
#### Data.
We split the two-operand addition dataset (see Section 2.1 ) into train (n=4500) and test (n=500) for the probing experiments. The two-operand addition dataset is designed such that correct results for the addition tasks are triple-digit numbers between 200 and 999. We use the zero-shot prompt setting for the probing experiment.
#### Probing Setup.
Our goal is to determine which result digits are available at the prediction step of s 2 s_{2} . We thus train probes to predict the result digits s 2 s_{2} , s 1 s_{1} , and s 0 s_{0} from hidden states of the model during the prediction step of s 2 s_{2} .
Specifically, we train one-layer linear probes to predict individual digit values of the results from the hidden state of the last token at each model layer. Probes are trained on the train split of the two-operand addition dataset and evaluated on the test split. We train separate probes to predict individual result digits s 2 s_{2} , s 1 s_{1} , and s 0 s_{0} , for all models at all layers. 1 1 1 We choose a low temperature of 0.1 during model inference to ensure deterministic and consistent outputs, reducing randomness in token generation and improving the reliability of numerical calculations.
### 3.2 Results
The probing accuracy of individual result digits is shown in Figure 3 . Gemma and Mistral with their digit-wise tokenization internally represent only s 2 s_{2} with high accuracy. In contrast, there is a high probing accuracy across all result digits in Llama-3. This is due to the fact that Llama-3 tokenizes numbers into 3-digit numeric tokens: It is forced by its tokenization to generate all result digits ( s 2 s_{2} , s 1 s_{1} , and s 0 s_{0} ) in one step as a single token.
The single-digit tokenization models Mistral and Gemma exhibit a low probing accuracy on s 0 s_{0} ( < 0.24 <0.24 ) in all layers. Recall that s 0 s_{0} is probed from the models’ hidden states while they autoregressively generate s 2 s_{2} . We interpret the lack of internal representation of s 0 s_{0} as evidence that these models disregard the potential influence of s 0 s_{0} (including any cascading carry) when generating s 2 s_{2} .
In line with this, Gemma and Mistral show notably higher probing accuracy on s 1 s_{1} compared to s 0 s_{0} , when probing from the models’ hidden states as they generate s 2 s_{2} . We thus conjecture that the single-digit-token models seem to recognize the potential influence of the carry resulting from the sum of the 10 1 10^{1} operand digits. Simply put, generating the digit at 10 2 10^{2} might employ a lookahead of one digit to the 10 1 10^{1} intermediate result. Based on this observation, we formulate a hypothesis for a heuristic used by LLMs: H1: LLMs employ a look ahead of one digit to generate the current digit of an addition task.
H1 would explain why LLMs cannot effectively represent each necessary digit of the result during generation, making it difficult to anticipate later carry values correctly. We first formalize H1 , which explains the patterns observed in Figure 3 , in the next Section, and then verify the fit of H1 with empirical addition outcomes generated by the models in Sections 5 , 6 , and 7 .
## 4 The Carry Heuristic of LLMs
Since LLMs generate numbers from left to right, they must anticipate whether a carry from later digits (with lower bases further on in the result) will impact the current digit they are generating. In this section, we evaluate the maximum accuracy LLMs can achieve in addition tasks, assuming they rely on H1 , given the limited lookahead of one digit.
### 4.1 Formalization of Left-to-Right Addition in Base 10
We first formalize a recursive algorithm for solving addition of k k operands-where each operand is a base 10 integer- in a left-to-right manner.
We define: • k k : Number of operands.
• n 1 , n 2 , … , n k n_{1},n_{2},\dots,n_{k} : Operands, each represented as digit sequences in base 10 10 , with 0 ≤ i < d \quad 0\leq i<d , where d d is the number of digits in the operands: n j = [ n j , d − 1 , … , n j , 0 ] , n j , i ∈ { 0 , … , 9 } n_{j}=[n_{j,d-1},\dots,n_{j,0}],\quad n_{j,i}\in\{0,\dots,9\}
• S S : The result of the addition. S = [ s d , s d − 1 , … s 0 ] S=[s_{d},s_{d-1},\dots s_{0}] , where s d = c d s_{d}=c_{d} , i.e., the final carry.
We recursively define the calculation of individual result digits: • Total Sum at Digit Position i i : t i = ∑ j = 1 k n j , i t_{i}=\sum_{j=1}^{k}n_{j,i} T i = t i + c i T_{i}=t_{i}+c_{i} where t i t_{i} is the digit sum at the current position, c i c_{i} the carry from the previous digit position, and k k the number of operands. Base case: c 0 = 0 c_{0}=0 , no carry at the least significant digit.
• Result Digit at Position i i : s i = T i mod 10 s_{i}=T_{i}\mod 10
• Carry to the Next Digit Position: c i + 1 = ⌊ T i 10 ⌋ c_{i+1}=\left\lfloor\frac{T_{i}}{10}\right\rfloor
A worked example is provided in Appendix A .
### 4.2 A Naive Heuristic for Solving Addition Left-to-Right
Due to the recursive nature of left-to-right addition, a lookahead of i − 1 i-1 digits is needed to determine any result digit s i s_{i} . There is however a simple, non-recursive heuristic for the estimation of s i s_{i} with only a one-digit lookahead, to the digit sum of the next position, i.e. only considering t i − 1 t_{i-1} .
We define c m i n c_{min} and c m a x c_{max} to be the minimal and maximal possible value for a carry, where trivially for all cases, c m i n = 0 c_{min}=0 , and c m a x ( k ) = ⌊ ∑ j = 1 k 9 10 ⌋ c_{max}(k)=\left\lfloor\frac{\sum_{j=1}^{k}9}{10}\right\rfloor in base 10 10 and for k k operands. We then define the carry heuristic c i h c_{i}^{h} as follows: c i h ∈ { ⌊ t i − 1 + c m i n 10 ⌋ , ⌊ t i − 1 + c m a x 10 ⌋ } c_{i}^{h}\in\{\left\lfloor\frac{t_{i-1}+c_{min}}{10}\right\rfloor,\left\lfloor\frac{t_{i-1}+c_{max}}{10}\right\rfloor\} Where c i h c_{i}^{h} is chosen uniformly at random. We then accordingly define the predicted total sum at digit position i T i h = t i + c i h T_{i}^{h}=t_{i}+c_{i}^{h}
and the predicted result digit
s i h = T i h mod 10 s_{i}^{h}=T_{i}^{h}\mod 10
#### Examples.
We show two examples of two-operand addition, one in which H1 is successful, and one in which it fails. For k = 2 k=2 , i.e., in two-operand addition: c m a x ( 2 ) = ⌊ ∑ j = 1 2 9 10 ⌋ = 1 c_{max}(2)=\left\lfloor\frac{\sum_{j=1}^{2}9}{10}\right\rfloor=1
#### 147 + 293.