Are Emergent Abilities of Large Language Models a Mirage? — part 1/3
Cleaned-text code points [0, 15285). Bibliography omitted here; source.md includes it.
# Are Emergent Abilities of Large Language Models a Mirage?
###### Abstract
Recent work claims that large language models display emergent abilities , abilities not present in smaller-scale models that are present in larger-scale models. What makes emergent abilities intriguing is two-fold: their sharpness , transitioning seemingly instantaneously from not present to present, and their unpredictability , appearing at seemingly unforeseeable model scales. Here, we present an alternative explanation for emergent abilities: that for a particular task and model family, when analyzing fixed model outputs, emergent abilities appear due the researcher’s choice of metric rather than due to fundamental changes in model behavior with scale. Specifically, nonlinear or discontinuous metrics produce apparent emergent abilities, whereas linear or continuous metrics produce smooth, continuous, predictable changes in model performance. We present our alternative explanation in a simple mathematical model, then test it in three complementary ways: we (1) make, test and confirm three predictions on the effect of metric choice using the InstructGPT/GPT-3 family on tasks with claimed emergent abilities, (2) make, test and confirm two predictions about metric choices in a meta-analysis of emergent abilities on BIG-Bench; and (3) show how to choose metrics to produce never-before-seen seemingly emergent abilities in multiple vision tasks across diverse deep networks. Via all three analyses, we provide evidence that alleged emergent abilities evaporate with different metrics or with better statistics, and may not be a fundamental property of scaling AI models.
## 1 Introduction
Emergent properties of complex systems have long been studied across disciplines, from physics to biology to mathematics. The idea of emergence was popularized by Nobel Prize-winning physicist P.W. Anderson’s “More Is Different” ( 1 ) , which argues that as the complexity of a system increases, new properties may materialize that cannot be predicted even from a precise quantitative understanding of the system’s microscopic details. Recently, the idea of emergence gained significant attention in machine learning due to observations that large language models (LLMs) such as GPT ( 3 ) , PaLM ( 6 ) and LaMDA [ 30 ] exhibit so-called “emergent abilities” ( 33 , 8 , 28 , 3 ) (Fig. 1 ).
The term “emergent abilities of LLMs” was recently and crisply defined as “abilities that are not present in smaller-scale models but are present in large-scale models; thus they cannot be predicted by simply extrapolating the performance improvements on smaller-scale models” [ 33 ] . Such emergent abilities were first discovered in the GPT-3 family [ 3 ] . Subsequent work emphasized the discovery, writing that “[although model] performance is predictable at a general level, performance on a specific task can sometimes emerge quite unpredictably and abruptly at scale” [ 8 ] . These quotations collectively identify the two defining properties of emergent abilities in LLMs: 1. Sharpness , transitioning seemingly instantaneously from not present to present
2. Unpredictability , transitioning at seemingly unforeseeable model scales
These emergent abilities have garnered significant interest, raising questions such as: What controls which abilities will emerge? What controls when abilities will emerge? How can we make desirable abilities emerge faster, and ensure undesirable abilities never emerge? These questions are especially pertinent to AI safety and alignment, as emergent abilities forewarn that larger models might one day, without warning, acquire undesired mastery over dangerous capabilities [ 29 , 10 , 17 , 18 ] .
In this paper, we call into question the claim that LLMs possess emergent abilities, by which we specifically mean sharp and unpredictable changes in model outputs as a function of model scale on specific tasks. Our doubt stems from the observation that emergent abilities seem to appear only under metrics that nonlinearly or discontinuously scale any model’s per-token error rate. For instance, as we later show, > 92 % >92\% of emergent abilities on BIG-Bench tasks [ 28 ] (hand-annotated by [ 32 ] ) appear under either of these two metrics: Multiple Choice Grade = def { 1 if highest probability mass on correct option 0 otherwise \displaystyle\defeq\,\begin{cases}1&\text{if highest probability mass on correct option}\\ 0&\text{otherwise}\end{cases} Exact String Match = def { 1 if output string exactly matches target string 0 otherwise \displaystyle\defeq\,\begin{cases}1&\text{if output string exactly matches target string}\\ 0&\text{otherwise}\end{cases}
This raises the possibility of an alternative explanation for the origin of LLMs’ emergent abilities: sharp and unpredictable changes might be induced by the researcher’s choice of measurement, even though the model family’s per-token error rate changes smoothly, continuously and predictably with increasing scale. Specifically, our alternative posits that emergent abilities are a mirage caused primarily by the researcher choosing a metric that nonlinearly or discontinuously deforms per-token error rates, and secondarily by possessing too few test data to accurately estimate the performance of smaller models, thereby causing smaller models to appear wholly unable to perform the task.
To communicate our alternative explanation, we present it as a simple mathematical model and demonstrate how it quantitatively reproduces the evidence offered in support of emergent abilities of LLMs. We then test our alternative explanation in three complementary ways: 1. We make, test and confirm three predictions based on our alternative hypotheses using the InstructGPT [ 24 ] / GPT-3 [ 3 ] model family.
2. We meta-analyze published benchmarks [ 28 , 33 ] to reveal that emergent abilities only appear for specific metrics, not for model families on particular tasks, and that changing the metric causes the emergence phenomenon to evaporate.
3. We induce never-before-seen, seemingly emergent abilities in multiple architectures across various vision tasks by intentionally changing the metrics used for evaluation.
## 2 Alternative Explanation for Emergent Abilities
How might smooth, continuous, predictable changes in model family performance appear sharp and unpredictable? The answer is that the researcher’s choice of a nonlinear or discontinuous metric can distort the model family’s performance to appear sharp and unpredictable.
To expound, suppose that within a model family, the test loss falls smoothly, continuously and predictably with the number of model parameters. One reason to believe this is the phenomenon known as neural scaling laws: empirical observations that deep networks exhibit power law scaling in the test loss as a function of training dataset size, number of parameters or compute ( 13 , 27 , 11 , 16 , 9 , 12 , 15 , 34 , 14 , 7 , 26 ) . For concreteness, suppose we have a model family of different numbers of parameters N > 0 N>0 and assume that each model’s per-token cross entropy falls as a power law with the number of parameters N N for constants c > 0 , α < 0 c>0,\alpha<0 (Fig. 2 A):
ℒ C E ( N ) = ( N c ) α \mathcal{L}_{CE}(N)=\Big(\frac{N}{c}\Big)^{\alpha}
To be clear, we do not require this particular functional form to hold; rather, we use it for illustrative purposes. Let V V denote the set of possible tokens, p ∈ Δ | V | − 1 p\in\Delta^{|V|-1} denote the true but unknown probability distribution, and p ^ N ∈ Δ | V | − 1 \hat{p}_{N}\in\Delta^{|V|-1} denote the N N -parameter model’s predicted probability distribution. The per-token cross entropy as a function of number of parameters N N is:
ℒ C E ( N ) = def − ∑ v ∈ V p ( v ) log p ^ N ( v ) \mathcal{L}_{CE}(N)\;\defeq\;-\sum_{v\in V}p(v)\log\hat{p}_{N}(v)
In practice, p p is unknown, so we substitute a one-hot distribution of the observed token v ∗ v^{*} :
ℒ C E ( N ) = − log p ^ N ( v ∗ ) \mathcal{L}_{CE}(N)=-\log\hat{p}_{N}(v^{*})
A model with N N parameters then has a per-token probability of selecting the correct token (Fig. 2 B):
p ( single token correct ) = exp ( − ℒ C E ( N ) ) = exp ( − ( N / c ) α ) p(\text{single token correct})=\exp\Big(-\mathcal{L}_{CE}(N)\Big)=\exp\Big(-(N/c)^{\alpha}\Big)
Suppose the researcher then chooses a metric that requires selecting L L tokens correctly. For example, our task might be L L -digit integer addition, and a model’s output is scored 1 1 if all L L output digits exactly match all target digits with no additions, deletions or substitutions, 0 0 otherwise. If the probability each token is correct is independent 1 1 1 While the independence assumption is not true, the approximation yields results qualitatively matching the observed emergence claims. , the probability of scoring 1 1 is:
Accuracy ( N ) ≈ p N ( single token correct ) num. of tokens = exp ( − ( N / c ) α ) L \text{Accuracy}(N)\approx p_{N}(\text{single token correct})^{\text{num. of tokens}}=\exp\Big(-(N/c)^{\alpha}\Big)^{L}
This choice of metric nonlinearly scales performance with increasing token sequence length. When plotting performance on a linear-log plot, one sees a sharp, unpredictable emergent ability on longer sequences (Fig. 2 C) that closely matches claimed emergent abilities (inset). What happens if the researcher switches from a nonlinear metric like Accuracy, under which the per-token error rate scales geometrically in target length (App. A.3 ), to an approximately linear metric like Token Edit Distance, under which the per-token error rate scales quasi-linearly in target length (App. A.2 )?
Token Edit Distance ( N ) ≈ L ( 1 − p N ( single token correct ) ) = L ( 1 − exp ( − ( N / c ) α ) ) \text{Token Edit Distance}(N)\approx L\,\Big(1-p_{N}(\text{single token correct})\Big)=L\,\Big(1-\exp\big(-(N/c)^{\alpha}\big)\Big)
The linear metric reveals smooth, continuous, predictable changes in model performance (Fig. 2 E). Similarly, if the researcher uses a discontinuous metric like Multiple Choice Grade, the researcher can find emergent abilities (Fig. 2 D), but switching to a continuous metric like Brier Score removes the emergent ability (Fig. 2 F). In summary, sharp and unpredictable changes with increasing scale can be fully explained by three interpretable factors: (1) the researcher choosing a metric that nonlinearly or discontinuously scales the per-token error rate, (2) having insufficient resolution to estimate model performance in the smaller parameter regime, with resolution 2 2 2 Resolution is defined as “The smallest interval measurable by a scientific instrument; the resolving power.” set by 1 / test dataset size 1/\text{test dataset size} , and (3) insufficiently sampling the larger parameter regime.
## 3 Analyzing InstructGPT/GPT-3’s Emergent Arithmetic Abilities
Previous papers prominently claimed the GPT [ 3 , 24 ] family 3 3 3 As of 2023-03-15, 4 models with 350M, 1.3B, 6.7B, 175B parameters are available via the OpenAI API. displays emergent abilities at integer arithmetic tasks [ 8 , 28 , 33 ] (Fig. 2 E). We chose these tasks as they were prominently presented [ 3 , 8 , 28 , 33 ] , and we focused on the GPT family due to it being publicly queryable. As explained mathematically and visually in Sec. 2 , our alternative explanation makes three predictions: 1. Changing the metric from a nonlinear or discontinuous metric (Fig. 2 CD) to a linear or continuous metric (Fig. 2 EF) should reveal smooth, continuous, predictable performance improvement with model scale.
2. For nonlinear metrics, increasing the resolution of measured model performance by increasing the test dataset size should reveal smooth, continuous, predictable model improvements commensurate with the predictable nonlinear effect of the chosen metric .
3. Regardless of metric, increasing the target string length should predictably affect the model’s performance as a function of the length-1 target performance: approximately geometrically for accuracy and approximately quasilinearly for token edit distance.
To test these predictions, we collected outputs from the InstructGPT/GPT-3 family on two tasks: 2-shot multiplication between two 2-digit integers and 2-shot addition between two 4-digit integers.
#### Prediction: Emergent Abilities Disappear With Different Metrics
On both arithmetic tasks, the GPT family displays emergent abilities if the target has 4 or 5 digits and if the metric is Accuracy (Fig. 3 , top) [ 3 , 8 , 33 ] . However, if one changes from nonlinear Accuracy to linear Token Edit Distance while keeping the models’ outputs fixed , the family’s performance smoothly, continuously and predictably improves with increasing scale (Fig. 3 , bottom). This confirms our first prediction and supports our alternative explanation that the source of emergent abilities is the researcher’s choice of metric, not changes in the model family’s outputs . We also observe that under Token Edit Distance, increasing the length of the target string from 1 to 5 predictably decreases the family’s performance in an approximately quasilinear manner, confirming the first half of our third prediction.
#### Prediction: Emergent Abilities Disappear With Better Statistics
We next tested our second prediction: that even on nonlinear metrics such as accuracy, smaller models do not have zero accuracy, but rather have non-zero above-chance accuracy commensurate with choosing to use accuracy as the metric . In order to accurately measure models’ accuracy, we increased the resolution by generating additional test data, and found that on both arithmetic tasks, all models in the InstructGPT/GPT-3 family achieve above-chance accuracy (Fig. 4 ). This confirms our second prediction. We also observe that as the target string length increases, the accuracy falls approximately geometrically with the length of the target string, confirming the second half of our third prediction. These results additionally demonstrate that the researcher’s choice of metric has the effect that one should predict accuracy to have, i.e., geometric decay with the target length.
## 4 Meta-Analysis of Claimed Emergent Abilities
Analyzing the GPT family is possible because the models are publicly queryable. However, other model families claimed to exhibit emergent abilities are not publicly queryable, nor are their generated outputs publicly available, meaning we are limited to analyzing the published results themselves [ 8 , 33 , 32 ] . Our alternative explanation makes two predictions. 1. At the “population level” of Task-Metric-Model Family triplets, emergent abilities should appear predominantly on specific metrics , not task-model family pairs, and specifically with nonlinear and/or discontinuous metrics.
2. On individual Task-Metric-Model Family triplets that display an emergent ability, changing the metric to a linear and/or continuous metric should remove the emergent ability.
To test these predictions, we used to claimed emergent abilities on BIG-Bench [ 28 , 33 ] due to the benchmark being pertinent and publicly available.
#### Prediction: Emergent Abilities Should Appear with Metrics, not Task-Model Families