Emergent Capabilities Arise Randomly from Learning Sparse Attention Patterns — part 2/4

Cleaned-text code points [15967, 31878). Bibliography omitted here; source.md includes it.

Figure 3 (left) visualizes the transition dynamics for our linear map task. We always use C = 2 C=2 and T = 2 T=2 , so that cells contain binary values and the model is trained solely on an initial random state and the subsequent transition state. Formally, the linear map transition function is f ⁡ ( x ) = A ​ x mod 2 f(x)=Ax\bmod 2 given a sparse matrix A ∈ { 0 , 1 } S × S A\in\{0,1\}^{S\times S} . As shown in Figure 3 , predicting each output bit x t + 1 , i x_{t+1,i} is equivalent to computing the bitwise parity of a subset of bits in the previous state indexed by A i , : A_{i,:} . We sample a constant A A for each training run, with the constraint that each row of A A must contain exactly s s nonzero entries for some sparsity level s ∈ { 1 , 2 , … , S } s\in\{1,2,\dots,S\} . Unless otherwise specified, we set S = 16 S=16 and s = 3 s=3 so that the task is sufficiently complex yet still learnable by a transformer (see Figure 5(a) ).

#### Cellular Automata.

Figure 3 (right) demonstrates our process for generating cellular automata datasets. A lookup table R : { 0 , … , C − 1 } 3 → { 0 , … , C − 1 } R:\{0,\dots,C-1\}^{3}\to\{0,\dots,C-1\} determines how to compute a single bit of x t + 1 x_{t+1} given a local window x t , i − 1 : i + 1 x_{t,i-1:i+1} . Each lookup table defines a rule which maps r R ( x ) i = R ( x t , i − 1 : i + 1 ) r_{R}(x)_{i}=R(x_{t,i-1:i+1}) for all i ∈ { 0 , 1 , … , S − 1 } i\in\{0,1,\dots,S-1\} . We introduce two complexity parameters: the number of total rules N N and the recursive depth k k . We sample N N lookup tables for each training run, where each lookup table is composed k k times for each state transition. To generate a single training example, we sample a single rule and apply the transition function x t + 1 = f ⁡ ( x ) = r k ​ ( x ) x_{t+1}=f(x)=r^{k}(x) iteratively starting with a random initial state. When N = 1 N=1 , this task involves learning the local window attention pattern and memorizing a static lookup table. However, for N > 1 N>1 , the model must first infer the rule r r from the prior context x 0 : t x_{0:t} before computing a posterior prediction for x t + 1 x_{t+1} .

These two tasks enable us to study two important factors of attention pattern learning in depth. In the linear map task, we vary the sparsity of the attention pattern by varying both the state size S S and sparsity parameter s s . We can also precisely measure how accurately a model has learned the ground-truth pattern by comparing to the matrix A A that generates the data. The cellular automata task consists of a simple, fixed local window attention pattern, but for N > 1 N>1 rules the model must apply this pattern across multiple states to infer the active rule for each sample. In contrast to the linear map task, we study the effect of context length by increasing the state size or trajectory length.

### 3.2 Abrupt Learning with Linear Maps

We begin by reproducing the abrupt learning phenomenon in Section 2 in our synthetic linear map setting. We train a single layer transformer model with hidden dimension 128 128 , MLP dimension 512 512 , 8 8 attention heads, and a binary vocabulary on the linear map task with state size S = 16 S=16 and sparsity s = 3 s=3 . Solving the sparse linear map directly corresponds to learning the ground-truth attention pattern , as we visualize in Figure 4 . We observe abrupt learning throughout training, characterized by long plateaus in the overall loss followed by sharp jumps. By tracking the loss on each individual output token, we see that loss jumps correspond to abruptly learning a single row of the ground-truth matrix A A .

We also track the entropy − ∑ i = S 2 ​ S − 1 ∑ j = 0 S − 1 s i ​ j log s i ​ j -\sum_{i=S}^{2S-1}\sum_{j=0}^{S-1}s_{ij}\log s_{ij} for each head across queries x S : 2 ​ S − 1 x_{S:2S-1} and keys x 0 : S − 1 x_{0:S-1} , where s i ​ j s_{ij} is the batch-averaged softmax score for query position i i and key position j j . In Figure 4 , the highlighted loss jump corresponds to a sharp decrease in entropy for two attention heads (see Appendix B.2 ). We visualize the attention heads before and after the loss jump and observe a transition from roughly uniform attention toward the ground-truth pattern for a single row. We note this abrupt learning is also random ; different initialization seeds learn the sparse linear map rows in different orders and at different times.

Since we define the ground-truth attention pattern by the matrix A A , we can apply the attention intervention proposed by Gopalani and Hu (2025) . We apply an attention bias to each attention head, where we add c ⋅ A c\cdot A to the attention logits for queries x S : 2 ​ S − 1 x_{S:2S-1} and keys x 0 : S − 1 x_{0:S-1} before computing softmax scores. We find that this intervention enables the model to learn the linear map task smoothly and almost instantly, with loss curves resembling those of a single jump in Figure 4 . We provide the complete training results in Appendix B.3 . This immediate learning under the attention intervention indicates that the linear map task is attention-heavy : the main bottleneck is in learning the correct attention pattern, and the subsequent computation of the dot-product mod 2 \bmod\ 2 is relatively trivial.

We now study how context length and attention sparsity affect the difficulty of learning the linear map task. We vary the state size S S to control context length, and sweep the sparsity ratio s / S s/S , which controls what proportion of the input state the model must attend to for predicting each output bit. In Figure 5(a) we illustrate final loss as a function of the ratio s / S s/S , sweeping across all s ∈ { 1 , 2 , … , S } s\in\{1,2,\dots,S\} for S ∈ { 8 , 16 , 32 } S\in\{8,16,32\} . We see that for a small state size ( S = 8 S=8 ), the model can learn any linear map regardless of the sparsity level. However, at S = 16 S=16 the task becomes more difficult, especially for linear maps with medium sparsity s / S ≈ 0.5 s/S\approx 0.5 . At S = 32 S=32 , the task is almost always unlearnable after 10,000 steps except for the simplest attention patterns ( s = 1 s=1 , s = S s=S ). We generally see that the difficulty of learning an attention pattern depends jointly on sparsity and context length; even extremely sparse or dense patterns become unlearnable at larger S S . We now demonstrate the same effect in the cellular automata task, where the attention pattern is simpler but increasing the context length has a multiplicative effect on learning difficulty.

### 3.3 Long-Context Attention Patterns are Difficult to Learn

In the cellular automata task, the model must attend to a fixed local window of the previous state in order to predict each subsequent output cell (Figure 3 ). However, even this relatively simple pattern is made arbitrarily difficult by increasing the state size. In Figure 5(b) we sweep state sizes S ∈ { 4 , 8 , 16 , … , 256 } S\in\{4,8,16,\dots,256\} , using C = 4 C=4 colors, T = 16 T=16 trajectory length, N = 256 N=256 rules, and recursive depth k = 1 k=1 . We train a 4-layer model for cellular automata, as a single layer is not sufficient to solve the multi-rule task. The task is relatively easy for small S S , but as we increase S S the loss curves become more abrupt. As the state size grows, identifying the relevant local window within a longer sequence becomes increasingly difficult, even though the pattern itself remains simple. With S = 512 S=512 , no model makes progress after 10,000 steps. Although the underlying algorithm generating the data is identical, varying the context length determines whether a model can easily learn the task or fail to make any progress at all.

We again apply an attention intervention where we artificially inflate attention logits in each head based on a ground-truth pattern. For this task, we encourage the model to attend to the window of cells from the previous state for each token, as illustrated in Figure 3 . In Figure 5(c) , we measure the loss plateau length as the step at which the model first achieves loss < 1.3 <1.3 . We see that increasing the state size has a multiplicative effect on the length of the loss plateau, but with our attention intervention the plateau length is negligible ( ≈ 100 \approx 100 steps) and remains constant regardless of context length. Our intervention confirms that learning a long-context attention pattern is a bottleneck to training on our cellular automata task.

In Appendix B.4 we also vary the context length by adjusting the trajectory length given a fixed state size. Similar to Figure 5(c) , increasing trajectory length leads to longer loss plateaus, although the relationship is non-monotonic as the model struggles to learn the cellular automata dynamics when the trajectory is too short. We also vary the number of rules N N , the recursive depth k k , and cell colors C C ; notably, these variables do not affect the loss plateau length, suggesting that context length is the primary factor.

## 4 The Role of Architecture in Learning Attention Patterns

We now investigate how model architecture interacts with learning attention patterns in our synthetic tasks. In all experiments we use a baseline with D = 128 D=128 with H = 8 H=8 attention heads. We use a single-layer model for the linear map task and a 4 4 -layer model for cellular automata. Surprisingly, deeper models perform worse on the linear map task, likely because composing attention layers makes it harder to express a single sparse pattern directly; we provide additional details in Appendix B.5 . In Sections 4.1 and 4.2 we study the effect of scaling width and show that including more attention heads improves performance on both synthetic tasks. Finally, in Section 4.3 we see that architectures with alternative attention mechanisms significantly outperform transformers on the linear map task, but this benefit does not translate to cellular automata.

### 4.1 Scaling Width

We scale the width for single-layer models on the linear map task and 4 4 -layer models on the cellular automata task. In Figure 6 we vary the width by scaling the number of heads H H with head dimensions 4 4 , 8 8 , and 16 16 . For both synthetic tasks, increasing the number of heads consistently reduces the final loss . We hypothesize that each attention head independently searches over possible patterns, so more heads increase the probability of finding the correct pattern. For cellular automata, we also observe that increasing the head dimension reduces training loss for a fixed number of heads H H ; we believe this is because each head needs sufficient capacity to store rule information from prior context. This tradeoff raises the question of how to optimally allocate capacity between head count and head dimension at a fixed model width.

### 4.2 Scaling Attention Heads

To control for the number of attention heads in isolation, we fix the width to D = 128 D=128 and sweep H = 1 , 2 , 4 , … , 128 H=1,2,4,\dots,128 heads with head dimension 128 / H 128/H . In Figure 6 we see that a model with H = 128 H=128 heads and head dimension 1 1 can still solve the linear map task, confirming that head count matters more than head capacity for attention-heavy tasks. However, in the cellular automata task, setting H = 128 H=128 results in significantly slower learning and slightly higher final loss, likely due to the reduced capacity for storing rule information. For both tasks, the optimal configuration favors many heads with a small head dimension, but there is a task-dependent minimum head dimension below which increasing head count is no longer beneficial.

### 4.3 Alternative Architectures

We now compare a standard transformer to MLP-Mixer ( Tolstikhin et al., 2021 ) . MLP-Mixer replaces dot-product attention with a static learned matrix that mixes information across sequence positions. In Figure 7 , we show that MLP Mixer learns the linear map task significantly faster than a transformer , demonstrating that it is possible to design architectures that outperform transformers on attention-heavy tasks. However, MLP Mixer underperforms on the cellular automata task, likely because it must learn the local-window pattern independently at each position, whereas dot-product attention can parameterize it directly. This performance gap motivates designing architectures that learn positional patterns as efficiently as MLP-Mixer while retaining a transformer’s ability to learn context-dependent token-mixing patterns.

We also compare the transformer to architectures with alternative token-mixing patterns, including Mamba ( Gu and Dao, 2024 ) , Gated DeltaNet ( Yang et al., 2025 ) , and RWKV ( Peng et al., 2023 ) . All other architectures outside of MLP-Mixer underperform relative to transformers on both the linear map and cellular automata tasks; full details are in Appendix B.6 .

## 5 Related Work

#### Emergent Capabilities.

Many downstream capabilities of language models emerge sharply past certain model scales ( Wei et al., 2022 ) . Schaeffer et al. (2023) argue emergence is a mirage caused by discontinuous metrics. We show that, even on the continuous metric of correct-token probability, capabilities arise sharply throughout training and across model scales. Zhao et al. (2026) show emergence is random across initialization seeds at a fixed scale; we confirm these results on Pythia models and further demonstrate that emergence is random across training steps. Zucchet et al. (2025) study how sparse attention patterns relate to emergence, developing a theoretical framework for how task structure governs emergence in small transformer models. We complement this framework with causal evidence in pretrained language models, showing that patching a small set of attention heads can directly elicit capabilities before they arise naturally. We further isolate the factors that make a pattern hard to learn, namely context length and sparsity, and show that architectural choices control the rate at which these patterns are learned.

#### Abrupt Learning.

Training transformers on algorithmic tasks is known to produce abrupt learning, where loss plateaus end in sudden drops ( Gopalani et al., 2024 ) . Gopalani and Hu (2025) show that loss plateaus persist until the model learns the correct attention map, at which point training converges rapidly; intervening on attention maps can trigger this transition earlier. Abrupt learning also appears in natural language pretraining: Chen et al. (2025) observe abrupt learning in BERT when specific attention heads learn syntactic structure, and Olsson et al. (2022) show that induction heads in autoregressive models produce a sharp phase transition in in-context learning ability. Kangaslahti et al. (2026) develop an unsupervised method for detecting these hidden breakthroughs from smooth pretraining loss curves. We note abrupt learning is distinct from grokking ( Power et al., 2022 ; Nanda et al., 2023 ) ; abrupt learning is a sudden drop in training loss, while grokking is a sudden improvement in generalization after training loss has already converged.

#### Neural Scaling Laws.

While language model capabilities emerge randomly and abruptly, pretraining loss decreases smoothly as a power law with respect to model scale and dataset size ( Hestness et al., 2017 ; Kaplan et al., 2020 ) . The quantization model of neural scaling ( Michaud et al., 2024 ) reconciles these two observations, hypothesizing that language models learn discrete skills (quanta) abruptly, and that the aggregate of many such phase changes produces smooth scaling curves. Our findings support this view; we further demonstrate that phase changes correspond to the abrupt learning of attention patterns, and improved emergence rates in larger models are due to acquiring these quanta faster on average.

#### Our Contributions.