In-Context Algebra — part 3/5

Cleaned-text code points [31693, 47688). Bibliography omitted here; source.md includes it.

We find that models undergo distinct phase transitions during training (Figure 6 ; see also Appendix C ). Across seeds and configurations, the same sequence of stages marked by drops or plateaus in loss recurs (Figure 7 ). We study the training loss by computing several metrics at each model checkpoint. For each hypothesized mechanism (§ 4 ), we evaluate the model using 128 randomly sampled data points from the corresponding datasets described in Section 4 . For structural tokens, we compute the model’s accuracy of predicting ‘ = = ’ and ‘,’ tokens across a batch of 128 prompts. For computing group closure, we measure the top-K matching accuracy at the left-slot position (more details are provided in Appendix F ).

The earliest ability to emerge is prediction of structural tokens : ‘=’ and ‘,’ (Figure 6 a, gray). This is followed closely by group closure (§ 5.4 ): the model learns that combining two elements always yields another valid group element (Figure 6 b, orange). This ability appears in left-slot predictions of facts, where the model distributes probability nearly uniformly across all valid candidates (Appendix F ). At the same time, the model learns the query promotion submechanism (Figure 6 b, yellow), achieving around 50 % 50\% on identity sequences (§ 5.3 ). The next sharp drop in loss corresponds to the model learning contextual copying (§ 5.2 ), first reproducing facts verbatim (Figure 6 c, green) and then extending to commutative copying (Figure 6 c, purple).

Later mechanisms emerge more gradually. The model develops identity recognition , steadily improving on identity-related facts and acquires elimination reasoning in parallel, applying cancellation laws and closure constraints to rule out inconsistent candidates. Unlike closure and copying, these abilities do not show sharp transitions but appear jointly, suggesting they build on top of copying: once the model can retrieve and recombine facts, it can also infer identities and apply elimination strategies (Appendix B ). We hypothesize these are learned jointly because the identity demotion mechanism (§ 5.3 ) and the elimination subspace (§ 5.4 ) perform similar functions, and their “promotion” submechanism counterparts are learned at similar times earlier in training. Finally, models begin to solve some associative sequences, after all other mechanisms have been learned.

## 7 Related Work

Arithmetic as a testbed for interpretability. Arithmetic tasks have long served as controlled settings for studying and interpreting transformers ( Liu et al., 2023 ) . Small transformers trained on modular arithmetic exhibit “grokking” where they first memorize training data before converging to interpretable, generalizing solutions with periodic embeddings ( Power et al., 2022 ; Liu et al., 2022 ; Nanda et al., 2023 ; Zhong et al., 2023 ; Stander et al., 2024 ; Morwani et al., 2024 ) . Pretrained LLMs exhibit similar periodic structure in their number embeddings ( Zhou et al., 2024 ; Hu et al., 2025 ; Kantamneni & Tegmark, 2025 ; Nikankin et al., 2025 ) , enabling modular arithmetic without explicit training. Deng et al. (2026) find that arithmetic-fine-tuned LMs rely on symbolic subgroup patterns, instead of using partial products, but Bai et al. (2025) show that implicit chain-of-thought training does induce partial products and Fourier number representations. More closely related to our setting, He et al. (2024) show that transformers trained on permutations of one group develop hierarchical “circle-of-circles” representations, and Zhong & Andreas (2024) demonstrate that models with trained embeddings, but otherwise frozen random weights can still implement familiar geometric solutions. While these works study arithmetic settings where tokens have some fixed structure, our work examines a complementary setting where we remove fixed meanings of tokens altogether, requiring models to solve problems where token referents vary arbitrarily between sequences.

Mechanisms of in-context learning. The ability of transformers to learn from demonstrations has been attributed to several mechanisms. Early work identifies induction heads that underlie copying ( Elhage et al., 2021 ; Olsson et al., 2022 ; Feucht et al., 2025 ) , while theory frames ICL as Bayesian inference ( Xie et al., 2022 ; Akyürek et al., 2023 ; Wurgaft et al., 2025 ) or gradient-descent-like adaptation ( Von Oswald et al., 2023 ) . More recent studies show LM representations capture task-level structure ( Todd et al., 2024 ; Hendel et al., 2023 ; Yin & Steinhardt, 2025 ; Minegishi et al., 2025 ) , and token representations flexibly adapt to context ( Park et al., 2025a ; Marjieh et al., 2025 ) .

Symbolic reasoning and causal interpretability. Neural systems have long been studied as potential mechanisms for symbol manipulation, from tensor product ( Smolensky, 1990 ) and holographic reduced representations ( Plate, 1995 ) to recent cognitive-science studies of emergent symbolic reasoning in modern networks ( Swaminathan et al., 2023 ; Yang et al., 2025 ) . More recently, mechanistic interpretability has started mapping internal symbolic reasoning circuits in transformers ( Li et al., 2023 ; Brinkmann et al., 2024 ; Prakash et al., 2024 ; Saparov et al., 2025 ; Wu et al., 2025 ; Li et al., 2025 ) , using causal intervention techniques ( Mueller et al., 2025 ; Geiger et al., 2024 ; Geiger et al., 2025 ) .

Variables versus value processing in LMs. A few works have tried to disentangle the ability of LMs to solve math abstractly from their ability to perform arithmetic computation. Cheng et al. (2025) find that LMs are better at abstract variable-based formulation of solutions compared to numeric computation of the same word problems, while Calais et al. (2025) find that in other problem settings LMs struggle with textual comprehension more than equation solving. Mirzadeh et al. (2025) similarly find that LMs lack robustness to changes in numeric values of math problems.

## 8 Conclusion

We have studied LMs trained on a focused algebra task designed to isolate abstract in-context reasoning behavior in the absence of fixed-meaning symbols. Our findings suggest that the kinds of reasoning strategies learned by transformers are dependent on the task structure. In our in-context algebra setting, where tokens carry no fixed meaning, we have analyzed the mechanisms learned by transformer LMs in detail and found that models develop symbolic mechanisms instead of the familiar parametric or geometric strategies found in settings where tokens do have fixed meanings. We have seen that transformers can learn to manipulate symbols in-context without needing to refer to their underlying meaning, similar to the way that high-school algebra students learn to solve math problems by manipulating letter variables without constantly thinking about the values they might contain ( Usiskin, 1988 ) . Understanding when and why models choose different computational strategies remains an important open question for future interpretability work.

## Ethics Statement

This paper aims to advance the foundational understanding of in-context learning and transformers. While such research may influence future model development and deployment, we cannot meaningfully anticipate these downstream impacts within the scope of this work.

## Acknowledgments

The authors would like to thank Chris Wendler, Natalie Shapira, Nikhil Prakash, Arnab Sen Sharma, Freya Behrens, and Dana Arad for advice on various stages of the project and also Grace Proebsting and Michael Ripa for feedback on the paper. We are grateful for the generous support of Coefficient Giving (ET, RG, DB) and the National Science Foundation (Grant No. 2403303; RG, DB).

## Appendix A Group Theory

In this section, we review relevant terms from group theory that are used in our analysis.

A group ( G , ⋅ ) (G,\cdot) is a non-empty set G G equipped with a binary operation ⋅ : G × G → G \cdot:G\times G\rightarrow G that satisfies the following properties: • Associativity. For all elements x , y , z ∈ G x,y,z\in G : ( x ⋅ y ) ⋅ z = x ⋅ ( y ⋅ z ) (x\cdot y)\cdot z=x\cdot(y\cdot z)

• Identity. There exists an identity element e ∈ G e\in G such that e ⋅ g = g ⋅ e = g e\cdot g=g\cdot e=g for all g ∈ G g\in G

• Invertibility. For each element g ∈ G g\in G , there exists an inverse element g − 1 ∈ G g^{-1}\in G such that g ⋅ g − 1 = g − 1 ⋅ g = e g\cdot g^{-1}=g^{-1}\cdot g=e

The order of a group is the number of elements contained in the set G G , and we denote it as | G | |G| . For notational convenience, we refer to a group ( G , ⋅ ) (G,\cdot) simply as G G .

A group G G is called abelian (or commutative ) if for all x , y ∈ G x,y\in G , the following holds: x ⋅ y = y ⋅ x x\cdot y=y\cdot x .

Our training data consists of two main families of groups: cyclic groups and dihedral groups (§ 2 ).

A cyclic group of order n n , denoted C n C_{n} , consists of all powers of a single generator element: C n = { e , g , g 2 , … , g n − 1 } C_{n}=\{e,g,g^{2},\ldots,g^{n-1}\} where g n = e g^{n}=e is the identity. Every cyclic group C n C_{n} has the same structure as (i.e., is isomorphic to) doing arithmetic modulo n n . For example, in C 5 C_{5} , multiplying group elements (e.g., Equation 8 ) works exactly like adding numbers mod 5 5 (e.g., Equation 9 ): g 3 ⋅ g 4 \displaystyle g^{3}\cdot g^{4} = g 2 \displaystyle=g^{2} (8) ≡ 3 + 4 \displaystyle\equiv 3+4 = 2 ( mod 5 ) \displaystyle=2\pmod{5} (9)

A dihedral group D n D_{n} is the group of symmetries of a regular n n -gon (square, hexagon, etc.), with order | D n | = 2 ​ n |D_{n}|=2n . Its elements consist of n n rotations and n n reflections. We note that while all cyclic groups are abelian, dihedral groups are non-abelian for n ≥ 3 n\geq 3 .

An important consequence of group structure is the cancellation law , which states that we can “cancel” common terms in equations. Specifically, for any group G G with elements x , y , z x,y,z : • Left cancellation: If x ​ y = x ​ z xy=xz , then y = z y=z

• Right cancellation: If y ​ x = z ​ x yx=zx , then y = z y=z

Equivalently (by contrapositive): if y ≠ z y\neq z , then x ​ y ≠ x ​ z xy\neq xz and y ​ x ≠ z ​ x yx\neq zx . This rule guarantees that distinct group elements produce distinct products which helps ground our understanding of the closure-based cancellation mechanism described in Section 5.4 .

For completeness, we briefly describe other algebraic structures we test on that lack some subset of group properties: • A semigroup is a set with an associative binary operation, but does not require an identity element or inverses.

• A quasigroup is a set where equations a ​ x = b ax=b and y ​ a = b ya=b always have unique solutions for any a , b a,b , but the operation need not be associative. Finite quasigroups are equivalent to Latin squares ( Jacobson & Matthews, 1996 ) .

• A magma is simply a set equipped with a binary operation, with no other required structural properties.

## Appendix B Discussion: Contextual Reasoning is Richer Than Just Copying

In this section, we provide additional context about how our findings relate to prior work investigating the mechanisms underlying in-context learning.

Since the introduction of in-context learning ( Brown et al., 2020 , ICL;) , understanding the ICL capabilities of LLMs has become a major area of research ( Dong et al., 2024 ; Lampinen et al., 2025 ) . Seminal work by Elhage et al. (2021) and Olsson et al. (2022) studied ICL through the lens of context-dependent loss reduction and identified induction heads that copy tokens from earlier in the context. Beyond literal copying, Olsson et al. (2022) hypothesized that much of in-context learning could be explained as “in-context nearest neighbors” or “analogical sequence copying” via induction heads that operate on higher-level abstractions beyond literal tokens.

Subsequent research has revealed increasingly sophisticated variations of induction-like behavior showing models transport abstract contextual information such as multi-token concepts ( Feucht et al., 2025 ) , few-shot task representations (i.e., function vectors) ( Todd et al., 2024 ; Hendel et al., 2023 ; Yin & Steinhardt, 2025 ) , syntactic and semantic relations ( Ren et al., 2024 ) , or abstract variables ( Yang et al., 2025 ) . At the implementation level, these mechanisms all involve attention heads moving information between token positions. However, each mechanism differs substantially in the kind of information they transport and how it is processed.

Thus, it is unclear from prior work whether all forms of contextual learning are best characterized as some form of “fuzzy copying” — where characterizing them by the level of abstraction at which they operate is enough — or if qualitatively different computational strategies emerge. For instance, when studying how LLMs perform tasks that require entity binding, several prior works have observed models transport token positional information pertaining to where tokens appear in a sequence ( Feng & Steinhardt, 2024 ; Dai et al., 2024 ; Prakash et al., 2024 ; Prakash et al., 2025 ; Wu et al., 2025 ) . While it is still “copied” forward via attention, this form of contextual reasoning seems fairly distinct from traditional or “fuzzy” induction as it is related to a token’s relative position instead of its meaning.

Studying this question is challenging in pretrained LLMs because token hidden states often reflect both immediate context and parametric knowledge encoded during pretraining. This entanglement makes it difficult to isolate pure contextual reasoning. To study this in a more controlled setting, we design a contextual reasoning setting that eliminates parametric knowledge by removing fixed token meanings entirely. Our in-context algebra task assigns tokens to algebraic elements randomly within each sequence, forcing models to infer all structure from observed relationships alone (§ 2 ).

We analyze the mechanisms that transformers consistently develop when trained on our in-context algebra setting and find that they still learn both induction-style copying and “fuzzy” commutative copying (§ 5.2 ). This is expected, as copying can solve a large portion of the training data on its own (§ 4.1 , Figure 3 a). However, we also identify two additional mechanisms that seem qualitatively different from the fuzzy copying strategies seen in prior work: identity element recognition, and closure-based cancellation. At the level of individual operations, these are not entirely unprecedented; inhibition signals or demotion of tokens through attention heads have been documented before ( Wang et al., 2023 ; Merullo et al., 2024 ) , and closure-based cancellation relies on a similar elimination submechanism.

What seems distinctive is not the operations themselves but what they operate over: rather than suppressing based on fixed syntactic or semantic token information, both mechanisms are driven by relational structure derived entirely in-context. Identity recognition works by observing which element leaves others unchanged across multiple facts, then applying both query promotion and a conditional suppression rule to select the non-identity variable (see § 5.3 and Appendix E ). Closure-based cancellation infers group membership to construct a set of valid candidates, then systematically eliminates those ruled out by the cancellation law (§ 5.4 , and Appendix F ).

These mechanisms are examples of how models can reason about tokens as variables by manipulating references in-context without knowing their underlying meaning. Taken together, our findings provide additional evidence that the contextual strategies that transformers can develop look increasingly different from mere copying. Understanding when and why models learn different contextual strategies remains an important open question for future work.

## Appendix C Model Architecture and Training