Is the Reversal Curse a Binding Problem? Uncovering Limitations of Transformers from a Basic Generalization Failure — part 2/4
Cleaned-text code points [15520, 30793). Bibliography omitted here; source.md includes it.
Conceptual consistency is challenging to learn well within the design of current transformers even with an abundance of data, due to the dynamic and open-ended nature of concepts. Firstly, concepts in transformers could emerge at various locations/subspaces, necessitating a mechanism for dynamically tracking and connecting the concept representations across different regions of the network. Secondly, concepts are continuously assimilated instead of coming from a fixed vocabulary, which requires a systematic design that can establish the binding of newly acquired concepts automatically. This level of systematicity is not achieved even for the much simpler problem of binding tokens at the input/output levels, especially for models with untied input/output embeddings (which is common in most LLMs today)–—when new tokens are introduced to the vocabulary, transformers still have to rely on dedicated data to establish their binding.
### 3.2 Entanglements of Concept Representations
Whereas consistency is involved with connecting representations of the same concept, problems also arise on the other side of the same coin— separating representations of distinct concepts. We conjecture that transformers lack inductive biases to decouple abstract mental concepts from direct perceptions during learning, an issue which we call entanglement . The inability to maintain the separation of distinct concepts could influence the training dynamics and negatively affect generalization.
To illustrate, consider the last MLP layer in the recognition module before concept representations are formed, as shown in Figure 1 (b), Right. Suppose we have two activated concepts a a and b b in the current learning step, which have MLP hidden activations α , β \alpha,\beta respectively. Let the output projection matrix be V V where v i v_{i} is its i i -th column, and hence a = ∑ i α i v i , b = ∑ i β i v i a=\sum_{i}\alpha_{i}v_{i},b=\sum_{i}\beta_{i}v_{i} . 4 4 4 Here we ignore the residual connection and bias terms for simplicity. During learning when the loss is L L , the negative gradients − ∂ L / ∂ a -\partial L/\partial a and − ∂ L / ∂ b -\partial L/\partial b represent the “desired directions” for updating a a and b b . Assuming for now that α , β \alpha,\beta remain constant, we could compute the updates of concept representations Δ a \Delta a and Δ b \Delta b after a gradient descent step with step size η \eta : ∂ L ∂ v i = α i ∂ L ∂ a + β i ∂ L ∂ b , \displaystyle\frac{\partial L}{\partial v_{i}}=\alpha_{i}\frac{\partial L}{\partial a}+\beta_{i}\frac{\partial L}{\partial b}, Δ a = ∑ i α i ( v i − η ∂ L ∂ v i ) − a = − η ‖ α ‖ 2 ∂ L ∂ a − η α T β ∂ L ∂ b , \displaystyle\Delta a=\sum_{i}\alpha_{i}(v_{i}-\eta\frac{\partial L}{\partial v_{i}})-a=-\eta||\alpha||^{2}\frac{\partial L}{\partial a}-\eta\alpha^{T}\beta\frac{\partial L}{\partial b}, Δ b = ∑ i β i ( v i − η ∂ L ∂ v i ) − b = − η ‖ β ‖ 2 ∂ L ∂ b − η α T β ∂ L ∂ a . \displaystyle\Delta b=\sum_{i}\beta_{i}(v_{i}-\eta\frac{\partial L}{\partial v_{i}})-b=-\eta||\beta||^{2}\frac{\partial L}{\partial b}-\eta\alpha^{T}\beta\frac{\partial L}{\partial a}. We can see that each concept is not updated in the direction of its negative gradient; rather, the updates are mixed with gradients from other concepts, where the level of entanglements is decided by α T β \alpha^{T}\beta , i.e., how strong the hidden activations of a a and b b overlap (red neurons in Figure 1 (b), Right). 5 5 5 These entanglements clearly extend to momentum-based updates in modern optimizers. This overlap is in turn determined by the surface form names of a , b a,b and the configuration of lower-level weights, which are also subject to change and could add further complications. Note that while our analysis here focuses on one specific layer, the effect of entanglements naturally scales with model depth as the representational distortions accumulate throughout layers. This is very problematic, especially given that concept names could be almost arbitrary and exhibit all kinds of correlations. For example, imagine two different people with somewhat similar names. While there are pattern overlaps during recognition, after it is complete, they should become two distinct objects, and the information that we wish to store on each should be stored independently and not interfere. However, as we can see, for transformers with gradient-based optimization, the overlaps in activation patterns effectively cause the learning of different concepts to “mix”, which could adversely influence the training dynamics and generalization.
We note that the entanglements here are problematic only during learning . It is entirely fine and often beneficial for different concept representations to share latent structures, which can lead to more efficient storage and retrieval. However, it is undesirable for these shared structures, which inherit the arbitrariness of surface-form names and other correlations, to disrupt the learning itself.
### 3.3 Experiments
We perform a series of experiments to ground the above analysis. Scientific-wise, the experiments support the previous conjectures and arguments. Practical-wise, the explorations lead to a model design based on Joint-Embedding Predictive Architectures (JEPA) ( LeCun, 2022 ) which breaks the Reversal Curse with high performance given prior knowledge of the location of concept representations. This also unlocks a new kind of parametric memory integration that could solve large-scale arithmetic reasoning problems better than LLMs based on non-parametric memory, which we discuss in § 4 .
Attaching surface-level names to concepts . We build upon the setup in the abstract setting (§ 2 ) with | ℰ A | = 50 K |\mathcal{E}_{A}|=50K and attach a unique surface-form name for each concept, where the inputs now become regular token sequences concatenating the concept names. Our preliminary experiments show that the names of relations do not affect the result. For the entities, we choose not to use real-world entity names since they typically do not emit meaningful statistics to experiment with. Instead, we use a simple controllable way to create overlapping names inspired by human names. Specifically, each entity name has two tokens (resembling the first name and last name of a person) belonging separately to two disjoint sets, where each entity is randomly assigned a unique (ordered) pair of tokens. We define multiplicity to be the number of entities who share the same first/last token, which controls the overall degree of surface name overlaps. We keep the same multiplicity for each unique token. While there are distances from realistic settings, we believe that the notion of multiplicity here is a good abstraction for the overlaps in real-world entity names. By default, we experiment with a multiplicity of 10 10 , and also examine how varying the multiplicity affects the model’s learning.
We first conduct a series of experiments on models trained with standard language modeling objectives at the surface level, and confirm that transformers fail entirely (0.0% accuracy) to learn reversal from surface-level predictions , regardless of architectural variants (e.g., depth, tied vs. untied embeddings). These findings align with prior work ( Golovneva et al., 2024 ; Allen-Zhu, 2024 ) .
Next, we study the impact of inconsistency and entanglements by deliberately scaffolding targeted modifications into the model architecture.
We begin with explicitly encouraging conceptual consistency in reversal. One key observation is that the well-known Joint-Embedding Predictive Architecture (JEPA) ( LeCun, 2022 ) , which conducts predictions at the abstract representation space instead of raw input space, could perfectly serve this purpose if the abstract representations for prediction are at the concept level . This is somewhat a “coincidence” since the original motivation of JEPA is to ignore unpredictable/unimportant information in the inputs, which is not related to the focus here. We experiment with a simple instantiation of JEPA based on in-batch contrastive learning, illustrated in Figure 2 . Here, autoregressive prediction is done at the concept level encoded by the recognition module, where the representations of other “activated” concepts within the same batch (besides the ground truth) serve as negatives for the standard InfoNCE contrastive loss ( van den Oord et al., 2019 ) (additional details in Appendix B ). We evaluate the quality of learned representations by comparing the predicted state against representations of all concepts. We experiment with different configurations of the model in terms of the depth of the recognition module ( #Rec ) and the depth of the semantic module ( #Sem ), the semantic processing component on top of the recognition module.
JEPA unlocks generalization, but suffers from entanglements. Results are in Figure 3 (Left). By simply encouraging conceptual consistency (with JEPA), the model can achieve highly non-trivial generalization. To our knowledge, this is the first-ever model design that breaks the reversal curse without side-stepping the core problem. Another important observation is that the performance decreases when the model becomes deeper. Specifically, generalization consistently worsens when increasing either 1) the depth ratio between the semantic and recognition module, or 2) the overall depth of the model while keeping the ratio fixed. This strongly corroborates our analysis that the effect of entanglements scales with model depth . We also examine the impact of multiplicity, where the results are in Figure 3 (Right). It could be seen that increasing the multiplicity severely hurts generalization , especially with deeper models whose performance could drop to near zero with a mere multiplicity of 20 20 . Overall, the results here suggest that current models likely learn a low degree of conceptual consistency, and even if not, it is still challenging for them to learn reversal due to the effects of entanglements.
Mitigating entanglements. We next explore how mitigating entanglements affects generalization, based on the JEPA design above. A straightforward strategy is to increase the width of the model. Intuitively, with larger hidden dimensions and more hidden units, there should be a greater chance for different concept representations to be more separated from each other. To investigate this, we train models with hidden dimensions increased from 768 768 to 1280 1280 , and 20 20 attention heads. Another approach is to build specialized recognition modules with more discriminative hidden activations. Memory layers ( Sukhbaatar et al., 2015 ; Berges et al., 2024 ) exactly exemplify this approach, featuring ultra-wide hidden layers with top- k k sparsity and softmax activations. In particular, if we use a memory layer with small k k and/or high softmax temperature to replace the last MLP layer, the recognition module effectively reduces to having separate learnable embeddings for concepts with distinct names (same as the abstract setting (§ 2 )), eliminating entanglements. 6 6 6 Note that with the memory layer here, the depth of the recognition module does not matter. We experiment with this setup to see its effect on model performance.
It can be observed that increasing the width does aid generalization, but only incrementally (Figure 4 , Left). Meanwhile, the specially designed memory layer could significantly enhance performance, though generalization still mildly declines with more semantic layers (Figure 4 , Right). These results also confirm that the limited generalization observed with standard transformer layers as the recognition module is not due to insufficient capacity, since the module with two 1280 1280 -width transformer layers already has nearly the same amount of effective parameters as the memory layer ( 61.1 61.1 M vs. 61.4 61.4 M). We further validate that standard transformers equipped with memory layers also fail to generalize, confirming that the inconsistency inherent under surface-level prediction remains a primary blocker; memory layers are effective only when this inconsistency is resolved. Overall, these results underscore the importance of thoughtful designs in addressing the issue of entanglements apart from scaling . Our findings also provide a concrete example that memory layers can improve generalization , corroborating recent efforts on scaling memory layers that report enhanced performance on general-domain tasks ( Berges et al., 2024 ) .
## 4 Arithmetic Reasoning via Parametric Forward-Chaining
Our high-level goal is to improve the parametric memory of current AI models, which we believe is important for handling difficult knowledge and reasoning tasks. While our previous explorations expose obstacles and pathways for models to break the Reversal Curse (and beyond), the benefits towards tackling more ambitious challenges seem rather unclear—take reversal as an example, a natural question would be: What exactly can be achieved if the model does acquire reversal, other than knowing some more simple facts that we could have just retrieved from somewhere else?
In this section, we show that reversal enables a new kind of parametric memory integration that allows models to solve large-scale arithmetic reasoning problems with much better performance than frontier LLMs based on non-parametric memory.
We are inspired by recent work that formalizes and scales the complexity of arithmetic reasoning problems in similar styles with popular benchmarks such as GSM8K ( Ye et al., 2024 ; Zhou et al., 2025 ) . An important observation is that reversal is a key skill needed for a kind of parametric variable binding that allows the model to infer and implicitly chain different pieces of information in parametric memory. To illustrate, imagine we are given three pieces of information: “ X X equals 5 5 ” , “ Y Y equals 3 3 ” , and “ X X plus Y Y equals Z Z ” . Here, X , Y , Z X,Y,Z could be any phrase that corresponds to a numerical value (prevalent in arithmetic problems), such as “Tom’s id” or “the amount of apples Bruce has” . Given these facts and basic arithmetic knowledge, we could naturally know “ Z Z equals 8 8 ” . Importantly, this simple skill requires reversal to perform if we wish to store this information parametrically , since after retrieving and adding the values of X X and Y Y ( 5 + 3 = 8 5+3=8 ), a reversal step is needed to go from “ 8 8 equals Z Z ” to “ Z Z equals 8 8 ” (Figure 5 , Left). Here, the recognition module effectively acts as a variable-binding module, which maps a variable name to its value.
The significance of this skill lies in its ability to not only infer unknown values, but also propagate these inferences through a chain of deductions : when the inferred value is properly bound to the variable, it can then serve as a stepping stone for uncovering additional unknowns that the variable connects with, triggering a cascading effect. This enables the model to perform parametric forward-chaining while internalizing the information, allowing it to bridge increasingly distant knowledge gaps over multiple steps in parametric memory.