The Illusion of Latent Generalization: Bi-directionality and the Reversal Curse — part 2/2

Cleaned-text code points [15888, 24940). Bibliography omitted here; source.md includes it.

To ensure the robustness of our findings, we utilize four distinct datasets ranging from synthetic to semantic knowledge. Importantly, all datasets are novel, i.e. not contaminated from pre-training.

##### Simple Reversal [ Lampinen et al., 2025b ] :

This dataset is explicitly designed to have sufficient diversity for minimal training from scratch. The universe consists of 1,000 entities and 20 relations (plus a reverse relation for each). • Data Volume: The total universe of facts is 1,000 ​ entities × 20 ​ relations × 2 ​ directions = 40,000 1,000\text{ entities}\times 20\text{ relations}\times 2\text{ directions}=40,000 sequences.

• Split: We hold out 200 specific facts for testing. For these 200 facts, the model is trained only on the forward sequences. The remaining 39,800 sequences are included in the training set to establish the structural patterns of the relations.

• Augmentation: To create diverse training documents, each relation sentence is augmented with up to 5 random prefix and suffix tokens.

##### Nonsense Entities [ Lampinen et al., 2025a ] :

This dataset consists of synthetic nonsense words constructed to test structural learning without any potential semantic leakage from pre-training. • Example: “ femp are more dangerous than glon .”

• Structure: There are 100 such comparisons provided, with comparison words sampled from a set of 28 (e.g., “brighter”, “heavier”). Each comparison is repeated across 10 different training examples, paired with randomly sampled preambles.

• Split: To facilitate structural learning, 40% of the forward sentences in the training data are explicitly augmented with their reverse counterparts. The test set consists of the remaining facts where the reverse direction was strictly held out.

##### Fictional Celebrity [ Berglund et al., 2024 ] :

This dataset consists of fictional celebrity names and their descriptions. • Example: “ Daphne Barrington is the director of ’A Journey through Time’ .”

• Split details: 50 %.

##### Semantic Structure Lampinen et al. [2025a] :

This benchmark employs a relational semantic hierarchy with nonsense terms to test deductive inference. • Example: “ gruds ” (dogs) are a type of “ abmes ” (mammals).

• Structure: The hierarchy includes 110 categories with naturalistic, asymmetric properties. To mitigate tokenization artifacts, nonsense terms are formed from plausible English phoneme combinations (4-5 letters).

## Appendix B Objective details and hyperparameters

##### Optimization.

Unless stated otherwise, we fine-tune pretrained models with learning rate 2 × 10 − 5 2\times 10^{-5} for 10 5 10^{5} steps. For Simple Reversal , we train from scratch using the architecture and training setup in Lampinen et al. [2025b] .

##### MLM masking rates.

For BERT-style MLM, we use the standard masking procedure: each token is selected for masking with probability 0.15 0.15 Devlin et al. [2019] . (For multi-token entities and natural language prompts, this yields diverse corrupted contexts.)

##### Decoder-only masking rates (NTP + Masking).

For NTP+Masking on language-based datasets, we follow Pan et al. [2025] and sample a masking ratio uniformly from ( 0.05 , 0.95 ) (0.05,0.95) per example, then mask tokens accordingly in the context segment placed before the unmasked target segment.

##### Special case: Simple Reversal masking.

In Simple Reversal , each fact is exactly three tokens ( A A , relation, B B ). Rather than applying a 15% token-level rate (which would rarely mask anything), we construct controlled variants by masking exactly one of the three positions (source, relation, or target), or pairs thereof, matching the sweep in Fig. 2 .

### B.1 Illustration of training samples on Simple Reversal

Table 2 illustrates, for a single fact “ A > B A>B ”, what training sequences look like under each objective (for Simple Reversal ). This is only meant as an intuition pump; for multi-token entities, MLM uses token-level masking with probability 0.15 and NTP+Masking uses a sampled masking ratio.

## Appendix C Auxiliary ablations

### C.1 Ablation II: Probing for Representational Collapse

Given that the successful models require predicting A A in the context of B B to succeed, we investigated the nature of this link. Do these models learn a structured relationship, or does the bidirectional attention (or its approximation) simply cause a “representational collapse” where A A and B B become strongly associated in a bag-of-words manner, ignoring syntax?

If the representations had collapsed (i.e., the model simply learns “ A A goes with B B ”), we would expect the model to predict A A regardless of the syntactic frame. We tested this by probing the successful MLM and “NTP + Masking” models with the novel structure: “ B > … ​ ? B>...? ”. Note that this is a false statement in our setup (as B B is the object), but if the model had merely associated the entities, the presence of B B should trigger the retrieval of A A even in this incorrect frame. Table 3 lists representative probe instances.

We report this probe for Simple Reversal and Nonsense . We omit Celebrity because that dataset does not train/test on an explicit inverse relational operator: many prompts are effectively definitional (“ X is the director of Y ” → \rightarrow “ Who is X? ”), so the “wrong-frame” construction does not cleanly correspond to flipping the syntactic direction while keeping the same relation token. We also omit Semantic Structure for the same reason as above: the hierarchical templates make it ambiguous what constitutes a comparable single-slot “false frame” probe.

The models achieved 0% accuracy on this probe (Table 3 ). This result indicates that the representations have not collapsed. The models distinguish the contexts: they retrieve A A when B B appears in a reverse frame (or in the “[MASK] > > ” slot), but correctly refuse to associate them in the wrong syntactic direction. Given this, one could argue that MLM is in fact doing latent learning behaviorally. Whether it does so by encoding forward and reverse facts in separate memories is an implementation debate.

### C.2 Ablation III: The Prediction Bias

Finally, we investigated the limits of what information an MLM model actually encodes. In Ablation I, we observed that predicting the source is necessary for reversal. This raises a fundamental question about latent generalization: If a model attends to a token constantly but is never forced to predict it, does it learn that token at all?

To test this, we trained models on the forward sequences “ A > B A>B ”. We used a modified MLM objective where we masked A A and the relation tokens, but strictly never masked B B . We then tested the model on the forward prediction: “ A > [MASK] A>\text{[MASK]} ” (expecting B B ). Note that this is not a reversal task; it tests the model’s ability to recall the exact sequence seen during training.

Despite B B being fully visible in the attention mechanism during every training step (while predicting A A or > > ), the model achieved 0% accuracy when asked to predict B B at test time.

This reveals a severe Prediction Bias : the model fails to learn representations for tokens that are not explicitly targets of the loss function. Even though B B is required to predict A A , the model treats B B merely as a conditioning context and does not encode it as a generatable output. This suggests a bleak outlook for latent generalization in current architectures: information encoded solely in activations (via attention) without a corresponding gradient update on the token itself is effectively invisible to the generation mechanism.

##### Connection to post-training.

This offers a lens on a standard post-training choice: many SFT/RLHF pipelines compute loss only on the answer tokens, not on the prompt/question . This is often desirable (it discourages parroting the prompt), but it may also reinforce the same bias: prompt-side spans can condition behavior without being learned as reliably generatable outputs when those spans are later required.

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .