Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior — part 3/6

Cleaned-text code points [31707, 46977). Bibliography omitted here; source.md includes it.

Our experiments thus far have been limited to one dimensional conceptual spaces arising from training data imbued with real-world structure, i.e., days, months, ages, and letters. In turn, the manifolds we found have been one dimensional curves with a single intrinsic coordinate. Now, we extend our results to a setting with two dimensional conceptual spaces whose geometry are defined via in-context learning. We fit manifolds and show there is a two dimensional intrinsic coordinate system for the manifold, where steering along each coordinate controls an independent dimension of the conceptual space.

#### In-context learning tasks with synthetic conceptual spaces.

Park et al. (2025b) introduce a family of tasks to study the in-context learning of representations (ICLR). For each task, arbitrary tokens are assigned to a discrete graphical structure and language models are supplied with sequences of tokens derived from a random walk on that graph. They show that the statistical patterns in the random walk of tokens induce a reorganization of representations that recapitulates the graphical structure used to generate data. This in turn enables the language model to match the next token distribution, i.e., predict tokens adjacent to the current location of the random walk on the grid. For our two experiments, we assign arbitrary tokens to grid and cylinder graph structures. Thus, the conceptual domain 𝒵 \mathcal{Z} is the set of tokens and the distance metric d 𝒵 d_{\mathcal{Z}} is distance on the graph used to generate the random walk. Fig. 6 (a) shows an example grid and an input prompt generated by a random walk.

#### Manifold fitting.

The ICLR grid manifold ℳ h \mathcal{M}_{h} is topologically described by a two dimensional surface with no holes or tears. The activation geometry of ℳ h \mathcal{M}_{h} is more complex. Its semi-spherical shape shown in Fig. 6 (c) is induced by task statistics: the random walk visits inner sites more frequently than peripheral sites, leading to slight distortions with respect to the ground truth geometry ( Park et al., 2025b ; Yang et al., 2025 ; Karkada et al., 2026 ) . We fit two-dimensional sheets to internal activations and output distributions via thin plate splines (TPS; Duchon 1977 ; Bookstein 1989 ), which can be seen as the 2 2 D analog of the cubic splines used previously. In this case, we use activations corresponding to the last token in the context, and compute centroids according to graph location at a given timestep. Then, TPS finds the smoothest surface interpolating through the centroids (see App. A.3 for further details).

#### Isometry results.

For each ICLR domain, we compute pairwise distance matrices over graph-node centroids under three metrics: Euclidean (linear) distance in the activation subspace, geodesic distance along the fitted activation manifold ℳ h \mathcal{M}_{h} , and geodesic distance along the behavior manifold ℳ y \mathcal{M}_{y} . We find very high correlations between geodesic paths on the activation and behavior manifolds ( r = .99 r=.99 for both the 5 × 5 5\times 5 grid and 9 × 9 9\times 9 cylinder domains) and reduced correlations for linear paths ( 5 × 5 5\times 5 grid r = 0.90 r=0.90 ; 9 × 9 9\times 9 cylinder r = 0.81 r=0.81 ). To further examine these results, we embed each distance matrix with multidimensional scaling (MDS). Fig. 6 (b) shows a clean grid structure in the activation manifold and behavior manifold embeddings, while the linear paths in activation space yield a warped surface. Again, we see the conceptual space recapitulated in both representation and behavior.

#### Manifold vs. Linear steering.

We next test whether the fitted two-dimensional activation manifold affords coherent, factored control over the graph geometry used to generate the ICLR inputs. In particular, we assess whether we can control the position of the random walk input via intervention, and, moreover, whether there is an intrinsic coordinate system where steering along each coordinate independently controls the horizontal and vertical position on the grid. For each ordered pair of nodes, we use manifold steering and linear steering to interpolate between the start and end centroid and average results over 5 input prompts.

The top panel of Fig. 6 (c) shows that manifold steering produces smooth transitions vertically along the steered graph dimension while remaining at the same horizontal position. The bottom panel of Fig. 6 (c) shows similar smooth transitions but along a horizontal dimension while keeping the same vertical position. This demonstrates the manifold has an intrinsic coordinate system corresponding to the two dimensions of the grid, enabling factored control. Furthermore, this shows the smooth and ordered transitions of manifold steering generalize to multi-dimensional spaces. In contrast, linear steering again fails to provide ordered transitions through grid locations, and shows very clear ‘teleportation’ behavior between the endpoint locations along its path.

## 5 Manifold Steering on a Visual World Model: Mountain Car Task

We now ask whether the same principles of geometry-aware steering extend to the visual domain of world models. This question is practically motivated: learned world models that predict future observations from past frames and actions are central to model-based reinforcement learning and robotic planning ( Ha and Schmidhuber, 2018 ; Hafner et al., 2020 ; Team, 2025 ; Black et al., 2024 ) . If the internal representations of such models admit geometric structure, manifold-based steering could provide a principled mechanism for intervening on a model’s behavior through changing its beliefs about the state of the world.

#### Environment and model architecture.

We train a recurrent world model on the Mountain Car environment ( Moore, 1990 ; Sutton and Barto, 2018 ) , a classical control task in which a car must escape a valley by building momentum. The environment has continuous position p ∈ [ − 1.2 , 0.6 ] p\in[-1.2,0.6] , continuous velocity v ∈ [ − 0.07 , 0.07 ] v\in[-0.07,0.07] , and three discrete actions (left, no-op, right). The model predicts the next frame x t + 1 x_{t+1} given the previous frame x t x_{t} and action a t a_{t} (see Fig. 7 (a) for an illustration). The full architecture is shown in Fig. 8 : A convolutional encoder maps each 128 × 128 × 3 128\times 128\times 3 RGB frame to a latent vector, v t v_{t} , which is concatenated with a learned action embedding e ⁡ ( a t ) ∈ ℝ 16 e(a_{t})\in\mathbb{R}^{16} and fed to a Gated Recurrent Unit (GRU; Cho et al. 2014 ): 𝐡 t = GRU ⁡ ( [ v t ; e ⁡ ( a t ) ] , 𝐡 t − 1 ) ∈ ℝ n ; v t = LayerNorm ⁡ ( f enc ​ ( x t ) ) ∈ ℝ n , \mathbf{h}_{t}=\mathrm{GRU}\!\bigl([v_{t};\,e(a_{t})],\;\mathbf{h}_{t-1}\bigr)\in\mathbb{R}^{n};\;\;v_{t}=\mathrm{LayerNorm}\!\bigl(f_{\mathrm{enc}}(x_{t})\bigr)\in\mathbb{R}^{n}, (8) where n = 64 n=64 . A convolutional decoder produces a residual image from the hidden state, yielding the prediction x ^ t + 1 = x t + f dec ​ ( 𝐡 t ) \hat{x}_{t+1}=x_{t}+f_{\mathrm{dec}}(\mathbf{h}_{t}) .

#### Activation and behavior manifold fitting.

For this setting, we consider position to play the role of the conceptual domain 𝒵 = [ p min , p max ] \mathcal{Z}=[p_{\text{min}},p_{\text{max}}] and aim to capture the manifold structure in both activation and behavior space of the vision encoder. To start, we first collect encoder activations from 100 rollouts in the environment (see § B.1 for details) and observe they occupy a curved, low-dimensional manifold ℳ h ⊂ ℝ n \mathcal{M}_{h}\subset\mathbb{R}^{n} (Fig. 7 (c)). We parameterize this manifold by partitioning the position range into bins and fit a smooth spline through the means { μ b } b = 1 B ⊂ ℝ n \{\mu_{b}\}_{b=1}^{B}\subset\mathbb{R}^{n} . For a given input, x x , we compute the output distribution over positions, 𝐩 ⁡ ( x ) \mathbf{p}(x) using the distance of the activations v ⁡ ( x ) v(x) to the centroid for each position: 𝐩 ⁡ ( x ) = softmax ​ ( − ‖ v − μ b ‖ 2 τ ) b = 1 B ∈ Δ B − 1 , \mathbf{p}(x)\;=\;\mathrm{softmax}\!\left(-\frac{\|v-\mu_{b}\|_{2}}{\tau}\right)_{b=1}^{B}\;\in\;\Delta^{B-1}, (9) with temperature τ = 0.5 \tau=0.5 . We follow § 3.2 to parameterize the behavior manifold ℳ y \mathcal{M}_{y} by embedding each bin in Hellinger coordinates on the unit sphere and fit a 1D smoothing spline γ ℳ y : 𝒵 → ℝ B \gamma_{\mathcal{M}_{y}}\colon\mathcal{Z}\to\mathbb{R}^{B} parameterized by position (full details in App. B ). Note that both ℳ h \mathcal{M}_{h} and ℳ y \mathcal{M}_{y} are 1D structures parameterized by the conceptual coordinate p p ; under PCA visualization (Fig. 12 ), both trace closed curves in their respective ambient spaces. The curves are closed because visually distinctive states at the wall, p ≈ − 1.2 p\approx-1.2 , and goal, p ≈ 0.4 p\approx 0.4 , are mapped to neighboring activations.

#### Geometry-aware steering in activation space.

Fig. 7 compares linear (Eq. 1 ) and manifold (Eq. 2 ) steering through K = 20 K=20 waypoints between encoder states corresponding to positions p A = − 0.4 p_{A}=-0.4 and p B = 0.4 p_{B}=0.4 , projected into the first three principal components of encoder space. The geodesic path closely tracks ℳ h \mathcal{M}_{h} , and the corresponding decoded frames display a smooth, coherent progression of the car through intermediate positions. The linear path departs from ℳ h \mathcal{M}_{h} at intermediate points and decoded frames exhibit blurred or ambiguous car placement, reflecting an incoherent superposition. Then, a ‘teleportation’ to the endpoint is observed, analogous to the behavior of linear steering in the language model experiments. Moreover, linear steering causes the probability distribution over position to show greater spread compared to on-manifold paths, yielding the ambiguous car placement seen in intermediate points along the path. Finally, we reproduce the pullback procedure in § C.3 and show that optimizing paths in the output distribution over possible car positions results in activation space paths that closely track the ℳ h \mathcal{M}_{h} (Fig. 12 ).

#### The geometry assumed by linear steering is not faithful to the conceptual ordering.

To make the difference between the two steering metrics visually concrete, we apply multidimensional scaling (MDS) to the pairwise distance matrices induced by three different distance functions over W = 50 W=50 anchor positions evenly spaced along [ p min , p max ] [p_{\text{min}},p_{\text{max}}] (Fig. 7 (b)). Both activation space and behavior space on-manifold distance embeddings recover a clean one-dimensional rainbow ordering of positions, while the linear distance embedding produces a scrambled three-dimensional structure whose colors are visibly out of order. This is because ℳ h \mathcal{M}_{h} folds back on itself in the encoder’s ambient space, so two activations whose underlying positions are far apart can sit arbitrarily close in ambient space. Quantitatively, the Pearson correlation between the two arc-length distance matrices is r = 0.99 r=0.99 , while correlation between activation-space linear paths and behavior manifold arc-length falls to r = 0.06 r=0.06 , confirming that the linear-steering metric is not a faithful proxy for the conceptual ordering that the encoder has learned.

## 6 Related Work

#### Activation Steering and the Linear Representation Hypothesis.

Activation steering protocols ( Bau et al., 2018 ; Subramani et al., 2022 ; Marks and Tegmark, 2023 ; Panickssery et al., 2024 ; Turner et al., 2024 ) are often motivated by the linear representation hypothesis (LRH)—a geometric assumption on model representations ( Smolensky, 1986 ; Elhage et al., 2022a ; Park et al., 2023 ; Costa et al., 2025 ; Zheng et al., 2025 ) . In particular, LRH argues neural networks encode concepts, i.e., latent variables underlying the data distribution ( Wang et al., 2023b ; Rajendran et al., 2024a ; Rajendran et al., 2024b ; Okawa et al., 2024 ) , along directions. This motivates tools like linear probing ( Belinkov, 2022 ; Guerner et al., 2023 ) and sparse autoencoders ( Cunningham et al., 2023 ; Bricken et al., 2023 ; Gao et al., 2025 ; Bussmann et al., 2024 ; Fel et al., 2025a ) . In the case where the representation geometry for a concept truly aligns with LRH, Bigelow et al. (2025) showed the effects of activation steering on model behavior can be accurately captured by a linear increase in concept log-probability. However, in the general scenario where geometry of representations does not abide by LRH, the effects of linear steering protocols are less clear. Recent work has started to fill this gap: e.g., work by Rodriguez et al. (2025) ; Ravfogel et al. (2022) has shown that linear steering protocols match the first moments of the current output distribution produced by the model with the target distribution; however, the effects on higher order moments can be unconstrained and adversarial (see results by Sarfati et al. (2026) ), possibly explaining why it produces incoherent outputs.

In contrast, when the representation geometry is fully respected, our work shows that steering smoothly interpolates the source and the target distributions. Prior work has shown similar results to this effect in narrow domains, e.g., Engels et al. (2024) ablate representations and write the days-of-the-week circle directly and Kantamneni and Tegmark (2025) follow a similar protocol for a helix representing numbers, but they lack a more general account of how representation geometry and output behavior map on to each other. The closest work to ours is the contemporary paper by Park et al. (2026) , who study a toy model where the representation-to-output distribution mapping is described via a simple softmax operation.

#### Activation Geometry and its Origins.

A large body of recent work has shown neural networks encode concepts along nonlinear, curved geometries embedded in low-dimensional subspaces across both modalities and architectures ( Fel et al., 2025b ; Pearce et al., 2025 ; Yocum et al., 2025 ; Modell et al., 2025a ; Lubana et al., 2025 ; Costa et al., 2025 ; Park et al., 2025b ; Engels et al., 2024 ; Karkada et al., 2026 ; Shai et al., 2024a ; Shai et al., 2026 ; Saxe et al., 2019 ; Park et al., 2025c ; Morwani et al., 2024 ; Kantamneni and Tegmark, 2025 ; Song and Zhong, 2023 ; Zhou et al., 2025 ; Maheswaranathan et al., 2019 ) . While earlier work ( Saxe et al., 2019 ; Arora et al., 2018 ; Park et al., 2023 ; Yocum et al., 2025 ) concretized, in toy settings, how structure in the data-generating process imposes geometric constraints on a neural network’s representations, only recently have such accounts been extended to make predictions about the geometry of neural representations at scale (e.g., Merullo et al. (2025) ). Karkada et al. (2026) and Korchinski et al. (2025) argue that symmetries in data statistics enforce geometries best suited for reflecting the uncertainty of the distribution in a model’s representation (cf. Prieto et al. 2026 ), and offer plausible accounts for the formation of representations in-context, as shown by works such as Park et al. (2025a) and Lepori et al. (2026) .

#### Causal Analysis of Neural Networks.