Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior — part 4/6
Cleaned-text code points [46977, 62763). Bibliography omitted here; source.md includes it.
Several works have convincingly argued that tools like probing or visualization of representations are insufficient to make claims about model behavior, i.e., artifacts produced via these tools can yield misleading explanations for why a model behaves the way it does ( Geiger et al., 2020 ; Belinkov, 2022 ; Bolukbasi et al., 2021 ; Saphra and Wiegreffe, 2024 ) . As such, a vast array of research has used intervention on activations to study model internals ( Li et al., 2017 ; Giulianelli et al., 2018 ; Cammarata et al., 2020 ; Elazar et al., 2020 ; Ravfogel et al., 2022 ; Ravfogel et al., 2023a ; Ravfogel et al., 2023b ; Belrose et al., 2023 ; Geva et al., 2023 ; Meng et al., 2022 ; Meng et al., 2023 ; Vig et al., 2020 ; Geiger et al., 2020 ; Davies et al., 2023 ; Stolfo et al., 2023 ; Guerner et al., 2023 ; Wang et al., 2023a ; Todd et al., 2024 ; Arora et al., 2024 ; Huang et al., 2024 ; Feng and Steinhardt, 2024 ; Mueller et al., 2025 ; Prakash et al., 2025 ; Gur-Arieh et al., 2025 ; Grant et al., 2025 ; Rodriguez et al., 2024 ) . This interpretability research leverages the frameworks of causal mediation ( Pearl, 2001 ; Vig et al., 2020 ; Mueller et al., 2026 ) and causal abstraction ( Rubenstein et al., 2017 ; Beckers and Halpern, 2019 ; Geiger et al., 2021 ; Geiger et al., 2025a ; Geiger et al., 2025b ) to ground understanding of model internals in the theory of causality ( Hume, 1748 ; Pearl, 1999 ; Spirtes et al., 2000 ) .
## 7 Discussion
#### Geometry-aware steering reveals the shared structure of behavior and representation.
We build out an empirical phenomenology that relates structure in activation space to the model output behavior. First, we show an isometry between representations and behavior manifolds, i.e., distance between two points on the activation manifold ℳ h \mathcal{M}_{h} aligns with distance between the distributions induced by those points on the behavior manifold ℳ y \mathcal{M}_{y} . Second, we show that steering representations along geodesics on ℳ h \mathcal{M}_{h} induces smooth, coherent transitions in behavior that follow geodesics on ℳ y \mathcal{M}_{y} . Third, we show that optimizing interventions to produce behaviors following geodesics on ℳ y \mathcal{M}_{y} recover trajectories in activation space that follow ℳ h \mathcal{M}_{h} . Thus, we establish a causal bridge between representation and behavior that reveals shared structure reflecting underlying conceptual geometry.
Our results also suggest that pathologies of linear steering—brittleness, incoherence, off-target effects ( Wu et al., 2025 ; Bigelow et al., 2025 ; Da Silva et al., 2025 ; Bhalla et al., 2024 ; Tan et al., 2024 ) —stem from the mismatch between assumed flat geometry and the true curved geometry of representation space, rather than an inherent challenge with representation-based intervention. This reframes the challenge of steering from “finding the right direction” to “finding the right geometry”.
#### Where does the shared geometry of behavior and representation come from?
While we do not study the origins of the shared geometry between behavior and representation, our experimental results are consistent with the hypothesis that conceptual structure constrains the geometry of both representation and behavior. While data statistics shape the geometry of neural representations ( Merullo et al., 2025 ; Karkada et al., 2026 ; Prieto et al., 2026 ) , this fails to explain how geometric structure is formed for out-of-distribution inputs. For example, our in-context learning tasks (Sec. 4 ) have synthetically defined geometries that imbue tokens with contextual meaning that is wildly different from the meaning learned during training. As such, the model must form novel representations and produce novel behaviors. The fact that we are able to establish a shared geometry for representation and behavior in these novel in-context learning tasks suggests that regardless of how training data statistics inform the geometries seen in the model, the output behavior is now computationally constrained by the activation geometry (see the contemporary work by Yocum et al. (2025) for a formalization of this claim).
#### Intrinsic coordinates of representation manifolds as units of causal analysis.
Mueller et al. (2024) frames the field of mechanistic interpretability as being on a quest to discover a primitive unit of representation best suited for the causal analysis of neural network internals. Causal abstraction provides a theoretical framework for defining such units of analysis ( Geiger et al., 2025a ; Geiger et al., 2025b ) , however Sutter et al. (2025) point out that allowing arbitrarily complex units admits degenerate solutions. Our work suggests a path toward both answering Mueller et al. (2024) and addressing the problem identified by Sutter et al. (2025) : the appropriate units of causal analysis are intrinsic coordinates on manifolds in activation space, and fitting these manifolds to naturally occurring activations provides a constraint that helps rule out degenerate solutions (cf. Grant et al. 2026 ).
## 8 Future Work and Limitations
The goal of our paper was to understand the role of geometry in neural networks and, subsequently, use this understanding to concretize what it means to steer model behavior via representations. We have shown that the geometry of neural network representations provides a blueprint for effective control. When interventions respect the geometry of activation space, the change in behavior is smooth and coherent; when they ignore it, they risk producing states with no natural behavioral counterpart. While we believe our results have enabled significant progress towards the motivating goals, there are remaining limitations that need to be addressed in future work.
• Expanding experimental validation to more complex domains. To illustrate our arguments, we focused on simple settings for which the concept of interest had a well-defined domain (e.g., weekdays), and the expected task outputs are the concepts themselves, therefore the conceptual geometry is directly displayed in the outputs. To further validate the claims posited in this paper, future work is needed to explore more abstract concepts, e.g., refusals ( Arditi et al., 2024 ) , sycophancy ( Vennemeyer et al., 2025 ) , and persuasion ( Costello et al., 2026 ) . For such concepts, output behavior will likely reflect conceptual structure in subtler ways, and it remains to be tested whether the conceptual geometry of such concepts can be inferred from behavior and related to representations as we did in this work. Moreover, in these more complex cases, it is unclear what are the right primitives for a representational account; we may need a notion of dynamics over manifolds, a view of representation as an aggregation of several geometric structures (similar to results seen by Fel et al. (2025b) in a vision context), or perhaps an altogether different object. Even if geometry is the right substrate to work with, we emphasize the simplicity of our domains allowed us to easily isolate the target concept’s geometry via synthetic, template-based text. Moving to more complex scenarios will require isolating the geometry of concepts from in-the-wild data.
• Moving from token to sequence-level outputs. Another way in which our tasks are simplified is our focus on the next-token distribution. This makes analysis feasible and helps avoid the combinatorial complexity involved in studying multi-token sequences. The obvious way to expand from our work’s token-level focus to sequence-level focus involves formalizing arguments in the language of “beliefs”, i.e., latent variables underlying the posterior predictive induced by a model in response to an input ( Bigelow et al., 2023 ; Bigelow et al., 2025 ; Wurgaft et al., 2025 ) . Correspondingly, what we expect to see via geometry-aware interventions is the nature of output sequences produced by a model will change as we perform steering: e.g., navigating the geometry of sycophancy (were it to exist) should allow us to alter the extent or type of sycophancy exhibited in model outputs; however, this property will be latent, rather than a concrete token-level change.
• Fitting the geometry. While we used a specific protocol to fit the observed geometries ( Bookstein, 1989 ) , we note there is a rich literature on fitting low-dimensional manifolds ( Coifman and Lafon, 2006 ; Brand, 2002 ; Schölkopf et al., 1997 ; Roweis and Saul, 2000 ; Jones, 2024 ; Jones and Lanners, 2026 ; Meilă and Zhang, 2023 ) . Critically, beyond just fitting the manifold, what we seek is an operator that allows us to navigate the manifold. For the domains analyzed in this work, we have ground-truth knowledge about how different states of the concept relate to each other, which allows us to define intrinsic coordinates for spline fitting. An unsupervised protocol would however significantly broaden the applicability of our methods.
• Manipulating intermediate algorithmic variables. All of our experiments are about manipulating the output behavior of neural networks directly. However, the most interesting control protocols will require manipulating intermediate quantities that mediate the flow of information from input to output, e.g., an image model determining the shape of an object in service of predicting its weight.
## Acknowledgments
The authors thank David Klindt, David Bau, Thomas Icard, Jing Huang, and the Mechanisms team at Goodfire for helpful conversations during the course of this project.
## Appendix A Experimental Details for Language Tasks
This Appendix describes the procedures behind the experiments of Sections § 2 , § 3 , and § 4 . The following section (Appendix § B ) will provide details for the mountain-car experiment in § 5 .
### A.1 Tasks and Datasets
#### Natural domain tasks.
We use four natural-domain addition tasks: weekdays and months (cyclic), and letters and ages (sequential). The full templates and entity sets are listed in Table 1 . For each task, we enumerate every (entity, increment) pair whose result lies in the task’s target set, dropping pairs whose result would fall outside it (e.g. letters past Z, or ages outside [ 10,100 ] [10,100] ). The reported activations and output distributions are computed at the answer-token position, and concept centroids are obtained by averaging across all prompts whose ground-truth result is the same value of 𝒵 \mathcal{Z} .
#### In-context learning of representations.
For the multi-dimensional setting of § 4 , we use the in-context learning of representations (ICLR) family of Park et al. (2025b) : arbitrary tokens corresponding to nouns (e.g., ”film”, ”rain”) are assigned to the nodes of a graph, and prompts are random walks on that graph. We study a 5 × 5 5\times 5 grid and a 9 × 9 9\times 9 cylinder, with random walks of 2048 2048 entity tokens. As in Park et al. (2025b) ’s setup, the random walks we sample do not allow backtracking, which we find aids models in learning the underlying structure.
### A.2 Model, Intervention Site, and Output Distribution
We investigate Llama 3.1 8B ( Touvron et al., 2023 ) activations at layer 28 in bfloat16 for all tasks. All interventions are performed on the residual stream at the last-token position. We chose to examine a late layer of the model to ensure that concept geometries are fully computed.
For an input x x , the output distribution 𝒑 ( x ) ∈ 𝒴 \bm{p}(x)\in\mathcal{Y} used throughout the paper is constructed as follows: we softmax over the full vocabulary logit distribution and aggregate probability mass over each concept value’s variant token spellings (e.g. the tokens ‘ Monday’ , ‘Monday’ , and ‘monday’ are all summed into the Monday entry). The remaining probability mass on tokens not associated with any concept value is collected into a single ‘other’ bin, yielding a distribution on the open simplex Δ | 𝒵 | \Delta^{|\mathcal{Z}|} over | 𝒵 | + 1 |\mathcal{Z}|+1 classes.
### A.3 Fitting the Activation Manifold ℳ h \mathcal{M}_{h}
To identify the activation manifold ℳ h \mathcal{M}_{h} , we first obtain points in full activation space and transform them into a 64-dimensional subspace obtained via PCA over the activations 𝒉 ( x ) \bm{h}(x) across all prompts in the task. The manifold lives entirely in the 64-dimensional PCA subspace; the orthogonal complement is preserved during all subsequent interventions (§ A.6 ).
We compute concept centroids c i c_{i} as the mean of the projected activations across all prompts whose ground-truth result equals the i i -th concept value, and fit a smooth interpolant through them. For the four natural-domain tasks the interpolant is a one-dimensional cubic spline ( Reinsch, 1967 ) : a natural cubic spline (with vanishing second derivatives at the endpoints) for the sequential tasks (letters, ages), and a periodic cubic spline for the cyclic tasks (weekdays, months) so that the curve closes smoothly. For the sequential tasks we use the ground-truth ordinal index of each concept as its intrinsic coordinate. For the cyclic tasks the centroids form a near-circular loop in the top two principal components of the activation subspace, so we instead derive the intrinsic coordinate θ = atan2 ( PC 2 , PC 1 ) \theta=\operatorname{atan2}(\mathrm{PC}_{2},\mathrm{PC}_{1}) in an unsupervised manner.
The interpolant for the ICLR tasks is a thin-plate spline (TPS; Duchon (1977) ; Bookstein (1989) ), a multi-dimensional generalisation of the cubic spline which minimizes the bending energy ∫ ‖ ∇ 2 f ‖ 2 \int\|\nabla^{2}f\|^{2} . Thin-plate splines map points in a lower-dimensional intrinsic space to the full ambient space. The TPS parameterisation requires a choice of intrinsic coordinates for the centroids. We use the ground-truth graph coordinates of each node in the ICLR task as intrinsic coordinates. Both the grid and cylinder tasks use the standard TPS kernel r 2 log r r^{2}\log r ; for the cylinder, which has both a linear and a periodic dimension, we additionally apply a ghost-point procedure where each control point is duplicated at one period above and below its θ \theta value (and we drop the linear-in- θ \theta polynomial column) to enforce closure across the periodic dimension. In every case the spline interpolates the centroids exactly, so ℳ h \mathcal{M}_{h} passes through every c i c_{i} .
### A.4 Fitting the Behavior Manifold ℳ y \mathcal{M}_{y}
Behavior centroids b i = 𝒑 ¯ i b_{i}=\bar{\bm{p}}_{i} are computed analogously to the activation centroids, by averaging the model’s output distributions across all prompts whose ground-truth result equals the i i -th concept value. Because the probability simplex is not a proper metric space, we map each centroid into Hellinger coordinates via b i ↦ b i b_{i}\mapsto\sqrt{b_{i}} , placing it on the non-negative orthant of the unit ℓ 2 \ell_{2} sphere in ℝ | 𝒵 | + 1 \mathbb{R}^{|\mathcal{Z}|+1} . We then fit the same family of splines used for ℳ h \mathcal{M}_{h} to the Hellinger-embedded centroids: a 1D cubic spline (natural or periodic) for the natural-domain tasks, and a thin-plate spline for the ICLR tasks. Analogous to the activation manifold, the fit passes exactly through every b i \sqrt{b_{i}} and we do not apply a smoothing penalty.
The spline is fit in Euclidean space, but valid b i \sqrt{b_{i}} points lie on a curved sphere. A naive fit to their ambient coordinates would leave the sphere between centroids, and an off-sphere vector does not square to a valid distribution. We therefore fit the spline in the tangent plane of the sphere at a base point b ∗ b_{*} – a flat space that touches the sphere at b ∗ b_{*} – and lift back to the sphere at decode time.