Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior — part 5/6

Cleaned-text code points [62763, 77466). Bibliography omitted here; source.md includes it.

We take b ∗ b_{*} to be the Euclidean mean of { b i } \{\sqrt{b_{i}}\} , re-normalized to unit length; because every b i \sqrt{b_{i}} lies in the non-negative orthant, so does b ∗ b_{*} . The log-map t i = log b ∗ ⁡ ( b i ) t_{i}=\log_{b_{*}}\!(\sqrt{b_{i}}) projects each centroid onto the tangent plane: it returns a vector whose direction points from b ∗ b_{*} along the geodesic to b i \sqrt{b_{i}} and whose length equals that geodesic distance. We fit the spline to the tangent vectors { t i } \{t_{i}\} . To decode at a query coordinate u u , we evaluate the spline to obtain a tangent vector t t and apply the exponential map exp b ∗ ⁡ ( t ) \exp_{b_{*}}(t) , the inverse of the log-map: it walks distance ‖ t ‖ \|t\| along the geodesic on the sphere starting at b ∗ b_{*} in direction t t . The result is unit-norm by construction, so ℳ y \mathcal{M}_{y} stays on the sphere everywhere, and because exp b ∗ ∘ log b ∗ \exp_{b_{*}}\!\circ\log_{b_{*}} is the identity on the sphere, the decoded curve passes through every b i \sqrt{b_{i}} exactly.

### A.5 Geodesic Distances and the Isometry Test

To compute the geodesic distance d ℳ h ​ ( c i , c j ) d_{\mathcal{M}_{h}}(c_{i},c_{j}) between two concept centroids, we discretize the line segment between 𝒔 − 1 ​ ( c i ) \bm{s}^{-1}(c_{i}) and 𝒔 − 1 ​ ( c j ) \bm{s}^{-1}(c_{j}) in intrinsic coordinates into 150 equal sub-intervals, decode each waypoint through 𝒔 \bm{s} , and accumulate consecutive ambient distances. Each waypoint therefore lies on ℳ h \mathcal{M}_{h} by construction, and the resulting arc length is measured in the 64-dimensional PCA subspace in which the manifold lives, with the Euclidean norm as the ambient norm. For the behavior manifold ℳ y \mathcal{M}_{y} we follow the same procedure but compute distances in the full sqrt-probability ambient space ℝ | 𝒵 | + 1 \mathbb{R}^{|\mathcal{Z}|+1} rather than a PCA-reduced subspace, using the Hellinger distance d H ​ ( p , q ) = 1 2 ​ ‖ p − q ‖ 2 d_{H}(p,q)=\tfrac{1}{\sqrt{2}}\|\sqrt{p}-\sqrt{q}\|_{2} directly on the sqrt-embedded waypoints.

The isometry score reported in § 2 is the Pearson correlation between the upper-triangular entries of the resulting pairwise distance matrices. We augment the W W centroid vertices with K K interior points sampled at equally spaced fractions of the u-space geodesic between each centroid pair, decoded onto the manifold via 𝒔 \bm{s} so that every vertex lives on ℳ h \mathcal{M}_{h} or ℳ y \mathcal{M}_{y} . We choose K K so that the vertex set is dense enough to probe the geometry between centroids: K = 4 K=4 for weekdays ( W = 7 W=7 ); K = 1 K=1 for months ( W = 12 W=12 ), alphabet ( W = 24 W=24 , letters C ​ – ​ Z C\text{--}Z ), and the grid 5 × 5 5{\times}5 task ( W = 25 W=25 ); and K = 0 K=0 for age ( W = 91 W=91 , ages 10 ​ – ​ 100 10\text{--}100 ) and the cylinder 9 × 9 9{\times}9 task ( W = 81 W=81 ), whose centroids are already dense. We then correlate every off-diagonal pair in the full vertex set except those whose two vertices lie on a common centroid-pair geodesic, since those distances are sub-arcs of the same geodesic and would inflate the correlation by construction. To visualise the resulting pairwise structure, we embed each distance matrix into three dimensions via classical multidimensional scaling, as shown in Figs. 2 , 3 , and 6 .

### A.6 Steering Interventions

For each pair of concept values ( z a , z b ) (z_{a},z_{b}) , we steer the model from the centroid c a c_{a} to the centroid c b c_{b} via a path of K = 50 K=50 waypoints. We use a fixed set of base prompts sampled randomly from the task’s input distribution (the prompts’ ground-truth results vary, and the same set is reused across all pairs): 16 16 prompts for the natural-domain tasks and 5 5 for the ICLR tasks. At each waypoint 𝝅 ⁡ ( t ) \bm{\pi}(t) , we intervene at the last-token residual-stream activation of the target layer and continue the forward pass to obtain 𝒑 𝒉 ← 𝝅 ⁡ ( t ) ​ ( x ) \bm{p}_{\bm{h}\leftarrow\bm{\pi}(t)}(x) . Every reported behavioral trajectory is the pointwise mean over the base prompts. We use up to 50 randomly-sampled pairs per task. On the smaller-domain tasks where W ⋅ ( W − 1 ) < 50 W\cdot(W-1)<50 , all pairs are used.

The two steering strategies differ both in how the waypoints 𝝅 ⁡ ( t ) \bm{\pi}(t) are constructed and in what the intervention replaces (Eqs. 1 , 2 ). For manifold steering, c a , c b c_{a},c_{b} live in intrinsic coordinates and the path is the manifold geodesic between them; at each waypoint we decode 𝝅 ⁡ ( t ) \bm{\pi}(t) onto ℳ h \mathcal{M}_{h} in the 64-dimensional PCA subspace, lift it back to the residual-stream basis via the PCA inverse, and combine it with the prompt’s unchanged off-subspace residual – so the steered activation differs from the base only in its top-64 PCA components. For linear steering, c a , c b c_{a},c_{b} are the raw activation centroids in the full residual stream, the path is the straight line between them, and the entire residual-stream activation is replaced by 𝝅 ⁡ ( t ) \bm{\pi}(t) at each step.

### A.7 Naturalness Metric

The cumulative output energy E BC E_{\text{BC}} of § 3.2 (Eq. 3 ) is computed as the sum of the Bhattacharyya distances D BC ( 𝜸 ( t ) , ℳ y ) = − log ∑ i γ i ​ ( t ) ​ q i ​ ( t ) D_{\text{BC}}(\bm{\gamma}(t),\mathcal{M}_{y})=-\log\sum_{i}\sqrt{\gamma_{i}(t)\,q_{i}(t)} between the induced output distribution at each of the K = 50 K=50 waypoints along steering paths and the closest point q ⁡ ( t ) q(t) on ℳ y \mathcal{M}_{y} . We use the Bhattacharyya distance because ℳ y \mathcal{M}_{y} is fit in Hellinger geometry and the two are tightly related, D BC = − log ⁡ ( 1 − d H 2 ) D_{\text{BC}}=-\log(1-d_{H}^{2}) , so D BC D_{\text{BC}} stays inside the same geometry the manifold was constructed in. For each of the up to 50 sampled centroid pairs, we average the per-waypoint cumulative sum across the base prompts to obtain one scalar per pair, and report the mean and standard error of these per-pair scalars.

### A.8 Pullback Optimization

The pullback procedure of § 3.3 consists of two stages. First, we fix a behavioral target by evaluating the spline geodesic on ℳ y \mathcal{M}_{y} between the two behavior centroids b a b_{a} and b b b_{b} at K = 20 K=20 uniform fractions, yielding a sequence of target distributions 𝒑 ^ t ∈ ℳ y \hat{\bm{p}}_{t}\in\mathcal{M}_{y} . Second, we optimize an activation-space path π h pullback \pi_{h}^{\text{pullback}} which, when used to intervene at each waypoint, induces a behavioral trajectory matching 𝒑 ^ 0 : K \hat{\bm{p}}_{0:K} .

#### Path Parameterization.

We parameterise π h pullback \pi_{h}^{\text{pullback}} as a one-dimensional natural cubic spline through 10 control vectors at uniform t t -positions, all of which are optimisation variables. The path is evaluated at the same K = 20 K=20 uniform fractions used to generate the target. Each control vector is restricted to the first 32 PCA components of the 64-dimensional subspace; the remaining 32 components, together with the orthogonal residual, are held at the base prompt’s activation values during the intervention. We note that the linear and manifold-steering paths used as comparisons span the full 64-dimensional subspace, so the pullback optimization is operating within a strictly smaller search space.

#### Loss and optimizer.

The loss at each waypoint t t is the squared Hellinger distance d H 2 ​ ( 𝒑 𝒉 ← 𝝅 ⁡ ( t ) ​ ( x n ) , 𝒑 ^ t ) d_{H}^{2}(\bm{p}_{\bm{h}\leftarrow\bm{\pi}(t)}(x_{n}),\hat{\bm{p}}_{t}) between the induced output distribution and the target, averaged over 16 base prompts { x n } \{x_{n}\} sampled freshly per pair, each conditioned on ground-truth z a z_{a} (in contrast to the steering setup of § A.6 , where a fixed unfiltered set is reused across pairs). We minimize the sum of these per-waypoint losses with L-BFGS using strong-Wolfe line search, running 50 outer steps with up to 5 inner iterations each. The optimization is initialized by linearly interpolating between the two centroids in the 32-dimensional subspace and then sampling the resulting line at the 10 control- t t positions. We stop early when the relative change in loss between two consecutive outer steps falls below 10 − 3 10^{-3} . We disable the path-norm regularizer for weekdays; on the other three natural-domain tasks we add a small regularizer that penalizes deviations of ‖ 𝝅 ⁡ ( t ) ‖ \|\bm{\pi}(t)\| from the linear interpolation between the endpoint centroid norms | c a | , | c b | |c_{a}|,|c_{b}| . We use weight 10 − 3 10^{-3} for age and 5 × 10 − 4 5\times 10^{-4} for months and alphabet. This discourages the optimizer from drifting into a high-norm shortcut basin.

### A.9 Pullback Recovery R 2 R^{2}

To compare the optimised pullback path π h pullback \pi_{h}^{\text{pullback}} to the manifold-steering path π h ∗ \pi_{h}^{*} along ℳ h \mathcal{M}_{h} , we project both into the SVD basis of π h ∗ \pi_{h}^{*} that captures at least 99 % 99\% of π h ∗ \pi_{h}^{*} ’s variance. In this basis we define the residual at each pullback waypoint as its orthogonal closest-point distance to π h ∗ \pi_{h}^{*} , and report R 2 = 1 − ∑ t ‖ π h pullback ​ ( t ) − proj π h ∗ ​ π h pullback ​ ( t ) ‖ 2 ∑ t ‖ π h pullback ​ ( t ) − π ¯ h pullback ‖ 2 . R^{2}\;=\;1-\frac{\sum_{t}\|\pi_{h}^{\text{pullback}}(t)-\mathrm{proj}_{\pi_{h}^{*}}\pi_{h}^{\text{pullback}}(t)\|^{2}}{\sum_{t}\|\pi_{h}^{\text{pullback}}(t)-\bar{\pi}_{h}^{\text{pullback}}\|^{2}}. The linear baseline used in the same comparison is the straight chord between c a c_{a} and c b c_{b} in the 64-dimensional PCA subspace—not the linear-steering trajectory after intervention. As in § A.7 , the values reported in § 3.3 are mean ± \pm standard error across the per-pair scalars, with p p -values from paired t t -tests against the linear baseline.

## Appendix B Experimental Details for the Vision Task

This section contains additional details on the experiment from § 5 .

### B.1 Mountain Car.

#### Data collection.

To recover the encoder’s position manifold we harvest activations on 100 100 rollouts collected in MountainCar-v0 (max 200 200 steps per episode) under a mixed stochastic policy chosen to give broad coverage of the position-velocity state space. At the start of each episode we sample one of two policies: with probability 0.7 0.7 , a noisy momentum policy that pushes in the direction of the current velocity but, at each step, replaces the action with a uniform random action with probability 0.4 0.4 ; with probability 0.3 0.3 , an oscillating square-wave policy that alternates between full-left and full-right thrust on a fixed period sampled uniformly from { 5 , … , 25 } \{5,\dots,25\} steps. We then pass each rendered frame through the trained encoder, label the resulting activation with the underlying ground-truth position, and fit the manifold to this collection of position-labelled activations.

#### Manifold fitting.

To parameterize the activation manifold, ℳ \mathcal{M} , we partition the position range into B = 100 B=100 bins, compute the mean encoder output per occupied bin, and fit a smoothing spline γ ℳ : [ 0 , 1 ] → ℳ \gamma_{\mathcal{M}}\colon[0,1]\to\mathcal{M} through these means (one univariate spline per coordinate, weighted by the square root of bin counts to regularize sparse regions). We additionally verify via linear probing that the encoder representations z t z_{t} encode the ground-truth physics: a Ridge regression probe recovers position with R 2 ≈ 0.95 R^{2}\approx 0.95 and velocity with R 2 ≈ 0.90 R^{2}\approx 0.90 . A three-component PCA of the encoder outputs reveals the spline γ ℳ \gamma_{\mathcal{M}} as a curve that closely tracks the data manifold, while the chord ℓ \ell between the same endpoints cuts through its interior.

To parameterize the behavior manifold, ℳ y \mathcal{M}_{y} , discretize 𝒵 \mathcal{Z} into B B bins with centers { μ b } b = 1 B ⊂ ℝ n \{\mu_{b}\}_{b=1}^{B}\subset\mathbb{R}^{n} obtained by evaluating the activation-manifold spline at B B evenly spaced positions, μ b = γ ℳ ​ ( p b ) \mu_{b}=\gamma_{\mathcal{M}}(p_{b}) . The mapping to behavior is F ⁡ ( z ) = softmax ​ ( − ‖ z − μ b ‖ 2 τ ) b = 1 B ∈ Δ B − 1 , F(z)\;=\;\mathrm{softmax}\!\left(-\frac{\|z-\mu_{b}\|_{2}}{\tau}\right)_{b=1}^{B}\;\in\;\Delta^{B-1}, (10) with temperature τ = 0.5 \tau=0.5 . F F is a smooth, deterministic map from activations to position distributions. We use B = 128 B=128 , which makes F F ’s Jacobian full column-rank, ensuring the inverse problem has a locally unique solution. For each bin i i , the natural centroid on the behavior manifold is the model’s average output distribution conditioned on samples in that bin: b i = 𝔼 ⁡ [ F ⁡ ( z ) ∣ bin ⁡ ( z ) = i ] ∈ Δ B − 1 . b_{i}\;=\;\mathbb{E}\!\left[F(z)\mid\mathrm{bin}(z)=i\right]\;\in\;\Delta^{B-1}. (11) Because the bin grid is dense relative to the data manifold’s intrinsic dimension, b i b_{i} is well-defined for all bins. We embed each b i b_{i} in Hellinger coordinates h i = b i h_{i}=\sqrt{b_{i}} on the unit sphere of ℝ B \mathbb{R}^{B} and fit a 1D smoothing spline γ ℳ y : 𝒵 → ℝ B \gamma_{\mathcal{M}_{y}}\colon\mathcal{Z}\to\mathbb{R}^{B} through { h i } \{h_{i}\} parameterized by position. This is the behavior manifold ℳ y \mathcal{M}_{y} .

## Appendix C Additional Results

### C.1 In-Context Learning of Representations.

In addition to results provided in the main text, we test a 9 × 9 9\times 9 cylinder in the ICLR domains, and find that despite the added complexity of a periodic dimension and substantially more graph nodes, when Llama 3.1 8B is provided sufficient context (2048 tokens in this case), it reaches above 80 % 80\% neighborhood accuracy (probability mass on valid neighbors). We fit a manifold and steer along this domain, finding that the result of factored control generalizes beyond the graph domain.

### C.2 Manifold steering allows manipulation of uncertainty without loss of structure.

To examine multi-dimensional concepts in known domains, we partition weekday addition centroids by addition value ( 1 1 – 5 5 , 6 6 – 10 10 , 11 11 – 15 15 , 16 16 – 20 20 ), revealing concentric circles along a second manifold dimension forming a cylinder-like structure (Fig. 11 ). Manifold steering along the circular dimension maintains ordered weekday transitions with increasing entropy per group. This suggests manifold geometry can serve as a handle for calibrating model confidence in a controlled fashion. The experiment was conducted with Llama 3.1 3.1 70 70 B layer 70 70 .

### C.3 Mountain Car

#### Structural Correspondence between ℳ h \mathcal{M}_{h} and ℳ y \mathcal{M}_{y} .