Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior — part 6/6

Cleaned-text code points [77466, 82318). Bibliography omitted here; source.md includes it.

If ℳ h \mathcal{M}_{h} encodes the model’s predictive distributions over 𝒵 \mathcal{Z} , then ℳ h \mathcal{M}_{h} and ℳ y \mathcal{M}_{y} should be approximately isometric—distances along one manifold should correlate with distances along the other. We test this by sampling W = 50 W=50 anchor positions inside the shared parameter range and computing pairwise arc lengths along each manifold: d ℳ ​ ( p i , p j ) = ∫ p i p j ‖ γ ℳ h ′ ​ ( p ) ‖ 2 ​ 𝑑 p , d ℳ y ​ ( p i , p j ) = 1 2 ​ ∫ p i p j ‖ γ ℳ y ′ ​ ( p ) ‖ 2 ​ 𝑑 p , d_{\mathcal{M}}(p_{i},p_{j})=\int_{p_{i}}^{p_{j}}\|\gamma_{\mathcal{M}_{h}}^{\prime}(p)\|_{2}\,dp,\quad d_{\mathcal{M}_{y}}(p_{i},p_{j})=\frac{1}{\sqrt{2}}\int_{p_{i}}^{p_{j}}\|\gamma_{\mathcal{M}_{y}}^{\prime}(p)\|_{2}\,dp, (12) where the 1 / 2 1/\sqrt{2} on the behavior side converts the Euclidean integral in Hellinger ambient space to Hellinger units. The Pearson correlation between { d ℳ h ​ ( p i , p j ) } \{d_{\mathcal{M}_{h}}(p_{i},p_{j})\} and { d ℳ y ​ ( p i , p j ) } \{d_{\mathcal{M}_{y}}(p_{i},p_{j})\} over all ( 50 2 ) = 1225 \binom{50}{2}=1225 pairs is r = 0.996 r=\mathbf{0.996} ; the chord distances used by linear steering correlate far less ( r = 0.06 r=0.06 between activation chord and behavior arc length), since chords cut across the encoder loop and are structurally divorced from the encoded conceptual geometry.

#### Pullback: Behavior space steering.

Having established the bottom-up direction in Fig. 7 , i.e., paths along ℳ h \mathcal{M}_{h} produce behavior trajectories on ℳ y \mathcal{M}_{y} , We now test the top-down direction: starting from a behaviorally-natural trajectory in ℳ y \mathcal{M}_{y} , do we naturally recover an activation path that traces ℳ h \mathcal{M}_{h} ? For each endpoint pair ( p a , p b ) (p_{a},p_{b}) we construct the conformal behavior target γ ^ α \hat{\gamma}_{\alpha} via the procedure of § 3.3 (geodesic on the simplex under cost c ⁡ ( p ) = exp ⁡ ( α ⋅ d H ​ ( p , ℳ y ) ) c(p)=\exp(\alpha\cdot d_{H}(p,\mathcal{M}_{y})) ). We then optimize an activation path π α = ( v 0 , … , v K ) \pi_{\alpha}=(v_{0},\dots,v_{K}) in ℝ n \mathbb{R}^{n} to minimize L ⁡ ( π ) = ∑ t = 0 K ‖ F ⁡ ( v t ) − γ ^ α ​ ( t ) ‖ 2 2 . L(\pi)\;=\;\sum_{t=0}^{K}\bigl\|\sqrt{F(v_{t})}-\sqrt{\hat{\gamma}_{\alpha}(t)}\bigr\|_{2}^{2}. (13) Following the language-model setup, all K + 1 K+1 waypoints (including endpoints) are free parameters, initialized at the linear chord and optimized jointly via L-BFGS with strong-Wolfe line search. We use K = 30 K=30 waypoints and run independent optimizations for each of 30 endpoint pairs.

Across all 30 endpoint pairs, the pullback paths π α \pi_{\alpha} closely trace ℳ h \mathcal{M}_{h} (Fig. 12 ): The mean Euclidean distance from π \pi to ℳ h \mathcal{M}_{h} , averaged over waypoints and pairs: linear ​ chord : 2.22 , geometric ⁡ ( ℳ h ) : 0.20 , pullback : 0.29 . \mathrm{linear\ chord}\!:\;\;2.22,\qquad\mathrm{geometric}\;(\mathcal{M}_{h})\!:\;\;0.20,\qquad\mathrm{pullback}\!:\;\;0.29. The pullback path is at 95.4 % \mathbf{95.4\%} of the chord-to-geometric recovery and dominates the chord baseline on 30 / 30 30/30 pairs. The aggregate degradation is concentrated on pairs with one endpoint at the extreme wall position ( p ≈ − 1.2 p\approx-1.2 ), where the encoder geometry has tighter curvature; on the remaining ∼ \sim 20 pairs, π ∞ \pi_{\infty} is essentially indistinguishable from ℳ h \mathcal{M}_{h} itself. The α \alpha -sweep traces the same family of trajectories observed in the language-model experiments: at α = 0 \alpha=0 the conformal target is the unrestricted Hellinger geodesic on the simplex, and the recovered π 0 \pi_{0} leaves ℳ h \mathcal{M}_{h} in order to match this off-manifold target; as α \alpha grows the target is pushed onto ℳ y \mathcal{M}_{y} and the recovered π α \pi_{\alpha} correspondingly tracks ℳ h \mathcal{M}_{h} .

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .