Revisiting the Platonic Representation Hypothesis: An Aristotelian View — part 2/8

Cleaned-text code points [15749, 31277). Bibliography omitted here; source.md includes it.

Since CKA is scaled by the normalized self-similarity terms, which each scale as 𝒪 ⁡ ( d ) \mathcal{O}(\sqrt{d}) , the resulting null baseline for the metric is thus 𝒪 ⁡ ( d / n ) \mathcal{O}(d/n) to leading order. We refer to this 𝒪 ⁡ ( d / n ) \mathcal{O}(d/n) effect as the width confounder : at a fixed sample size n n it grows with the representation width d d , and it persists even when two representations share a genuine signal ( Proposition C.5 ).

This aligns with insights from random matrix theory: in high-dimensional regimes ( d ∼ n d\sim n ), the null singular spectrum of interaction operators (after centering/whitening) concentrates into a non-trivial “noise bulk” whose upper edge depends on d / n d/n and preprocessing, rather than collapsing to zero ( Wachter, 1978 ; Müller, 2002 ; Livan et al., 2018 ) . Our framework estimates this null baseline directly via permutation, providing a metric- and pipeline-independent alternative to asymptotic formulas.

##### Neighborhood metrics follow a different regime.

While spectral metrics have null baselines scaling as 𝒪 ⁡ ( d / n ) \mathcal{O}(d/n) , neighborhood-based metrics such as mutual k k -NN exhibit different behavior, as they rely on set comparisons rather than interactions.

###### Proposition 4.2 (Null baseline for neighborhood metrics) .

Assume the rows are i.i.d. with 𝐱 i \mathbf{x}_{i} and 𝐲 i \mathbf{y}_{i} independent, and that pairwise distances are almost surely distinct (e.g., under absolutely continuous distributions). Then for any k < n k<n , 𝔼 H 0 ​ [ mKNN ⁡ ( 𝐗 , 𝐘 ) ] = k n − 1 . \mathbb{E}_{H_{0}}\bigg[\mathrm{mKNN}(\mathbf{X},\mathbf{Y})\bigg]\;=\;\frac{k}{n-1}. (4)

Proof. See Section C.6 .

In particular, neighborhood metrics have null baselines scaling as 𝒪 ⁡ ( k / n ) \mathcal{O}(k/n) .

The difference in null baseline between spectral and neighborhood metrics is substantial: (i) The neighborhood scale k k can be fixed consistently across experiments, whereas the embedding dimension d d is determined by the model architecture, making it difficult to control in comparison studies. (ii) The neighborhood metrics are much less confounded since k ≪ d k\ll d in typical settings.

Geometric metrics ( e.g. , Procrustes), similarly to the spectral metrics, are functions of all-pairs distances or inner products, which are the interaction quantities of Proposition 4.1 . They thus inherit a width-dependent 𝒪 ⁡ ( d / n ) \mathcal{O}(d/n) baseline rather than the 𝒪 ⁡ ( k / n ) \mathcal{O}(k/n) baseline of rank-based neighborhood metrics. We verify this empirically in Section E.7 .

### 4.2 The depth confounder

A subtle yet pervasive issue is the comparison of selection-based alignment summaries across models. Let S ℓ , ℓ ′ := s ⁡ ( 𝐗 ℓ ( A ) , 𝐘 ℓ ′ ( B ) ) S_{\ell,\ell^{\prime}}:=s(\mathbf{X}^{(A)}_{\ell},\mathbf{Y}^{(B)}_{\ell^{\prime}}) be the similarity between layer ℓ \ell of model A A and layer ℓ ′ \ell^{\prime} of model B B . It is common to summarize the similarity between two models by the maximum alignment score T max = max ℓ , ℓ ′ ⁡ S ℓ , ℓ ′ T_{\max}=\max_{\ell,\ell^{\prime}}S_{\ell,\ell^{\prime}} ( Schrimpf et al., 2018 ; Huh et al., 2024 ; Raghu et al., 2017 ; Kornblith et al., 2019 ) . Let M = L A ​ L B M=L_{A}L_{B} be the number of layer pairs searched, where L A L_{A} and L B L_{B} are the depths of models A A and B B . Even under H 0 H_{0} , taking a maximum over M M comparisons inflates the reported score, a “look-elsewhere” effect. This is an instance of the classical multiple comparisons problem ( Benjamini and Hochberg, 1995 ; Bonferroni, 1936 ) : as M M increases, the probability that at least one null similarity exceeds any fixed threshold grows, inflating the expected maximum. Consequently, when alignment is summarized via a max or top- k k statistic without correction, unrelated representations can exhibit spuriously high reported similarity, as the inflation depends on model depth, making raw summaries non-comparable across architectures.

Characterizing this inflation does not require independence across pairs. It follows from a uniform right-tail bound. Assume there exist a common mean μ ∈ ℝ \mu\in\mathbb{R} and σ > 0 \sigma>0 such that the null fluctuations satisfy, for all ( ℓ , ℓ ′ ) (\ell,\ell^{\prime}) and all t ≥ 0 t\geq 0 , ℙ ⁡ ( S ℓ , ℓ ′ − μ ≥ t ) ≤ exp ⁡ ( − t 2 2 ​ σ 2 ) . \mathbb{P}(S_{\ell,\ell^{\prime}}-\mu\geq t)\leq\exp\!\left(-\frac{t^{2}}{2\sigma^{2}}\right). (5) For bounded similarities S ℓ , ℓ ′ ∈ [ s min , s max ] S_{\ell,\ell^{\prime}}\in[s_{\min},s_{\max}] , Hoeffding’s inequality implies a sub-Gaussian right-tail bound of the form Equation 5 with σ ≤ ( s max − s min ) / 2 \sigma\leq(s_{\max}-s_{\min})/2 . This covers many common bounded metrics ( e.g. , CKA / RSA / mKNN ). Crucially, only the right tail is needed for bounding the maximum. Then a union bound gives ℙ ⁡ ( T max − μ ≥ t ) ≤ M ​ exp ⁡ ( − t 2 2 ​ σ 2 ) , \mathbb{P}(T_{\max}-\mu\geq t)\leq M\exp\!\left(-\frac{t^{2}}{2\sigma^{2}}\right), (6) and consequently for a constant C C 𝔼 H 0 ​ [ T max ] ≤ μ + C ​ σ ​ log ⁡ M . \mathbb{E}_{H_{0}}\left[T_{\max}\right]\;\leq\;\mu+C\,\sigma\sqrt{\log M}. (7) Proof. See Section C.5 .

This creates a depth confounder. Deeper models (larger M = L A ​ L B M=L_{A}L_{B} ) can attain higher raw “max-alignment” scores purely because of a larger search space. Correlations across neighboring layers reduce the effective number of comparisons, but the inflation remains monotone in the search space size in typical workflows. Therefore, raw scaling plots of T max T_{\max} (or top- k k summaries) are not comparable across architectures unless the selection step itself is calibrated.

The two confounders are not independent but hierarchical: the depth confounder operates on top of the width confounder. The width confounder gives each individual layer pair a positive chance similarity, and deeper models supply a larger pool of such chance similarities, so the selection maximum drifts higher.

## 5 Representational similarity calibration

To overcome the issues of the width and depth confounders, we introduce the null-calibration for representational similarity. The key idea is to compare observed similarity scores against an empirical null distribution obtained by permuting sample correspondences, thereby establishing a principled zero point that accounts for finite-sample, high-dimensional artifacts.

### 5.1 Null-calibrated similarity

We propose null-calibrated similarity measures to correct for width and depth confounders by transforming raw similarity scores into an effect size with a principled zero point.

Given representations 𝐗 ∈ ℝ n × d x \mathbf{X}\in\mathbb{R}^{n\times d_{x}} and 𝐘 ∈ ℝ n × d y \mathbf{Y}\in\mathbb{R}^{n\times d_{y}} aligned by rows, we operationalize the null hypothesis H 0 H_{0} (no relationship beyond marginal statistics) by permuting sample correspondences. For permutations π k ∈ Π n \pi_{k}\in\Pi_{n} drawn i.i.d. uniformly from Π n \Pi_{n} and independently of ( 𝐗 , 𝐘 ) (\mathbf{X},\mathbf{Y}) , we form null scores s ( k ) = s ( 𝐗 , π k ( 𝐘 ) ) , k = 1 , … , K . s^{(k)}=s(\mathbf{X},\pi_{k}(\mathbf{Y})),\qquad k=1,\dots,K. (8) Let s obs := s ⁡ ( 𝐗 , 𝐘 ) s_{\mathrm{obs}}:=s(\mathbf{X},\mathbf{Y}) denote the observed score. Let s ( 1 ) ≤ s ( 2 ) ≤ ⋯ ≤ s ( K + 1 ) s_{(1)}\leq s_{(2)}\leq\cdots\leq s_{(K+1)} denote the order statistics of the combined multiset { s obs , s ( 1 ) , … , s ( K ) } \{s_{\mathrm{obs}},s^{(1)},\dots,s^{(K)}\} (with ties allowed). We define a right-tail rank-based critical value: τ α := s ( ⌈ ( 1 − α ) ​ ( K + 1 ) ⌉ ) , \tau_{\alpha}\;:=\;s_{(\lceil(1-\alpha)(K+1)\rceil)}, (9) where ⌈ ( 1 − α ) ​ ( K + 1 ) ⌉ \lceil(1-\alpha)(K+1)\rceil is the ( 1 − α ) (1-\alpha) -quantile of the ( K + 1 ) (K+1) -sized multiset and the empirical right-tail p p -value: p = 1 + # ⁡ { k ∈ { 1 , … , K } : s ( k ) ≥ s obs } K + 1 . p\;=\;\frac{1+\#\{k\in\{1,\dots,K\}:s^{(k)}\geq s_{\mathrm{obs}}\}}{K+1}. (10)

The critical value τ α \tau_{\alpha} defines a robust zero point: values below τ α \tau_{\alpha} are typical under H 0 H_{0} at level α \alpha , while p p provides an evidence measure that can be combined with multiple-testing correction when many comparisons are performed.

The proposed calibration framework relies on randomization (permutation) to construct a null distribution for any similarity statistic. This yields finite-sample guarantees under an exchangeability condition ( Assumption 3.1 ), and it implies useful invariances that make calibrated scores comparable across metrics and implementations.

The permutation p p -value in Equation 10 is super-uniform under H 0 H_{0} ( i.e. , ℙ H 0 ​ ( p ≤ α ) ≤ α \mathbb{P}_{H_{0}}(p\leq\alpha)\leq\alpha for all α ∈ [ 0 , 1 ] \alpha\in[0,1] ), a standard consequence of randomization inference ( Nichols and Holmes, 2002 ; Phipson and Smyth, 2010 ; Good, 2005 ) (see Section C.1 for formal definitions and proofs).

###### Corollary 5.1 (Type-I control for calibrated scores) .

Let s obs = s ⁡ ( 𝐗 , 𝐘 ) s_{\mathrm{obs}}=s(\mathbf{X},\mathbf{Y}) and s ( k ) = s ⁡ ( 𝐗 , π k ​ ( 𝐘 ) ) s^{(k)}=s(\mathbf{X},\pi_{k}(\mathbf{Y})) for k = 1 , … , K k=1,\dots,K . Define the add-one permutation p p -value p p as in Equation 10 , and equivalently define the rank-based critical value τ α := s ( ⌈ ( 1 − α ) ​ ( K + 1 ) ⌉ ) \tau_{\alpha}:=s_{(\lceil(1-\alpha)(K+1)\rceil)} from the sorted combined set { s obs , s ( 1 ) , … , s ( K ) } \{s_{\mathrm{obs}},s^{(1)},\dots,s^{(K)}\} . Under Assumption 3.1 , ℙ H 0 ​ ( p ≤ α ) ≤ α and hence ℙ H 0 ​ ( s obs > τ α ) ≤ α , \mathbb{P}_{H_{0}}\big(p\leq\alpha\big)\leq\alpha\quad\text{and hence}\quad\mathbb{P}_{H_{0}}\big(s_{\mathrm{obs}}>\tau_{\alpha}\big)\leq\alpha, (11) so the gating rule “ s cal > 0 s_{\mathrm{cal}}>0 ” (where s cal s_{\mathrm{cal}} is the calibrated score defined in Equation 12 , which implies p ≤ α p\leq\alpha ) is a finite-sample α \alpha -level declaration of similarity above chance.

Proof. Follows directly from Lemma C.2 ; see Section C.1 .

##### Calibrated score (scalar case).

While p p -values and null percentiles are rank-based and therefore invariant under monotone transformations of the raw score ( Proposition C.3 ; see Section C.2 ), effect sizes serve a complementary purpose: they quantify how much similarity exceeds chance on an interpretable scale. The calibrated score achieves this by rescaling the excess over the null threshold τ α \tau_{\alpha} to the interval [ 0 , 1 ] [0,1] . This rescaling is not monotone-invariant, by design. A purely rank-based calibration would be equivalent to a score shift and would be unable to correct for the scale-dependent null baselines identified in Section 4 . The calibrated score instead adapts to the actual null distribution, providing a meaningful zero point.

For similarity metrics with a known maximum (upper bound) s max s_{\max} (often s max = 1 s_{\max}=1 ), we define a max-preserving calibrated score s cal = max ⁡ ( s obs − τ α s max − τ α , 0 ) . s_{\mathrm{cal}}=\max\!\left(\frac{s_{\mathrm{obs}}-\tau_{\alpha}}{s_{\max}-\tau_{\alpha}},0\right). (12) This calibrated score depends on the chosen level α \alpha through τ α \tau_{\alpha} ( Equation 9 ). We therefore also report the corresponding permutation p p -value and/or null percentile for an α \alpha -free summary. This score satisfies s cal = 0 s_{\mathrm{cal}}=0 whenever s obs ≤ τ α s_{\mathrm{obs}}\leq\tau_{\alpha} ( i.e. , below the estimated right-tail critical value of the permutation null), and s cal = 1 s_{\mathrm{cal}}=1 when s obs = s max s_{\mathrm{obs}}=s_{\max} ( i.e. , perfect similarity remains 1 1 ). When s max s_{\max} is unknown, or the metric is unbounded, we default to the unnormalized effect size [ s − τ α ] + = max ⁡ ( s − τ α , 0 ) [s-\tau_{\alpha}]_{+}=\max(s-\tau_{\alpha},0) .

### 5.2 Aggregation-aware null-calibration

To analyze the similarity between two models A A and B B with depths L A L_{A} and L B L_{B} , a common approach is to compute a layer-by-layer similarity matrix 𝐒 ∈ ℝ L A × L B \mathbf{S}\in\mathbb{R}^{L_{A}\times L_{B}} by evaluating a similarity score for every pair of layers: S ℓ , ℓ ′ = s ⁡ ( 𝐗 ℓ ( A ) , 𝐘 ℓ ′ ( B ) ) , S_{\ell,\ell^{\prime}}=s\!\left(\mathbf{X}^{(A)}_{\ell},\mathbf{Y}^{(B)}_{\ell^{\prime}}\right), (13) where 𝐗 ℓ ( A ) ∈ ℝ n × d ℓ \mathbf{X}^{(A)}_{\ell}\in\mathbb{R}^{n\times d_{\ell}} and 𝐘 ℓ ′ ( B ) ∈ ℝ n × d ℓ ′ \mathbf{Y}^{(B)}_{\ell^{\prime}}\in\mathbb{R}^{n\times d_{\ell^{\prime}}} are the representations of models A A and B B at layers ℓ \ell and ℓ ′ \ell^{\prime} respectively, evaluated on n n samples, and s ⁡ ( ⋅ , ⋅ ) s(\cdot,\cdot) is a similarity metric. A common practice is then to summarize 𝐒 \mathbf{S} by a selection-based aggregation operator, such as taking the maximum. These summaries are attractive because they support statements such as “there exists a layer in A A that matches some layer in B B ” or “each layer of A A best matches a layer in B B ”. However, selection introduces a statistical effect: even under the null hypothesis of no relationship between representations, selection-based summaries are systematically inflated.

As analyzed in Section 4.2 , this inflation grows with the number of layer pairs and makes naïve post-selection p p -values anti-conservative. Our aggregation-aware calibration addresses this by calibrating the reported statistic directly: the null distribution must match the entire analysis pipeline. Let the aggregate score be T ⁡ ( 𝐒 ) T(\mathbf{S}) ( e.g. , a maximum), then the appropriate null is the distribution of T ⁡ ( 𝐒 ) T(\mathbf{S}) under a valid null transformation ( e.g. , permuting sample correspondences). We therefore define an aggregation-aware permutation null. When T T is the maximum over layer pairs, this recovers the classical “maxT” procedure of Westfall and Young (1993) . Our formulation generalizes it to arbitrary selection-based aggregates ( e.g. , top- k k ) and to any similarity metric.

##### Consistency of permutations across layers.

For each draw π k ∈ Π n \pi_{k}\in\Pi_{n} , we apply the same sample permutation to all layers of model B B and define S ℓ , ℓ ′ ( k ) := s ⁡ ( 𝐗 ℓ ( A ) , π k ​ ( 𝐘 ℓ ′ ( B ) ) ) , \displaystyle S^{(k)}_{\ell,\ell^{\prime}}:=s\!\left(\mathbf{X}^{(A)}_{\ell},\pi_{k}\!\left(\mathbf{Y}^{(B)}_{\ell^{\prime}}\right)\right), (14) ℓ = 1 , … , L A , ℓ ′ = 1 , … , L B , \displaystyle\ell=1,\dots,L_{A},\quad\ell^{\prime}=1,\dots,L_{B}, then compute T ( k ) := T ⁡ ( 𝐒 ( k ) ) T^{(k)}:=T(\mathbf{S}^{(k)}) . Let T obs := T ⁡ ( 𝐒 ) T_{\mathrm{obs}}:=T(\mathbf{S}) denote the observed aggregate. Let T ( 1 ) ≤ ⋯ ≤ T ( K + 1 ) T_{(1)}\leq\cdots\leq T_{(K+1)} denote the order statistics of the combined set { T obs , T ( 1 ) , … , T ( K ) } \{T_{\mathrm{obs}},T^{(1)},\dots,T^{(K)}\} (with ties allowed). We define τ α agg := T ( ⌈ ( 1 − α ) ​ ( K + 1 ) ⌉ ) , \tau_{\alpha}^{\mathrm{agg}}:=T_{(\lceil(1-\alpha)(K+1)\rceil)}, (15) where ⌈ ( 1 − α ) ​ ( K + 1 ) ⌉ \lceil(1-\alpha)(K+1)\rceil is the ( 1 − α ) (1-\alpha) -quantile of the ( K + 1 ) (K+1) -sized multiset. We report the right-tail permutation p p -value p agg = 1 + # ⁡ { k ∈ { 1 , … , K } : T ( k ) ≥ T obs } K + 1 , p_{\mathrm{agg}}=\frac{1+\#\{k\in\{1,\ldots,K\}:T^{(k)}\geq T_{\mathrm{obs}}\}}{K+1}, (16) By the same exchangeability argument as for scalar calibration, p agg p_{\mathrm{agg}} is super-uniform under H 0 H_{0} (see Proposition C.4 ).

##### Calibrated score (aggregate case).