Revisiting the Platonic Representation Hypothesis: An Aristotelian View — part 4/8

Cleaned-text code points [46820, 62625). Bibliography omitted here; source.md includes it.

This section provides formal definitions of the similarity metrics used throughout the paper. In the main text, we primarily use CKA (linear and RBF kernel) ( Kornblith et al., 2019 ) , RSA ( Kriegeskorte et al., 2008 ) , and mutual k k -NN ( Huh et al., 2024 ) as representative metrics from the spectral, geometric, and neighborhood families, respectively. Additional metrics (SVCCA, PWCCA, cycle- k k NN, CKNNA, RV coefficient, Procrustes) are included for completeness and used in supplementary experiments.

#### B.2.1 Spectral metrics

##### Linear Centered Kernel Alignment (CKA) .

Linear CKA ( Kornblith et al., 2019 ) can be written as a normalized Frobenius energy of the sample cross-covariance operator. With 𝐗 c , 𝐘 c \mathbf{X}_{c},\mathbf{Y}_{c} as above, define the sample (cross-)covariances 𝚺 ~ X ​ X := 1 n − 1 ​ 𝐗 c ⊤ ​ 𝐗 c , 𝚺 ~ Y ​ Y := 1 n − 1 ​ 𝐘 c ⊤ ​ 𝐘 c , 𝐂 ~ := 𝚺 ~ X ​ Y := 1 n − 1 ​ 𝐗 c ⊤ ​ 𝐘 c . \widetilde{\mathbf{\Sigma}}_{XX}:=\frac{1}{n-1}\mathbf{X}_{c}^{\top}\mathbf{X}_{c},\qquad\widetilde{\mathbf{\Sigma}}_{YY}:=\frac{1}{n-1}\mathbf{Y}_{c}^{\top}\mathbf{Y}_{c},\qquad\widetilde{\mathbf{C}}:=\widetilde{\mathbf{\Sigma}}_{XY}:=\frac{1}{n-1}\mathbf{X}_{c}^{\top}\mathbf{Y}_{c}. (19) The biased linear Hilbert-Schmidt Independence Criterion (HSIC) energy equals ‖ 𝐂 ~ ‖ F 2 \|\widetilde{\mathbf{C}}\|_{F}^{2} . The commonly used linear CKA normalization can be written as CKA lin ​ ( 𝐗 , 𝐘 ) = ‖ 𝐂 ~ ‖ F 2 ‖ 𝚺 ~ X ​ X ‖ F ​ ‖ 𝚺 ~ Y ​ Y ‖ F = ‖ 𝐗 c ⊤ ​ 𝐘 c ‖ F 2 ‖ 𝐗 c ⊤ ​ 𝐗 c ‖ F ​ ‖ 𝐘 c ⊤ ​ 𝐘 c ‖ F ∈ [ 0 , 1 ] , \mathrm{CKA}_{\mathrm{lin}}(\mathbf{X},\mathbf{Y})\;=\;\frac{\|\widetilde{\mathbf{C}}\|_{F}^{2}}{\|\widetilde{\mathbf{\Sigma}}_{XX}\|_{F}\,\|\widetilde{\mathbf{\Sigma}}_{YY}\|_{F}}\;=\;\frac{\|\mathbf{X}_{c}^{\top}\mathbf{Y}_{c}\|_{F}^{2}}{\|\mathbf{X}_{c}^{\top}\mathbf{X}_{c}\|_{F}\,\|\mathbf{Y}_{c}^{\top}\mathbf{Y}_{c}\|_{F}}\;\in\;[0,1], (20) where the second equality follows by cancellation of common 1 / ( n − 1 ) 1/(n-1) factors. What it measures: linear CKA treats two representations as similar when pairs of points with high inner product in one space also have high inner product in the other, i.e. , when their global second-order geometry agrees. It is invariant to rotations and isotropic rescaling, but not to general invertible linear maps.

##### Kernel Centered Kernel Alignment .

Kernel CKA ( Kornblith et al., 2019 ) generalizes linear CKA by replacing dot products with kernel functions. Let k X : ℝ d x × ℝ d x → ℝ k_{X}:\mathbb{R}^{d_{x}}\times\mathbb{R}^{d_{x}}\to\mathbb{R} and k Y : ℝ d y × ℝ d y → ℝ k_{Y}:\mathbb{R}^{d_{y}}\times\mathbb{R}^{d_{y}}\to\mathbb{R} be positive semidefinite kernel functions ( e.g. , RBF kernel k X ( 𝐱 , 𝐱 ′ ) = exp ( − ∥ 𝐱 − 𝐱 ′ ∥ 2 / 2 σ 2 ) k_{X}(\mathbf{x},\mathbf{x}^{\prime})=\exp(-\|\mathbf{x}-\mathbf{x}^{\prime}\|^{2}/2\sigma^{2}) ). Let 𝐊 X ∈ ℝ n × n \mathbf{K}_{X}\in\mathbb{R}^{n\times n} and 𝐊 Y ∈ ℝ n × n \mathbf{K}_{Y}\in\mathbb{R}^{n\times n} be Gram matrices with entries ( 𝐊 X ) i ​ j = k X ​ ( 𝐱 i , 𝐱 j ) (\mathbf{K}_{X})_{ij}=k_{X}(\mathbf{x}_{i},\mathbf{x}_{j}) and ( 𝐊 Y ) i ​ j = k Y ​ ( 𝐲 i , 𝐲 j ) (\mathbf{K}_{Y})_{ij}=k_{Y}(\mathbf{y}_{i},\mathbf{y}_{j}) . Let 𝐊 ~ X = 𝐇𝐊 X ​ 𝐇 \widetilde{\mathbf{K}}_{X}=\mathbf{H}\mathbf{K}_{X}\mathbf{H} and 𝐊 ~ Y = 𝐇𝐊 Y ​ 𝐇 \widetilde{\mathbf{K}}_{Y}=\mathbf{H}\mathbf{K}_{Y}\mathbf{H} denote centered Gram matrices. Kernel CKA is defined as: CKA k X , k Y ​ ( 𝐗 , 𝐘 ) = ⟨ 𝐊 ~ X , 𝐊 ~ Y ⟩ F ‖ 𝐊 ~ X ‖ F ​ ‖ 𝐊 ~ Y ‖ F . \mathrm{CKA}_{k_{X},k_{Y}}(\mathbf{X},\mathbf{Y})\;=\;\frac{\langle\widetilde{\mathbf{K}}_{X},\widetilde{\mathbf{K}}_{Y}\rangle_{F}}{\|\widetilde{\mathbf{K}}_{X}\|_{F}\,\|\widetilde{\mathbf{K}}_{Y}\|_{F}}. (21) where ⟨ A , B ⟩ F = tr ⁡ ( A ⊤ ​ B ) \langle A,B\rangle_{F}=\mathrm{tr}(A^{\top}B) . With positive semidefinite kernels and the biased HSIC estimator, the numerator is nonnegative, and kernel CKA typically lies in [ 0 , 1 ] [0,1] . What it measures: kernel CKA treats representations as similar when their kernel-induced similarity patterns over points agree. With an RBF kernel of small bandwidth, this emphasizes local, nonlinear similarity structure rather than global linear geometry.

##### Unbiased Centered Kernel Alignment .

The biased HSIC estimator can yield inflated similarity scores at finite sample sizes. Song et al. (2012) derived an unbiased HSIC estimator by recognizing that HSIC can be formulated as a U-statistic. Following Kornblith et al. (2019) , we substitute the unbiased estimator into the CKA formula. Let 𝐊 ̊ X \mathring{\mathbf{K}}_{X} denote the Gram matrix 𝐊 X \mathbf{K}_{X} with its diagonal entries set to zero (and likewise 𝐊 ̊ Y \mathring{\mathbf{K}}_{Y} ). The unbiased HSIC estimator is: HSIC u ​ ( 𝐊 X , 𝐊 Y ) = 1 n ⁡ ( n − 3 ) ​ ( tr ⁡ ( 𝐊 ̊ X ​ 𝐊 ̊ Y ) + 𝟏 ⊤ ​ 𝐊 ̊ X ​ 𝟏 ⋅ 𝟏 ⊤ ​ 𝐊 ̊ Y ​ 𝟏 ( n − 1 ) ​ ( n − 2 ) − 2 n − 2 ​ 𝟏 ⊤ ​ 𝐊 ̊ X ​ 𝐊 ̊ Y ​ 𝟏 ) . \mathrm{HSIC}_{u}(\mathbf{K}_{X},\mathbf{K}_{Y})=\frac{1}{n(n-3)}\left(\mathrm{tr}(\mathring{\mathbf{K}}_{X}\mathring{\mathbf{K}}_{Y})+\frac{\mathbf{1}^{\top}\mathring{\mathbf{K}}_{X}\mathbf{1}\cdot\mathbf{1}^{\top}\mathring{\mathbf{K}}_{Y}\mathbf{1}}{(n-1)(n-2)}-\frac{2}{n-2}\mathbf{1}^{\top}\mathring{\mathbf{K}}_{X}\mathring{\mathbf{K}}_{Y}\mathbf{1}\right). (22) Unbiased CKA replaces both numerator and denominator of Equation 21 with this estimator. Unlike the biased version, unbiased CKA can take small negative values at finite n n . What it measures: the same agreement of similarity patterns as kernel CKA , but with the finite-sample inflation of HSIC removed, so a value near zero indicates no more agreement than expected by chance.

##### Canonical Correlation Analysis (CCA) -based similarity.

CCA ( Weenink, 2003 ) measures linear subspace alignment. The sample canonical correlations { ρ i } i = 1 r \{\rho_{i}\}_{i=1}^{r} (with r = rank ⁡ ( 𝚺 ~ X ​ Y ) r=\mathrm{rank}(\widetilde{\mathbf{\Sigma}}_{XY}) ) are the singular values of the whitened cross-covariance operator 𝐓 ~ CCA = 𝚺 ~ X ​ X − 1 2 ​ 𝚺 ~ X ​ Y ​ 𝚺 ~ Y ​ Y − 1 2 . \widetilde{\mathbf{T}}_{\mathrm{CCA}}\;=\;\widetilde{\mathbf{\Sigma}}_{XX}^{-\tfrac{1}{2}}\,\widetilde{\mathbf{\Sigma}}_{XY}\,\widetilde{\mathbf{\Sigma}}_{YY}^{-\tfrac{1}{2}}. (23) Common scalar summaries include the mean canonical correlation 1 r ​ ∑ i = 1 r ρ i \frac{1}{r}\sum_{i=1}^{r}\rho_{i} or a weighted average as used in SVCCA ( Raghu et al., 2017 ) and PWCCA ( Morcos et al., 2018 ) . What it measures: CCA treats two representations as similar when one can be mapped onto the other by an invertible linear transformation, i.e. , when they span the same linear subspace. It ignores rotations, scaling, and other invertible linear changes within each space.

##### Singular Vector Canonical Correlation Analysis (SVCCA) .

SVCCA ( Raghu et al., 2017 ) combines dimensionality reduction via singular value decomposition (SVD) with CCA . First, truncated SVD is applied to each representation to retain the top principal components, yielding 𝐗 ′ ∈ ℝ n × p \mathbf{X}^{\prime}\in\mathbb{R}^{n\times p} and 𝐘 ′ ∈ ℝ n × q \mathbf{Y}^{\prime}\in\mathbb{R}^{n\times q} . Then CCA is applied to the reduced representations, yielding canonical correlations { ρ i } i = 1 r \{\rho_{i}\}_{i=1}^{r} . The SVCCA similarity is the mean canonical correlation: SVCCA ⁡ ( 𝐗 , 𝐘 ) = 1 r ​ ∑ i = 1 r ρ i . \mathrm{SVCCA}(\mathbf{X},\mathbf{Y})=\frac{1}{r}\sum_{i=1}^{r}\rho_{i}. (24) What it measures: SVCCA treats representations as similar when their high-variance subspaces are linearly aligned, discarding low-variance (noise) directions before measuring linear correspondence.

##### Projection Weighted Canonical Correlation Analysis (PWCCA) .

PWCCA ( Morcos et al., 2018 ) improves upon SVCCA by weighting canonical correlations according to their importance in explaining the original representations. Let 𝐡 i X \mathbf{h}_{i}^{X} and 𝐡 i Y \mathbf{h}_{i}^{Y} denote the i i -th canonical variables (projections onto canonical directions). The weight for the i i -th canonical correlation is the overall magnitude of the corresponding canonical variable across the dataset: α i = ∑ m = 1 n | ( 𝐡 i X ) m | = ∥ 𝐡 i X ∥ 1 , \alpha_{i}=\sum_{m=1}^{n}\left|(\mathbf{h}_{i}^{X})_{m}\right|=\lVert\mathbf{h}_{i}^{X}\rVert_{1}, (25) the ℓ 1 \ell_{1} norm of the i i -th canonical variable over the n n samples, following the reference implementation of Morcos et al. (2018) . The PWCCA similarity is the weighted mean: PWCCA ⁡ ( 𝐗 , 𝐘 ) = ∑ i = 1 r α i ​ ρ i ∑ i = 1 r α i . \mathrm{PWCCA}(\mathbf{X},\mathbf{Y})=\frac{\sum_{i=1}^{r}\alpha_{i}\rho_{i}}{\sum_{i=1}^{r}\alpha_{i}}. (26) This weighting ensures that canonical correlations corresponding to principal directions receive higher weight than those corresponding to noise dimensions. What it measures: like CCA , PWCCA treats representations as similar when they linearly correspond, but it counts agreement along high-variance directions more, so two representations are similar when their dominant directions align.

##### RV coefficient.

The RV (“Relation between two sets of Variables”) coefficient ( Robert and Escoufier, 1976 ; Smilde et al., 2009 ) is a multivariate generalization of the squared Pearson correlation. It measures the similarity between two configuration matrices via their inner-product (Gram) matrices. Let 𝐖 X = 𝐗 c ​ 𝐗 c ⊤ \mathbf{W}_{X}=\mathbf{X}_{c}\mathbf{X}_{c}^{\top} and 𝐖 Y = 𝐘 c ​ 𝐘 c ⊤ \mathbf{W}_{Y}=\mathbf{Y}_{c}\mathbf{Y}_{c}^{\top} be the inner-product (Gram) matrices of the centered representations 𝐗 c , 𝐘 c \mathbf{X}_{c},\mathbf{Y}_{c} . The RV coefficient is: RV ⁡ ( 𝐗 , 𝐘 ) = tr ⁡ ( 𝐖 X ​ 𝐖 Y ) tr ⁡ ( 𝐖 X 2 ) ​ tr ​ ( 𝐖 Y 2 ) ∈ [ 0 , 1 ] . \mathrm{RV}(\mathbf{X},\mathbf{Y})=\frac{\mathrm{tr}(\mathbf{W}_{X}\mathbf{W}_{Y})}{\sqrt{\mathrm{tr}(\mathbf{W}_{X}^{2})\ \mathrm{tr}(\mathbf{W}_{Y}^{2})}}\;\in\;[0,1]. (27) What it measures: the RV coefficient treats representations as similar when their point-by-point inner-product (Gram) matrices match. Like linear CKA , it captures global second-order geometry and is invariant to rotation but sensitive to scaling. Computed on centered representations (our default convention), the RV coefficient coincides exactly with linear CKA ; we list it separately as it arises from a different historical motivation and include it for completeness.

#### B.2.2 Geometric metrics

##### Representational Similarity Analysis (RSA) via Spearman correlation of dissimilarity matrices.

RSA ( Kriegeskorte et al., 2008 ) compares the geometry induced by pairwise dissimilarities. Let δ ⁡ ( ⋅ , ⋅ ) \delta(\cdot,\cdot) be a dissimilarity on representation vectors ( e.g. , correlation distance δ ⁡ ( 𝐮 , 𝐯 ) = 1 − corr ⁡ ( 𝐮 , 𝐯 ) \delta(\mathbf{u},\mathbf{v})=1-\mathrm{corr}(\mathbf{u},\mathbf{v}) , cosine distance). Define Representational Dissimilarity Matrices (RDMs) ( 𝐃 X ) i ​ j = δ ⁡ ( 𝐱 i , 𝐱 j ) , ( 𝐃 Y ) i ​ j = δ ⁡ ( 𝐲 i , 𝐲 j ) , (\mathbf{D}_{X})_{ij}=\delta(\mathbf{x}_{i},\mathbf{x}_{j}),\qquad(\mathbf{D}_{Y})_{ij}=\delta(\mathbf{y}_{i},\mathbf{y}_{j}), (28) and let vec △ ​ ( 𝐃 ) ∈ ℝ n ⁡ ( n − 1 ) / 2 \mathrm{vec}_{\triangle}(\mathbf{D})\in\mathbb{R}^{n(n-1)/2} denote vectorization of the strict upper triangle. RSA is then computed as a rank correlation between the two RDM vectors: RSA ⁡ ( 𝐗 , 𝐘 ) = ρ S ​ ( vec △ ​ ( 𝐃 X ) , vec △ ​ ( 𝐃 Y ) ) , \mathrm{RSA}(\mathbf{X},\mathbf{Y})\;=\;\rho_{S}\!\left(\mathrm{vec}_{\triangle}(\mathbf{D}_{X}),\;\mathrm{vec}_{\triangle}(\mathbf{D}_{Y})\right), (29) where Spearman’s ρ \rho can be expressed as Pearson correlation of ranks, ρ S ​ ( 𝐮 , 𝐯 ) = corr ⁡ ( rank ⁡ ( 𝐮 ) , rank ⁡ ( 𝐯 ) ) . \rho_{S}(\mathbf{u},\mathbf{v})\;=\;\mathrm{corr}\!\left(\mathrm{rank}(\mathbf{u}),\;\mathrm{rank}(\mathbf{v})\right). (30) What it measures: RSA treats two representations as similar when they rank pairs of points in the same order of (dis)similarity. It compares the relational geometry of distances rather than the coordinates, and depends only on the pattern of pairwise dissimilarities.

##### Procrustes distance.

The orthogonal Procrustes distance ( Williams et al., 2021 ) measures the minimal Euclidean distance between two representations after optimal orthogonal alignment. Assuming d x = d y = d d_{x}=d_{y}=d , the optimal orthogonal matrix 𝐐 ∗ ∈ 𝒪 ⁡ ( d ) \mathbf{Q}^{*}\in\mathcal{O}(d) is: 𝐐 ∗ = argmin 𝐐 ∈ 𝒪 ⁡ ( d ) ‖ 𝐗 − 𝐘𝐐 ‖ F 2 , \mathbf{Q}^{*}=\operatorname*{argmin}_{\mathbf{Q}\in\mathcal{O}(d)}\|\mathbf{X}-\mathbf{Y}\mathbf{Q}\|_{F}^{2}, (31) which has the closed-form solution 𝐐 ∗ = 𝐕𝐔 ⊤ \mathbf{Q}^{*}=\mathbf{V}\mathbf{U}^{\top} where 𝐔 ​ 𝚺 ​ 𝐕 ⊤ = 𝐗 ⊤ ​ 𝐘 \mathbf{U}\mathbf{\Sigma}\mathbf{V}^{\top}=\mathbf{X}^{\top}\mathbf{Y} is the SVD. The Procrustes distance is: d Proc ​ ( 𝐗 , 𝐘 ) = ‖ 𝐗 − 𝐘𝐐 ∗ ‖ F . d_{\mathrm{Proc}}(\mathbf{X},\mathbf{Y})=\|\mathbf{X}-\mathbf{Y}\mathbf{Q}^{*}\|_{F}. (32) We convert this distance to a similarity score s Proc = 1 − d Proc / ‖ 𝐘 c ‖ F s_{\mathrm{Proc}}=1-d_{\mathrm{Proc}}/\|\mathbf{Y}_{c}\|_{F} , evaluated on centered representations, where 𝐘 c \mathbf{Y}_{c} denotes the centered 𝐘 \mathbf{Y} . It equals 1 1 at perfect orthogonal alignment ( d Proc = 0 d_{\mathrm{Proc}}=0 ) and can be negative for poorly aligned pairs. What it measures: Procrustes treats two representations as similar when one can be rotated and reflected onto the other with small residual. It compares absolute geometry up to rigid motion and, unlike neighborhood metrics, is sensitive to the actual pairwise distances.

#### B.2.3 Neighborhood metrics

##### Mutual k k -Nearest Neighbors (mKNN) .

mKNN ( Huh et al., 2024 ) focuses on local topology. For each anchor sample i i , define the set of its k k nearest neighbors according to a distance measure dist ⁡ ( ⋅ , ⋅ ) \operatorname{dist}(\cdot,\cdot) in 𝐗 \mathbf{X} and 𝐘 \mathbf{Y} , N 𝐗 ​ ( i ) = KNN k ⁡ ( i ; 𝐗 ) , N 𝐘 ​ ( i ) = KNN k ⁡ ( i ; 𝐘 ) , N_{\mathbf{X}}(i)=\operatorname{KNN}_{k}(i;\mathbf{X}),\qquad N_{\mathbf{Y}}(i)=\operatorname{KNN}_{k}(i;\mathbf{Y}), (33) where KNN k ⁡ ( i , 𝐗 ) \operatorname{KNN}_{k}(i;\mathbf{X}) denotes the indices of the k k samples (excluding i i ) that minimize dist ⁡ ( 𝐱 i , 𝐱 j ) \mathrm{dist}(\mathbf{x}_{i},\mathbf{x}_{j}) . mKNN is then defined as the average fraction of shared neighbors: mKNN k ​ ( 𝐗 , 𝐘 ) = 1 n ​ ∑ i = 1 n | N 𝐗 ​ ( i ) ∩ N 𝐘 ​ ( i ) | k ∈ [ 0 , 1 ] . \mathrm{mKNN}_{k}(\mathbf{X},\mathbf{Y})\;=\;\frac{1}{n}\sum_{i=1}^{n}\frac{|N_{\mathbf{X}}(i)\cap N_{\mathbf{Y}}(i)|}{k}\;\in\;[0,1]. (34) What it measures: mKNN treats two representations as similar when each point keeps the same set of nearest neighbors in both spaces. It captures local topology, depends only on the rank order of distances, and is therefore invariant to any transformation that preserves k k -nearest-neighbor sets, ignoring exact distances and global geometry.

##### Cycle- k k NN (bidirectional k k -NN).

While mKNN measures one-directional neighborhood overlap, cycle- k k NN enforces a round-trip consistency between the two spaces ( Huh et al., 2024 ) . An anchor i i counts as consistent if at least one of its nearest neighbors j j in 𝐘 \mathbf{Y} has i i among its nearest neighbors in 𝐗 \mathbf{X} : cycle - kNN k ( 𝐗 , 𝐘 ) = 1 n ∑ i = 1 n [ i ∈ ⋃ j ∈ N 𝐘 ​ ( i ) N 𝐗 ( j ) ] ∈ [ 0 , 1 ] . \mathrm{cycle\text{-}kNN}_{k}(\mathbf{X},\mathbf{Y})\;=\;\frac{1}{n}\sum_{i=1}^{n}\mathbbm{1}\!\left[\,i\in\bigcup_{j\in N_{\mathbf{Y}}(i)}N_{\mathbf{X}}(j)\,\right]\;\in\;[0,1]. (35) This is stricter than mKNN , as it requires neighbor relations to be mutually recognized across the two spaces.

What it measures: cycle- k k NN treats points as similar only when neighborhood membership is reciprocated across spaces, a stricter local-topology criterion than mKNN .

##### CKA with Neighborhood Alignment (CKNNA).