Revisiting the Platonic Representation Hypothesis: An Aristotelian View — part 5/8

Cleaned-text code points [62625, 78332). Bibliography omitted here; source.md includes it.

CKNNA ( Huh et al., 2024 ) combines the kernel formulation of CKA with local neighborhood structure, restricting the kernel interaction to mutual k k -nearest-neighbor edges. Let 𝐊 X = 𝐗𝐗 ⊤ \mathbf{K}_{X}=\mathbf{X}\mathbf{X}^{\top} and 𝐊 Y = 𝐘𝐘 ⊤ \mathbf{K}_{Y}=\mathbf{Y}\mathbf{Y}^{\top} be the (linear) Gram matrices, and let 𝐌 X , 𝐌 Y ∈ { 0 , 1 } n × n \mathbf{M}_{X},\mathbf{M}_{Y}\in\{0,1\}^{n\times n} be the k k -NN indicator matrices with ( 𝐌 X ) i ​ j = 𝟙 [ j ∈ N 𝐗 ( i ) ] (\mathbf{M}_{X})_{ij}=\mathbbm{1}[\,j\in N_{\mathbf{X}}(i)\,] and ( 𝐌 Y ) i ​ j = 𝟙 [ j ∈ N 𝐘 ( i ) ] (\mathbf{M}_{Y})_{ij}=\mathbbm{1}[\,j\in N_{\mathbf{Y}}(i)\,] . Let 𝐀 = 𝐌 X ⊙ 𝐌 Y \mathbf{A}=\mathbf{M}_{X}\odot\mathbf{M}_{Y} retain only edges that are k k -NN in both spaces, where ⊙ \odot is the Hadamard product. CKNNA aligns the neighbor-masked Gram matrices with CKA ’s normalization: CKNNA k ​ ( 𝐗 , 𝐘 ) = ⟨ 𝐇 ⁡ ( 𝐀 ⊙ 𝐊 X ) ​ 𝐇 , 𝐇 ⁡ ( 𝐀 ⊙ 𝐊 Y ) ​ 𝐇 ⟩ F ‖ 𝐇 ⁡ ( 𝐌 X ⊙ 𝐊 X ) ​ 𝐇 ‖ F ​ ‖ 𝐇 ⁡ ( 𝐌 Y ⊙ 𝐊 Y ) ​ 𝐇 ‖ F . \mathrm{CKNNA}_{k}(\mathbf{X},\mathbf{Y})\;=\;\frac{\langle\mathbf{H}(\mathbf{A}\odot\mathbf{K}_{X})\mathbf{H},\;\mathbf{H}(\mathbf{A}\odot\mathbf{K}_{Y})\mathbf{H}\rangle_{F}}{\|\mathbf{H}(\mathbf{M}_{X}\odot\mathbf{K}_{X})\mathbf{H}\|_{F}\,\|\mathbf{H}(\mathbf{M}_{Y}\odot\mathbf{K}_{Y})\mathbf{H}\|_{F}}. (36) What it measures: CKNNA treats representations as similar when kernel values on shared (mutual) nearest-neighbor edges agree, combining CKA ’s normalization with a purely local notion of neighborhood structure.

## Appendix C Theoretical Derivations

In this section, we provide the theoretical justification for the confounding factors identified in Section 4 .

### C.1 Permutation validity, super-uniformity, and gating

This section formalizes the finite-sample validity of permutation calibration.

###### Definition C.1 (Super-uniformity) .

A p p -value p p is super-uniform under H 0 H_{0} if for all t ∈ [ 0 , 1 ] t\in[0,1] , ℙ H 0 ​ ( p ≤ t ) ≤ t . \mathbb{P}_{H_{0}}(p\leq t)\leq t. (37) Equivalently, p p -values under H 0 H_{0} are stochastically larger than Unif ⁡ ( 0 , 1 ) \mathrm{Unif}(0,1) , which is sufficient for valid Type-I error control.

###### Lemma C.2 (Permutation p p -values are super-uniform) .

Under Assumption 3.1 , the permutation p p -value in Equation 10 satisfies super-uniformity: ℙ H 0 ​ ( p ≤ α ) ≤ α \mathbb{P}_{H_{0}}(p\leq\alpha)\leq\alpha for all α ∈ [ 0 , 1 ] \alpha\in[0,1] (finite-sample validity).

###### Proof of Lemma C.2 .

Let s obs = s ⁡ ( 𝐗 , 𝐘 ) s_{\mathrm{obs}}=s(\mathbf{X},\mathbf{Y}) be the observed statistic and let s ( k ) = s ⁡ ( 𝐗 , π k ​ ( 𝐘 ) ) s^{(k)}=s(\mathbf{X},\pi_{k}(\mathbf{Y})) for k = 1 , … , K k=1,\dots,K be the statistics computed on permuted pairings. Under Assumption 3.1 , the vector ( s obs , s ( 1 ) , … , s ( K ) ) (s_{\mathrm{obs}},s^{(1)},\dots,s^{(K)}) is exchangeable : its joint distribution is invariant to permutations of the indices. Consider the (upper) rank R = 1 + # ⁡ { k ∈ { 1 , … , K } : s ( k ) ≥ s obs } ∈ { 1 , … , K + 1 } . R\;=\;1+\#\{k\in\{1,\dots,K\}:s^{(k)}\geq s_{\mathrm{obs}}\}\ \in\ \{1,\dots,K+1\}. (38) If the scores are almost surely distinct, exchangeability implies that the rank of s obs s_{\mathrm{obs}} among { s obs , s ( 1 ) , … , s ( K ) } \{s_{\mathrm{obs}},s^{(1)},\dots,s^{(K)}\} is uniform on { 1 , … , K + 1 } \{1,\dots,K+1\} . With possible ties, the add-one p p -value of Phipson and Smyth (2010) , p = R K + 1 , p=\frac{R}{K+1}, (39) is conservative, implying ℙ H 0 ​ ( p ≤ α ) ≤ α \mathbb{P}_{H_{0}}(p\leq\alpha)\leq\alpha for all α ∈ [ 0 , 1 ] \alpha\in[0,1] . ∎

###### Proof of Corollary 5.1 .

Let s obs = s ⁡ ( 𝐗 , 𝐘 ) s_{\mathrm{obs}}=s(\mathbf{X},\mathbf{Y}) and s ( k ) = s ⁡ ( 𝐗 , π k ​ ( 𝐘 ) ) s^{(k)}=s(\mathbf{X},\pi_{k}(\mathbf{Y})) for k = 1 , … , K k=1,\dots,K . Under Assumption 3.1 , the vector ( s obs , s ( 1 ) , … , s ( K ) ) (s_{\mathrm{obs}},s^{(1)},\dots,s^{(K)}) is exchangeable. Let τ α := s ( ⌈ ( 1 − α ) ​ ( K + 1 ) ⌉ ) \tau_{\alpha}:=s_{(\lceil(1-\alpha)(K+1)\rceil)} be the ( 1 − α ) (1-\alpha) -quantile defined via the order statistic of the combined multiset { s obs , s ( 1 ) , … , s ( K ) } \{s_{\mathrm{obs}},s^{(1)},\dots,s^{(K)}\} . Define the (upper) rank R = 1 + # ⁡ { k ∈ { 1 , … , K } : s ( k ) ≥ s obs } ∈ { 1 , … , K + 1 } , R\;=\;1+\#\{k\in\{1,\dots,K\}:s^{(k)}\geq s_{\mathrm{obs}}\}\ \in\ \{1,\dots,K+1\}, and the corresponding add-one p p -value p = R / ( K + 1 ) p=R/(K+1) . By construction of τ α \tau_{\alpha} , the rejection event { s obs > τ α } \{s_{\mathrm{obs}}>\tau_{\alpha}\} implies that s obs s_{\mathrm{obs}} lies among the largest ⌊ α ⁡ ( K + 1 ) ⌋ \lfloor\alpha(K+1)\rfloor values of { s obs , s ( 1 ) , … , s ( K ) } \{s_{\mathrm{obs}},s^{(1)},\dots,s^{(K)}\} , hence R ≤ α ⁡ ( K + 1 ) R\leq\alpha(K+1) and therefore p ≤ α p\leq\alpha . By Lemma C.2 , ℙ H 0 ​ ( p ≤ α ) ≤ α \mathbb{P}_{H_{0}}(p\leq\alpha)\leq\alpha , which yields ℙ H 0 ​ ( s obs > τ α ) ≤ ℙ H 0 ​ ( p ≤ α ) ≤ α . \mathbb{P}_{H_{0}}(s_{\mathrm{obs}}>\tau_{\alpha})\leq\mathbb{P}_{H_{0}}(p\leq\alpha)\leq\alpha. ∎

##### Restricted permutations under dependence.

Assumption 3.1 treats the n n row pairs as exchangeable units. In practice, exchangeability can be violated even without sequential structure ( e.g. , grouped or clustered samples). Validity is then recovered by using restricted permutations that preserve the dependence structure ( e.g. , permuting within blocks or permuting block labels) and re-running under each restricted permutation.

### C.2 Monotone invariance of rank-based calibration

The following proposition is a standard result in randomization inference; we state it here for completeness and to clarify its role in justifying the calibrated score design.

###### Proposition C.3 (Monotone invariance of rank-based calibration ( Lehmann and Romano, 2005 ) ) .

Let g : ℝ → ℝ g:\mathbb{R}\to\mathbb{R} be strictly increasing. Define p g p_{g} by applying Equation 10 to the transformed statistic g ∘ s g\circ s using the same permutations. Then p g = p p_{g}=p , and likewise the null percentile (the rank of s obs s_{\mathrm{obs}} among the combined set) is invariant under g g .

###### Proof.

Let g g be strictly increasing. For any two real numbers a , b a,b , we have a ≥ b a\geq b if and only if g ⁡ ( a ) ≥ g ⁡ ( b ) g(a)\geq g(b) . Therefore, for each permutation draw k k , 𝟙 { s ( k ) ≥ s obs } = 𝟙 { g ( s ( k ) ) ≥ g ( s obs ) } . \mathbbm{1}\{s^{(k)}\geq s_{\mathrm{obs}}\}=\mathbbm{1}\{g(s^{(k)})\geq g(s_{\mathrm{obs}})\}. (40) Summing over k k shows that the permutation rank R R (and thus the add-one p p -value) is unchanged by applying g g to both the observed and permuted statistics. The same argument applies to the null percentile, since the ordering of samples is preserved under g g . ∎

### C.3 Post-selection inflation and aggregation-aware validity

###### Proposition C.4 (Validity for aggregation-aware calibration) .

Let T T be any measurable aggregation operator applied to a layer-wise similarity matrix 𝐒 \mathbf{S} (e.g., max, row-max, top- k k ). If T obs = T ⁡ ( 𝐒 ) T_{\mathrm{obs}}=T(\mathbf{S}) is calibrated against the permutation null { T ⁡ ( 𝐒 ( k ) ) } k = 1 K \{T(\mathbf{S}^{(k)})\}_{k=1}^{K} as in Equation 16 , then the resulting p agg p_{\mathrm{agg}} is super-uniform under H 0 H_{0} .

###### Proof of Proposition C.4 .

Let T T be any measurable functional of the full data (representations across all layers), producing the scalar report T obs T_{\mathrm{obs}} . Under Assumption 3.1 and consistent layer-wise permutation of sample correspondences, the vector ( T obs , T ( 1 ) , … , T ( K ) ) (T_{\mathrm{obs}},T^{(1)},\dots,T^{(K)}) is exchangeable. Applying the same rank argument as in Section C.1 yields super-uniformity for the add-one p p -value in Equation 16 . ∎

### C.4 The width confounder

This appendix provides concrete calculations that justify the width confounder using Random Matrix Theory (RMT) : even under independence, interaction operators have non-trivial magnitude and spectrum when d d is not negligible relative to n n .

###### Proof of Proposition 4.1 .

Let 𝐗 ∈ ℝ n × d x \mathbf{X}\in\mathbb{R}^{n\times d_{x}} and 𝐘 ∈ ℝ n × d y \mathbf{Y}\in\mathbb{R}^{n\times d_{y}} have i.i.d. rows with mean 0 0 , identity covariance, and 𝐱 i \mathbf{x}_{i} and 𝐲 i \mathbf{y}_{i} independent. Let 𝐇 = 𝐈 n − 1 n ​ 𝟙 n ​ 𝟙 n ⊤ \mathbf{H}=\mathbf{I}_{n}-\frac{1}{n}\mathbbm{1}_{n}\mathbbm{1}_{n}^{\top} be the centering matrix, so 𝐗 c = 𝐇𝐗 \mathbf{X}_{c}=\mathbf{H}\mathbf{X} and 𝐘 c = 𝐇𝐘 \mathbf{Y}_{c}=\mathbf{H}\mathbf{Y} . Since 𝐇 \mathbf{H} is symmetric and idempotent ( 𝐇 2 = 𝐇 \mathbf{H}^{2}=\mathbf{H} ), the sample cross-covariance is 𝐂 ~ = 1 n − 1 ​ 𝐗 c ⊤ ​ 𝐘 c = 1 n − 1 ​ 𝐗 ⊤ ​ 𝐇𝐘 . \widetilde{\mathbf{C}}=\frac{1}{n-1}\mathbf{X}_{c}^{\top}\mathbf{Y}_{c}=\frac{1}{n-1}\mathbf{X}^{\top}\mathbf{H}\mathbf{Y}. (41) Denote entry ( a , b ) (a,b) as C ~ a ​ b \widetilde{C}_{ab} . Expanding via H i ​ j = δ i ​ j − 1 n H_{ij}=\delta_{ij}-\frac{1}{n} : C ~ a ​ b = 1 n − 1 ​ ( ∑ i = 1 n X i ​ a ​ Y i ​ b − 1 n ​ ( ∑ i = 1 n X i ​ a ) ​ ( ∑ j = 1 n Y j ​ b ) ) . \widetilde{C}_{ab}=\frac{1}{n-1}\left(\sum_{i=1}^{n}X_{ia}Y_{ib}-\frac{1}{n}\Bigl(\sum_{i=1}^{n}X_{ia}\Bigr)\Bigl(\sum_{j=1}^{n}Y_{jb}\Bigr)\right). (42) We compute 𝔼 ⁡ [ C ~ a ​ b 2 ] \mathbb{E}[\widetilde{C}_{ab}^{2}] using independence of 𝐱 i \mathbf{x}_{i} and 𝐲 j \mathbf{y}_{j} for all i , j i,j , zero means, and identity covariance.

Term 1: 𝔼 ⁡ [ ( ∑ i X i ​ a ​ Y i ​ b ) 2 ] = ∑ i , j 𝔼 ⁡ [ X i ​ a ​ X j ​ a ] ​ 𝔼 ​ [ Y i ​ b ​ Y j ​ b ] \mathbb{E}\bigl[(\sum_{i}X_{ia}Y_{ib})^{2}\bigr]=\sum_{i,j}\mathbb{E}[X_{ia}X_{ja}]\mathbb{E}[Y_{ib}Y_{jb}] . For i ≠ j i\!\neq\!j , independence across rows and zero mean give 𝔼 ⁡ [ X i ​ a ​ X j ​ a ] = 𝔼 ⁡ [ X i ​ a ] ​ 𝔼 ​ [ X j ​ a ] = 0 \mathbb{E}[X_{ia}X_{ja}]=\mathbb{E}[X_{ia}]\mathbb{E}[X_{ja}]=0 . For i = j i=j , we have 𝔼 ⁡ [ X i ​ a 2 ] ​ 𝔼 ​ [ Y i ​ b 2 ] = 1 \mathbb{E}[X_{ia}^{2}]\mathbb{E}[Y_{ib}^{2}]=1 . Thus 𝔼 ⁡ [ ( ∑ i X i ​ a ​ Y i ​ b ) 2 ] = n \mathbb{E}\bigl[(\sum_{i}X_{ia}Y_{ib})^{2}\bigr]=n .

𝔼 ⁡ [ ( ∑ i X i ​ a ​ Y i ​ b ) ​ ( ∑ j X j ​ a ) ​ ( ∑ k Y k ​ b ) ] = ∑ i , j , k 𝔼 ⁡ [ X i ​ a ​ X j ​ a ] ​ 𝔼 ​ [ Y i ​ b ​ Y k ​ b ] \mathbb{E}\bigl[(\sum_{i}X_{ia}Y_{ib})(\sum_{j}X_{ja})(\sum_{k}Y_{kb})\bigr]=\sum_{i,j,k}\mathbb{E}[X_{ia}X_{ja}]\mathbb{E}[Y_{ib}Y_{kb}] . This is nonzero only when i = j i=j and i = k i=k , yielding ∑ i 1 ⋅ 1 = n \sum_{i}1\cdot 1=n .

𝔼 ⁡ [ ( ∑ i X i ​ a ) 2 ​ ( ∑ j Y j ​ b ) 2 ] = 𝔼 ⁡ [ ( ∑ i X i ​ a ) 2 ] ​ 𝔼 ​ [ ( ∑ j Y j ​ b ) 2 ] = n ⋅ n = n 2 \mathbb{E}\bigl[(\sum_{i}X_{ia})^{2}(\sum_{j}Y_{jb})^{2}\bigr]=\mathbb{E}[(\sum_{i}X_{ia})^{2}]\mathbb{E}[(\sum_{j}Y_{jb})^{2}]=n\cdot n=n^{2} .

Combining: 𝔼 ⁡ [ C ~ a ​ b 2 ] \displaystyle\mathbb{E}\left[\widetilde{C}_{ab}^{2}\right] = 1 ( n − 1 ) 2 ​ ( n − 2 n ⋅ n + n 2 n 2 ) = 1 ( n − 1 ) 2 ​ ( n − 2 + 1 ) = 1 n − 1 . \displaystyle=\frac{1}{(n-1)^{2}}\left(n-\frac{2}{n}\cdot n+\frac{n^{2}}{n^{2}}\right)=\frac{1}{(n-1)^{2}}(n-2+1)=\frac{1}{n-1}. (43) Summing over all entries: 𝔼 ⁡ [ ‖ 𝐂 ~ ‖ F 2 ] = ∑ a = 1 d x ∑ b = 1 d y 𝔼 ⁡ [ C ~ a ​ b 2 ] = d x ​ d y n − 1 . \mathbb{E}\left[\|\widetilde{\mathbf{C}}\|_{F}^{2}\right]=\sum_{a=1}^{d_{x}}\sum_{b=1}^{d_{y}}\mathbb{E}[\widetilde{C}_{ab}^{2}]=\frac{d_{x}d_{y}}{n-1}. (44) ∎

##### Interpretation.

The null interaction energy is 𝒪 ⁡ ( d x ​ d y / n ) \mathcal{O}(d_{x}d_{y}/n) . In the common regime d x , d y ≍ n d_{x},d_{y}\asymp n , the null energy is 𝒪 ⁡ ( n ) \mathcal{O}(n) and therefore does not vanish . Since many spectral similarity metrics aggregate singular values ( e.g. , via ‖ 𝐂 ~ ‖ F 2 = ∑ i σ i 2 ​ ( 𝐂 ~ ) \|\widetilde{\mathbf{C}}\|_{F}^{2}=\sum_{i}\sigma_{i}^{2}(\widetilde{\mathbf{C}}) ), this already explains a positive baseline under H 0 H_{0} and its dependence on ( n , d x , d y ) (n,d_{x},d_{y}) .

##### Width inflation persists under genuine signal ( H 1 H_{1} ).

The width confounder is not specific to the null hypothesis. Even when two representations share a fixed, genuine signal, finite-sample CKA is inflated by width, because increasing d d grows the diagonal and off-diagonal Gram entries at different rates and the CKA denominator does not cancel the resulting width-dependent self-similarity.

###### Proposition C.5 (Width inflation under genuine signal) .

Fix n ≥ 2 n\geq 2 and a target alignment ρ ∈ ( 0 , 1 ) \rho\in(0,1) . For each width d d , draw 𝐮 i , 𝐯 i , 𝐰 i ∼ i . i . d . 𝒩 ⁡ ( 𝟎 , 𝐈 d ) \mathbf{u}_{i},\mathbf{v}_{i},\mathbf{w}_{i}\stackrel{{\scriptstyle\mathrm{i.i.d.}}}{{\sim}}\mathcal{N}(\mathbf{0},\mathbf{I}_{d}) independently across i = 1 , … , n i=1,\dots,n , and set 𝐱 i = ρ ​ 𝐮 i + 1 − ρ ​ 𝐯 i , 𝐲 i = ρ ​ 𝐮 i + 1 − ρ ​ 𝐰 i , \mathbf{x}_{i}=\sqrt{\rho}\,\mathbf{u}_{i}+\sqrt{1-\rho}\,\mathbf{v}_{i},\qquad\mathbf{y}_{i}=\sqrt{\rho}\,\mathbf{u}_{i}+\sqrt{1-\rho}\,\mathbf{w}_{i}, (45) so that 𝐱 i \mathbf{x}_{i} and 𝐲 i \mathbf{y}_{i} share the latent signal 𝐮 i \mathbf{u}_{i} . Then the population linear CKA equals ρ 2 < 1 \rho^{2}<1 , whereas for fixed n n the sample centered linear CKA satisfies CKA ⁡ ( 𝐗 , 𝐘 ) → 1 \mathrm{CKA}(\mathbf{X},\mathbf{Y})\to 1 almost surely as d → ∞ d\to\infty . Hence the finite-sample estimate is inflated by width, with CKA ⁡ ( 𝐗 , 𝐘 ) − ρ 2 → 1 − ρ 2 > 0 \mathrm{CKA}(\mathbf{X},\mathbf{Y})-\rho^{2}\to 1-\rho^{2}>0 .

###### Proof.

By independence of 𝐮 i , 𝐯 i , 𝐰 i \mathbf{u}_{i},\mathbf{v}_{i},\mathbf{w}_{i} , the marginal covariances are Σ x = Σ y = 𝐈 d \Sigma_{x}=\Sigma_{y}=\mathbf{I}_{d} and the cross-covariance is Σ x ​ y = Cov ⁡ ( 𝐱 i , 𝐲 i ) = ρ ​ 𝐈 d \Sigma_{xy}=\mathrm{Cov}(\mathbf{x}_{i},\mathbf{y}_{i})=\rho\,\mathbf{I}_{d} . The population linear CKA is therefore CKA pop = ‖ Σ x ​ y ‖ F 2 ‖ Σ x ‖ F ​ ‖ Σ y ‖ F = ρ 2 ​ d d ​ d = ρ 2 . \mathrm{CKA}_{\mathrm{pop}}=\frac{\|\Sigma_{xy}\|_{F}^{2}}{\|\Sigma_{x}\|_{F}\,\|\Sigma_{y}\|_{F}}=\frac{\rho^{2}d}{\sqrt{d}\,\sqrt{d}}=\rho^{2}. (46) For the sample statistic at fixed n n , write the columns of 𝐗 \mathbf{X} as 𝐠 1 , … , 𝐠 d ∈ ℝ n \mathbf{g}_{1},\dots,\mathbf{g}_{d}\in\mathbb{R}^{n} . Each entry of 𝐗 \mathbf{X} is 𝒩 ⁡ ( 0 , 1 ) \mathcal{N}(0,1) (since ρ + ( 1 − ρ ) = 1 \rho+(1-\rho)=1 ), and the columns are i.i.d. across j j with 𝐠 j ∼ 𝒩 ⁡ ( 𝟎 , 𝐈 n ) \mathbf{g}_{j}\sim\mathcal{N}(\mathbf{0},\mathbf{I}_{n}) . By the strong law of large numbers, 1 d ​ 𝐗𝐗 ⊤ = 1 d ​ ∑ j = 1 d 𝐠 j ​ 𝐠 j ⊤ → a . s . 𝔼 ⁡ [ 𝐠 1 ​ 𝐠 1 ⊤ ] = 𝐈 n . \frac{1}{d}\mathbf{X}\mathbf{X}^{\top}=\frac{1}{d}\sum_{j=1}^{d}\mathbf{g}_{j}\mathbf{g}_{j}^{\top}\xrightarrow{\mathrm{a.s.}}\mathbb{E}[\mathbf{g}_{1}\mathbf{g}_{1}^{\top}]=\mathbf{I}_{n}. (47) Since n n is fixed, this convergence also holds in Frobenius norm, so 1 d ​ 𝐇𝐗𝐗 ⊤ ​ 𝐇 → 𝐇 \tfrac{1}{d}\mathbf{H}\mathbf{X}\mathbf{X}^{\top}\mathbf{H}\to\mathbf{H} a.s.; the identical argument gives 1 d ​ 𝐇𝐘𝐘 ⊤ ​ 𝐇 → 𝐇 \tfrac{1}{d}\mathbf{H}\mathbf{Y}\mathbf{Y}^{\top}\mathbf{H}\to\mathbf{H} . By scale invariance of CKA , CKA ⁡ ( 𝐗 , 𝐘 ) = ⟨ 1 d ​ 𝐇𝐗𝐗 ⊤ ​ 𝐇 , 1 d ​ 𝐇𝐘𝐘 ⊤ ​ 𝐇 ⟩ F ‖ 1 d ​ 𝐇𝐗𝐗 ⊤ ​ 𝐇 ‖ F ​ ‖ 1 d ​ 𝐇𝐘𝐘 ⊤ ​ 𝐇 ‖ F → a . s . ⟨ 𝐇 , 𝐇 ⟩ F ‖ 𝐇 ‖ F 2 = 1 . \mathrm{CKA}(\mathbf{X},\mathbf{Y})=\frac{\langle\tfrac{1}{d}\mathbf{H}\mathbf{X}\mathbf{X}^{\top}\mathbf{H},\,\tfrac{1}{d}\mathbf{H}\mathbf{Y}\mathbf{Y}^{\top}\mathbf{H}\rangle_{F}}{\|\tfrac{1}{d}\mathbf{H}\mathbf{X}\mathbf{X}^{\top}\mathbf{H}\|_{F}\,\|\tfrac{1}{d}\mathbf{H}\mathbf{Y}\mathbf{Y}^{\top}\mathbf{H}\|_{F}}\xrightarrow{\mathrm{a.s.}}\frac{\langle\mathbf{H},\mathbf{H}\rangle_{F}}{\|\mathbf{H}\|_{F}^{2}}=1. (48) Subtracting the population value gives CKA ⁡ ( 𝐗 , 𝐘 ) − ρ 2 → 1 − ρ 2 > 0 \mathrm{CKA}(\mathbf{X},\mathbf{Y})-\rho^{2}\to 1-\rho^{2}>0 . ∎