Revisiting the Platonic Representation Hypothesis: An Aristotelian View — part 7/8

Cleaned-text code points [94047, 109382). Bibliography omitted here; source.md includes it.

The theoretical analysis in Section 4 assumes Gaussian entries for tractability, but real neural network activations rarely follow Gaussian distributions. Instead, they often exhibit heavy tails, sparsity, or multimodality. A critical question is whether our calibration, which makes no distributional assumptions, remains effective under such deviations.

Figure 8 shows phase diagrams under different noise distributions: Gaussian, Student- t t ( ν = 3 \nu=3 ), Laplace, and Gaussian mixtures. Each panel shows raw scores (left) and calibrated scores (right) across the ( d / n , σ ) (d/n,\sigma) grid, where σ \sigma controls the noise level added to a fixed shared signal. At low σ \sigma , the signal dominates and both raw and calibrated scores correctly indicate high similarity. At high σ \sigma , noise overwhelms the signal, and similarity should approach zero. The key finding is that raw scores remain elevated (around 0.4–0.6) even at high noise levels where no detectable signal remains, while calibrated scores correctly collapse to near-zero. This pattern holds across all noise distributions tested, confirming that permutation-based calibration adapts to the data-generating process without requiring explicit distributional modeling.

### E.2 SNR heatmaps

The experiments of the main paper ( Figure 4 ) demonstrated that calibration eliminates false positives under H 0 H_{0} while preserving sensitivity to fixed signals. This section extends the analysis by characterizing how calibrated similarity varies jointly with signal strength, noise level, and dimensionality ratio, thereby delineating the regimes in which similarity estimation remains reliable.

Figure 9 presents heatmaps of raw scores (top row) and calibrated scores (bottom row) across the (Noise level, Signal strength) grid for three signal ranks ( r ∈ { 1 , 5 , 10 } r\in\{1,5,10\} ). The results reveal a clear phase transition structure. Raw scores (top) show uniformly high values across most of the grid, obscuring the true detection boundary. Calibrated scores (bottom) reveal the underlying signal: high scores concentrate in the low-noise, high-signal corner (bottom-left), while scores correctly collapse to zero as noise increases (moving right) or signal weakens (moving down). The detection boundary shifts rightward (tolerating higher noise) as signal rank increases. This phase structure is meaningful: it delineates when similarity measurements carry information about shared structure versus when they reflect only finite-sample artifacts.

Figure 10 provides a complementary view by collapsing the 2D heatmaps into 1D curves, plotting calibrated score against noise level for different signal strengths s s . As expected, calibrated scores decrease monotonically with noise level: at low noise, scores are high (reflecting the detectable shared signal), while at high noise, scores collapse to zero (reflecting that the signal is buried). Stronger signals (larger s s ) maintain elevated scores across a wider range of noise levels before eventually succumbing. Higher-rank signals ( r = 5 , 10 r=5,10 ) show more gradual decay compared to r = 1 r=1 , consistent with their greater statistical detectability. All curves converge to zero at high noise, confirming that the null floor is correctly calibrated regardless of signal strength or rank.

### E.3 Comparing calibration approaches

A natural question is whether the choice of calibration summary affects the correction. We consider several approaches: (i) gated score , which thresholds at a significance level and rescales ( α ∈ { 0.05 , 0.1 } \alpha\in\{0.05,0.1\} ); (ii) null-centered , subtracting the null mean; (iii) z-score , standardizing by null mean and standard deviation; and (iv) ARI-style , applying the chance-correction formula ( s − 𝔼 ⁡ [ s ] ) / ( s max − 𝔼 ⁡ [ s ] ) (s-\mathbb{E}[s])/(s_{\max}-\mathbb{E}[s]) . Figure 11 evaluates these variants across metrics as d / n d/n increases.

The results demonstrate that the gated score, null-centered, and ARI-style corrections all successfully collapse to appropriate null baselines across all metrics, regardless of whether the raw metric exhibits severe inflation (CKA, approaching 0.8) or mild inflation (RSA and mKNN, below 0.1). The z-score calibration, while correcting the mean, can exhibit artifacts when the null distribution is skewed, as occurs for bounded metrics like CKA at high d / n d/n , making it less suitable as a universal correction.

### E.4 Comparison with analytical debiasing

We validate our empirical null calibration by comparing it to existing analytical bias corrections for CKA . Figure 12 shows the difference between our calibrated CKA and two existing estimators: the debiased CKA of Murphy et al. (2024) and the dep-cols CKA of Chun et al. (2025) .

Our calibrated CKA closely matches the debiased CKA estimator, indicating that our calibration automatically corrects the dominant width-induced bias without requiring a metric-specific derivation. In contrast, dep-cols CKA is designed to correct column dependence, which is not present in our experimental setup (columns are independent by construction), so it does not address the dominant width-induced inflation. Under genuine signal ( H 1 H_{1} ) it stays near its maximal value across all d / n d/n , far above the closely agreeing debiased and calibrated estimates, while under the null ( H 0 H_{0} ) it fluctuates around zero with high variance.

Our calibration targets the same source of finite-sample bias that the unbiased HSIC estimator of Song et al. (2012) subtracts analytically: the self-similarity (diagonal) terms. This is consistent with the close agreement between the two corrections in Figure 12 . Unlike such analytical estimators, our permutation calibration requires no metric-specific derivation and applies to metrics for which no debiasing exists.

### E.5 Width confounder under genuine signal on real networks

We now verify on real pretrained networks that calibration corrects the width confounder when a genuine signal is present, complementing the analytical result of Proposition C.5 . Because the bias scales as 𝒪 ⁡ ( d / n ) \mathcal{O}(d/n) ( Proposition 4.1 ), width and sample size can be separated only by varying one while holding the other fixed, which this experiment does.

We extract last-layer features from the DINOv2 and AugReg ViT families. Within a family the models share training objective, data, and architecture and differ only in width d d ( e.g. , ViT-S/B/L/g for DINOv2), so a within-family pair isolates the effect of width. All pairs use the same n = 1024 n=1024 WIT images as the PRH setting, and for each pair we compute raw and calibrated linear CKA . At fixed n = 1024 n=1024 , the permutation threshold τ \tau (the magnitude of the width correction) increases with the total representation width d X + d Y d_{X}+d_{Y} , with Pearson correlation r = 0.88 r=0.88 ( Figure 13 ). Since n n is held constant, this dependence is attributable to width alone: calibration adapts its correction to representation width rather than applying a fixed offset. Together with Proposition C.5 , this confirms that calibration corrects the width confounder on real networks with genuine similarity.

### E.6 Permutation budget analysis

Permutation-based calibration introduces a computational-statistical tradeoff: more permutations yield more stable threshold estimates but increase runtime. Practitioners need guidance on the minimum budget required for reliable inference.

We analyze the stability of threshold estimates τ α \tau_{\alpha} and calibrated scores as a function of the permutation budget K K across 50 random seeds. Figure 14 shows two panels: the left panel displays threshold estimates, while the right panel shows calibrated scores under H 0 H_{0} . Threshold estimates (left) stabilize rapidly, reaching stable values by approximately K = 50 K=50 for all metrics tested. Calibrated scores (right) exhibit more variability at very low budgets ( K < 50 K<50 ), with occasional spikes due to unstable threshold estimation, but converge to near-zero by K ≈ 100 K\approx 100 – 200 200 .

Based on these results, we recommend K ≥ 200 K\geq 200 . The computational cost scales linearly with K K , so this recommendation represents a favorable tradeoff between precision and efficiency.

### E.7 Full null drift results

The main text presents null drift results for a representative subset of metrics under Gaussian noise. Here, we present additional results across all metrics evaluated in this work, including RSA, the RV coefficient, and Procrustes distance, as well as results under heavy-tailed noise distributions.

Figure 15 presents results under Gaussian noise for all metrics. The severity and shape of the null baseline vary substantially across metric families: among the spectral metrics, CKA variants and the RV coefficient show the strongest monotonic inflation with width, whereas the CCA family follows a distinct rank-deficiency pattern (mean CCA peaks near d ≈ n d\approx n then decays, PWCCA saturates, and SVCCA stays width-insensitive); the neighborhood metrics show the mildest drift. This reflects the structural sensitivity of the metrics to high-dimensional spurious correlations. RSA is the exception: as a self-normalized correlation of within-space dissimilarities, its null stays near zero and shows no width-driven drift. Critically, calibration eliminates drift across all metrics, collapsing scores to zero regardless of the raw bias magnitude.

Figure 16 extends these results to heavy-tailed noise (Student- t t , ν = 3 \nu=3 ). The qualitative pattern is preserved: the confounded metrics exhibit positive drift under the null, and calibration eliminates this drift. The magnitude of raw bias under heavy-tailed noise is comparable to the Gaussian case (marginally lower for the spectral metrics), and calibration adapts automatically without requiring distributional knowledge.

##### Robustness to the generative process.

The experiments above use i.i.d. Gaussian and heavy-tailed noise. To confirm that the width confounder is not tied to these specific generators, we also produce representations as 𝐱 i = f x ​ ( 𝐚 i ) \mathbf{x}_{i}=f_{x}(\mathbf{a}_{i}) and 𝐲 i = f y ​ ( 𝐚 i ) \mathbf{y}_{i}=f_{y}(\mathbf{a}_{i}) , where 𝐚 i \mathbf{a}_{i} is a shared Gaussian input and f x , f y f_{x},f_{y} are independent random linear maps or MLPs, so the two representations share inputs but no systematic structure. Figure 17 shows that raw CKA still inflates with dimensionality d d in every regime, while calibrated CKA stays at zero. Sharing inputs through unrelated networks thus produces width-driven inflation rather than genuine convergence, and calibration removes it regardless of how the representations are generated.

### E.8 Extended PRH alignment results (image–text)

The main text establishes a divergence between local and global similarity metrics when applied to the Platonic Representation Hypothesis (PRH) : neighborhood-based metrics retain significant cross-modal alignment after calibration, while spectral metrics lose their apparent convergence trend. A natural question is whether this finding is robust across model families and metric variants.

Here we present comprehensive results across all five vision model families in the PRH setting (DINOv2, CLIP, ImageNet-21K, MAE, and CLIP-finetuned) and a broad range of metrics spanning the local-to-global spectrum ( Figures 18 , 19 and 20 ).

The results reinforce and extend the main text findings. Neighborhood metrics (mKNN, cycle- k k NN, CKNNA) show a consistent alignment trend across all vision families with a neighborhood size of 10. This pattern holds for both self-supervised (DINOv2, MAE) and supervised (ImageNet-21K) pretraining objectives, as well as for both CLIP-aligned and CLIP-finetuned variants. Spectral metrics (CKA linear, CKA RBF, unbiased CKA, RV coefficient, SVCCA ) show a different pattern: raw scores suggest increasing alignment with model scale, but calibrated scores show no such scaling trend.

##### Trend with model scale, before versus after calibration.

For each metric separately, we measure the Pearson correlation between language-model capability (the model ranking of Huh et al. (2024) ) and the similarity score across all model pairs, before and after calibration ( Table 2 ). Global metrics lose most of their trend with scale after calibration, while local (neighborhood) metrics retain it across neighborhood sizes.

##### Continuous language-performance axis.

The preceding figures place each language model at its performance rank . Because Huh et al. (2024) instead plot cross-modal alignment against a continuous measure of language performance, we reproduce all of the above results on that axis for direct comparison. We quantify language performance by bits-per-byte (BPB) over OpenWebText, as in Huh et al. (2024) , oriented so that larger values, max ⁡ ( BPB ) − BPB \max(\mathrm{BPB})-\mathrm{BPB} , indicate stronger next-token prediction, and place each model at its measured value with a per-encoder least-squares fit ( Figures 21 , 22 and 23 ). The local–global divergence is unchanged on this axis: after calibration, neighborhood metrics retain their alignment trend with language performance while spectral metrics do not, so the finding does not depend on whether capability is expressed as a rank or as a continuous performance value.

##### Statistical significance.

Beyond calibrated scores, we report permutation p p -values to quantify statistical evidence against the null hypothesis of no cross-modal alignment ( Figure 24 ). All 204 vision–language model pairs are significant at p < 0.05 p<0.05 , with most achieving p ≈ 0.002 p\approx 0.002 (the minimum attainable with K = 500 K=500 permutations) for both local and global metrics. This confirms that cross-modal similarity is statistically significant ( i.e. , has some alignment) across all model pairs. The critical distinction between local and global metrics lies not in statistical significance but in the magnitude and trends of calibrated scores. Local metrics show substantial alignment above the null threshold that persists across scales, whereas global metrics, although significant, show no convergence in calibrated effect sizes.

### E.9 Extended video–language alignment results

The main text extends the PRH analysis to video–language alignment following Zhu et al. (2026) . Here, we provide additional results to verify that the local-vs-global pattern observed for image–language alignment extends to the video modality.

We use 1024 samples from the PVD ( Bolya et al., 2025 ; Cho et al., 2025 ) test set. We evaluate video-native models (VideoMAE ( Tong et al., 2022 ) ) and, as a frame-level baseline, image models (DINOv2 and CLIP) applied to the middle frame of each video, comparing all against the same three language model families used in the image–language experiments (BLOOM, OpenLLaMA, LLaMA) at multiple scales. For VideoMAE we include both the fine-tuned scale series (small/base/large/huge, fine-tuned on Kinetics) used in the main text and the corresponding non-fine-tuned checkpoints. Figure 25 shows results for spectral ( CKA RBF) and neighborhood ( mKNN , CKNNA) metrics.