Correcting Biased Centered Kernel Alignment Measures in Biological and Artificial Neural Networks — part 2/3

Cleaned-text code points [14662, 30101). Bibliography omitted here; source.md includes it.

In order to test the effect of bias correction on CKA used as an alignment metric to quantify similarity of neural network responses with human neural data, we used images, fMRI and MEG responses from the THINGS dataset ( Hebart et al., 2023 ) . This is a public repository of neural responses to a suite of naturalistic object images on relatively clear backgrounds, designed to represent solely the specified class label with as little confounding background information as possible. The fMRI data consist of three participants, while the MEG data consist of four participants. We used data from all three fMRI participants and selected participants one, two and four from the MEG data (randomly chosen). Participants in the fMRI experiment saw a set of 720 image classes (12 images in each class) from a larger image dataset consisting of 1,854 image classes. MEG participants viewed all images in the larger image dataset. The fMRI data were preprocessed using a variant of GLMSingle ( Prince et al., 2022 ) , resulting in voxelwise beta regressors per image and required no further preprocessing. Further details relating to the acquisition protocol are available in Hebart et al. (2023) . Details relating to preprocessing steps we applied to the raw MEG data are given in Appendix A.2 . We extracted fMRI responses across the following ROIs: V1, V2, V3, hV4, VO1, TO1, lFFA, rFFA, lPPA, rPPA, lEBA and rEBA (functional ROIs prefixed with l-/r- designate left/right brain hemispheres, while retinotopic visual regions implicitly encompass both). For the MEG data, we restrict our analysis to occipital electrodes ( N = 39 N=39 ). Our choice of sensors / ROIs is motivated by capturing neural responses along the visual processing hierarchy. Further information relating to per-participant neural data dimensions is given in Appendix A.4 .

### 3.2 Model Representations

We employed two convolutional neural network (CNN) models in order to extract layer-wise activations to a subset of the THINGS stimuli set mentioned in Section 3.1 . Specifically, we used ResNet18 ( He et al., 2015 ) and CORnet-S ( Kubilius et al., 2018 ) . ResNet models have been widely used to study alignment between ANN representations and neural data ( Bennett & Baldassano, 2023 ; Han et al., 2023 ) . We specifically selected ResNet18 due to its computational tractability, prominence in prior work and for its similarity in layer count to the CORnet model family, which was specifically designed to mimic properties of the mammalian visual processing hierarchy during Core Object Recognition and has also been widely used in brain-model alignment research ( Federer et al., 2020 ; Han et al., 2023 ) . We selected the CORnet-S variant because it had the highest score on the Brain-Score benchmark ( Schrimpf et al., 2018 ; Schrimpf et al., 2020 ) . CORnet-S is subdivided into block-like structures: V1, V2, V4 & IT (analogous to the notion of stages in ResNet models). Both models were trained using the ImageNet dataset ( Deng et al., 2009 ) . Layer activations were extracted and saved using the Python toolbox THINGSvision ( Muttenthaler & Hebart, 2021 ) for a single image from each of the 720 classes used in the fMRI experiment and a single image from each of the 1,854 classes used in the MEG experiment.

## 4 CKA Sensitivity to Random Data

As mentioned in Section 2.2 , one undesirable property is the fact standard CKA is sensitive to large discrepancies between the dimensionality of the feature spaces being compared. We use a reference matrix of random values A ∈ ℝ 1024 × 1024 A\in\mathbb{R}^{1024\times 1024} and compute the similarity with matrices B ∈ ℝ 1024 × P B\in\mathbb{R}^{1024\times P} where P P runs from [10, 250k] on a log scale (geometric progression). We calculate similarities using: (i) biased CKA (red), debiased CKA (green) and RV2 (black). As the number of features increases, we move from the high-data low-dimensionality regime (Figure 1 A) up to a commonly-occurring scenario in neural / biological data analysis where feature dimensionality vastly outnumbers the samples (Figure 1 C).

As shown in Figure 1 , the RV2 and debiased CKA metrics are not sensitive to large feature-sample ratio differences and accurately detect that there is no similarity among the matrix representations, since the dependence on the sample-feature ratio is removed. The debiasing step in CKA is more widely applicable to any choice of kernel, while RV2 is strictly limited to the linear case, so we do not perform any further comparisons using the RV2 metric.

Neural measurements can exhibit low signal-to-noise ratios resulting from, for example, loose sensors, misaligned voxel locations due to excessive head movement or just background activity not connected to any experimental stimuli, which has the potential to resemble random data. This fact suggests that biased CKA is not an ideal metric to directly compare among representational spaces if there is a large difference between their cardinalities. There are other instances in which inflated similarity results can arise when using biased CKA: (i) when fixing neural data (e.g. ROIs in fMRI or sensor groups / temporal windows in M/EEG) and comparing the alignment with each layer in an ANN and (ii) when fixing an ANN layer and comparing alignment across ROIs, temporal windows or sensor groups. In these scenarios the ratio of the dimensionality of the data is not fixed, which could cause issues when using biased CKA to quantify representational alignment. We examine this in the next section.

## 5 CKA Sensitivity to Stimuli-Driven Neural Responses

The previous section showed how biased CKA is sensitive to changes in the ratio of sample-feature dimensionality for random matrices. We now show how this issue can lead to erroneous conclusions in a standard alignment experiment between biological and artificial neural networks across two types of brain recordings, matched on the same stimuli set and two CNNs trained on the same data (see Methods for full description). We selected primary visual cortex (V1) and occipital sensors for each of the three participants in the fMRI / MEG data, respectively. ANN responses were calculated across layers of ResNet18 and CORnet-S while keeping neural data fixed (Figure 2 a-b). This was also done when fixing a CNN layer and calculating similarity scores across varying neural data sizes (fMRI ROIs in Figure 2 e and increasing temporal windows of occipital MEG responses in Figure 2 f). In each plot, we contrast biased CKA with debiased CKA across three data manipulations: (i) random data (See Appendix A.1 ), (ii) shuffling the rows of the neural data such that there is no correspondence between CNN representations and image-level neural responses and (iii) with the original data order, where sample-feature pairs are matched.

We first see that CNN representations across both ResNet18 and CORnet-S result in high alignment values to random (”neural”) matrices, where the CNN representations are unchanged. This inflated result could lead to the erroneous conclusion with noisy / near-random neural data that both CNN models extract interesting representations that align well with the associated neural responses. When a CNN layer is fixed (Figure 2 e-f) and we sample random data to match the size of ROIs or sensor groups, we show how conclusions based on biased CKA results alone could be misinterpreted as meaningful alignment when this effect is driven by discrepancies in feature dimensions.

For true neural responses (both in the original order and shuffled) we see increasing biased CKA alignment over layers of both CNNs. For fMRI, we see a slightly elevated value over the corresponding shuffled version, but for MEG data these values are equal. For debiased CKA, it correctly reports zero alignment with random matrices and shuffled data. For fMRI, we do see small positive alignment values for debiased CKA in the original data order, while this is absent from the MEG data. Biased CKA without a shuffled-data control would result in a potential conclusion that both ResNet18 and CORnet-S extract similar information, however this would not be due to stimuli-driven responses. Only in the case of debiased CKA do we see a positive alignment value in this case (for fMRI). We now examine the effects of biased and debiased CKA on shuffled and unshuffled data.

## 6 CKA Sensitivity to Shuffled Data

In Figure 2 we observed that biased CKA consistently resulted in similar alignment scores for both shuffled / unshuffled data orderings across both fMRI / MEG and two CNN models (ResNet18 / CORnet-S), while debiased CKA only reported positive alignment scores for unshuffled data, i.e. the true stimuli-driven responses in the fMRI results. In this section we examine this finding in more detail and suggest an explanation for the observed results.

The Manifold Hypothesis states that real-world data are expected to concentrate in the vicinity of a manifold of lower dimensionality, embedded in a higher-dimensional space ( Bengio et al., 2013 ) . Deep learning models aim to learn a parameterization of the manifold of the data they are trained on ( Lei et al., 2020 ) . If a set of input data is presented to experimental participants, whose brain responses are recorded, it follows that the neural data will also lie on a lower-dimensional manifold within a higher dimensional space. In the context of alignment between biological and artificial neural networks, this can be seen when permuting neural responses and measuring the alignment with layer activations from deep learning models using a selected metric of interest. When both systems have seen the same input data, it remains an open question how much of the detectable alignment is sensitive to true stimuli-driven neural responses, as well as to a more generalized response that detects the presence of structured neural data, e.g. prototypical fMRI responses during image viewing.

Figure 3 shows a comparison between biased and debiased CKA across ResNet18 (CORnet-S results reported in Appendix A.6 ) for a subset of fMRI ROIs, categorized into early (V1, V2, V3), intermediate (hV4, VO1, TO1) and late (lFFA, lPPA, lEBA) groups ( Witthoft et al., 2013 ) . For the original data order, the reported value is the mean across the three fMRI participants ( N = 3 N=3 ), while for the shuffled order, the reported value is the mean across five random shuffles per participant ( N = 3 × 5 = 15 N=3\times 5=15 ). For all ROIs and ResNet18 layers, biased CKA indicates an alignment for the original data order which is only slightly larger than for shuffled data. This indicates that biased CKA is not primarily sensitive to stimuli-driven responses. Debiased CKA does not suffer from this undesirable side-effect and returns a positive value only for stimuli-driven neural responses. It is only via debiased CKA that we recover previously-reported observations that early visual regions are more aligned with lower layers of CNNs, while later regions are more aligned with the upper layers ( Yamins et al., 2014 ; Yamins & DiCarlo, 2016 ) . Though this pattern is clear, the values we report are small in magnitude compared to results obtained with biased CKA, which we expect might be associated with the limited dataset of 720 images. We performed an additional analysis (See Appendix A.7 ) between fMRI areas V1 & left PPA (lPPA) and the first three layers of ResNet18. We gradually varied the amount of shuffling and see that only in the case of positive stimuli-driven alignment scores is there sensitivity to the amount of shuffling (in V1). For lPPA, biased and debiased CKA are insensitive to varying amounts of shuffling. This supports our interpretation that biased CKA is not sensitive to stimuli-driven activity.

An interesting observation in Figure 3 is that with respect to the original data order, biased CKA alignment exhibits an increased value over the corresponding shuffled version in line with the magnitude of the debiased CKA score. This is predominantly seen in the early ROIs (V1, V2 & V3), while when debiased CKA does not detect alignment (e.g. lPPA), biased CKA is near-identical for both the original and shuffled data orders. This indicates that when stimuli-driven alignment exists between biological and ANNs, biased CKA is able to detect it as roughly the additional alignment given over and above that which is given for shuffled data. We have posited a hypothesis to explain what biased CKA might be additionally sensitive to, in light of the similarity to shuffled sample-feature pairs. As data exhibit more of an unstructured random profile, biased CKA scores rapidly increase (See Figure 1). The similarity between shuffled and unshuffled data pairs points to a sensitivity based on the structure of the neural data itself, which might be characterized by structure that looks like fMRI data , for example. Even when using random data sampled from Normal distributions where voxel-wise / sensor-wise means and standard deviations are used to parameterize these distributions (See Appendix A.1 ) in order to closer approximate structured neural data, biased CKA results are indistinguishable from random results reported in Figure 2 . The fact that shuffled responses are treated differently lends further support to our idea that biased CKA sensitivity is driven by a generalized neural response pattern.

## 7 Conclusion

Using fMRI and MEG data paired with stimuli from the THINGS dataset, we showed that measuring alignment with biased CKA has several weaknesses that are corrected by implementing a debiasing step. Biased CKA can be increased arbitrarily by varying the feature ratio w.r.t. a fixed dataset size (e.g. examining fMRI ROIs w.r.t. to a single CNN layer). This prevents direct comparisons across varying feature dimensionalities which often arises in comparisons between neural data and ANNs (e.g. different ROI sizes / sensor groups, temporal windows, parameter counts). We postulate that biased CKA is sensitive in part to the inherent structure of neural data since biased CKA reveals similar alignment values between shuffled and unshuffled data when debiased CKA results are close to zero. We show that alignment based on the true stimuli-driven responses is only obtained when using the debiased version of CKA. Our analysis did not show debiased CKA alignment with MEG data. We chose one method of preprocessing, but other methods that boost the signal-to-noise ratio (e.g. more data, different settings) will likely increase our ability to detect stimuli-driven alignment with ANN representations. It will be informative to quantify the same analysis across other common alignment metrics, which we leave to future work. As biological and ANN alignment research is an emerging topic in representation learning, methods like CKA are being used not only to quantify alignment but also to induce brain-like representations in neural networks. We have demonstrated some issues when using CKA as an alignment metric in the hope that the community can sidestep erroneous conclusions and spot potential issues when assessing reported alignment metrics in their work and the work of others.

#### Acknowledgments

We would like to thank Jessica Thompson for a constructive discussion that has better informed our work.

## Appendix A Appendix

### A.1 Generating Random Data