Comparing representational geometries using whitened unbiased-distance-matrix similarity — part 1/6

Cleaned-text code points [0, 15310). Bibliography omitted here; source.md includes it.

\papertype Original Article \paperfield Journal Section \corraddress Jörn Diedrichsen, Brain and Mind Institute, Western University, London, Ontario, N6A3K7 \corremail jdiedric@uwo.ca \fundinginfo National Science and Engineering Research Council: RGPIN-2016-04890; the Canada First Research Excellence Fund (BrainsCAN); German Research Foundation: DFG SCHU 3351/1-1

# Comparing representational geometries using whitened unbiased-distance-matrix similarity

###### Abstract

Representational similarity analysis (RSA) tests models of brain computation by investigating how neural activity patterns reflect experimental conditions. Instead of predicting activity patterns directly, the models predict the geometry of the representation, as defined by the representational dissimilarity matrix (RDM), which captures how similar or dissimilar different activity patterns associated with different experimental conditions are. RSA therefore first quantifies the representational geometry by calculating a dissimilarity measure for each pair of conditions, and then compares the estimated representational dissimilarities to those predicted by each model. Here we address two central challenges of RSA: First, dissimilarity measures such as the Euclidean, Mahalanobis, and correlation distance, are biased by measurement noise, which can lead to incorrect inferences. Unbiased dissimilarity estimates can be obtained by crossvalidation, at the price of increased variance. Second, the pairwise dissimilarity estimates are not statistically independent, and ignoring this dependency makes model comparison statistically suboptimal. We present an analytical expression for the mean and (co)variance of both biased and unbiased estimators of the squared Euclidean and Mahalanobis distance, allowing us to quantify the bias-variance trade-off. We also use the analytical expression of the covariance of the dissimilarity estimates to whiten the RDM estimation errors. This results in a new criterion for RDM similarity, the whitened unbiased RDM cosine similarity (WUC), which allows for near-optimal model selection combined with robustness to correlated measurement noise.

###### keywords

## 1 Introduction

Systems neuroscience investigates how patterns of brain activity implement the computational processes that support behavior. The computations can be understood as transformations of representations that reflect task-relevant information about the external world, the state of the animal’s body, its needs, goals, plans, or actions. Models of brain computation seek to explain how the brain processes information [ 1 ] . One way to test such models formally is to compare the representations in the models to the activity patterns measured in the brain. An essential challenge for computational neuroscience, therefore, is to develop methods for comparing representations between brains and models.

We focus here on the approach to characterize brain representations at the level of the neural population [ 2 , 3 ] , which abstracts from the roles of individual neurons, and makes it easier to compare representations between brains and models [ 4 ] . Brain representations are characterized by measuring patterns of activity across a brain region. Each activity pattern is associated with an experimental condition, for example the presentation of a particular sensory stimulus, and defines a point in the multivariate response space [ 5 ] . The distances among these points define the geometry of the representation. If the noise is isotropic and homoscedastic, each distance determines how well one could discriminate between two patterns. The distance matrix therefore determines the encoded information. The representational geometry additionally captures aspects of the format of the code, revealing, for example, what subset of the encoded information is amenable to linear readout [ 1 ] .

The analysis of representational geometries has come to be called representational similarity analysis (RSA, [ 4 ] ), and has been applied to data from invasive electrophysiology, fMRI, electroencephalography (EEG), or other methods. RSA proceeds in three steps: In the first step, the estimated activity patterns are used to compute a condition-by-condition representational dissimilarity matrix (RDM, see Figure 1). An important decision here is the choice of dissimilarity measure. Choices include the accuracy of pairwise decoders, correlation distance, or Euclidean and Mahalanobis distances [ 6 , 7 ] . In a second step, the data and models are compared by relating the vector of upper-triangular elements of the data RDM (Figure 1) and the corresponding vectors for the model RDMs. Because the dissimilarity estimates typically lack units, models cannot predict the values of the dissimilarities directly. Instead models predict the ratios or ranks of the dissimilarities. Therefore, the off-diagonal elements are compared using cosine similarity, the Pearson correlation, or the Spearman or Kendall τ a \tau_{a} rank correlation [ 8 ] . In the third and final step, models are inferentially compared using frequentist parametric [ 9 ] or non-parametric tests [ 8 ] .

In this paper, we address two closely related problems for inference using RSA. The first problem is that distances (including correlation, Euclidean, and Mahalanobis distances) are positively biased when directly estimated from noisy data [ 6 ] . This can be easily seen when considering the following case: if the true activity patterns are identical, and so the true distance is zero, the measured activity patterns will still differ by virtue of the measurement noise, and the estimated distance will be larger than zero. If different conditions are measured with different noise levels, or if measurement noise is correlated across conditions, different distances will be biased to different extents. This can distort the representational geometry and potentially lead to systematic errors in the inference [ 10 ] . To avoid the noise-induced bias in distance estimates, we have previously proposed crossvalidated dissimilarity estimators [ 8 , 6 ] , which provide distance estimates with an interpretable 0 point. The removal of bias by crossvalidation comes at the cost of slightly increased variance. In this paper, we derive analytical expressions for the bias and variance of both biased (non-crossvalidated) and unbiased (crossvalidated) distance estimates. This allows us to gain analytic insights into when the use of unbiased distance estimates is advantageous.

The second problem is that the elements of an estimated RDM have a complex covariance structure across different samples. This covariance, if not accounted for, can make model selection sub-optimal. The comparison between RDMs can be visualized in a space in which each unique dissimilarity of the RDM defines a dimension (Fig. 2a). When comparing RDMs with the cosine similarity, the model that has the smaller angle with the data RDM will be considered the better fit. While this approach is not systematically wrong, model comparison will not be optimal. In the second part of the paper, we therefore propose a simple method to address this issue: Using the analytical expression for the covariance of the different dissimilarity estimates, we can effectively calculate a cosine similarity in a “whitened" space, in which the measurement error is isotropic (Fig. 2b). We show that this whitened RDM cosine similarity based on biased distance estimates is equivalent to the RV coefficient [ 11 ] , which is also known as the linear Centered Kernel Alignment [ 12 ] . By combining the whitened RDM cosine similarity with unbiased distance estimates, we define a novel criterion, the whitened unbiased RDM cosine similarity (WUC). The WUC substantially improves the power of inferential model comparisons, performing close to the theoretical maximum of the likelihood-ratio test [ 13 , 14 , 15 ] for normally distributed data. At the same time, the WUC is robust to violations of noise assumptions, making it the method of choice for RSA model comparison in many applications. The techniques described in this paper are all implemented in a new Python-toolbox implemented by our group [ 16 ] .

## 2 Results

### 2.1 Basic Definitions

Let matrix 𝐁 \mathbf{B} be the matrix of true activation values for K K experimental conditions measured over P P measurement channels. Each row of 𝐁 \mathbf{B} contains an activity pattern across channels, elicited by a single experimental condition. Each column of 𝐁 \mathbf{B} contains an activity profile across conditions, for a single channel. As common for experimental settings, we assume that the each condition is measured repeatedly, specifically that the data can be subdivided into M M partition, indexed by m m , such that we have a full estimate of the activity patterns 𝐁 ^ m \hat{\mathbf{B}}_{m} from each data partition.

For example, in an fMRI experiment, the raw data are time series of blood-oxygenation-level-dependent (BOLD) signal measurements for every voxel (our measurement channels). These data is separated into M M different scanner runs. From these runs we can estimate, using a linear model, the activity patterns across all conditions for each run. Thus, the "measured" activity patterns would be the regression coefficients from this time series model. In other cases, such as for MEG or electrophysiological data, the activity patterns are estimated by simple averaging activity over a specific time window of recording.

The measured activity patterns would differ slightly from run-to-run, a variation that we ascribe here to measurement noise. We consider the relatively general situation where the true signal 𝐁 \mathbf{B} can have arbitrary structure and the measurement errors can come from any distribution. Across the different partitions, the measured activity patterns are assumed to have covariance 𝚺 K \bm{\Sigma}_{K} between conditions, and a covariance 𝚺 P \bm{\Sigma}_{P} between measurement channels (Figure 1). Note that for the derivation of the expected variance of distance estimates, we need to make the assumption of a matrix normal distribution of the measurement noise.

The first step in RSA is to compute the dissimilarities between activity patterns. Let 𝐛 i \mathbf{b}_{i} be the i th i^{\rm th} row of 𝐁 \mathbf{B} , that is, the true activity pattern for the i th i^{\rm th} condition across voxels. We define the k th k^{\rm th} dissimilarity to be between conditions i i and j j . In this Results section, we consider the squared Euclidean distance, but we show in the Methods section how the results generalize to the squared Mahalanobis distance. Both dissimilarities are based on the difference between activity patterns 𝜹 k = 𝐛 i − 𝐛 j \bm{\delta}_{k}=\mathbf{b}_{i}-\mathbf{b}_{j} . Specifically the squared Euclidean distance is

d k = ( 𝐛 i − 𝐛 j ) ​ ( 𝐛 i − 𝐛 j ) T / P = 𝜹 k ​ 𝜹 k T / P . d_{k}=(\mathbf{b}_{i}-\mathbf{b}_{j})(\mathbf{b}_{i}-\mathbf{b}_{j})^{T}/P=\bm{\delta}_{k}\bm{\delta}_{k}^{T}/P. (1)

Note that we are normalizing all dissimilarities by the number of channels ( P P ) to make the measures comparable across regions of different sizes. In an experiment with K K conditions, we have a total of K ⁡ ( K − 1 ) / 2 ≡ D K(K-1)/2\equiv D unique pairwise distances.

### 2.2 Biased and unbiased estimates for the squared Euclidean distance

The simplest estimator of the squared Euclidean distance can be obtained by first averaging the estimated pattern-differences across partitions

𝜹 ¯ k = 1 M ​ ∑ m M 𝜹 ^ k , m , \bar{\bm{\delta}}_{k}=\frac{1}{M}\sum_{m}^{M}\hat{\bm{\delta}}_{k,m}, (2) and then taking the inner product of these estimated pattern-difference vectors. When plugging Eq. 2 into Eq. 1 , we can see that the distance estimate relies on all the pairwise products of pattern differences:

d k ^ = 𝜹 ¯ k ​ 𝜹 ¯ k T P = 1 M 2 ​ ∑ m M ∑ n M 𝜹 ^ k , m ​ 𝜹 ^ k , n T / P \hat{d_{k}}=\frac{\bar{\bm{\delta}}_{k}\bar{\bm{\delta}}_{k}^{T}}{P}=\frac{1}{M^{2}}\sum_{m}^{M}{\sum_{n}^{M}{\hat{\bm{\delta}}_{k,m}\hat{\bm{\delta}}_{k,n}^{T}/P}} (3)

This estimate is positively biased by measurement noise:

E ⁡ ( d ^ k ) = 𝜹 k ​ 𝜹 k T / P + 𝚵 k ​ k / M . \begin{split}\mathrm{E}(\hat{d}_{k})&=\bm{\delta}_{k}\bm{\delta}_{k}^{T}/P+\bm{\Xi}_{kk}/M.\end{split} (4)

As shown in the methods, the expected value ( E ⁡ ( ) \mathrm{E}() ) of the estimator is the true distance plus the measurement variance of the pattern difference, var ⁡ ( 𝜹 ^ k ) = 𝚵 k ​ k \mathrm{var}(\hat{\bm{\delta}}_{k})=\bm{\Xi}_{kk} . This positive arises because we are multiplying a noisy pattern estimate with itself.

If the measurement variance across all pattern differences is the same, the bias is a constant value across all dissimilarities, and can be accounted for by using Pearson or rank correlations to compare RDMs (see below). However, if the variance differs, the bias will systematically differ across dissimilarities, and possibly distort the representational geometry in favour of the wrong model [ 10 ] .

To avoid this bias, we can estimate squared distances by only multiplying pattern estimates from different, and hence independent, partitions [ 8 , 6 ] with each other. Thus, we drop from Eq. 3 all pairs where m = n m=n .

d k ~ = 1 M ⁡ ( M − 1 ) ​ ∑ m M ∑ n ≠ m M 𝜹 ^ k , m ​ 𝜹 ^ k , n T / P \tilde{d_{k}}=\frac{1}{M(M-1)}\sum_{m}^{M}\sum_{n\neq m}^{M}\hat{\bm{\delta}}_{k,m}\hat{\bm{\delta}}_{k,n}^{T}/P (5)

In contrast to the biased estimate, d k ^ \hat{d_{k}} , we denote the unbiased estimate as d k ~ \tilde{d_{k}} . The bias is removed, as only independent partitions enter the product (for details, see Methods). Avoiding products where noise is multiplied with itself ensures that the expected value of the estimator is the distance we want to estimate:

E ⁡ ( d ~ k ) = 𝜹 k ​ 𝜹 k T / P \mathrm{E}(\tilde{d}_{k})=\bm{\delta}_{k}\bm{\delta}_{k}^{T}/P (6) In other words, the crossvalidated distance estimator is unbiased.

### 2.3 Variance of distance estimates

The removal of the bias, however, does not come for free: As can be seen when comparing Eq. 3 and Eq. 5 , the unbiased estimate uses fewer pairs of activity patterns to estimate the true distance. We therefore expect this estimate to have a higher variance than the unbiased estimate. Indeed, as shown in the methods, we can derive a analytical expression for the variance of the biased distance if we assume normality of the measurement noise:

var ⁡ ( d ^ k ) \displaystyle\mathrm{var}(\hat{d}_{k}) = \displaystyle= 1 P 2 ​ ( 2 ​ t ​ r ​ ( 𝚺 P ​ 𝚺 P ) M 2 ​ 𝚵 k ​ k 2 + 4 ​ P M ​ 𝜹 k ​ 𝚺 P ​ 𝜹 k T ​ 𝚵 k ​ k ) . \displaystyle\frac{1}{P^{2}}\left(\frac{2tr(\bm{\Sigma}_{P}\bm{\Sigma}_{P})}{M^{2}}\bm{\Xi}_{kk}^{2}+\frac{4P}{M}\bm{\delta}_{k}\bm{\Sigma}_{P}\bm{\delta}_{k}^{T}\bm{\Xi}_{kk}\right). (7)

A very similar expression is obtained for the unbiased estimate of the distance:

var ⁡ ( d ~ k ) \displaystyle\mathrm{var}(\tilde{d}_{k}) = \displaystyle= 1 P 2 ​ ( 2 ​ t ​ r ​ ( 𝚺 P ​ 𝚺 P ) M ⁡ ( M − 1 ) ​ 𝚵 k ​ k 2 + 4 ​ P M ​ 𝜹 k ​ 𝚺 P ​ 𝜹 k T ​ 𝚵 k ​ k ) . \displaystyle\frac{1}{P^{2}}\left(\frac{2tr(\bm{\Sigma}_{P}\bm{\Sigma}_{P})}{M(M-1)}\bm{\Xi}_{kk}^{2}+\frac{4P}{M}\bm{\delta}_{k}\bm{\Sigma}_{P}\bm{\delta}_{k}^{T}\bm{\Xi}_{kk}\right). (8)