Similarity of Neural Network Models: A Survey of Functional and
  Representational Measures
Measuring similarity of neural networks to understand and improve their
behavior has become an issue of great importance and research interest. In this
survey, we provide a comprehensive overview of two complementary perspectives
of measuring neural network similarity: (i) representational similarity, which
considers how activations of intermediate layers differ, and (ii) functional
similarity, which considers how models differ in their outputs. In addition to
providing detailed descriptions of existing measures, we summarize and discuss
results on the properties of and relationships between these measures, and
point to open research problems. We hope our work lays a foundation for more
systematic research on the properties and applicability of similarity measures
for neural network models.
Max Klabunde, Tobias Schumacher, Markus Strohmaier, Florian Lemmerich
# Similarity of Neural Network Models:

###### Abstract

Measuring similarity of neural networks has become an issue of great importance and research interest to understand and utilize differences of neural networks. While there are several perspectives on how neural networks can be similar, we specifically focus on two complementing perspectives, i.e., (i) representational similarity, which considers how _activations_ of intermediate neural layers differ, and (ii) functional similarity, which considers how models differ in their _outputs_. In this survey, we provide a comprehensive overview of these two families of similarity measures for neural network models. In addition to providing detailed descriptions of existing measures, we summarize and discuss results on the properties and relationships of these measures, and point to open research problems. Further, we provide practical recommendations that can guide researchers as well as practitioners in applying the measures. We hope our work lays a foundation for our community to engage in more systematic research on the properties, nature and applicability of similarity measures for neural network models.

## 1 Introduction

Understanding and measuring similarity of neural networks is a complex problem, as there are many perspectives on how such models can be similar. In this work, we specifically focus on two broad and complementary perspectives: _representational_ and _functional measures of similarity_ (see Figure 1). Representational similarity measures assess how activations of intermediate neural layers differ, whereas functional similarity measures specifically compare the outputs of neural networks with respect to their learning task. Both perspectives on their own are not sufficient to gain detailed insights into similarity of neural network models. Seemingly similar representations can still yield different outputs, and conversely, similar outputs can result from very different representations. In that sense, combining these two perspectives provides a more comprehensive approach to analyze similarity between neural networks at all layers.

Measures to quantify similarity of neural network models have been widely applied in the literature, including research on learning dynamics [1, 2], effect of width and depth [3], differences between supervised and unsupervised models [4], robustness [5, 6], effect of data and model updates [7, 8, 9, 10], evaluating knowledge distillation [11], designing ensembles [12], language representation [13, 14, 15], and generalizability [16, 17, 18].

Given this broad range of research on neural network similarity, numerous corresponding measures have been proposed and applied, often times with many lines of research being disconnected from each other. With this work, we provide a comprehensive overview of measures for representational similarity and functional similarity that gives a
unified perspective on the existing literature and can inform and guide both researchers and practitioners interested in understanding and comparing neural network models.

Measures for representational or functional similarity have been covered in prior work to some extent. Regarding representational similarity, measures for matrix correlation have been reviewed by [19, 20]. These surveys however do not cover more recent measures or do not consider the context of deep learning. A recent survey by Rauker et al. [21] reviews methods to interpret inner workings of neural networks, but discusses representational similarity measures only very briefly. Functional similarity measures have been surveyed in context of ensemble learning [22, 23], inter-rater agreement [24, 25, 26], and image and text generation scenarios [27, 28], with each considering differing subsets of the measures considered in context of our survey. Consequently, to the best of our knowledge, this survey represents the first comprehensive review of representational and functional similarity measures for neural networks.

This survey makes the following contributions:

1. **Systematic and comprehensive overview:** We formally define the problem of measuring functional and representational similarity in neural networks and provide a systematic and comprehensive overview of existing measures in the context of classification.
2. **Unified terminology:** We provide detailed definitions, explanations and categorizations for each measure in a unified manner, facilitating the understanding of commonalities and differences between measures.
3. **Practical properties and applicability:** We discuss the practical properties of existing measures, such as robustness to noise or confounding issues, and connections between existing measures to guide researchers and practitioners in applying these measures.
4. **Open research challenges:** We highlight unresolved issues of similarity measures and point out research gaps that can be addressed in the future to improve our understanding of neural networks in general.

While we focus on measures for representational and functional similarity, we acknowledge various other approaches to comparing neural networks, such as probing [29], weight masking [30], and visualization [31], that will not be discussed in this work.

The rest of this article is structured as follows. In Section 2, we formally introduce the problem of measuring functional and representational similarity in neural network models. Afterward, we provide thorough overviews about both measures for representational similarity (Section 3) and functional similarity (Section 4). In Section 5, we summarize research on connections between these measures as well as practical properties of similarity measures. Section 6 provides a discussion on the relationship between both perspectives on similarity, open research problems, and practical considerations when applying the measures. Finally, in Section 7, we conclude our survey and provide pointers to future research on open problems.

## 2 Similarity of Neural Network Models

We consider the problem of comparing neural networks, which we assume to have the form

\[f=f^{(L)}\circ f^{(L-1)}\circ\cdots\circ f^{(1)}, \tag{1}\]

with each function \(f^{(l)}:\mathbb{R}^{D}\longrightarrow\mathbb{R}^{D^{\prime}}\) denoting a single layer, and a total number of \(L\in\mathbb{N}\) layers. These networks operate on a set of \(N\) given inputs \(\{\mathbf{X}_{i}\}_{i=1}^{N}\), which we typically assume to be vectors in \(\mathbb{R}^{p}\), \(p\in\mathbb{N}\), although these can also be higher-dimensional structures as occurring in image or video data. For simplicity, we collect these inputs in a matrix \(\mathbf{X}\in\mathbb{R}^{N\times p}\) so that the \(i\)-th row \(\mathbf{X}_{i}\) corresponds to the \(i\)-th input. To further simplify notation, we will also denote individual inputs \(\mathbf{X}_{i}\) as _instances_\(i\in\{1,\dots,N\}\). We generally do not make any assumption on the number of features \(p\), the depth of the network \(L\), the width or activation function of any layer \(f^{(l)}\), or the training objective.

Similarity of neural network models is then quantified by similarity measures \(m\). For simplicity, we also consider dissimilarity measures that quantify _distance_ between models as similarity measures, since these concepts are generally equivalent. While there are other approaches to analyze the similarity of neural network models (e.g., probing [29], or visualization [31]), in our survey we specifically consider two kinds of similarity, namely _representational similarity_ and _functional similarity_. Representational similarity measures consider how the inner activations of neural network models differ, whereas functional similarity measures compare the output behavior of neural networks with respect to the given (classification) task. Compared to other approaches, these kinds of similarity focus on how given inputs \(\mathbf{X}_{i}\) are processed, rather than considering architecture or plain weights of neural network models.

Generally, there is no ground-truth as to whether two neural network models are similar. Hence, representational and functional similarity measures can offer complementing notions of similarity, allowing for more nuanced insights
into model similarity when combining these two perspectives. In the following, we give more thorough definitions of representational and functional similarity, and provide a set of properties to categorize these measures.

In general, we introduce notations for each variable that we use once at the first time the variable is needed. However, we provide an overview over notation conventions as well as a table of notations in Appendix A.

### Representational Similarity

A representational similarity measure compares neural networks by measuring similarity between activations of a fixed set of inputs at intermediate layers \(f^{(l)}\). Given such inputs \(\mathbf{X}\), we define the representations of model \(f\) at layer \(l\) as a matrix

\[\mathbf{R}:=\mathbf{R}^{(l)}=\begin{bmatrix}\left(f^{(l)}\circ f^{(l-1)}\circ\cdots \circ f^{(1)}\right)\left(\mathbf{X}_{1}\right)\\ \vdots\\ \left(f^{(l)}\circ f^{(l-1)}\circ\cdots\circ f^{(1)}\right)\left(\mathbf{X}_{N} \right)\end{bmatrix}\in\mathbb{R}^{N\times D}, \tag{2}\]

where \(D:=D^{(l)}\) denotes the number of neurons in layer \(l\). The activations of instance \(i\) then correspond to the \(i\)-th row of \(\mathbf{R}\), which we denote as _instance representation_\(\mathbf{R}_{i}:=\left(f^{(l)}\circ\cdots\circ f^{(1)}\right)\left(\mathbf{X}_{i} \right)\in\mathbb{R}^{D}\). The activations of single neurons over all instances correspond to the columns of \(\mathbf{R}\), and we denote the \(j\)-th column of \(\mathbf{R}\) as \(\mathbf{R}_{-\cdot j}\). Like the inputs, we also consider the instance representations to be vectors even though in practice, for instance in convolutional neural networks, these activations can also be matrices. In such a case, these representations can be flattened (see Section 2.1.2).

A representational similarity measure is then a mapping \(m:\mathbb{R}^{N\times D}\times\mathbb{R}^{N\times D^{\prime}}\longrightarrow \mathbb{R}\) that assigns a similarity score \(m(\mathbf{R},\mathbf{R^{\prime}})\) to a pair of representations \(\mathbf{R}\), \(\mathbf{R^{\prime}}\), which are derived from different models \(f,f^{\prime}\), but use the same inputs \(\mathbf{X}\). Without loss of generality, we assume that \(D\leq D^{\prime}\), though some measures require that \(D=D^{\prime}\). In such cases, preprocessing techniques can be applied (see Section 2.1.2).

Figure 1: A conceptual overview of representational and functional similarity. In this illustration, two neural network models \(f,f^{\prime}\) are compared, but one could also consider similarity of more than two models. Functional similarity measures mainly consider the outputs \(\mathbf{O},\mathbf{O^{\prime}}\) of the compared models, whereas representational similarity measures consider their intermediate representations \(\mathbf{R},\mathbf{R^{\prime}}\). All models get the same inputs. Specifically in classification tasks, outputs have clear and universal semantics, so that they can be compared in a straightforward manner. In contrast, the geometry of the representations requires more care when measuring their similarity. In the illustration above, for instance, rotating \(\mathbf{R^{\prime}}\) by 90 degrees would yield an alignment of representations after which they would appear much more similar. Combined, representational and functional measures cover all layers of the models.
#### 2.1.1 Equivalence of Representations

Even if two representation matrices \(\mathbf{R},\mathbf{R^{\prime}}\in\mathbb{R}^{N\times D}\) are different on an element-per-element basis, one may still consider them to be equivalent, i.e., perfectly similar, written as \(\mathbf{R}\sim\mathbf{R^{\prime}}\). An intuitive example for such a case would be when representations only differ in their sign, i.e., \(\mathbf{R}=-\mathbf{R^{\prime}}\), or when representations can be rotated onto another. Such notions of equivalence can be formalized in terms of bijective mappings (transformations) \(\varphi:\mathbb{R}^{N\times D}\longrightarrow\mathbb{R}^{N\times D}\), for which it then holds that \(\varphi(\mathbf{R})=\mathbf{R^{\prime}}\). What kind of transformations constitute equivalence between representations may vary depending on the context at hand. For instance, equivalence up to rotation does not make sense if some feature dimensions are already aligned with fixed axes, e.g., in interpretable word embeddings where a dimension could represent a scale between two polar opposites like "bright" and "dark" [32]. Thus, we define equivalence of representations in terms of classes of transformations \(\mathcal{T}\), and call two representations \(\mathbf{R},\mathbf{R^{\prime}}\) equivalent with respect to a class of transformations \(\mathcal{T}\), if there is a \(\varphi\in\mathcal{T}\) such that \(\varphi(\mathbf{R})=\mathbf{R^{\prime}}\). One might argue that a similarity measure should not be able to distinguish two representations if the representations are considered equivalent. We call a representational similarity measure \(m\) invariant to a class of transformations \(\mathcal{T}\), if for all \(\mathbf{R},\mathbf{R^{\prime}}\in\mathbb{R}^{N\times D}\) and all \(\varphi\in\mathcal{T}\) it holds that \(m(\mathbf{R},\mathbf{R^{\prime}})=m(\mathbf{R},\varphi(\mathbf{R^{\prime}}))=m(\varphi(\mathbf{R} ),\mathbf{R^{\prime}})\). In related literature, there are six main classes of transformations under which representations are considered equivalent, and that representational similarity measures are often designed to be invariant to:

* **Permutations (PT).** A similarity measure \(m\) is invariant to permutations if swapping columns of the representation matrices \(\mathbf{R}\), that is, reordering neurons, does not affect the resulting similarity score. Letting \(S_{D}\) denote the set of all permutations on \(\{1,\ldots,D\}\), and for \(\pi\in S_{D}\), \(\mathbf{P}_{\pi}=(p_{i,j})\in\mathbb{R}^{D\times D}\) denote the permutation matrix where \(p_{i,j}=1\) if \(\pi(i)=j\) and \(p_{i,j}=0\) otherwise, the class of all permutation transformations is given by the set \[\mathcal{T}_{\text{PT}}=\{\mathbf{R}\mapsto\mathbf{R}\mathbf{P}_{\pi}:\pi\in S_{D}\}.\] (3)
* **Orthogonal transformations (OT).** As noted in an earlier example, one might intuitively consider two representations equivalent if they can be rotated onto each other. Next to rotations, the class of orthogonal transformations also includes permutations and reflections. Letting \(\mathrm{O}(D):=\{\mathbf{Q}\in\mathbb{R}^{D\times D},\mathbf{Q^{\mathsf{T}}Q}=\mathbf{I}_{D}\}\) denote the orthogonal group, the set of these transformations is given by \[\mathcal{T}_{\text{OT}}=\{\mathbf{R}\mapsto\mathbf{R}\mathbf{Q}:\mathbf{Q}\in\mathrm{O}(D)\}.\] (4)
* **Scaling (IS).** Scaling all elements of a representation \(\mathbf{R}\) identically (isotropic scaling) does not change the angles between instance representations \(\mathbf{R}_{i}\). Thus, for some metrics it can be beneficial if it is invariant to such rescalings. The set of all isotropic scaling transformations is defined as \[\mathcal{T}_{\text{IS}}=\{\mathbf{R}\mapsto a\cdot\mathbf{R}:a\in\mathbb{R}_{+}\}.\] (5)
* **Invertible linear transformations (ILT).** A broader class of transformations is given by the invertible linear transformations \[\mathcal{T}_{\text{ILT}}=\{\mathbf{R}\mapsto\mathbf{R}\mathbf{A}:\mathbf{A}\in\mathrm{GL}(D, \mathbb{R}),\] (6) where \(\mathrm{GL}(D,\mathbb{R})\) denotes the general linear group of all invertible matrices \(\mathbf{A}\in\mathbb{R}^{D\times D}\). This class of transformations also includes orthogonal transformations and rescalings. In case we have representations with \(N<D\), i.e., fewer instances than features, equivalence or invariance with respect to invertible linear transformations is not desirable, since any pair of representations \(\mathbf{R},\mathbf{R^{\prime}}\) with full rank will be equivalent [33]. Further, invertible linear transformations include normalization layers, which demonstrably influence neural network convergence [34]. Thus, invariance to invertible linear transformations may lead to overestimated representational similarity.
* **Translations (TR).** If the angles between instance representations \(\mathbf{R}_{i}\) are not of concern, one might argue that two representations are equivalent if they can be mapped onto each other by adding a constant vector. In that regard, a measure \(m\) is invariant to translations if is invariant to the set of all mappings \[\mathcal{T}_{\text{TR}}=\{\mathbf{R}\mapsto\mathbf{R}+\mathbf{1}\mathbf{b}^{\mathsf{T}}:\mathbf{b} \in\mathbb{R}^{D}\},\] (7) where \(\mathbf{1}:=\mathbf{1}_{D}\in\{1\}^{D}\) is a vector of \(D\) ones.
* **Affine transformations (AT).** The most general class of transformations that is typically considered for representations is given by the set of affine transformations \[\mathcal{T}_{\text{AT}}=\{\mathbf{R}\mapsto\mathbf{R}\mathbf{A}+\mathbf{1}\mathbf{b}^{\mathsf{T}}: \mathbf{A}\in\mathrm{GL}(D,\mathbb{R}),\mathbf{b}\in\mathbb{R}^{D}\}.\] (8) This class of transformations in particular also includes rescaling, translations, orthogonal transformations, and invertible linear transformations. If \(\mathbf{b}=0\), this class is the class of invertible linear transformations.

We depict the hierarchy of these measures in Figure 2. In practice, representational similarity measures are designed with specific invariances dependent on context. Many of these measures are invariant to orthogonal transformations and isotropic scaling.
#### 2.1.2 Preprocessing of Representations

Many of the representational similarity measures will assume certain properties of the representations \(\mathbf{R},\mathbf{R^{\prime}}\), that in practice are not always given. In these cases, the representations need to be preprocessed depending on the given assumptions. Overall, there are three kinds of preprocessing that may have to be applied, namely a) adjusting dimensionality, b) normalization, and c) flattening of representations. We briefly describe common techniques for these preprocessing problems.

Adjusting Dimensionality.Many of the representational similarity measures presented in Section 3 implicitly assume that the representations \(\mathbf{R},\mathbf{R^{\prime}}\) have the same dimensionality, i.e., \(D=D^{\prime}\). In practice, this however isn't always the case. Thus, if \(D<D^{\prime}\), some preprocessing technique must be applied to match the dimensionality. There are two techniques that are commonly recommended for preprocessing, namely zero-padding and dimensionality reduction, such as principal component analysis (PCA) [35, 36]. When zero-padding, the dimension \(D\) of representation \(\mathbf{R}\) is inflated by appending \(D^{\prime}-D\) columns of zeros to \(\mathbf{R}\). PCA conversely reduces the dimension of the representation \(\mathbf{R}^{\prime}\) by removing the \(D^{\prime}-D\) lowest-information components from the representation.

Normalization.Some representational similarity measures assume that the representations are mean-centered in the columns [33, 36, 1]. This normalization technique effectively constitutes a translation of the representations, which alters the angles between instance representations. Similarly, double centering in both rows and columns changes the angles. Another common assumption is that the individual representations have unit norm, which can only be achieved by rescaling each instance representation accordingly [37, 38]. While this normalization approach preserves angles between single instance representations, the Euclidean distances are altered. Therefore, any normalization has to be applied with caution, as preprocessing might alter the inner representation structure -- for instance, neural embedding models such as skip-gram based embeddings model distance between representations in terms of angles.

Flattening.Representational similarity measures assume input matrices \(\mathbf{R}\in\mathbb{R}^{N\times D}\) as input. However, some models such as convolutional neural networks (CNNs) produce representations of more than two dimensions, making them incompatible with these measures. Therefore, such multidimensional representations would have to be flattened. For this preprocessing, the specific properties of these representations need to be taken into account. As an example, representations of CNNs are usually modelled as elements of \(\mathbb{R}^{N\times h\times w\times c}\), where \(h,w\) are the height and the width of the feature maps, and \(c\) is the number of channels. While it is possible to flatten these representations into matrices \(\mathbf{R}\in\mathbb{R}^{N\times hwc}\), permuting the features of these flattened representations disregards the spatial information in the original feature map. As a solution, Williams et al. [36] recommend flattening CNN representations to the shape of \(Nhw\times c\), so that a permutation only affects the channels, and spatial differences can still be recognized. However, this flattening is only possible if the resulting effective number of inputs \(Nhw\) matches in both models or a feature map is upsampled.

Aside from issues regarding the structure of representations, the computational cost of the representational similarity measure may also be strongly affected by the format of the flattened representations.

Figure 2: Invariances of representational similarity measures build a hierarchy. Arrows describe implication, with the top invariance being the more general one.
### Functional Similarity Measures

Functional similarity measures compare neural networks by measuring similarity of their output behavior [39].

Given a set of inputs \(\mathbf{X}\) and a neural network \(f\) that is trained for a classification task on \(C\) classes, we denote the matrix of its outputs as:

\[\mathbf{O}:=\begin{bmatrix}f(\mathbf{X}_{1})\\ \vdots\\ f(\mathbf{X}_{N})\end{bmatrix}\in\mathbb{R}^{N\times C}. \tag{9}\]

Each row \(\mathbf{O}_{i}=f(\mathbf{X}_{i})\in\mathbb{R}^{C}\) denotes the output with respect to input \(\mathbf{X}_{i}\). This vector-based output naturally includes _soft predictions_ where each element \(\mathbf{O}_{i,c}\) denotes the probabilities or decision function scores for class \(c\), as well as _hard predictions_ for multiclass problems, where \(\mathbf{O}_{i,c}=1\) if the model predicts class \(c\) for input \(\mathbf{X}_{i}\), and \(\mathbf{O}_{i,c}=0\) otherwise. Moreover, in the most common cases where the outputs are given as hard predictions or as soft predictions modeling probabilities, one can consider the individual rows \(\mathbf{O}^{\prime}_{i}\) to be elements of the probability simplex

\[\Delta_{C}=\bigg{\{}\mathbf{p}\in[0,1]^{C}:\sum_{i=1}^{C}\mathbf{p}_{i}=1\bigg{\}}. \tag{10}\]

In contrast to representational similarity, analyzing functional similarity generally does not require consideration of preprocessing or alignment issues. Even more, for many functional similarity measures, black-box access to the given model \(f\) is sufficient. However, functional similarity measures may include additional information aside from the raw outputs \(\mathbf{O}_{i}\). For instance, in practice a set of ground-truth labels \(\mathbf{y}\in\mathbb{R}^{N}\) is often given, which is typically used by a quality function \(q\) that quantifies how well the output matches the ground-truth. Another kind of additional information that can inform model similarity would be task-based gradients, which, for simplicity, we will assume to be directly included in the output matrix \(\mathbf{O}\). This information, however, can only be obtained with white-box access to the model.

Overall, we broadly distinguish functional similarity measures into first-order and second-order similarity measures. First-order similarity measures \(m\) directly operate on the raw outputs and have the form \(m(\mathbf{O},\mathbf{O}^{\prime})\), whereas second-order similarity measures consider the outputs of quality functions \(q\) with respect to ground-truth labels \(\mathbf{y}\), and therefore have the form \(m(q(\mathbf{O},\mathbf{y}),q(\mathbf{O}^{\prime},\mathbf{y}))\). The most commonly used quality function in that context is the error rate, which is defined as

\[q_{\mathrm{Err}}(\mathbf{O}):=q_{\mathrm{Err}}(\mathbf{O},\mathbf{y}):=\frac{1}{N}\sum_ {i=1}^{N}\mathbb{1}\{\arg\max_{j}\mathbf{O}_{i,j}\neq\mathbf{y}_{i}\}. \tag{11}\]

Finally, some measures operate on sets of outputs, which we denote as \(m(\mathcal{O})\), with \(\mathcal{O}=\{\mathbf{O},\mathbf{O}^{\prime},\mathbf{O}^{\prime\prime},\dots\}\).

### Properties of Similarity Measures

Within our survey, we provide an overview of a broad range of representational and functional similarity measures, which we will also categorize by some properties that may affect their practical applicability. In the following, we will briefly discuss two of such properties of measures, namely whether they can take groupwise inputs, and whether they are formal metrics. To accommodate both representational and functional similarity measures, we will use the notation that measures \(m\) consider input matrices \(\mathbf{A}\in\mathbb{R}^{N\times d},d\in\mathbb{N}\), for the rest of this section.

#### 2.3.1 Groupwise Inputs

The majority of measures discussed in this survey are mappings \(m:\mathbb{R}^{N\times d}\times\mathbb{R}^{N\times d^{\prime}}\longrightarrow \mathbb{R}\) that take a pair of matrices \(\mathbf{A},\mathbf{A}^{\prime}\) as input. In some applications, however, one might need to compare more a set \(\mathcal{A}\) consisting of more than two representations or outputs. In this case, the few measures that do not take _pairwise_, but _groupwise_ inputs, would be of specific interest. An alternative to using these measures would be to aggregate similarity scores over all pairs in \(\mathcal{A}\), for instance, by taking their mean:

\[m(\mathcal{A})=\frac{2}{|\mathcal{A}|\cdot(|\mathcal{A}|-1)}\sum_{\mathbf{A},\mathbf{ A}^{\prime}\in\mathcal{A}}m(\mathbf{A},\mathbf{A}^{\prime}). \tag{12}\]

However, this approach has the downside of not being able to find commonalities across the whole set of representations or outputs.
#### 2.3.2 Distance Metrics

For each measure \(m\) that is mentioned in this survey, there is a unique value \(c\in\mathbb{R}\) that indicates equality (or equivalence) of the inputs, i.e., for all input matrices \(\mathbf{A}\in\mathbb{R}^{N\times d}\) it holds that \(m(\mathbf{A},\mathbf{A})=c\). This value is also typically either the upper or lower bound of the range of a measure. In the case of \(c=0\), the corresponding measure will quantify distance rather than similarity. While -as noted before- in the terminology of this survey we do not differentiate between the notions of similarity and distance measures, some of the measures \(m:\mathbb{R}^{N\times d}\longrightarrow\mathbb{R}\) will satisfy the formal axioms of a (distance) metric, which includes that \(m(\mathbf{A},\mathbf{A}^{\prime})=0\) if \(\mathbf{A}\sim\mathbf{A}^{\prime}\), and \(m(\mathbf{A},\mathbf{A}^{\prime})>0\) otherwise. The second property of a metric is _symmetry_, which requires that for all \(\mathbf{A},\mathbf{A}\in\mathbb{R}^{N\times d}\) if holds that

\[m(\mathbf{A},\mathbf{A}^{\prime})=m(\mathbf{A}^{\prime},\mathbf{A}). \tag{13}\]

Unless noted otherwise, one can assume that measures presented in this survey are symmetric. The third and final property of a metric is the satisfaction of the _triangle inequality_, which states that for all \(\mathbf{A},\mathbf{A}^{\prime},\mathbf{Z}\in\mathbb{R}^{N\times d}\) it has to hold that

\[m(\mathbf{A},\mathbf{A}^{\prime})\leq m(\mathbf{A},\mathbf{Z})+m(\mathbf{A}^{\prime},\mathbf{Z}). \tag{14}\]

This property formalizes the intuitive notion that if two matrices are considered close to a reference matrix by a measure, the measure should also consider them close to each other. It is generally considered favorable for a similarity measure to satisfy these properties, since they benefit consistency and interpretability of what is measured. Hence, we will explicitly note when a representational similarity measure formally satisfies all these properties.

## 3 Representational Similarity Measures

We now review existing the representational similarity measures, organized by their methodological approach. An overview of all representational similarity measures can be found in Table 1.

\begin{table}
\begin{tabular}{l|l c c c c c c c c c c} \hline \hline \multirow{2}{*}{Type} & \multirow{2}{*}{Measure} & \multirow{2}{*}{\begin{tabular}{c} Model \\ Apostice \\ \end{tabular} } & \multicolumn{6}{c}{Invariances} & \multirow{2}{*}{\begin{tabular}{c} \\ \end{tabular} } \\  & & & PT & OT & IS & LT & TR & AT & Other & \(D\neq D^{\prime}\) & Metric & Similarity \(\uparrow\) \\ \hline \multirow{4}{*}{\begin{tabular}{c} Canonical \\ Conical \\ Analysis \\ \end{tabular} } & Mean Canonical Correlation [40] & ✓ & ✓ & ✓ & ✓ & ✓ & ✓ & ✓ & ✗ & ✗ & ✗ & ✓ \\  & Mean Canonical Correlation [33, 41] & ✓ & ✓ & ✓ & ✓ & ✓ & ✓ & ✗ & ✓ & ✗ & ✓ \\  & Singular Vector Canonical Correlation Analysis (SVCCA) [40] & ✓ & ✓ & ✓ & ✗ & ✗ & ✗ & ✗ & ✗ & ✓ & ✗ & ✓ \\  & Projection-Weighted Canonical Correlation Analysis (PVCCA) [1] & ✓ & ✗ & ✗ & ✗ & ✗ & ✗ & ✗ & ✓ & ✗ & ✓ \\ \hline \multirow{6}{*}{\begin{tabular}{c} Alignment \\ Alignment \\ \end{tabular} } & Proximites [38, 36] & ✓ & ✓ & ✓ & ✗ & ✗ & ✗ & ✗ & ✗ & ✗ & ✗ & ✗ \\  & \(\alpha_{\text{LL}}\)-Prectures [42] & ✗ & ✓ & ✗ & ✗ & ✗ & ✗ & ✗ & ✗ & ✓ \\  & Singular Cone Similarity [15] & ✓ & ✓ & ✓ & ✓ & ✓ & ✗ & ✗ & ✗ & ✗ & ✓ \\  & Partial-Wheening Map Metric [36] & ✓ & ✓ & ✓ & ✓ & ✓ & ✓ & ✗ & ✗ & ✗ & ✓ \\  & Shift Shape Metric [36] & ✓ & ✓ & ✓ & ✗ & ✗ & ✗ & ✗ & ✗ & ✗ & ✗ \\  & Correlation Match [43] & ✓ & ✓ & ✗ & ✗ & ✗ & ✗ & ✗ & ✗ & ✗ & ✓ \\  & Maximum Matching [48] & ✓ & ✓ & ✓ & ✓ & ✓ & ✗ & ✗ & ✗ & ✓ \\  & Linear Regression [33, 33] & ✓ & ✓ & ✓ & ✓ & ✗ & ✗ & ✗ & ✗ & ✗ & ✗ \\ \hline \multirow{6}{*}{
\begin{tabular}{c} Representation \\ Representation \\ \end{tabular} } & \multicolumn{1}{c}{✓
### Canonical Correlation Analysis-Based Measures

Canonical Correlation Analysis (CCA) [56] is a classical method to compare two sets of values of random variables. The goal is to find vectors \(\mathbf{w_{R}}\in\mathbb{R}^{D},\mathbf{w_{R^{\prime}}}\in\mathbb{R}^{D^{\prime}}\) in the representation space (weightings of the representation dimensions) such that their projections to the unit ball in \(\mathbb{R}^{N}\) via their corresponding representation matrices \(\mathbf{R},\mathbf{R^{\prime}}\) have minimal angle between them (or equivalently maximal correlation \(\rho\)):

\[\rho:=\rho(\mathbf{R},\mathbf{R^{\prime}}):=\max_{\mathbf{w_{R}},\mathbf{w_{R^{ \prime}}}}\frac{\langle\mathbf{R}\mathbf{w_{R}},\mathbf{R^{\prime}}\mathbf{w_{R^{\prime}}} \rangle}{\|\mathbf{R}\mathbf{w_{R}}\|\cdot\|\mathbf{R^{\prime}}\mathbf{w_{R^{\prime}}}\|}. \tag{15}\]

The vectors \(\mathbf{w_{R}^{(j)}}\mathbf{w_{R^{\prime}}^{(j)}}\) can be understood as weightings of the representation dimensions. \(\rho\) is called a _canonical correlation_. Once one has determined this maximum, one can find additional canonical correlations \(\rho_{i}\), with new vectors \(\mathbf{w_{R}^{(i)}},\mathbf{w_{R^{\prime}}^{(i)}}\) being orthogonal, and thus uncorrelated, to the previous ones. This yields a system of \(D\) canonical correlations \(\rho_{i}\) defined as

\[\rho_{i}:=\max_{\mathbf{w_{R}^{(i)}},\mathbf{w_{R^{\prime}}^{(i)}}}\frac {\langle\mathbf{R}\mathbf{w_{R}^{(i)}},\mathbf{R^{\prime}}\mathbf{w_{R^{\prime}}^{(i)}} \rangle}{\|\mathbf{R}\mathbf{w_{R}^{(i)}}\|\cdot\|\mathbf{R^{\prime}}\mathbf{w_{R^{\prime}}^{( i)}}\|} \tag{16}\] \[\mathrm{s.t.}\ \mathbf{R}\mathbf{w_{R}^{(j)}}\bot\mathbf{R}\mathbf{w_{R}^{(i)}},\ \mathbf{R^{\prime}}\mathbf{w_{R^{ \prime}}^{(j)}}\bot\mathbf{R^{\prime}}\mathbf{w_{R^{\prime}}^{(i)}}\ \forall j<i,\]

where \(\bot\) means orthogonality. For the first canonical correlation, we have \(\rho=\rho_{1}\).

A single similarity score \(m(\mathbf{R},\mathbf{R^{\prime}})\) is then computed by aggregating the canonical correlations \(\rho_{i}\). Standard choices for this aggregation that have been used to quantify representational similarity for neural networks are the mean canonical correlation [40, 33, 57]

\[m_{\mathrm{CCA}}(\mathbf{R},\mathbf{R^{\prime}})=\frac{1}{D}\sum_{i=1}^{D}\rho_{i}, \tag{17}\]

and the mean squared canonical correlations [33, 57]

\[m_{\mathrm{CCA}^{2}}(\mathbf{R},\mathbf{R^{\prime}})=\frac{1}{D}\sum_{i=1}^{D}\rho_{ i}^{2}, \tag{18}\]

which are also known as Yanai's generalized coefficient of determination [41, 19]. Other prominent aggregation schemes, though not applied for neural representational similarity, include the sum of the squared canonical correlations (also known as Pillai's trace [58]), Wilk's lambda statistic [59], and the Lawley-Hotelling trace [60, 61]. Overall, several more aggregation methods can be applied, and there is a wide range of variants of CCA measures, including non-linear methods, or methods that consider more than two input matrices. Within this work, however, we only consider those CCA-based measures that have been used to measure representational similarity of neural networks. A broader overview on existing CCA measures is provided in the recent survey by Yang et al. [20] or the tutorial by Uurtio et al. [62].

CCA is invariant to affine transformations [1]. If the representations \(\mathbf{R},\mathbf{R^{\prime}}\) are equivalent, it holds that \(\rho_{i}=1\) for all \(i\in\{1,\dots,D\}\) and thus \(m_{\mathrm{CCA}}(\mathbf{R},\mathbf{R^{\prime}})=1\) and \(m_{\mathrm{CCA}^{2}}(\mathbf{R},\mathbf{R^{\prime}})=1\).

We now describe CCA-based measures that have been used to measure neural network similarity.

#### 3.1.1 Singular Value CCA

Singular Value CCA (SVCCA) combines preprocessing of the representations via Singular Value Decomposition (SVD) with CCA [40]. Raghu et al. [40] argue that representations are noisy and that this noise should be removed before conducting CCA on the representations \(\mathbf{R},\mathbf{R^{\prime}}\). Their SVD-based approach to obtain denoised representations is equivalent to performing PCA on the representations. The number \(k\) of principal components that are kept is selected such that a fixed relative amount \(t\) of the variance in the data, usually 99 percent, is explained.

Formally, assuming that the SVD of each representation \(\mathbf{R},\mathbf{R^{\prime}}\) has the form \(\mathbf{R}=\mathbf{U}\mathbf{\Sigma}\mathbf{V}^{\mathsf{T}}\) where \(\mathbf{U}\in\mathrm{O}(N)\), \(\mathbf{V}\in\mathrm{O}(D)\), and \(\Sigma=\mathrm{diag}(\sigma_{1},\dots,\sigma_{\min(N,D)})\in\mathbb{R}^{N\times D}\) is the rectangular diagonal matrix of singular values \(\sigma_{i}\), they propose to select the smallest value \(k\) such that \(\sum_{j=1}^{k}\sigma_{j}>t\cdot\sum_{j=1}^{\min(N,D)}\sigma_{j}\) for a relative threshold \(t\in(0,1)\). The resulting value \(k\) is then used to compute the denoised representations

\[\mathbf{\tilde{R}}=\mathbf{U}_{-,-k}\mathbf{\Sigma}_{-k,-k}\in\mathbb{R}^{N\times k}, \tag{19}\]

where \(\mathbf{U}_{-,-k}\in\mathbb{R}^{N\times k}\) consists of the first \(k\) columns of \(\mathbf{U}\), and \(\mathbf{\Sigma}_{-k,-k}=\mathrm{diag}(\sigma_{1},\dots,\sigma_{k})\in\mathbb{R}^{k \times k}\) is the diagonal matrix of the \(k\) largest singular values.
Afterward, they use standard CCA on the denoised representations. The average canonical correlation is then used as the final similarity measure:

\[m_{\mathrm{SVCCA}}(\mathbf{R},\mathbf{R^{\prime}})=m_{\mathrm{CCA}}(\mathbf{\tilde{R}},\mathbf{ \tilde{R^{\prime}}}). \tag{20}\]

Unlike CCA, SVCCA is not invariant to affine transformations, since such transformations generally alter the singular value decomposition. However, SVCCA is still invariant to orthogonal transformations and isotropic scaling. SVCCA is bounded in the interval \([0,1]\), with a score of one indicating perfectly similar representations, and a score of zero indicating perfectly dissimilar representations.

To compute SVCCA efficiently for CNNs with many features, Raghu et al. [40] apply a Discrete Fourier Transform on each channel, yielding block-diagonal matrices for CCA computation, which eliminates unneeded operations.

#### 3.1.2 Projection Weighted CCA

Morcos et al. [1] proposed Projection Weighted CCA (PWCCA) as an alternative to SVCCA. They argue that a representational similarity measure should weigh the individual canonical correlations \(\rho_{i}\) by their importance, i.e., the similarity of the _canonical variables_\(\mathbf{Rw}_{\mathbf{R}}^{(i)}\) with the raw representation \(\mathbf{R}\).

For that purpose, for every canonical correlation \(\rho_{i}\), they formally define a weighting coefficient

\[\tilde{\alpha}^{(i)}=\sum_{j=1}^{D}|\langle\mathbf{Rw}_{\mathbf{R}}^{(i)},\mathbf{R}_{-,j} \rangle|, \tag{21}\]

which can be understood as measuring how much of the neurons is represented by a canonical variable. These coefficients are then normalized to weights \(\alpha^{(i)}=\tilde{\alpha}^{(i)}/\sum_{j}\tilde{\alpha}^{(j)}\), yielding the final representational similarity measure defined as

\[m_{\mathrm{PWCCA}}(\mathbf{R},\mathbf{R^{\prime}})=\sum_{i=1}^{D}\alpha^{(i)}\rho_{i}. \tag{22}\]

Since the weighting coefficients are only calculated based on \(\mathbf{R}\), without taking \(\mathbf{R^{\prime}}\) into account, this representational similarity measure is asymmetric. Further, this measure is not invariant to any class of matrix transformations, since in the computation of the weighting coefficients (21), only the first factor \(\mathbf{Rw}_{\mathbf{R}}^{(i)}\) in the scalar product will be unaltered, whereas the second factor will change. However, the normalization of the weighting coefficients makes PWCCA invariant to scaling. PWCCA is bounded in the interval \([0,1]\), with a value of one indicating equivalent representations.

### Alignment-Based Measures

The next group of measures stipulates that a pair of representations \(\mathbf{R},\mathbf{R^{\prime}}\) can be compared directly, once the corresponding representation spaces have been aligned to each other. Such alignment is usually realized by finding an optimal transformation \(\varphi\in\mathcal{T}\) that minimizes a difference of the form \(\|\varphi(\mathbf{R})-\mathbf{R^{\prime}}\|\). In that context, the exact group of transformations \(\mathcal{T}\) that can be used for alignment also directly determines and usually corresponds to the group of transformations that the corresponding measure will be invariant to. Such direct alignment is only possible if the number of neurons in both representations are equal, and thus we will assume that throughout the next section it holds that \(D=D^{\prime}\), unless otherwise mentioned. We will now discuss existing measures from that category.

#### 3.2.1 Orthogonal Procrustes

The orthogonal Procrustes problem is a classical problem of finding the best orthogonal transformation to align two matrices, and can be formulated as

\[\mathbf{Q}^{*}=\arg\min_{\mathbf{Q}\in\mathrm{O}(D)}\|\mathbf{RQ}-\mathbf{R^{\prime}}\|_{F}, \tag{23}\]

where \(\|\cdot\|_{F}\) denotes the Frobenius norm. Ding et al. [38] use the square of the minimum value obtained in (23) as a measure for representational similarity, which is given by

\[m_{\mathrm{Ortho-Proc}}(\mathbf{R},\mathbf{R^{\prime}})=\|\mathbf{R}\|_{F}^{2}+\|\mathbf{R^{ \prime}}\|_{F}^{2}-2\|\mathbf{R^{\mathrm{T}}}\mathbf{R^{\prime}}\|_{*}, \tag{24}\]

where \(\|\cdot\|_{*}\) is the nuclear norm that is defined as the sum of the singular values of a matrix [63].

By design, this measure is invariant to orthogonal transformations. Further, Williams et al. [36] have proven that this measure satisfies the properties of a distance metric. This is also the case when the solution is constrained to come from a subgroup \(\mathrm{G}(D)\) of the orthogonal group \(\mathrm{O}(D)\) yielding the measure

\[m_{\mathrm{Proc}}(\mathbf{R},\mathbf{R^{\prime}})=\min_{\mathbf{Q}\in\mathrm{G}(D)}\|\mathbf{RQ }-\mathbf{R^{\prime}}\|_{F}. \tag{25}\]
This gives options for finer notions of similarity. Both for Orthogonal Procrustes and the more general Procrustes measure, a value of zero indicates equivalence and larger values dissimilar representations.

Also, to consider translation equivariance of CNNs, Williams et al. [36] propose a metric that optimizes over orthogonal transformations along the channels and spatial shifts in the feature map. However, in practice the spatial shifts are ignored, and the metric reduces to Orthogonal Procrustes.

#### 3.2.2 \(G_{\mathrm{ReLU}}\)-Procrustes

\(G_{\mathrm{ReLU}}\)-Procrustes is a model-specific instantiation of Procrustes with invariance to a subset of the orthogonal transformations [42]. The work is motivated by understanding _symmetries_ of neural networks, such as different sets of weights that have identical function. The transformations, that \(G_{\mathrm{ReLU}}\)-Procrustes is invariant to, are a group consisting of the transformations applied before activation that have a counterpart after activation, such that the overall transformation is identical. Formally, Godfrey et al. [42] describe a group of transformations, called _intertwiner group_, for an element-wise applied activation function \(\sigma\):

\[G_{\sigma}:=G_{\sigma,D}=\{\mathbf{A}\in\mathrm{GL}(D,\mathbb{R}):\exists\mathbf{B} \in\mathrm{GL}(D,\mathbb{R})\text{ s.t. }\sigma\circ\mathbf{A}=\mathbf{B}\circ\sigma\}, \tag{26}\]

where \(\mathrm{GL}(D,\mathbb{R})\) are the invertible matrices in \(\mathbb{R}^{D\times D}\). We highlight the case where \(\sigma=\mathrm{ReLU}\), which yields the group \(G_{\mathrm{ReLU}}\), because Godfrey et al. [42] detail similarity measures for \(G_{\mathrm{ReLU}}\), but they also describe intertwiner groups for other popular activation functions.

\(G_{\mathrm{ReLU}}\)-Procrustes is closely related to Procrustes alignment with permutations as \(G_{\mathrm{ReLU}}\) consists of matrices of the form \(\mathbf{PD}\), where \(\mathbf{P}\in\mathcal{P}\) is a permutation matrix and \(\mathbf{D}\) is a diagonal matrix with positive elements. Given column-wise normalized representations \(\mathbf{\tilde{R}}=\mathbf{RD_{R}^{-1}}\), where \(\mathbf{D_{R}}\) denotes the diagonal matrix of column norms of \(\mathbf{R}\), the measure is defined as:

\[m_{\mathrm{G_{ReLU}}\text{-}\mathrm{P}\text{-}\mathrm{P}\text{-}\mathrm{P} \text{-}\mathrm{P}\text{-}\mathbf{R^{\prime}}}=1-\frac{\min_{\mathbf{P}\in\mathcal{P} }\|\mathbf{\tilde{R}}\mathbf{P}-\mathbf{\tilde{R}^{\prime}}\|_{F}}{2\sqrt{D}}. \tag{27}\]

The denominator \(2\sqrt{D}\) is added to scale the measure between zero and one. If the representations are equivalent up to \(G_{\mathrm{ReLU}}\) transformation, the measure equals one. Practically, the minimization can be solved via the linear sum assignment problem [36].

#### 3.2.3 Aligned Cosine Similarity

This measure has been used to quantify similarity of instance-wise representations, such as embeddings of individual words over time [15]. Its idea is to first align the representations by the orthogonal Procrustes transformation, and then to use cosine similarity to measure similarity between the aligned representations. Generally, the cosine similarity between two vectors \(\mathbf{v},\mathbf{v}^{\prime}\in\mathbb{R}^{n}\), \(n\in\mathbb{N}\), is defined as

\[\text{cos-sim}(\mathbf{v},\mathbf{v}^{\prime})=\frac{\mathbf{v}^{\mathsf{T}}\mathbf{v}^{ \prime}}{\|\mathbf{v}\|_{2}\|\mathbf{v}^{\prime}\|_{2}}, \tag{28}\]

and bounded in the interval \([-1,1]\), with \(\text{cos-sim}(\mathbf{v},\mathbf{v}^{\prime})=1\) indicating that both vectors point in the exact same direction, and \(\text{cos-sim}(\mathbf{v},\mathbf{v}^{\prime})=0\) indicating orthogonality. Letting \(\mathbf{Q}^{*}\) denote the solution of the Procrustes problem (23), the similarity of two instance representations is given by

\[s_{\mathrm{Aligned-Cossim}}(\mathbf{R}_{i},\mathbf{R}_{i}^{\prime})=\text{cos-sim} \left((\mathbf{RQ^{*}})_{i},\mathbf{R}_{i}^{\prime}\right). \tag{29}\]

Overall similarity can then be analyzed by comparing the overall distribution of similarity scores, or aggregating them by, for instance, taking their mean value [37]. The latter option yields a similarity measure

\[m_{\mathrm{Aligned-Cossim}}(\mathbf{R},\mathbf{R}^{\prime})=\frac{1}{N}\sum_{i=1}^{N }\text{cos-sim}\left((\mathbf{RQ^{*}})_{i},\mathbf{R}_{i}^{\prime}\right). \tag{30}\]

Due to the properties of cosine similarity, \(m_{\mathrm{Aligned-Cossim}}(\mathbf{R},\mathbf{R}^{\prime})=1\) indicates equivalence of representations, and lower values indicate less similarity.

#### 3.2.4 Generalized Shape Metrics

Williams et al. [36] apply theory of statistical shape analysis on the problem of representational similarity of neural network models. Next to several other findings, they also define a novel similarity measure. For this measure, they apply the partial whitening function \(\phi_{\alpha}:\mathbb{R}^{N\times D}\longrightarrow\mathbb{R}^{N\times D}\), \(\alpha\in[0,1]\), defined as

\[\phi_{\alpha}(\mathbf{R})=\mathbf{HR}(\alpha\mathbf{I}_{D}+(1-\alpha)(\mathbf{R}^{\mathsf{T}} \mathbf{HR})^{-1/2}), \tag{31}\]
where \(\mathbf{H}:=\mathbf{I}_{N}-\frac{1}{N}\mathbf{1}\mathbf{1}^{\mathsf{T}}\) is a centering matrix. The partial whitening makes neurons (partially) uncorrelated and have unit variance. Due to this transformation, the _partial whitening (PW) shape metric_ defined as

\[m_{\theta,\alpha}(\mathbf{R},\mathbf{R}^{\prime})=\min_{\mathbf{Q}\in\mathrm{O}(D)}\arccos \frac{\langle\phi_{\alpha}(\mathbf{R})\mathbf{Q},\phi_{\alpha}(\mathbf{R}^{\prime})\rangle} {\|\phi_{\alpha}(\mathbf{R})\|\|\phi_{\alpha}(\mathbf{R}^{\prime})\|}, \tag{32}\]

satisfies the properties of a distance metric for all \(\alpha\in[0,1]\), where \(\langle\cdot,\cdot\rangle\) denotes the inner Frobenius product [64]. Because it is a distance metric, a value of zero indicates equivalence and larger values dissimilar representations, similar to Procrustes. If the hyperparameter \(\alpha=0\), then this metric is invariant to affine transformation, if \(\alpha=1\) it is only invariant to orthogonal transformations. Williams et al. [36] further show that this metric is related to canonical correlations.

Duong et al. [65] recently generalized these metrics to stochastic neural networks, which map to distributions of representations instead of deterministic representations, such as variational autoencoders [66].

#### 3.2.5 Correlation Match

Li et al. [43] measure representational similarity by creating a correlation matrix between the neuron activations of two representations. They match neurons by permutation. This permutation step is achieved by viewing the correlation matrix as the adjacency matrix of a graph, on which they perform a (semi-)matching. The average along the diagonal of the aligned correlation matrix yields a summary of representational similarity:

\[m_{\text{Corr-Match}}(\mathbf{R},\mathbf{R}^{\prime})=\frac{1}{D}\sum_{j=1}^{D}\frac{ \langle\mathbf{R}_{-,j},(\mathbf{R}^{\prime}\mathbf{P})_{-,j}\rangle}{\|\mathbf{R}_{-,j}\|_{ 2}\|(\mathbf{R}^{\prime}\mathbf{P})_{-,j}\|_{2}}, \tag{33}\]

with mean centered representations and the permutation \(\mathbf{P}\) from the matching procedure.

This measure is invariant to permutations, isotropic scaling, and translations. A value of one indicates equivalent representations, a value of zero uncorrelated ones.

#### 3.2.6 Maximum Matching Similarity

In contrast to the previous measures, maximum matching similarity [44] aligns representations only implicitly, by testing whether neuron activations of one representation, i.e., columns of the representation matrix, (approximately) lie in a subspace spanned from neuron activations of the other representation. Every neuron, of which the activation vector can be approximated by such a subspace, is then considered part of a match between the representations. Following this intuition, the main idea of the measure proposed by Wang et al. [44] is to find the maximal set of neurons in each representation that can be matched with the other subspace. Formally, for an index subset \(\mathcal{J}\subseteq\{1,\dots,D\}\), let \(\mathbf{R}_{-,\mathcal{J}}=\{\mathbf{R}_{-,j},j\in\mathcal{J}\}\) denote the set of corresponding neuron activation vectors. Then a pair \((\mathcal{J},\mathcal{J}^{\prime})\) forms an \(\varepsilon\)-approximate match, \(\varepsilon\in(0,1]\), on the representations \(\mathbf{R},\mathbf{R}^{\prime}\) if for all \(j\in\mathcal{J},j^{\prime}\in\mathcal{J}^{\prime}\) it holds that

\[\min_{\mathbf{r}\in\mathrm{span}(\mathbf{R}_{-,\mathcal{J}})}\|\mathbf{R}^{ \prime}_{-,j^{\prime}}-\mathbf{r}\|\leq\varepsilon\cdot\|\mathbf{R}^{\prime}_{-,j^{ \prime}}\| \tag{34}\] \[\text{and}\quad\min_{\mathbf{r}^{\prime}\in\mathrm{span}(\mathbf{R}^{ \prime}_{-,\mathcal{J}^{\prime}})}\|\mathbf{R}_{-,j}-\mathbf{r}^{\prime}\|\leq \varepsilon\cdot\|\mathbf{R}_{-,j}\|. \tag{35}\]

A pair \((\mathcal{J}_{\max},\mathcal{J}^{\prime}_{\max})\) is considered a maximum match, if for all \(\varepsilon\)-matches \((\mathcal{J},\mathcal{J}^{\prime})\) it holds that \(\mathcal{J}\subseteq\mathcal{J}_{\max}\) and \(\mathcal{J}^{\prime}\subseteq\mathcal{J}^{\prime}_{\max}\). Wang et al. [44] show that this maximum match is unique and provide algorithms to determine this match. Based on the maximum match, the maximum-match similarity is defined as

\[m^{\varepsilon}_{\text{maximum-match}}(\mathbf{R},\mathbf{R}^{\prime})=\frac{| \mathcal{J}_{\max}|+|\mathcal{J}^{\prime}_{\max}|}{D+D^{\prime}}. \tag{36}\]

Contrary to prior measures of this category, maximum matching similarity can operate on representations of different dimension.

This measure is invariant to invertible linear transformation, since such transformations do not alter the subspaces. It is bounded in the interval \([0,1]\), with a similarity score of 1 indicating maximum similarity.

#### 3.2.7 Linear Regression

Another approach to match representations is by linear regression that predicts one representation from the other [43, 33]. Given a weight matrix \(\mathbf{W}\in\mathbb{R}^{D\times D}\), the R-squared of the optimal fit can be used to measure similarity [33]:

\[m_{R^{2}}(\mathbf{R},\mathbf{R}^{\prime})=1-\frac{\min_{\mathbf{W}}\|\mathbf{R}^{\prime}-\mathbf{R} \mathbf{W}\|_{F}^{2}}{\|\mathbf{R}^{\prime}\|_{F}^{2}}. \tag{37}\]
Li et al. [43] add a L1 penalty to the optimization to encourage a sparse mapping between neurons. This measure is also similar to _model stitching_ (cf. Section 4.5), though here the focus is on the quality of matching instead of effect on functional behavior.

This asymmetric measure is invariant to orthogonal transformation and isotropic scaling. A value of one indicates maximal similarity, lower values indicate lower similarity. This measure has no lower bound.

### Representational Similarity Matrix-Based Measures

A common approach to avoid alignment issues in direct comparisons of representations is to use _representational similarity matrices_ (RSMs). Intuitively, an RSM describes how each instance \(i\) is represented in relation to all other instances, in a given representation \(\mathbf{R}\). Given the RSM is computed suitably, these relations are invariant to alignment transformations such as rotation. One can then apply the RSMs of two different representations \(\mathbf{R},\mathbf{R^{\prime}}\) to quantify representational similarity in terms of the difference between these RSMs.

Formally, given an instance-wise similarity function \(s:\mathbb{R}^{D}\times\mathbb{R}^{D}\longrightarrow\mathbb{R}\), the representational similarity matrix (RSM) \(\mathbf{S}\in\mathbb{R}^{N\times N}\) of a representation \(\mathbf{R}\) can be defined in terms of its entries via

\[\mathbf{S}_{i,j}:=s(\mathbf{R}_{i},\mathbf{R}_{j}). \tag{38}\]

Each row \(\mathbf{S}_{i}\) then corresponds to the similarity between the representations of instance \(i\) and the representations of all other inputs, including itself.

RSMs can be computed with a variety of similarity measures \(s\) including correlation [45], Euclidean distance [67], cosine similarity [68], and kernels [33] -- like before, we do not differentiate between the equivalent concepts of similarity and distance functions. Naturally, the choice of the underlying similarity measure \(s\) has to suit the geometry of the representations, and has a large impact on the kind of transformations that the representational similarity measures \(m\) are invariant to. If the RSM is unchanged by a transformation, then the representational similarity will not change either. For instance, each of the measures mentioned above, i.e., linear kernels, RBF kernels, cosine similarity, Pearson correlation and Euclidean distance are invariant to orthogonal transformations. In addition, cosine similarity is invariant to scaling, whereas Euclidean distance and RBF kernels are invariant to translations. Even more, Pearson correlation is invariant to both of these classes of transformations.

After selecting a suitable similarity measure \(s\), two RSMs \(\mathbf{S},\mathbf{S^{\prime}}\) are compared. One could directly compute their difference and apply some matrix norm \(\|\cdot\|\) to obtain a measure

\[m_{\mathrm{Norm}}(\mathbf{R},\mathbf{R^{\prime}})=\|\mathbf{S}-\mathbf{S^{\prime}}\|. \tag{39}\]

There are several more RSM-based representational similarity measures, which we will review in the following.

#### 3.3.1 Representational Similarity Analysis

Kriegeskorte et al. [45] propose Representational Similarity Analysis (RSA) in neuroscience. RSA is a general framework that utilizes RSMs to compare sets of measurements, such as neural representations. In the first step of this framework, RSMs with respect to an inner similarity measure \(s_{\text{in}}\) are computed. Since the RSMs are symmetric, their lower triangles can then be vectorized in a next step to vectors \(\mathsf{v}(\mathbf{S})\in\mathbb{R}^{\frac{N(N-1)}{2}}\). Finally, these vectors are compared by an outer similarity function \(s_{\text{out}}\), giving the following general representation of RSA similarity measures:

\[m_{\mathrm{RSA}}(\mathbf{R},\mathbf{R^{\prime}})=s_{\text{out}}(\mathsf{v}(\mathbf{S}), \mathsf{v}(\mathbf{S^{\prime}})), \tag{40}\]

This framework can be instantiated with various choices for the similarity functions \(s_{\text{in}}\) and \(s_{\text{out}}\). Kriegeskorte et al. [45] use Pearson correlation as inner similarity function \(s_{\text{in}}\) to compute the RSMs, and Spearman correlation as outer similarity function \(s_{\text{out}}\), since these correlation measures are also invariant to scaling and translations. As alternatives, Kriegeskorte et al. [45] further suggest measures such as Euclidean distance or Mahalanobis distance.

As noted before, the choice of both inner and outer similarity function determines the kind of transformations that the overall representational similarity measure is invariant to. Further, these functions also determine the range and interpretation of this measure.

#### 3.3.2 Centered Kernel Alignment

Kornblith et al. [33] propose Centered Kernel Alignment (CKA) [69, 70] to measure representational similarity. CKA uses the Hilbert-Schmidt Independence Criterion (HSIC) [71] to test statistical independence between the RSMs. The
similarity measure is defined as:

\[m_{\rm{CKA}}(\mathbf{R},\mathbf{R^{\prime}})=\frac{\rm{HSIC}(\mathbf{S},\mathbf{S^{\prime}})}{ \sqrt{\rm{HSIC}(\mathbf{S},\mathbf{S})\rm{HSIC}(\mathbf{S^{\prime}},\mathbf{S^{\prime}})}}, \tag{41}\]

where \(\rm{HSIC}(\mathbf{S},\mathbf{S^{\prime}})=\frac{1}{(N-1)^{2}}\,\rm{tr}(\mathbf{SHS^{\prime} }\mathbf{H})\), \(\mathbf{H}=\mathbf{I}-\frac{1}{N}\mathbf{1}\mathbf{1}^{\sf{T}}\) is a centering matrix, and \(\mathbf{1}\) is a vector of \(N\) ones. The denominator is introduced to scale CKA between zero and one, where a value of one indicates equivalent representations. Kornblith et al. [33] assume centered representations to compute RSMs, that is, all columns of the representation matrix have zero mean. The RSMs are computed with kernel functions. Specifically, Kornblith et al. [33] use the linear kernel and test the RBF kernel without reporting large differences in results. CKA with linear kernel has an alternative formulation that is more efficient if there are more neurons than inputs.

CKA is invariant to orthogonal transformations and isotropic scaling, assuming invariant similarity measures for RSM computation. If a kernel with hyperparameters, such as RBF, is used, then the hyperparameters must be selected dependent on the data. To be invariant to isotropic scaling, they select the bandwidth proportional to the median distance within each set of representations.

CKA with linear kernel is equivalent to the RV coefficient, a statistical measure to compare data matrices [72, 33]. Further, linear CKA is closely related to CCA and can be seen as an alternative weighting scheme of individual canonical correlations similar to PWCCA. The advantage is that linear CKA is symmetric and does not require a matrix decomposition to be computed [33].

#### 3.3.3 \(G_{\rm{ReLU}}\)-Cka

Similar to \(G_{\rm{ReLU}}\)-Procrustes, \(G_{\rm{ReLU}}\)-CKA is a representational similarity measure that is invariant to \(G_{\rm{ReLU}}\) transformations [42] (see Section 3.2.2). \(G_{\rm{ReLU}}\)-CKA can be understood as a model-specific instantiation of CKA.

To compute \(G_{\rm{ReLU}}\)-CKA, the representations are mean centered and then column-wise normalized by their column norm: \(\mathbf{\tilde{R}}=\mathbf{R}\mathbf{D}_{\mathbf{R}}^{-1}\), where \(\mathbf{D}_{\mathbf{R}}\) is the diagonal matrix of column norms of \(\mathbf{R}\). Then the RSMs are computed as \(\mathbf{S}_{i,j}=\max_{k}(\mathbf{\tilde{R}}_{i,k}\cdot\mathbf{\tilde{R}}_{j,k})\). The final score is computed using the standard CKA formulation, Equation (41), with an unbiased estimator of HSIC [73] in the sense that its expectation matches the value of HSIC if infinite amounts of data were available. Effectively, \(G_{\rm{ReLU}}\)-CKA is CKA with a maximum kernel [42]. \(G_{\rm{ReLU}}\)-CKA is invariant to \(G_{\rm{ReLU}}\), but otherwise inherits the properties of CKA.

#### 3.3.4 Riemannian Distance

This measure considers the special geometry of symmetric positive definite (SPD) matrices, which lie on a Riemannian manifold [e.g., 74]. Every inner product defined on a Riemannian manifold induces a distance metric that considers the special curvature of these structures. For the manifold of SPD matrices, such a metric is given by

\[m_{\rm{Riemann}}(\mathbf{R},\mathbf{R^{\prime}})=\sqrt{\sum_{i=1}^{N}\log^{2}( \lambda_{i})}, \tag{42}\]

where \(\lambda_{i}\) is the \(i\)-th eigenvalue of \(\mathbf{S}^{-1}\mathbf{S^{\prime}}\). Shahbazi et al. [46] have proposed this measure using RSMs defined as \(\mathbf{S}=\mathbf{R}\mathbf{R}^{\sf{T}}/D\). This matrix however can only be positive definite if \(D>N\), which limits applicability of this measure. Equivalence is indicated by a value of zero, and larger values indicate dissimilarity.

#### 3.3.5 Adaptive Geo-Topological Independence Criterion

Lin and Kriegeskorte [47] proposed this measure as an adaptation of distance correlation [75] that disregards the smallest and greatest distances in RSMs. This adaptation is motivated by the notion that short distances are often susceptible to noise, whereas the exact values of the longest distances may be somewhat arbitrary: whether two items are far or very far away from each other may be irrelevant to conclude that these items are not close to each other.

Distance correlation is a non-linear correlation measure that tests dependence of two random variables \(X\) and \(Y\) with finite mean. In the context of our survey, we consider the individual representations as samples of such random variables. To determine the distance correlation of two representation matrices \(\mathbf{R},\mathbf{R^{\prime}}\), one first computes the RSMs \(\mathbf{S},\mathbf{S^{\prime}}\) using Euclidean distance as similarity function \(s\). Next, these RSMs are double centered, i.e., for each RSM \(\mathbf{S}\) one computes the matrix \(\tilde{\mathbf{S}}\) via

\[\tilde{\mathbf{S}}_{i,j}:=\mathbf{S}_{i,j}-\frac{1}{N}\sum_{k=1}^{N}\mathbf{S}_{i,k}-\frac {1}{N}\sum_{k=1}^{N}\mathbf{S}_{k,j}+\frac{1}{N^{2}}\sum_{k=1}^{N}\sum_{n=1}^{N} \mathbf{S}_{k,n}. \tag{43}\]
Then the squared sample distance covariance of the RSMs \(\mathbf{S},\mathbf{S^{\prime}}\) can be computed via:

\[\mathrm{dCov}^{2}(\mathbf{S},\mathbf{S^{\prime}})=\frac{1}{N^{2}}\sum_{i=1}^{N}\sum_{j=1} ^{N}\tilde{\mathbf{S}}_{i,j}\tilde{\mathbf{S^{\prime}}}_{i,j}. \tag{44}\]

Finally, the squared distance correlation is defined in [75] as

\[m_{\mathrm{dCor}}^{2}(\mathbf{R},\mathbf{R^{\prime}})=\frac{\mathrm{dCov}^{2}(\mathbf{S}, \mathbf{S^{\prime}})}{\sqrt{\mathrm{dCov}^{2}(\mathbf{S},\mathbf{S})\,\mathrm{dCov}^{2}( \mathbf{S^{\prime}},\mathbf{S^{\prime}})}}. \tag{45}\]

A distance correlation of zero would then indicate statistical independence between the representations \(\mathbf{R}\) and \(\mathbf{R^{\prime}}\).

Lin and Kriegeskorte [47] modify this procedure by introducing a so-called "geo-topological" transformation function \(g_{l,u}:\mathbb{R}^{N\times N}\longrightarrow\mathbb{R}^{N\times N}\), which is a nonlinear monotonic transformation with two parameters \(u,l\), that will act on the elements of the RSMs. Letting \(\mathbf{S}_{\max}:=\max_{i,j}\mathbf{S}_{i,j}\) denote the maximum element of \(\mathbf{S}\), the transformation is defined as

\[(g_{l,u}(\mathbf{S}))_{i,j}=\begin{cases}0&\text{if }0\leq\mathbf{S}_{i,j}<l\\ \mathbf{S}_{\max}\cdot\frac{\mathbf{S}_{i,j}-l}{u-l}&\text{if }l\leq\mathbf{S}_{i,j}<u\\ \mathbf{S}_{\max}&\text{if }u\leq\mathbf{S}_{i,j}.\end{cases} \tag{46}\]

All distances below the lower threshold \(l\) are set to zero and all distances above the upper threshold \(u\) are set to the maximal distance. In between, the values are linearly interpolated.

Letting \(\mathrm{dCov}^{2}(\mathbf{S}):=\mathrm{dCov}^{2}(\mathbf{S},\mathbf{S})\), the Adaptive Geo-Topological Independence Criterion (AGTIC) is defined as

\[m_{\mathrm{AGTIC}}^{2}(\mathbf{R},\mathbf{R^{\prime}})=\max_{\tiny\begin{subarray}{ c}l,l^{\prime},u,u^{\prime}\\ u,l\leq u,l^{\prime}\leq u^{\prime}\end{subarray}}\frac{\mathrm{dCov}^{2}(g_{ l,u}(\mathbf{S}),g_{l^{\prime},u^{\prime}}(\mathbf{S^{\prime}}))}{\sqrt{\mathrm{dCov}^{2}(g_{ l,u}(\mathbf{S}))\,\mathrm{dCov}^{2}(g_{l^{\prime},u^{\prime}}(\mathbf{S^{\prime}}))}}. \tag{47}\]

Due to the usage of Euclidean distance as similarity function \(s\), AGTIC is invariant to orthogonal transformations and translations. As with distance correlation, AGTIC of zero indicates statistical independence.

Lin and Kriegeskorte [47] propose several variations of this measure, e.g., using percentile based cutoffs in the geo-topological transform.

#### 3.3.6 Normalized Bures Similarity

This measure has been inspired by the Bures distance, that has its roots in quantum information theory [76] and satisfies the properties of a distance metric on the space of positive semi-definite matrices [77]. As Tang et al. [48] apply linear kernel functions to compute the RSMs \(\mathbf{S},\mathbf{S^{\prime}}\), these matrices are by design positive semi-definite, and in consequence, these matrices also have a unique square root. Therefore, they can define the normalized Bures similarity as

\[m_{\text{NBS}}(\mathbf{R},\mathbf{R^{\prime}})=\frac{\mathrm{tr}(\mathbf{S}^{\frac{1}{2}} \mathbf{S^{\prime}}\mathbf{S}^{\frac{1}{2}})^{\frac{1}{2}}}{\sqrt{\mathrm{tr}(\mathbf{S}) \,\mathrm{tr}(\mathbf{S^{\prime}})}}. \tag{48}\]

This measure is bounded in the interval \([0,1]\), with \(m_{\text{NBS}}(\mathbf{R},\mathbf{R^{\prime}})=1\) indicating perfect similarity. Due to the application of linear kernels, it is invariant to orthogonal transformations, and further invariant to isotropic scaling due to the normalization.

#### 3.3.7 Representation Topology Divergence

The main idea of Representation Topology Divergence (RTD) [49] is to consider representations as graphs \(\mathcal{G}=(\mathcal{V},\mathcal{E})\), where each instance \(i\) corresponds to a node \(v_{i}\in\mathcal{V}\) with nodes forming an edge \((v_{i},v_{j})\in\mathcal{E}\) if the corresponding representations have a short distance to each other, and then to apply tools from algebraic topology to quantify differences between these graphs.

Specifically, based on the RSMs computed from Euclidean distance, for a given distance threshold \(\alpha>0\) they construct a graph \(\mathcal{G}^{\alpha}(\mathbf{R})\) with adjacency matrix \(\mathbf{A}\) defined as

\[\mathbf{A}_{i,j}=\begin{cases}\mathbf{S}_{i,j},&\text{if }\mathbf{S}_{i,j}<\alpha,\\ 0,&\text{else},\end{cases} \tag{49}\]
and a union graph \(\mathcal{G}^{\alpha}(\mathbf{R},\mathbf{R}^{\prime})\) with its adjacency matrix \(\mathbf{A}\) defined as

\[\mathbf{A}_{i,j}=\begin{cases}\min(\mathbf{S}_{i,j},\mathbf{S}^{\prime}_{i,j})&\text{if }\min(\mathbf{S}_{i,j},\mathbf{S}^{\prime}_{i,j})<\alpha,\\ 0,&\text{else}.\end{cases} \tag{50}\]

If the number of connected components in \(\mathcal{G}^{\alpha}(\mathbf{R})\) is different from the number of connected components in \(\mathcal{G}^{\alpha}(\mathbf{R},\mathbf{R}^{\prime})\), this is considered a topological discrepancy. For each specific discrepancy that occurs for varying values of \(\alpha\), the shortest corresponding interval (bar) \((\alpha_{1},\alpha_{2})\), for which this discrepancy persists, is collected in a set \(\mathcal{B}(\mathbf{R},\mathbf{R}^{\prime})\) that is denoted as _barcode_. These barcodes are then summarized by the total length of their intervals, denoted as

\[b(\mathbf{R},\mathbf{R}^{\prime})=\sum_{(\alpha_{1},\alpha_{2})\in\mathcal{B}(\mathbf{R}, \mathbf{R}^{\prime})}\alpha_{2}-\alpha_{1}, \tag{51}\]

which ultimately quantifies representational similarity between two models. Their final RTD measure is then constructed by subsampling \(K\) subsets \(\mathcal{I}^{(k)}\subset\{1,\dots,N\}\) of \(n:=|\mathcal{I}^{(k)}|<N\) instances each, and collecting the barcodes derived from representations \(\mathbf{R}^{(k)}=(\mathbf{R}_{i})_{i\in\mathcal{I}^{(k)}}\in\mathbb{R}^{n\times D}\), forming a measure

\[RTD(\mathbf{R},\mathbf{R}^{\prime})=\frac{1}{K}\sum_{i=1}^{K}b\big{(}\mathbf{R}^{(k)},\mathbf{ R}^{\prime(k)}\big{)}. \tag{52}\]

Because \(RTD\) is asymmetric, they ultimately propose to use the following symmetrized variant:

\[m_{\text{RTD}}(\mathbf{R},\mathbf{R}^{\prime})=\frac{1}{2}(RTD(\mathbf{R},\mathbf{R}^{\prime} )+RTD(\mathbf{R}^{\prime},\mathbf{R})). \tag{53}\]

Since each interval (bar) of the barcode represents a topological discrepancy, increasing values of RTD indicate stronger dissimilarity. If the representations are equivalent, each barcode will be an empty set, and thus it will hold that \(m_{\text{RTD}}(\mathbf{R},\mathbf{R}^{\prime})=0\).

Barannikov et al. [49] make the RTD invariant to isotropic scaling by normalizing the RSM \(\mathbf{S}\) by the ninetieth percentile of its values. Further, this measure is invariant to orthogonal transformations, since Euclidean distance is used to compute the RSMs. Regarding the choice of parameters, they suggest using \(n=10\) subsets of \(k=500\) representations each as default values.

### Neighborhood-Based Measures

The measures in this section compare the nearest neighbors of instances in the representation space. Thus, each of these measures first determine the \(k\) nearest neighbors of each instance representation \(\mathbf{R}_{i}\) in the full representation matrix \(\mathbf{R}\) with respect to a given similarity measure \(s\). Letting \(\mathbf{S}\) denote the corresponding RSM of representation \(\mathbf{R}\), and w.l.o.g. assuming that lower values indicate more similar representations, we formally define the set of the \(k\) nearest neighbors of the instance representation \(\mathbf{R}_{i}\) as the set \(\mathcal{N}_{\mathbf{R}}^{k}(i):=\mathcal{N}_{\mathbf{R}}^{k}(i,s)\subset\{j:1\leq j \leq N,j\neq i\}\) with \(|\mathcal{N}_{\mathbf{R}}^{k}(i)|=k\) for which it holds that \(\mathbf{S}_{i,j}<\mathbf{S}_{i,l}\) for all \(j\in\mathcal{N}_{\mathbf{R}}^{k}(i),l\not\in\mathcal{N}_{\mathbf{R}}^{k}(i)\cup\{i\}\).

Similar to previous groups of measures, the choice of the similarity function \(s\) directly determines which transformations a measure is invariant to.

For all the measures that we introduce in the following, the neighborhood size \(k\) is a parameter that has to be chosen for the application at hand.

#### 3.4.1 \(k\)-NN Jaccard Similarity

This measure considers how many of the \(k\) nearest neighbors each instance has in common over a given pair of representations. It computes a vector of the instance-wise Jaccard similarities \(v_{\text{lac}}^{k}\left(\mathbf{R},\mathbf{R}^{\prime}\right)\), where the \(i\)-th element, \(1\leq i\leq N\), corresponds to the instance representations \(\mathbf{R}_{i},\mathbf{R}^{\prime}_{i}\) and is defined as

\[\left(\mathbf{v}_{\text{lac}}^{k}\left(\mathbf{R},\mathbf{R}^{\prime}\right)\right)_{i}:= \frac{\left|\mathcal{N}_{\mathbf{R}}^{k}(i)\cap\mathcal{N}_{\mathbf{R}^{\prime}}^{k}( i)\right|}{\left|\mathcal{N}_{\mathbf{R}}^{k}(i)\cup\mathcal{N}_{\mathbf{R}^{\prime}}^{k}( i)\right|}. \tag{54}\]

Its values are then averaged to obtain the final similarity measure

\[m_{\text{lac}}^{k}(\mathbf{R},\mathbf{R}^{\prime}):=\frac{1}{N}\sum_{i=1}^{N}\left(v_{ \text{lac}}^{k}\left(\mathbf{R},\mathbf{R}^{\prime}\right)\right)_{i}. \tag{55}\]
In practice, cosine similarity is a common choice for a similarity function \(s\) to determine the nearest neighbors of each instance [37, 50]. Hryniowski and Wong [51] use the same approach under the name of nearest neighbor topological similarity with Euclidean distance to compute nearest neighbors. Gwilliam and Shrivastava [4] use this measure under the name of nearest-neighbor graph similarity.

Jaccard similarity is bounded in the interval \([0,1]\). A value of zero indicates distinct nearest-neighbor sets, hence dissimilarity, whereas as value of one indicates equivalent representations.

#### 3.4.2 Second-Order Cosine Similarity

This method has been proposed by Hamilton et al. [14] when analyzing changes in word embeddings over time. It compares the cosine similarities (28) of each instance \(\mathbf{R}_{i}\) to its \(k\) nearest neighbors with the corresponding cosine similarities of \(\mathbf{R}^{\prime}_{i}\) to its nearest neighbors in \(\mathbf{R}^{\prime}\). Formally, the union of the \(k\) nearest neighbors is computed as an ordered set \(\{j_{1},\ldots,j_{K(i)}\}:=\mathcal{N}^{k}_{\mathbf{R}}(i)\cup\mathcal{N}^{k}_{ \mathbf{R}^{\prime}}(i)\). Given these neighbors, the cosine similarity RSMs \(\mathbf{S},\mathbf{S}^{\prime}\) of the representations \(\mathbf{R},\mathbf{R}^{\prime}\) are utilized. The vector of second-order cosine similarities \(\mathbf{v}^{k}_{\text{2nd-cos}}\left(\mathbf{R},\mathbf{R}^{\prime}\right)\) can then be defined element-wise for \(1\leq i\leq N\) as

\[\left(\mathbf{v}^{k}_{\text{2nd-cos}}\left(\mathbf{R},\mathbf{R}^{\prime}\right)\right)_{ i}:=\quad\text{cos-sim}\left(\left(\mathbf{S}_{i,j_{1}},\ldots,\mathbf{S}_{i,j_{K(i)}} \right),\left(\mathbf{S}^{\prime}_{i,j_{1}},\ldots,\mathbf{S}^{\prime}_{i,j_{K(i)}} \right)\right).\]

Again, averaging the values of this vector yields the final similarity measure

\[m^{k}_{\text{2nd-cos}}(\mathbf{R},\mathbf{R}^{\prime}):=\frac{1}{N}\sum_{i=1}^{N} \left(\mathbf{v}^{k}_{\text{2nd-cos}}\left(\mathbf{R},\mathbf{R}^{\prime}\right)\right)_{ i}. \tag{56}\]

This measure is bounded in the interval \([0,1]\), with \(m^{k}_{\text{2nd-cos}}(\mathbf{R},\mathbf{R}^{\prime})=1\) indicating equivalence of \(\mathbf{R}\) and \(\mathbf{R}^{\prime}\).

Rather than considering the union of the neighborhoods \(\mathcal{N}^{k}_{\mathbf{R}}(i),\mathcal{N}^{k}_{\mathbf{R}^{\prime}}(i)\), Chen et al. [68] essentially compute this second-order similarity over the intersection of the top-\(k\) neighborhoods. Their approach is based on graph similarity [78]. Another similar approach was presented by Moschella et al. [79] where a random fixed set of reference instances is used instead of neighbors.

#### 3.4.3 Rank Similarity

The \(k\)-NN Jaccard similarity captures the extent to which two neighborhoods \(\mathcal{N}^{k}_{\mathbf{R}}(i),\mathcal{N}^{k}_{\mathbf{R}^{\prime}}(i)\) overlap, but not the order of the common neighbors with respect to the distance to their reference representations \(\mathbf{R}_{i},\mathbf{R}^{\prime}_{i}\). To also assign stronger weights to closer neighbors, Wang et al. [50] determine distance-based ranks \(r_{\mathbf{R}_{i}}(j)\) to all \(j\in\mathcal{N}^{k}_{\mathbf{R}}(i)\), where \(r_{\mathbf{R}_{i}}(j)=n\) if \(\mathbf{R}_{j}\) is the \(n\)-th closest neighbor of \(\mathbf{R}_{i}\) with respect to a given similarity measure \(s\). Based on these ranks, one defines the vector of instance-wise ranking similarity \(\mathbf{v}^{k}_{\mathrm{ranksim}}(\mathbf{R},\mathbf{R}^{\prime})\) as

\[\left(\mathbf{v}^{k}_{\mathrm{ranksim}}(\mathbf{R},\mathbf{R}^{\prime})\right)_{i}=\frac{ 1}{(\mathbf{v}_{\mathrm{max}})_{i}}\times\sum_{j\in\mathcal{N}^{k}_{\mathbf{R}}(i) \cap\mathcal{N}^{k}_{\mathbf{R}^{\prime}}(i)}\frac{2}{(1+|r_{\mathbf{R}_{i}}(j)-r_{\bm {R}^{\prime}_{i}}(j)|)(r_{\mathbf{R}_{i}}(j)+r_{\mathbf{R}^{\prime}_{i}}(j))}, \tag{57}\]

where \((\mathbf{v}_{\mathrm{max}})_{i}=\sum_{k=1}^{K}\frac{1}{k}\), with \(K=|\mathcal{N}^{k}_{\mathbf{R}}(i)\cap\mathcal{N}^{k}_{\mathbf{R}^{\prime}}(i)|\), is a normalization factor that limits the maximum of the ranking similarity to one. Intuitively, the first factor of the denominator in Equation (57) measures the similarity of the ranks of an instance, whereas the second factor assigns rank-based weights to this similarity, with lower-ranked instances gaining less influence on \(\left(\mathbf{v}^{k}_{\mathrm{ranksim}}\big{(}\mathbf{R},\mathbf{R}^{\prime}\big{)}\right)_{i}\). Based on the instance-wise values, a similarity score for the full representation can be determined by averaging:

\[m^{k}_{\mathrm{rank}}(\mathbf{R},\mathbf{R}^{\prime})=\frac{1}{N}\sum_{i=1}^{N}\left( \mathbf{v}^{k}_{\mathrm{ranksim}}(\mathbf{R},\mathbf{R}^{\prime})\right)_{i}. \tag{58}\]

It holds that \(m^{k}_{\mathrm{rank}}(\mathbf{R},\mathbf{R}^{\prime})\in(0,1]\), with \(m^{k}_{\mathrm{rank}}(\mathbf{R},\mathbf{R}^{\prime})=1\) indicating perfect similarity.

#### 3.4.4 Joint Rank and k-NN Jaccard Similarity

Rank similarity has the issue that it is only calculated on the intersection of the \(k\)-nearest neighbor sets in different representations. That means rank similarity might be high, even if the \(k\)-NN sets have almost no overlap. Similarly, Jaccard similarity might be high, but the order of the nearest neighbors might be completely different. Therefore, Wang et al. [50] combine these two approaches to calculate the _embedding stability_, by considering the product of Jaccard and
rank similarity. Thus, using the instance vectors defined in Equation (54) and Equation (57), we can define the vector of instance-wise similarities as

\[\left(\mathbf{v}_{\mathrm{Jac-Rank}}^{k}(\mathbf{R},\mathbf{R^{\prime}})\right)_{i}=\left( \mathbf{v}_{\mathrm{Jac}}^{k}\left(\mathbf{R},\mathbf{R^{\prime}}\right)\right)_{i}\times \left(\mathbf{v}_{\mathrm{ranksim}}^{k}(\mathbf{R},\mathbf{R^{\prime}})\right)_{i}. \tag{59}\]

Overall similarity, considering all instances, is then once again obtained by averaging all instances:

\[m_{\mathrm{Jac-Rank}}^{k}(\mathbf{R},\mathbf{R^{\prime}})=\frac{1}{N}\sum_{i=1}^{N} \left(\mathbf{v}_{\mathrm{Jac-Rank}}^{k}(\mathbf{R},\mathbf{R^{\prime}})\right)_{i}. \tag{60}\]

By the properties of \(k\)-NN Jaccard similarity and rank similarity, it follows that \(m_{\mathrm{Jac-Rank}}^{k}(\mathbf{R},\mathbf{R^{\prime}})\in[0,1]\), with \(m_{\mathrm{Jac-Rank}}^{k}(\mathbf{R},\mathbf{R^{\prime}})=1\) indicating perfect similarity.

### Descriptive Statistics

Measures of this category deviate from all previous measures in a way that they describe statistical properties of either (i) individual representations \(\mathbf{R}\), or (ii) measures of variance in the instance representations \(\mathbf{R}_{i}\) over sets of more than two representations. In case of (i), the similarity scores can be directly compared over pairs or sets of representations. For case (ii), one could aggregate or analyze the distribution of the instance-wise variations. While there are numerous statistics that could be used to compare representations, in the following we specifically outline statistics that have already been used to characterize representations in existing literature.

#### 3.5.1 Magnitude

Wang et al. [50] characterize magnitude as the Euclidean length of instance representations \(\mathbf{R}_{i}\). Based on this intuition, they consider the mean representation \(\bar{\mathbf{R}}:=\|\frac{1}{N}\sum_{i=1}^{N}\mathbf{R}_{i}\|\) and define its length as one statistic to characterize an individual representation \(\mathbf{R}\):

\[m_{\mathrm{Mag}}(\mathbf{R}):=\|\bar{\mathbf{R}}\|_{2}. \tag{61}\]

Aside from aggregating magnitude over all instances, they further propose a measure to quantify the variance of the magnitude of instance-wise representations over multiple models. More precisely, given a set of representations \(\mathbf{R}\), Wang et al. [50] measure the variance in the magnitudes of individual instances \(i\) as

\[m_{\mathrm{Var-Mag}}(\mathcal{R},i)=\frac{1}{\max_{\mathbf{R}\in\mathcal{R}}\| \mathbf{R}_{i}\|_{2}-\min_{\mathbf{R}\in\mathcal{R}}\|\mathbf{R}_{i}\|_{2}}\times\sqrt{ \frac{1}{|\mathcal{R}|}\sum_{\mathbf{R}\in\mathcal{R}}(\|\mathbf{R}_{i}\|_{2}-\bar{d}_ {i})}, \tag{62}\]

where \(\bar{d}_{i}=\frac{1}{|\mathcal{R}|}\sum_{\mathbf{R}\in\mathcal{R}}\|\mathbf{R}_{i}\|_{2}\) is the average magnitude of the representations of instance \(i\) in \(\mathcal{R}\).

As magnitude is unchanged by distance-preserving transformation, this statistic is invariant to orthogonal transformation and translation.

#### 3.5.2 Concentricity

Wang et al. [50] propose concentricity as a measure of the density of representations. This measure is also defined on instance level, with the concentricity of instance \(i\) in representation \(\mathbf{R}\) being defined as the cosine similarity (28) of the representation \(\mathbf{R}_{i}\) and the mean representation \(\bar{\mathbf{R}}\):

\[\alpha_{i}(\mathbf{R})=\mathrm{cos-sim}(\mathbf{R}_{i},\bar{\mathbf{R}}). \tag{63}\]

Similar to magnitude, Wang et al. [50] consider the mean concentricity

\[m_{\mathrm{mConc}}(\mathbf{R}):=\frac{1}{N}\sum_{i=1}^{N}\alpha_{i}(\mathbf{R}) \tag{64}\]

as a statistic for a single model, and measure the instance-wise variance of concentricity via

\[m_{\mathrm{Var-Conc}}(\mathcal{R},i)=\frac{1}{\max_{\mathbf{R}\in\mathcal{R}} \alpha_{i}(\mathbf{R})\,-\min_{\mathbf{R}\in\mathcal{R}}\alpha_{i}(\mathbf{R})}\times\sqrt {\frac{1}{|\mathcal{R}|}\sum_{\mathbf{R}\in\mathcal{R}}(\|\alpha_{i}(\mathbf{R})\|_{2} -\bar{\alpha}_{i})}, \tag{65}\]

where \(\bar{\alpha}_{i}=\frac{1}{|\mathcal{R}|}\sum_{\mathbf{R}\in\mathcal{R}}\|\alpha_{i} \|_{2}\) is the average magnitude of the representation of input \(\mathbf{X}_{i}\) in \(\mathcal{R}\).

Concentricity inherits from cosine similarity the invariances to orthogonal transformations and isotropic scaling.
#### 3.5.3 Uniformity

Another approach to measure density of representations is uniformity [52, 4]. Uniformity measures how close the distribution of individual representations is to a uniform distribution on the unit hypersphere, and is defined as

\[m_{\mathrm{uniformity}}(\mathbf{R})=\log\left(\frac{1}{N^{2}}\sum_{i=1}^{N}\sum_{j= 1}^{N}e^{-t\|\mathbf{R}_{i}-\mathbf{R}_{j}\|_{2}^{2}}\right), \tag{66}\]

where \(t\) is a hyperparameter that is set to \(t=2\) by Wang et al. [52] and Gwilliam and Shrivastava [4].

\(m_{\mathrm{uniformity}}(\mathbf{R})=0\) indicates perfectly uniform representations. Uniformity is invariant to orthogonal transformations and translation, as these transformations preserve distances.

#### 3.5.4 Tolerance

This statistic measures how close representations of semantically similar inputs are [53]. In contrast to all previous measures, it specifically requires a vector of ground-truth labels \(\mathbf{y}\in\mathbb{R}^{N}\). Tolerance is computed as the mean similarity of inputs with the same class:

\[m_{\mathrm{tol}}(\mathbf{R})=\frac{1}{N^{2}}\sum_{i=1}^{N}\sum_{j=1}^{N}(\mathbf{R}_{i }^{\mathsf{T}}\mathbf{R}_{j})\cdot\mathbb{1}\{\mathbf{y}_{i}=\mathbf{y}_{j}\}. \tag{67}\]

It is assumed that all instance representations are preprocessed to have unit norm, which effectively makes this measure invariant to scaling and the dot product equivalent to cosine similarity. The dot product is also invariant to orthogonal transformation.

#### 3.5.5 Modularity

Similar to the RTD measure (cf. Section 3.3.7), this measure is based on building a graph from the representations, or more precisely, their representational similarity matrices. It is also related to tolerance, as Lu et al. [54] propose to consider modularity as a measure that quantifies whether semantically similar inputs are close together in the graph, and consequently, the representation space. They specifically suggest connecting each node to its \(k\) nearest neighbors, using cosine similarity as a distance metric. Thus, letting \(\mathbf{S}\) denote the corresponding RSM, the adjacency matrix \(\mathbf{A}\in\mathbb{R}^{N\times N}\) of the resulting graph \(\mathcal{G}\) is then defined as

\[\mathbf{A}_{i,j}=\begin{cases}\mathbf{S}_{i,j},&\text{if }j\in\mathcal{N}_{\mathbf{R}}^{k}( i),\\ 0,&\text{otherwise}.\end{cases} \tag{68}\]

The modularity of the network [80], and in consequence the statistic for \(\mathbf{R}\), is then defined as

\[m_{\text{Mod}}(\mathbf{R})=\frac{1}{2W}\sum_{i,j}\left(\mathbf{A}_{i,j}-\frac{d_{i}d_ {j}}{W}\right)\cdot\mathbb{1}\{\mathbf{y}_{i}=\mathbf{y}_{j}\}, \tag{69}\]

where \(d_{i}=\sum_{j}\mathbf{A}_{i,j}\) denotes the effective degree of node \(v_{i}\), \(W=\sum_{i,j}\mathbf{A}_{i,j}\) is a normalization factor, and \(\mathbf{y}\) is the vector of ground-truth labels. The maximum modularity is given by 1, and high modularity implies that nodes of the same label are highly connected with each other, with only few connections to nodes of another label. Since the adjacency matrix of the graph is based on cosine similarity, \(m_{\text{Mod}}\) is invariant to orthogonal transformation.

#### 3.5.6 Neuron-RSM Modularity

A variant of modularity is also used by Lange et al. [55] to describe the structure of representations, or more precisely, the pattern of neuron activations. They also define the adjacency matrix of a representation graph in terms of RSMs, however, in their case they consider RSMs that describe similarity between neuron activations instead of instance representations. Specifically, they propose four different variants of RSMs that either consider pure neuron activations in specific layers, or also gradients with respect to neuron activations at specific layers. In that latter case, one may also consider the modularity based on such RSMs as a hybrid measure that assesses characteristics of representations \(\mathbf{R}\) with respect to functional behavior of a neural network.

The first RSM that they propose, which does not consider gradients, is defined as

\[\mathbf{S}_{k,l}=\frac{1}{N-1}\left|\sum_{i=1}^{N}(\mathbf{R}_{i,k}-\bar{\mathbf{R}}_{-,k} )(\mathbf{R}_{i,l}-\bar{\mathbf{R}}_{-,l})\right|, \tag{70}\]
where \(\bar{\mathbf{R}}_{-,k}\) is the mean activation of neuron \(k\). The other three RSMs consider gradients, with the first of three remaining variants considering the gradients of a representational layer with respect to the inputs \(\mathbf{X}_{i}\):

\[\mathbf{S}_{k,l}=\frac{1}{N}\left|\sum_{i=1}^{N}(\nabla_{\mathbf{X}_{i}}\mathbf{R}_{i,k})^{ \mathsf{T}}\nabla_{\mathbf{X}_{i}}\mathbf{R}_{i,l}\right|. \tag{71}\]

The second gradient-based RSM uses the gradient \(\nabla_{\mathbf{R}_{i}}\mathbf{O}_{i,c}\) of the outputs with respect to the neurons of a representational layer:

\[\mathbf{S}_{k,l}=\frac{1}{N}\left|\sum_{i=1}^{N}\sum_{c=1}^{C}\frac{\partial\mathbf{ O}_{i,c}}{\partial\mathbf{R}_{i,c}}\frac{\partial\mathbf{O}_{i,c}}{\partial\mathbf{R}_{i,l} }\right|. \tag{72}\]

Finally, the last RSM uses the Hessian of the loss \(\mathcal{L}\) with respect to the neuron activations of a given layer:

\[\mathbf{S}_{k,l}=\frac{1}{N}\left|\sum_{i=1}^{N}\frac{\partial^{2}\mathcal{L}}{ \partial\mathbf{R}_{i,k}\partial\mathbf{R}_{i,l}}\right|. \tag{73}\]

Once the RSM has been computed, Lange et al. [55] construct the adjacency matrix of the networks they want to compute modularity of via

\[\mathbf{A}_{i,j}=\begin{cases}\mathbf{S}_{i,j},&\text{if }i\neq j,\\ 0,&\text{otherwise}.\end{cases} \tag{74}\]

Unlike Lu et al. [54], they do not consider hard ground-truth labels to allocate nodes to clusters, but rather determine an optimal soft assignment of \(n\) clusters that maximizes modularity. Specifically, they try to find an optimal cluster assignment matrix \(\mathbf{C}\in\mathbb{R}^{D\times n}\), where each entry \(\mathbf{C}_{j,k}\in[0,1]\) determines the assignment of neuron \(j\in\{1,\dots,D\}\) to cluster \(k\in\{1,\dots,n\}\). These assignments have to be normalized such that \(\mathbf{C}\mathbf{1}_{n}=\mathbf{1}_{D}\), and the number of clusters \(n\leq D\) of neuron activations is a parameter that is to be optimized as well. Given a definition of clustering from Girvan and Newman [81], their neuron modularity is then defined as

\[m_{\mathrm{nMod}}(\mathbf{R})=\max_{\mathbf{C}}\mathrm{tr}(\mathbf{C}^{\mathsf{T}}\mathbf{ \tilde{A}}\mathbf{C})-\mathrm{tr}(\mathbf{C}^{\mathsf{T}}\mathbf{1}^{\mathsf{T}}\mathbf{1} \mathbf{\tilde{A}}\mathbf{C}), \tag{75}\]

where \(\mathbf{\tilde{A}}=\frac{1}{\mathbf{1}_{D}\mathbf{A}\mathbf{1}_{D}}\mathbf{A}\) is the normalized adjacency matrix. To determine the cluster assignment \(\mathbf{C}\), they provide an approximation method based on Newman's modularity maximization algorithm.

Lange et al. [55] also propose normalizing the RSMs before constructing the corresponding graphs, but did not observe big differences in the modularity of the corresponding graphs. Further, they experiment with computing the modularity of untrained models, using their initial weights, and find that the resulting modularity was similar to the modularity of trained models. This implies that these kinds of RSMs are not suitable to study training dynamics.

Generally, \(m_{\mathrm{nMod}}\) is invariant to permutations, since these effectively only relabel the nodes in the resulting graph.

## 4 Functional Similarity Measures

We now present functional similarity measures. These can be categorized into four main approaches: performance-based, hard prediction-based, soft prediction-based, and gradient-based measures. We show an overview of all measures in Section 4. Both hard prediction and soft prediction measures fundamentally measure agreement of models as their output is directly compared without an oracle reference such as human labels. We sometimes collectively call them agreement-based measures. These measures are related to prior literature on ensemble diversity [22, 23, 82] and inter-rater agreement [24, 26, 25].

All measures can easily be used on subsets of inputs, e.g., of specific classes, to gain more detailed insights into functional behavior.

### Performance-Based Measures

A popular view on functional similarity is that models are similar if they reach similar performance on some task (e.g., [38, 95, 39, 94]). This approach is easy to implement, as the comparison of models is reduced to comparing two scalar performance scores, such as accuracy. However, this simplification also obfuscates more nuanced differences in functional behavior, which cannot directly be captured with a single number.
Most commonly, given some quality function \(q\) that evaluates the performance of a model, the (absolute) difference in performance is used for similarity:

\[m_{\mathrm{Perf}}(\mathbf{O},\mathbf{O^{\prime}})=|q(\mathbf{O})-q(\mathbf{O^{\prime}})|. \tag{76}\]

While this measure is symmetric, we can also define an asymmetric measure by leaving out the absolute value.

Although accuracy is the most commonly used quality function in literature [38, 95, 39, 94], other performance metrics such as F1 score [96] are suitable, too. However, choosing performance metrics that capture relevant aspects of functional behavior is non-trivial, as highlighted in the survey on performance metrics for vision tasks by Reinke et al. [97].

### Hard Prediction-Based Measures

The measures in this section quantify functional similarity by comparing hard predictions on instance-level. Each of the measure of this category will report high similarity for two models if their hard predictions agree for most inputs, regardless of whether these predictions are correct or not. Aspects like the prediction confidence are not considered.

In the following, we explain hard prediction-based measures in machine learning.

#### 4.2.1 Disagreement

Disagreement, also known as churn [7], jitter [8], or Hamming prediction differences [84], is the expected rate of conflicting hard predictions over inputs and models [98, 83]. Due to its simplicity, it is a particularly popular measure for functional similarity. Formally, disagreement between two models is defined as

\[m_{\mathrm{Dis}}(\mathbf{O},\mathbf{O^{\prime}})=\frac{1}{N}\sum_{i=1}^{N}\mathbb{1} \,\{\arg\max_{j}\mathbf{O}_{i,j}\neq\arg\max_{j}\mathbf{O^{\prime}}_{i,j}\}. \tag{77}\]

The measure is bounded in the interval \([0,1]\), with a disagreement of one indicating completely distinct functional behavior, and a disagreement of zero indicating perfect agreement in hard predictions. In practice, this range is however bounded by model quality, with high disagreement being impossible if the compared models are both very accurate. In that context, Bhojanapalli et al. [89] further present theoretical bounds on disagreement in terms of the confidence in the predictions that is encoded via the soft predictions. They show that disagreement can be expected to be low when the confidence in the predictions is high, or when the soft predictions of the compared models are very similar.

_Transferred discrepancy_ used disagreement of linear classifiers trained on intermediate representations as a proxy for representational similarity [99].

\begin{table}
\begin{tabular}{l|l c c c c} \hline \hline Type & Measure & Groupwise & Blackbox Access & Labels Required & Similarity \(\uparrow\) \\ \hline \multirow{2}{*}{Performance} & Performance Difference & ✗ & ✓ & ✓ & ✗ \\ \hline \multirow{6}{*}{Hard Prediction} & Disagreement [83, 7, 8, 84] & ✗ & ✓ & ✗ & ✗ \\ \cline{2-5}  & Error-Corrected Disagreement [85] & ✗ & ✓ & ✓ & ✗ \\ \cline{2-5}  & Minmax-normalized Disagreement [9] & ✗ & ✓ & ✓ & ✗ \\ \cline{2-5}  & Cohen’s Kappa [86] & ✗ & ✓ & ✗ & ✓ \\ \cline{2-5}  & Fleiss Kappa [87] & ✓ & ✓ & ✗ & ✓ \\ \cline{2-5}  & Ambiguity [37, 88] & ✓ & ✓ & ✗ & ✗ \\ \cline{2-5}  & Discrepancy [88] & ✓ & ✓ & ✗ & ✗ \\ \hline \multirow{3}{*}{Soft Prediction} & Surrogate Churn [89] & ✗ & ✓ & ✗ & ✗ \\ \cline{2-5}  & Jensen-Shannon Divergence [90] & ✗ & ✓ & ✗ & ✗ \\ \cline{1-1} \cline{2-5}  & Prediction Difference [84] & ✓ & ✓ & ✗ & ✗ \\ \cline{1-1} \cline{2-5}  & Rashomon Capacity [91] & ✓ & ✓ & ✗ & ✗ \\ \hline \multirow{3}{*}{Gradient} & ModelDiff [92] & ✗ & ✗ & ✓ & ✓ \\ \cline{2-5}  & Adversarial Transferability [93] & ✗ & ✗ & ✓ & ✓ \\ \cline{1-1} \cline{2-5}  & Saliency Map Similarity [5] & ✗ & ✗ & ✓ & ✓ \\ \hline Stitching & Performance Difference [94, 39, 95] & ✗ & ✗ & ✓ & ✗ \\ \hline \hline \end{tabular}
\end{table}
Table 2: Overview of functional similarity measures.
#### 4.2.2 Error-Corrected Disagreement

As the range of possible disagreement values is dependent on the accuracy of the compared models, Fort et al. [85] propose to correct for this influence by dividing disagreement by the error rate \(q_{\mathrm{Err}}\) (Eq. (11)) of one of the models:

\[m_{\mathrm{ErrCorDis}}(\mathbf{O},\mathbf{O^{\prime}})=\frac{m_{\mathrm{Dis}}(\mathbf{O}, \mathbf{O^{\prime}})}{q_{\mathrm{Err}}(\mathbf{O})}. \tag{78}\]

By design, this measure is not symmetric since the error rates of the outputs \(\mathbf{O},\mathbf{O^{\prime}}\) may vary. A normalized disagreement of zero indicates perfect agreement, whereas the upper limit is dependent on the error rate - exact limits are provided by Fort et al. [85], which helps to contextualize the similarity scores that are obtained.

A normalized variant of this measure, which also provides symmetry, has been used by Klabunde and Lemmerich [9]. In their _Min-Max-normalized disagreement_ measure, they relate the obtained disagreement \(m_{\mathrm{Dis}}\) to the minimum and maximum possible disagreement, which are given by

\[m_{\mathrm{Dis}}^{(\min)}(\mathbf{O},\mathbf{O^{\prime}})=|q_{\mathrm{Err}}(\mathbf{O})- q_{\mathrm{Err}}(\mathbf{O^{\prime}})|\quad\text{and}\quad m_{\mathrm{Dis}}^{( \max)}(\mathbf{O},\mathbf{O^{\prime}})=\min(q_{\mathrm{Err}}(\mathbf{O})+q_{\mathrm{Err}} (\mathbf{O^{\prime}}),1), \tag{79}\]

respectively. Based on these values, their measure is defined as

\[m_{\mathrm{MinMaxNormDis}}(\mathbf{O},\mathbf{O^{\prime}})=\frac{m_{\mathrm{Dis}}( \mathbf{O},\mathbf{O^{\prime}})-m_{\mathrm{Dis}}^{(\min)}(\mathbf{O},\mathbf{O^{\prime}})}{ m_{\mathrm{Dis}}^{(\max)}(\mathbf{O},\mathbf{O^{\prime}})-m_{\mathrm{Dis}}^{(\min)}( \mathbf{O},\mathbf{O^{\prime}})}. \tag{80}\]

This measure is bounded in the interval \([0,1]\), with \(m_{\mathrm{MinMaxNormDis}}(\mathbf{O},\mathbf{O^{\prime}})=0\) indicating perfect agreement between the models.

#### 4.2.3 Chance-Corrected Disagreement

Rather than correcting for accuracy of models, chance corrected disagreement measures correct for the rate of agreement that two or more classification models are expected to have by chance. The probably most prominent measures that follow this rationale are _Cohen's Kappa_[86] and Fleiss's Kappa [87], which were historically introduced as measures for inter-rater agreement, and can also be used to evaluate similarity in machine learning models to measure functional similarity [28, 100].

Both measures assume that the outputs that they are comparing are statistically independent, with the main difference being that Cohen's Kappa can only compare a pair of model outputs. Given a pair of models \(f,f^{\prime}\) with corresponding outputs \(\mathbf{O},\mathbf{O^{\prime}}\), and letting \(k_{c}=\sum_{i=1}^{N}\mathbbm{1}\{\arg\max_{j}\mathbf{O}_{i,j}=c\}\) denote the absolute amount of times that class \(c\) is predicted by model \(f\), the expected agreement rate of such models is given by \(p_{e}=\frac{1}{N^{2}}\sum_{c=1}^{C}k_{c}k_{c}^{\prime}\). Based on these values, Cohen's Kappa is defined as

\[m_{\text{Cohen}}(\mathbf{O},\mathbf{O^{\prime}})=1-\frac{m_{\mathrm{Dis}}(\mathbf{O},\mathbf{ O^{\prime}})}{1-p_{e}}=\frac{p_{o}-p_{e}}{1-p_{e}}, \tag{81}\]

where \(p_{o}=1-m_{\mathrm{Dis}}(\mathbf{O},\mathbf{O^{\prime}})\) denotes the observed agreement. When \(m_{\text{Cohen}}(\mathbf{O},\mathbf{O^{\prime}})=1\), perfect agreement of the models is indicated, a value \(m_{\text{Cohen}}(\mathbf{O},\mathbf{O^{\prime}})<0\) indicates less agreement than expected by chance. Interpreting Cohen's Kappa is non-trivial as Kappa values are influenced by accuracy of the models, number of classes, and class imbalance [101, 102] - with the latter issue being quite prevalent in application scenarios.

Fleiss's Kappa [87] can be seen as an extension of Cohen's Kappa to settings with multiple classification models. Letting \(k_{ic}=\sum_{\mathbf{O}\in\mathcal{O}}\mathbbm{1}\{\arg\max_{j}\mathbf{O}_{i,j}=c\}\) denote the number of times instance \(i\) is predicted as class \(c\), and \(p_{c}=\frac{1}{TN}\sum_{i=1}^{N}k_{ic}\) denote the share of class \(c\) over all predictions, the expected agreement over all models is given by \(\bar{P}_{e}=\sum_{c=1}^{C}p_{c}^{2}\). This expected agreement over all models is then related to the actual agreement \(\bar{P}=\frac{1}{N}\sum_{i=1}^{N}P_{i}\), where \(P_{i}=\frac{2}{T(T-1)}\sum_{c=1}^{C}\frac{k_{ic}(k_{ic}-1)}{2}\), \(i\in\{1,\dots,N\}\) denotes the actual instance-wise agreements.

Finally, the definition of the measure is similar to Cohen's Kappa:

\[m_{\text{Fleiss}}(\mathbf{O},\mathbf{O^{\prime}})=\frac{\bar{P}-\bar{P}_{e}}{1-\bar{P} _{e}}. \tag{82}\]

Similar to Cohen's Kappa, \(m_{\text{Fleiss}}(\mathbf{O},\mathbf{O^{\prime}})=1\) indicates perfect agreement of outputs, and values lower than zero indicate less agreement than expected by chance.

Aside from these two measures, there are several more measures that also correct for agreement by chance when measuring agreement. Cohen [103] proposes a weighted variant of his Kappa measure that assigns weights to different
kinds of disagreement - for example, in ordinal classification, disagreement between similar classes may weigh less compared to disagreement of more distinct classes. Conger [104] and Davies and Fleiss [105] propose alternatives to Fleiss' Kappa that relax the assumption of identical marginal prediction distributions to compute expected agreement. Krippendorff's Alpha [106] can be used as a generalization of several agreement measures.

#### 4.2.4 Groupwise Disagreement

Disagreement cannot identify commonalities across a whole set of models, as pairwise similarity of models does not imply groupwise similarity. The two following measures extend disagreement to identify functional similarity across sets of models.

Ambiguity [88], also called _linear prediction overlap_[4], is the share of instances that receive conflicting predictions by _any_ pair of models out of a given set of models. Ambiguity is defined formally as follows:

\[m_{\text{Ambiguity}}(\mathcal{O})=\frac{1}{N}\sum_{i=1}^{N}\max_{ \begin{subarray}{c}\boldsymbol{O},\boldsymbol{O}^{\prime}\in\mathcal{O}\\ s.t.\ \boldsymbol{O}\not=\boldsymbol{O}^{\prime}\end{subarray}}1\{\arg\max_{j} \boldsymbol{O}_{i,j}\neq\arg\max_{j}\boldsymbol{O}^{\prime}_{i,j}\}. \tag{83}\]

If all outputs have perfect agreement, it holds that \(m_{\text{Ambiguity}}(\mathcal{O})=0\). Marx et al. [88] originally proposed ambiguity to measure multiplicity of models with similar performance. Hence, in this formulation, one model was fixed for all comparisons and the maximum was taken over the set of models that have similar loss.

The counterpart to ambiguity is the _stable core_ measure proposed by Schumacher et al. [37], which counts the share of instances with consistent predictions, and can be defined as

\[m_{\text{StableCore}}(\mathcal{O})=1-m_{\text{Ambiguity}}(\mathcal{O}). \tag{84}\]

They also consider a relaxation of the stable core, where an instance was only required to obtain the same prediction by a fixed proportion of models to be considered stable, which they set to 90%.

Discrepancy [88] is defined as the maximum disagreement between two classifiers from a bigger set of models:

\[m_{\text{Discrepancy}}(\mathcal{O})=\max_{\begin{subarray}{c}\boldsymbol{O}, \boldsymbol{O}^{\prime}\in\mathcal{O}\\ s.t.\ \boldsymbol{O}\not=\boldsymbol{O}^{\prime}\end{subarray}}\frac{1}{N}\sum_{i=1}^{ N}\mathbb{1}\{\arg\max_{j}\boldsymbol{O}_{i,j}\neq\arg\max_{j}\boldsymbol{O}^{ \prime}_{i,j}\}. \tag{85}\]

Similar to ambiguity, Marx et al. [88] proposed discrepancy to measure model multiplicity. Again, one model was fixed for all comparisons and the maximum was taken over the set of models that have similar loss.

### Soft Prediction-Based Measures

This group of measures specifically compares soft prediction outputs, such as class-wise probabilities or scores from decision functions. Intuitively, this provides more nuance to the notion of similarity in outputs, since we can consider differences in confidence of individual predictions. The impact of confidence is specifically exemplified by cases where scores are close to the decision boundary. Even a minimal change in scores may cause a different classification in one case, whereas scores would need to change drastically for a different classification in another case. The following measures are thus particularly sensitive to such cases.

#### 4.3.1 Surrogate Churn

Bhojanapalli et al. [89] propose _surrogate churn_ (SChurn) as a relaxed version of disagreement, that takes into account the distribution of the soft predictions. For \(\alpha>0\), it is defined as

\[m_{\text{SChurn}}^{\alpha}(\boldsymbol{O},\boldsymbol{O^{\prime}})=\frac{1}{2N }\sum_{i=1}^{N}\left\|\left(\frac{\boldsymbol{O}_{i}}{\max_{c}\boldsymbol{O}_{ i,c}}\right)^{\alpha}-\left(\frac{\boldsymbol{O}^{\prime}_{i}}{\max_{c} \boldsymbol{O}^{\prime}_{i,c}}\right)^{\alpha}\right\|_{1}. \tag{86}\]

A value \(m_{\text{SChurn}}^{\alpha}(\boldsymbol{O},\boldsymbol{O^{\prime}})=0\) indicates perfect agreement of outputs. The authors show that when \(\alpha\to\infty\), this measure is equivalent to standard disagreement (cf. Sec. 4.2.1), and use \(\alpha=1\) as the default value.

#### 4.3.2 Jensen-Shannon Divergence

When soft predictions are specifically modelling class probabilities, several divergence measures for probability distributions could be applied to measure the difference between instance-level predictions. A popular choice of
measure for that case is Jensen-Shannon Divergence (JSD) [90], which, to measure functional similarity, is applied on every instance and then averaged [100, 85]. Thus, letting \(\mathrm{KL}(\cdot\|\cdot)\) denote the Kullback-Leibler divergence [107], this measure is defined as

\[m_{\text{JSD}}(\mathbf{O},\mathbf{O^{\prime}})=\frac{1}{2N}\sum_{i=1}^{N}\mathrm{KL}( \mathbf{O}_{i}\|\mathbf{O}_{i}^{\prime})+\mathrm{KL}(\mathbf{O}_{i}^{\prime}\|\mathbf{O}_{i}). \tag{87}\]

Equality of outputs is given when \(m_{\text{JSD}}(\mathbf{O},\mathbf{O^{\prime}})=0\), and higher values indicate more dissimilarity.

As noted above, several divergence measures could be applied to measure similarity of probability distributions. A comprehensive overview of such divergence measures is given by Cha [108].

#### 4.3.3 Prediction Difference

Shamir and Coviello [84] specifically consider differences in predictions over more than two models. Their _prediction difference_ (PD) intuitively quantifies variance in model predictions. Letting \(\mathbf{\bar{O}}=\frac{1}{|\mathcal{O}|}\sum_{\mathbf{O}\in\mathcal{O}}\mathbf{O}\) denote the average output matrix, their standard prediction difference measure aggregates instance-wise deviations from the average output in terms of a \(p\)-norm:

\[m_{\text{PD}}^{p}(\mathcal{O})=\frac{1}{N}\sum_{i=1}^{N}\frac{1}{|\mathcal{O} |}\sum_{\mathbf{O}\in\mathcal{O}}\|\mathbf{O}_{i}-\mathbf{\bar{O}}_{i}\|_{p}. \tag{88}\]

Shamir and Coviello [84] use \(p=1\) for interpretable differences of probability distributions. \(m_{\text{PD}}^{p}(\mathcal{O})=0\) indicates identical outputs of all models and is therefore not achieved in practical settings. Other than that, higher PD indicates higher dissimilarity between the compared models.

Next to norm-based prediction difference, Shamir and Coviello [84] further propose measures that relate the variance in the outputs to their average magnitude. That way, differences on low-confidence predictions are penalized stronger. Relative prediction difference is defined as

\[m_{\text{Rel-PD}}(\mathcal{O})=\frac{1}{N}\sum_{i=1}^{N}\frac{1}{|\mathcal{O} |}\sum_{\mathbf{O}\in\mathcal{O}}\left[\sum_{c=1}^{C}\frac{|\mathbf{O}_{i,c}-\mathbf{ \bar{O}}_{i,c}|}{\mathbf{\bar{O}}_{i,c}}\right]. \tag{89}\]

When ground-truth labels \(\mathbf{y}\) are given, Shamir and Coviello [84] further propose to specifically focus on discrepancies in the confidence of the predictions of the true labels \(\mathbf{y}_{i}\). The corresponding measure is defined as

\[m_{\text{Rel-True-PD}}(\mathcal{O})=\frac{1}{N}\sum_{i=1}^{N}\frac{1}{| \mathcal{O}|}\sum_{\mathbf{O}\in\mathcal{O}}\left[\frac{|\mathbf{O}_{i,\mathbf{y}_{i}}- \mathbf{\bar{O}}_{i,\mathbf{y}_{i}}|}{\mathbf{\bar{O}}_{i,\mathbf{y}_{i}}}\right]. \tag{90}\]

As for the standard prediction difference, both of these variants can only achieve values of zero when all models make identical predictions, and higher values indicate more dissimilarity.

#### 4.3.4 Rashomon Capacity

Rashomon Capacity (RC) [91] measures multiplicity in predictions on individual instances. It is rooted in information theory and has been proposed to specifically measure differences in distinct models that have very similar loss, so-called Rashomon sets. However, it can also be used to measure similarity of a general set of outputs \(\mathbf{O}\). Formally, letting \(P_{\mathbf{O}}\) denote a probability distribution over the set of outputs \(\mathcal{O}\), and \(\Delta_{C}\) the probability simplex (10), the output spread given this distribution is defined as

\[\inf_{\mathbf{p}\in\Delta_{C}}\mathbb{E}_{\mathbf{O}\sim P_{\mathcal{O}}}\mathrm{KL}( \mathbf{O}_{i}\|\mathbf{p}), \tag{91}\]

where \(\mathbf{p}\in\Delta_{C}\) is a reference distribution that is optimized to minimize distances to all outputs. The maximum spread, or _channel capacity_, over the outputs is then determined over all probability distributions on the outputs:

\[\mathrm{Capacity}(\mathcal{O},i)=\sup_{P_{\mathbf{O}}}\inf_{\mathbf{p}\in\Delta_{C}} \mathbb{E}_{\mathbf{O}\sim P_{\mathbf{O}}}\mathrm{KL}(\mathbf{O}_{i}\|\mathbf{p}). \tag{92}\]

Finally, the Rashomon Capacity over instance \(i\) is defined as

\[m_{\text{RC}}(\mathcal{O},i)=2^{\mathrm{Capacity}(\mathcal{O},i)}. \tag{93}\]

To approximate the Rashomon capacity of an instance, Hsu and Calmon [91] suggest using the Blahut-Arimoto algorithm [109, 110]. A similarity measure over all instances can be obtained by aggregation, such as taking the mean value.

It holds that \(m_{\text{RC}}(\mathcal{O},i)\in[1,C]\) with \(m_{\text{RC}}(\mathcal{O},i)=1\) if and only if all outputs are identical, and \(m_{\text{RC}}(\mathcal{O},i)=C\) if and only if every class is predicted once with perfect confidence. Further, the measure is monotonous, i.e., it holds that \(m_{\text{RC}}(\mathcal{O}^{\prime},i)\leq m_{\text{RC}}(\mathcal{O},i)\) for all \(\mathcal{O}^{\prime}\subseteq\mathcal{O}\).
### Gradient-Based Measures

The measures in this section use model gradients to characterize similarity. A core assumption of these methods is that similar models have similar gradients. This means that, for instance, adversarial examples created for one model should lead to similar effects in another model if they are similar.

#### 4.4.1 ModelDiff

In their _ModelDiff_ measure, Li et al. [92] use adversarial examples from perturbation attacks to characterize decision regions, which can then be compared across two models. Given a model \(f\), they first create adversarial examples \(\mathbf{\tilde{X}}_{i}\) for every input \(\mathbf{X}_{i}\), by adding noise to these inputs that steer the model away from a correct prediction. Such examples can be determined by methods such as projected gradient descent [111]. The difference between the original soft predictions \(\mathbf{O}_{i}=f(\mathbf{X}_{i})\) and the output of the adversarial example \(\tilde{\mathbf{O}}_{i}=f(\mathbf{\tilde{X}}_{i})\) is collected in a _decision distance vector_\(\mathbf{v}_{\mathrm{DDV}}\) (DDV). This difference is computed in terms of cosine similarity (28):

\[\big{(}\mathbf{v}_{\mathrm{DDV}}(\mathbf{O},\tilde{\mathbf{O}})\big{)}_{i}=\mathrm{cos-sim} (\mathbf{O}_{i},\tilde{\mathbf{O}}_{i}). \tag{94}\]

Finally, the DDVs of different models are compared, with the outputs \(\tilde{\mathbf{O}}_{i}^{\prime}=f^{\prime}(\mathbf{\tilde{X}}_{i})\) being computed from the same adversarial examples \(\mathbf{\tilde{X}}_{i}\), and again using cosine similarity:

\[m_{\text{ModelDiff}}(\mathbf{O},\mathbf{O}^{\prime})=\mathrm{cos-sim}(\mathbf{v}_{\mathrm{ DDV}}(\mathbf{O},\tilde{\mathbf{O}}),\mathbf{v}_{\mathrm{DDV}}(\mathbf{O}^{\prime},\tilde{\mathbf{O}}^{ \prime})). \tag{95}\]

A similarity score \(m_{\text{ModelDiff}}(\mathbf{O},\mathbf{O}^{\prime})=1\) indicates that both models are equivalent in their outputs. Since this measure uses adversarial examples of only one of the models, it is not symmetric. This asymmetry is rooted in the fact that ModelDiff was developed to identify (unauthorized) model reuse. In this scenario, access to a third-party model might be restricted and thus the generation of adversarial examples could become infeasible. Li et al. [92] also propose an approach to find adversarial examples without white-box access to either model.

#### 4.4.2 Adversarial Transferability

Similar to ModelDiff, Hwang et al. [93] measure the similarity of networks in terms of the transferability of adversarial attacks: if both models are susceptible to the same adversarial examples, then the networks are considered similar. Given two networks \(f,f^{\prime}\), for each input \(\mathbf{X}_{i}\) that is predicted correctly by both networks, a pair of corresponding adversarial examples \(\mathbf{\tilde{X}}_{i}\), \(\mathbf{\tilde{X}}_{i}^{\prime}\) is generated with projected gradient descent [111]. These adversarial examples are then fed into the opposite model, yielding outputs \(\tilde{\mathbf{O}}_{i}=f(\mathbf{\tilde{X}}_{i}^{\prime})\) and \(\tilde{\mathbf{O}}^{\prime}_{i}=f^{\prime}(\mathbf{\tilde{X}}_{i})\), for which it is then determined how often both are incorrect. Thus, given the vector of ground-truth labels \(\mathbf{y}\), and letting

\[\mathcal{X}_{\text{true}}:=\{i|\arg\max_{j}\mathbf{O}_{i,j}=\arg\max_{j}\mathbf{O}_{i,j}^{\prime}=\mathbf{y}_{i}\} \tag{96}\]

denote the set of instances that were predicted correctly by both models, Hwang et al. [93] define their measure as

\[m_{\text{AdvTrans}}(\tilde{\mathbf{O}},\tilde{\mathbf{O}}^{\prime})=\log\bigg{[}\max \bigg{\{}\varepsilon,\frac{100}{2\left|\mathcal{X}_{\text{true}}\right|}\sum_ {i\in\mathcal{X}_{\text{true}}}\big{(}\mathbb{1}(\arg\max_{j}\tilde{\mathbf{O}}_{ i,j}\neq\mathbf{y}_{i})+\mathbb{1}(\arg\max_{j}\tilde{\mathbf{O}}_{i,j}^{\prime}\neq\mathbf{y}_{i})) \bigg{\}}\bigg{]}, \tag{97}\]

where \(\varepsilon>0\) is introduced to avoid \(\log(0)\). A value of \(m_{\text{AdvTrans}}(\tilde{\mathbf{O}},\tilde{\mathbf{O}}^{\prime})=\log(100)\) indicates perfect model similarity, whereas \(m_{\text{AdvTrans}}(\tilde{\mathbf{O}},\tilde{\mathbf{O}}^{\prime})=\log(\varepsilon)\) indicates complete disagreement.

#### 4.4.3 Cosine Similarity of Saliency Maps

When investigating the relationship between model similarity and robustness, among other methods, Jones et al. [5] apply a direct approach to compare models in terms of their gradients. More precisely, they compute the cosine similarity (28) between (vectorized) saliency maps [112]. Saliency maps have been introduced in the context of image classification, where they conceptually displayed the influence of each pixel in an input image on the predictions of specific classes. More generally, these saliency maps can be depicted as the gradient of individual predictions \(\mathbf{O}_{i,c}\) with respect to their corresponding inputs \(\mathbf{X}_{i}\). Based on such individual gradients, a similarity score can be computed by aggregation:

\[m_{\text{SaliencyMap}}(\mathbf{O},\mathbf{O}^{\prime})=\frac{1}{nC}\sum_{i=1}^{N}\sum _{c=1}^{C}\mathrm{cos-sim}\left(|\nabla_{\mathbf{X}_{i}}\mathbf{O}_{i,c}|,|\nabla_{ \mathbf{X}_{i}}\mathbf{O}_{i,c}^{\prime}|\right), \tag{98}\]

where the absolute value \(|\cdot|\) is applied element-wise. A value \(m_{\text{SaliencyMap}}(\mathbf{O},\mathbf{O}^{\prime})=1\) indicates perfect similarity, with lower values indicating stronger differences between models.
### Stitching-Based Measures

The intuition behind _stitching_ is that similar models should be similar in their internal processes and, thus, swapping layers between such models should not result in big differences in the outputs if a layer that converts representations is introduced [95, 39, 94]. Given two models \(f,f^{\prime}\), stitching consists of training a _stitching layer_ (or network) \(g\) to convert representations \(\mathbf{R}\) of model \(f\) at layer \(l\) into representations \(\mathbf{R^{\prime}}\) of model \(f^{\prime}\) at some layer \(l^{\prime}\). One then considers the composed model

\[\tilde{f}:=f^{\prime(L^{\prime})}\circ\cdots\circ f^{\prime(l^{\prime}+1)}\circ f ^{\prime(l^{\prime})}\circ g\circ f^{(l)}\circ f^{(l-1)}\circ\cdots\circ f^{( 1)}, \tag{99}\]

which uses the bottom-most layers of \(f\) and the top-most layers of \(f^{\prime}\), and compares its performance with the original models. In existing literature, the most prevalent approach for that is to directly compare their performance in terms of a quality function \(q\) such as accuracy [39, 94]. This yields a measure

\[m_{\text{stitch}}(\tilde{\mathbf{O}},\mathbf{O^{\prime}})=q(\tilde{\mathbf{O}})-q(\mathbf{O^{ \prime}}), \tag{100}\]

where \(\tilde{\mathbf{O}}=\tilde{f}(\mathbf{X})\) is the output of the stitched model. However, we point out that other functional similarity measures may be suitable to identify more fine-grained similarity. Although stitching operates on representations, we classify it as a functional similarity measure because, in the end, one only considers differences in the outputs.

Both the design and the placement of the stitching layer are of high importance to obtain proper assessments of model similarity. Regarding design, Bansal et al. [94] generally choose stitching layers such that the architecture of the composed model is consistent with the architectures of the stitched models. For instance, they use a token-wise linear function between transformer blocks to stitch transformers. For CNNs, 1x1 convolutions are generally used in stitching layers [95, 39, 94]. Csiszarik et al. [39] further try orthogonal transformations, linear transformations, and low-rank linear transformations, to directly convert between representations, though SGD-trained stitching layers outperform these options.

To train the stitching layers, one typically freezes parameters of the stitched models and only optimizes the weights of the stitching layer via backpropagation, using ground truth labels or the output of \(f^{\prime}\) as soft labels [39, 94, 95]. To aid the optimization, Bansal et al. [94] suggest adding BatchNorm layers before and after the stitching layer. In case the stitching layer constitutes a simple linear transformation \(\mathbf{T}\), its weights can be directly computed by solving the least squares problem \(\|\mathbf{R}^{(l)}\mathbf{T}-\mathbf{R^{\prime}}^{(l^{\prime})}\|_{F}\). This objective can be modified with \(L1\) regularization [39] to encourage a sparse stitching, i.e., each neuron of the stitched layer takes in outputs of only few neurons of the bottom network.

The asymmetry of model stitching allows concluding whether parts of one model are better than another (does plugging in the representations of model \(f\) increase performance of the stitched model compared to model \(f^{\prime}\)?), whereas representational similarity measures can only indicate whether they are similar or not. Compared to other measures, model stitching requires training of an additional layer and thus might be more complicated and expensive to implement.

## 5 Meta-Analysis of Similarity Measures

In this section, we will summarize existing results regarding the properties of similarity measures and their relationships, to aid the readers in choosing appropriate measures for their application. Much of the analysis of similarity measures in deep learning is focused on representational similarity. However, popular functional similarity measures are used beyond deep learning and, thus, have been analyzed in more general contexts. Hence, to keep in scope of similarity measures for deep learning, we focus on the results for representational similarity measures, but we point to related work for functional measures [113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123].

### Correlation between Functional and Representational Measures

There has only been little work that investigates the relationship between representational and functional similarity over specific kinds of neural networks. Most prominently, Ding et al. [38] conduct a correlation analysis on BERT language models [124] and ResNet image recognition models [125], where they investigate whether diverging functional behavior can also be depicted through representational similarity measures. For that purpose, they induce functional changes on the given models, such as varying training seeds, decreasing layer depth, removing principal components of representations at certain layers, or applying out-of-distribution inputs, and investigate whether observed changes in accuracy on classification tasks correlate with changes in representational similarity as measured by CKA, PWCCA, and Orthogonal Procrustes. Overall, they observe that on these two neural network types, the Procrustes measure generally correlates well with changes in functional behavior, which conversely does not always hold for CKA and PWCCA. Specifically, they find when removing principal components from intermediate representations, CKA is much less sensitive to these changes than orthogonal Procrustes and PWCCA. CKA still indicates high similarity between the original and the modified representation when the accuracy of the model has already dropped by over 15 percent.
A similar analysis has been conducted by Hayne et al. [57]. They induce functional changes by deleting neurons in the linear layers of ImageNet-trained CNNs. Aside from using accuracy as a functional measure, they use another performance-based measure based on the rank of the true class in the soft predictions. They report that, on these CNNs, the orthogonal Procrustes measure and CKA correlate more with functional similarity than CCA measures. A key difference in their protocol compared to Ding et al. [38] is that they do not use the same preprocessing of presentations. They do not normalize the representations to unit norm, which makes their results not directly comparable.

Davari et al. [126] have specifically explored the connection between CKA and functional similarity measures in more detail, pointing out how this measure is sensitive to manipulations of its input representations that would not affect the functional similarity of the underlying models. As one of their main results, they have shown that manipulating the representation of only a single instance can strongly affect the CKA score. Specifically, they could alter the CKA of two identical representations to almost zero by translating the representation of a single instance in one of the copies, without affecting the separability of the corresponding classes by this translation. Further, they have shown how pairwise CKA scores between layers can be altered to a specified reference with a setup similar to distillation, while leaving functional similarity almost unaffected. This kind of sensitivity of the CKA to manipulations that would not affect functional similarity of models has also been reported by Csiszarik et al. [39]. CKA scores were also compared to disagreement of models [49], clearly correlating to a lower degree than RTD.

Finally, a number of works [33, 68, 46, 38] have explored the ability of representational similarity measures to match corresponding layers in pairs of models that only differ in their training seed. Since such models and their corresponding layers are typically very similar in terms of their performance [127], they are considered functionally similar in that context. In the experiments, the authors compare the similarities between all combinations of layers \(f^{(l)},f^{\prime(l^{\prime})}\), and consider a measure \(m\) to correctly match layers if the value \(m(\mathbf{R}^{(l)},\mathbf{R}^{\prime(l)})\) indicates higher similarity than \(m(\mathbf{R}^{(l)},\mathbf{R}^{\prime(l^{\prime})})\) for all \(l^{\prime}\neq l\). CKA and a variant of second-order cosine similarity overall outperform CCA-based measures in these experiments.

### Discriminative Abilities of Representational Similarity Measures

There are a number of works that have assessed different aspects regarding what kind of representations are considered similar by representational similarity measures.

Morcos et al. [1] test the robustness of CCA-based measures to noise in representations. They argue that such measures should identify two representations as similar if they have a fixed shared part, next to a number of dimensions that are random noise. Thus, they test the measures with varying share of noise dimensions in the overall representations, and find that PWCCA overall is most robust in indicating high similarity, even if half of the dimensions are noise. When the number of noise dimensions was smaller, SVCCA has also shown robust behavior, with only mean CCA falling behind.

Shahbazi et al. [46] test whether representations obtained by sampling a low number of dimensions from a given baseline still yield high similarity with the original representation, as well as other low-dimensional samples. They specifically compare CKA, Riemannian distance, RSA, and Frobenius norm of the RSM difference on a neuroscience dataset, where the sampled dimensions are varied between 10 and 50. Their findings indicate that Riemannian distance almost always assigns high similarity between a low-dimensional sample and its original representation. The other measures do not assign such high similarities when dimensionality was low, though CKA gave better than RSA and RSM norm. In higher dimensions, all measures perform equally well. Further, Riemannian distance generally indicated high similarity between two low-dimensional samples from the same baseline.

Lin and Kriegeskorte [47] have tested whether AGTIC, dCor, and HSIC, the statistic used in CKA, are able to discriminate between varying distributions of data patterns, such as spirals or circles. In lower-dimensional representations, AGTIC overall appeared to discriminate better than HSIC and dCor. When dimensions were higher, these measures yielded similar performance at this discrimination task.

A similar experiment has been conducted by Barannikov et al. [49], who also use synthetic data patterns to test the ability of RTD to discriminate between topologically different data. Specifically, they generate data points that come from an increasing number of different clusters that are arranged circularly in two-dimensional space. They argue that the similarity between the original data with one cluster and other datasets with more clusters should decrease with increasing number of clusters. In their results, their rank correlation between similarity score of measure and number of clusters in the data was perfect for the RTD, whereas he CKA and SVCCA had relatively low correlations.

Finally, Tang et al. [48] argued that models trained from two similar datasets, such as CIFAR-10 and CIFAR-100, should be more similar compared to models trained on dissimilar datasets, that for instance do not contain natural images. In their experiments, they consider the CKA and NBS measures. They find that CKA discriminates these types of data better.
### Influence of Inputs

Another relevant issue that has been studied in literature is the impact that the inputs \(\mathbf{X}\) have on the resulting similarity scores. Specifically when it comes to functional similarity, it is a well-known fact in machine learning research that similarity of outputs is strongly confounded by the accuracy of the models, the number of classes and the class distribution [85, 9, 89, 101, 102]. Similar confounding effects also exist with respect to representational similarity measures, and we will discuss some corresponding results in the following.

First, there are strong indications that increasing similarity of inputs also results in increasing similarity of the resulting outputs. Cui et al. [128] specifically point out this effect for the RSA and CKA measures, where they provide examples as to how strongly confounded inputs processed via random neural networks yield higher CKA scores than neural networks that were optimized for the given image recognition task. As a solution to this issue, they propose a regression-based approach to de-confound the inputs.

Second, it has been shown that representational similarity measures can be confounded by input features. Dujmovic et al. [129] show this effect for RSA measures. They compare a model trained on standard image data to models trained on images that were modified such that in every image there was a pixel that leaked the class of the image, which allowed these models to learn a shortcut to classification. Depending on where the leaking pixels were placed in the images, the representational similarity between the standard model and the tweaked model varied strongly.

Similarly, Jones et al. [5] find that feature co-occurrence in inputs leads to high representational similarity scores by CKA. Different input features may co-occur in the data used to compute representations, but models may use these features to different extents. For example, on a high level, the features "hair" and "eyes" co-occur in images of human faces, but one model may only use the hair for its task, whereas the other model may only use the eyes feature. In their analysis, they show that CKA scores ignore the difference in feature use with an image inversion approach: using data synthetically generated to produce the same representations in one model, similarity to the other model drops drastically as feature co-occurrences are eliminated.

Third, there are also indicators that the quantity \(N\) of input instances can influence similarity scores. In that context, Williams et al. [36] study the relation between similarity scores and the ratio \(N/D\) of input quantity over dimensionality of the representations. They conduct experiments on variants of the Procrustes measure, from permutation invariance to linear invariance. They find that invariance affects the ratio \(N/D\) needed for consistent similarity scores. Invariances, that allow for more transformations between representations, such as linear invariance, require a higher ratio than invariances with comparatively few allowed transformations, such as permutation.

## 6 Discussion

After presenting both representational and functional similarity measures, as well as results of meta-analyses on these measures, we will now discuss the relationship between these measures, present open research problems, and provide some practical considerations.

### Relationship between Representational Similarity and Functional Similarity

As noted in Section 2, representational and functional similarity are two complementary notions, which in combination can allow for nuanced insights into similarity of models. However, to use and interpret the corresponding measures correctly, the relationships between these measures have to be properly contextualized.

Functional outputs in classification tasks have a clear and universal semantic, which makes it intuitive to understand when two outputs are similar. In contrast, the semantics of representations may depend on the type of neural network, its activation function, or its objective. As a result, functional similarity measures are generally easier to interpret than representational measures.

This allows us to use functional similarity to (partially) validate representational similarity. When functional similarity measures indicate strong dissimilarity of models, there has to be some dissimilarity in the representations of the previous layers, assuming that differences in the final classification layer cannot fully explain the functional difference. The opposite is not true: two functionally similar models may reach their output with dissimilar representations. At the same time, representational measures may simply not be suited to pick up on representational equivalences that could lead seemingly dissimilar models to the same outputs. Further, if a functional similarity measure indicates strong similarity, this does also not imply that the models are indeed equal in general, both from a representational or from a functional perspective. There may be various ways for representations to keep separability of inputs, and if the model was applied to out-of-distribution inputs, the resulting outputs may vary drastically [16]. Finally, strong representational similarity may not imply strong functional similarity either, as functional outputs may be susceptible to noise in
representations that would not necessarily have strong effects on representational similarity measures or the models may use transformations that are less powerful than the invariances of the measure.

In conclusion, a single sound implication can be made: if there is significant functional dissimilarity in a model, there also should be a representational similarity measure indicating significant dissimilarity of the model. This specific relation between functional and representational similarity was already used in the quality evaluation of representational similarity measures by measuring correlations between functional and representational similarity scores, as done by Ding et al. [38]. We recommend this specific approach to evaluate applicability of representational similarity measures, and hope that additional understanding of representational similarity measures will be gained in the future.

### Open Research Problems

As can be seen from Section 5, there is a contrast between the numerous proposed measures, and the rather small amount of research dedicated to systematically analyzing and comparing the existing measures.

We argue that this exemplifies a significant gap in research, as deeper understanding of the properties of measures and their applicability is necessary to apply them properly. Otherwise, misinterpreted or faulty measurements may mislead future research.

This lack of understanding is a particular problem for representational similarity measures as the representational structure is usually difficult to understand, and strongly dependent on factors such as architecture, loss functions, layer depth, and activation functions. Therefore, we consider research on the applicability of representational similarity measures to specific models to be of particular value. As discussed in Section 5, only the works by Ding et al. [38] and Hayne et al. [57] have specifically addressed the issue of applicability of measures for specific language and image recognition models, where overall, Procrustes similarity was recommended and the behavior of CKA was different for differing neural network models. However, even these works only consider a small subset of representational similarity measures, and there are some differences in experimental settings, which impair the generalizability of these findings. As discussed in Section 6.1, we support the approach by the authors to determine applicability of measures, but argue that more research, considering a higher amount of both similarity measures and model types, is necessary.

Beyond the problem of determining suitable measures for given model architectures, further understanding of the behavior of the measures is necessary to improve the interpretability of similarity scores. Unless a measure takes on a value that indicates perfect similarity, there is typically no direct implication on whether a given similarity score indicates similarity or dissimilarity of models, as this strongly depends on the context. For instance, when analyzing functional similarity, a disagreement of \(m_{\text{Dis}}(\mathbf{O},\mathbf{O^{\prime}})=0.05\) can be considered low in a difficult classification problem with many classes, where no high accuracy is expected, and high in an easy binary classification problem where one expects near-perfect accuracy. While such contextualization is easy to grasp and establish for measures that are as intuitive as disagreement, for more opaque representational similarity measures, there are hardly any known baselines that help to classify a score as indicating similarity or dissimilarity. It is possible to create a baseline for unrelated representations by permutation of the rows of the representation matrices[45, 46], which allows for testing whether a score indicates statistically significant similarity. However, a more general contextualization of similarity scores is still out of reach with this approach.

Therefore, we argue that future research should put more emphasis on investigating practical properties of similarity measures, and on establishing baselines that help to interpret similarity scores. There are several aspects that could be considered, for instance:

* **Effects of input similarity**: As discussed in Section 5.3, representational similarity tends to be higher if inputs are similar. Are there boundaries of representational similarity in terms of some notion of input similarity?
* **Effects of representation perturbation**: It has already been demonstrated that CKA is sensitive to translating a single instance representation. At the same time, it is robust to the removal of principal components. For most other representational similarity measures, such analyses have not been conducted yet. To what extent do changes in a representation affect similarity scores?
* **Effects of dimensionality**: As an example, Orthogonal Procrustes scores between random representations change with increasing dimension (see Appendix B). What is the range a measure can be expected to take, depending on such contexts?

We argue that deeper understanding of such aspects is crucial to properly apply and interpret their output scores, which in consequence would strongly benefit the understanding of similarity of neural networks in general.
### Practical Considerations

We close this discussion with providing advice for the practical applications of similarity measures. Aside from general recommendations, we will also briefly discuss how functional similarity can be assessed in related learning tasks such as regression or multi-label classification, and what computational challenges exist aside from the computational cost of computing similarity scores.

#### 6.3.1 General Recommendations

We describe 30 representational and 16 functional similarity measures in this survey. Given this big number of similarity measures, selecting an appropriate measure for a specific context is not trivial, and depends on constraints such as data accessibility, model access, computational resources, and objectives of the analysis. Regarding some of these external factors, Table 1 and Section 4 already provide an overview of basic properties that may affect the applicability of measures in a context at hand. However, there are a number of more things to consider when trying to measure similarity of neural networks in practice.

Most importantly, as discussed in Section 6.1, one should generally consider both representational and functional similarity measures to assess similarity of neural networks in a holistic manner. Due to their generally low cost, high interpretability, and model agnostic semantics, it is also generally advisable to use as many performance-based and prediction-based functional similarity measures as the context, e.g., availability of soft predictions, allows. Agreement-based measures are to be preferred since they conceptually provide more nuanced information about the similarity of the outputs, but performance-based measures, and the plain performance values of the compared models themselves, should all be utilized as well to provide additional context, as agreement-based measures are confounded by model performance. Gradient-based measures are more expensive and require white-box access to the model, but can provide even more nuanced insights on functional similarity beyond plain agreement.

In contrast, the choice of representational similarity measures requires more care. Since the structure of the representation space strongly depends on factors such as the type of the neural network, its loss function, or its activation functions, the chosen representational similarity measure should be compatible with this structure. For instance, nearest-neighbor based measures that apply Euclidean distance to identify nearest neighbors may not be suitable when similarity of representations is modeled in terms of angles. Conversely, angle-based similarity measures such as cosine similarity may not be a good choice when representations are bound to the positive orthant due to activation functions such as ReLu. Thus, if notions of equivalence in the representation space are given, one could consider measures that are invariant to the corresponding class of transformations, or if appropriate distance functions in representation space are known, these could be utilized when applying RSM-based or nearest-neighbor-based measures. More research is required at determining appropriate measures for given neural network designs - as of now, there is only some evidence indicating that orthogonal Procrustes is relatively well-applicable on specific image recognition and language models [38]. Again, also the results on representational similarity of a given set of models should be contextualized by considering appropriate baselines such as similarity of random representations, or representations stemming from random inputs, as there are several factors that could potentially confound the observed similarities. Further, sensitivities to noise, as known for the CKA measure [126], have to be considered. Moreover, non-linearity of measures should be taken into account. For example, the widely-used cosine similarity changes non-linearly with respect to the angle between two compared vectors, which may highlight some changes of representation or function, but downplay others.

Finally, choosing appropriate inputs may also benefit the understanding of similarity between neural network models. Given that similarity of inputs can confound the resulting similarity scores (see Section 5.3), one may consider using inputs that do not correlate strongly with each other, or using the de-confounding approach for RSM-based measures by Cui et al. [128]. Even more, out-of-distribution inputs, if available, might be particularly well-suited to search for differences in behavior of models that may only have consistent behavior on the kinds of inputs it was trained on.

#### 6.3.2 Functional Similarity for Non-Classification Tasks

Although in this survey we focus on functional similarity with respect to classification, many of the functional similarity measures can be used for other tasks.

In particular, if a suitable performance measure is given, performance-based measures can be used in any other context. This is also the case for gradient-based and stitching measures if white-box access to the models is given, and, in case of gradient-based measures, adversarial examples can be constructed for the given context.

The agreement-based measures that we presented, conversely, are essentially limited to tasks where outputs are assigned discrete labels. For multi-label classification, the inter-rater agreement measures naturally generalize to this context, while most other measures are not applicable. For regression tasks, some measures can be adapted by comparing
the regression outputs with a suitable distance measure. In that context, we would like to point to prior surveys on agreement of continuous outputs [26, 25, 130].

Finally, if output is structured, e.g., text or image generation, functional similarity becomes less tractable as outputs do not share universally identical semantics as in classification tasks. For example, generated images may have differences that are not perceivable for the human eye, and one would have to reconsider the notion of agreement or similarity in such a context, taking such issues into account. The evaluation of these kinds of models, including comparison of outputs to a human reference, has been studied in prior surveys as well [27, 28].

#### 6.3.3 Computational Challenges

Similarity measures are typically used to understand neural networks, e.g., their changes upon modification of the architecture or training data. Because models can be diverse in their representational structure and functional behavior depending on arbitrary factors such as initialization seed [10, 127], multiple comparisons across a population of models are necessary to find generalizable results. Generating this model population represents a significant computation challenge added to the cost of comparison itself: training multiple models quickly becomes infeasible when models grow in size. Throughout the paper, we assumed that the models for a comparison are given - here we discuss options to generate a model population.

As already mentioned, a naive solution is to retrain models from scratch with different initialization and batch order. This approach usually generates diverse models, but restricts the experiments to relatively small models due to the computational cost.

One option to save computational costs when generating model populations is to fine-tune pre-initialized models with different seeds, as done by McCoy et al. [16] on language models. In the corresponding experiments, this approach yielded a population of models that was still functionally diverse in out-of-distribution tasks. Another alternative to fully training models is sampling models in the weight space neighborhood of a fully trained model, as suggested by Fort et al. [85]. These models are less diverse compared to populations obtained from full retraining, but allow for a much larger population of models as only a single model has to be trained. Possibly, the diversity can be increased by fine-tuning the sampled models, however, how the result would compare to different fully trained models or fine-tuned models from a single starting point is, to the best of our knowledge, unknown. Similar to fine-tuning, adversarial weight perturbation can also be used to generate functionally diverse models on specific inputs [91]. In this approach, a base model is fine-tuned with the objective of having a specific prediction for selected inputs. Compared to fine-tuning, this approach may be more efficient if the number of inputs that the model is perturbed for is relatively small.

Finally, in some applications it might be possible to utilize resources that were published in prior works. Such resources may include models trained or fine-tuned from different seeds, checkpoints obtained over the course of training, and variations in model architecture [e.g. 131, 16, 132, 133].

Generally, the diversity of compared models should be taken into account when making statements about neural networks similarity: Similarity estimates may be considered as lower bounds on the maximal discrepancy between two models as the diversity of models typically cannot be fully explored. Hence, the dependency of results on the studied set of models should not be ignored. At the same time, some models may be (almost) deterministic by design, allowing for reliable results from few model comparisons.

## 7 Conclusion

Representational similarity and functional similarity represent two complementing perspectives on analyzing and comparing neural networks. In this work, we provide a comprehensive overview of existing measures for both representational and functional similarity. We provide formal definitions for 46 similarity measures, along with a systematic categorization into different types of measures.

In addition, we conduct a meta-analysis of the literature to shed light on some of their salient properties. We specifically identify a lack of research that analyzes properties and applicability of representational similarity measures for specific neural network models in a unified manner. This gap in the literature also affects the quality of the recommendations that one can make about their practical applicability. We argue that additional research is necessary to enable the informed application of similarity measures to better understand similarity of neural network models. We hope our work lays a foundation for our community to engage in more systematic research on the properties, nature and applicability of similarity measures for neural network models. Further, with our categorization and meta-analysis, we believe that our work can assist researchers and practitioners in choosing appropriate measures for their applications at hand, even if we cannot recommend a one-fits-all solution specifically for representational similarity.