Morse Neural Networks for Uncertainty Quantification
We introduce a new deep generative model useful for uncertainty
quantification: the Morse neural network, which generalizes the unnormalized
Gaussian densities to have modes of high-dimensional submanifolds instead of
just discrete points. Fitting the Morse neural network via a KL-divergence loss
yields 1) a (unnormalized) generative density, 2) an OOD detector, 3) a
calibration temperature, 4) a generative sampler, along with in the supervised
case 5) a distance aware-classifier. The Morse network can be used on top of a
pre-trained network to bring distance-aware calibration w.r.t the training
data. Because of its versatility, the Morse neural networks unifies many
techniques: e.g., the Entropic Out-of-Distribution Detector of (Mac\^edo et
al., 2021) in OOD detection, the one class Deep Support Vector Description
method of (Ruff et al., 2018) in anomaly detection, or the Contrastive One
Class classifier in continuous learning (Sun et al., 2021). The Morse neural
network has connections to support vector machines, kernel methods, and Morse
theory in topology.
Benoit Dherin, Huiyi Hu, Jie Ren, Michael W. Dusenberry, Balaji Lakshminarayanan
# Morse Neural Networks for Uncertainty Quantification

###### Abstract

We introduce a new deep generative model useful for uncertainty quantification: the Morse neural network, which generalizes the unnormalized Gaussian densities to have modes of high-dimensional submanifolds instead of just discrete points. Fitting the Morse neural network via a KL-divergence loss yields 1) a (unnormalized) generative density, 2) an OOD detector, 3) a calibration temperature, 4) a generative sampler, along with in the supervised case 5) a distance aware-classifier. The Morse network can be used on top of a pre-trained network to bring distance-aware calibration w.r.t the training data. Because of its versatility, the Morse neural networks unifies many techniques: e.g., the Entropic Out-Of-Distribution Detector of (Macedo et al., 2021) in OOD detection, the one class Deep Support Vector Description method of (Ruff et al., 2018) in anomaly detection, or the Contrastive One Class classifier in continuous learning (Sun et al., 2021). The Morse neural network has connections to support vector machines, kernel methods, and Morse theory in topology.

Machine Learning, Morse neural networks, Morse neural networks, Morse neural networks

## 1 Introduction

Neural networks are becoming prevalent in a large number of applications both in industry and research (Larson et al., 2022; Jumper et al., 2021; Ren et al., 2019; Han et al., 2021; Kivlichan et al., 2021). Because of their impressive performances, these models are likely to become increasingly trusted in a wider range of domains, including sensitive applications like in the medical field (Roy et al., 2021; Kivlichan et al., 2021; Jumper et al., 2021; Han et al., 2021). This makes the ability to quantify when the predictions of a neural network should be considered uncertain a critical issue, especially since neural networks are known to deliver wrong predictions very confidently (Nguyen et al., 2015; Goodfellow et al., 2015; Ovadia et al., 2019; Guo et al., 2017; Hendrycks and Gimpel, 2017; Lakshminarayanan et al., 2017). As a result, the development of methods to quantify neural network uncertainty is an increasingly important subject in deep learning research (Amodei et al., 2016). In particular, neural networks tend to produce confidently wrong predictions when presented with Out-Of-Distribution (OOD) inputs, that is, inputs that are far away from the data distribution with which the model was trained (Murphy, 2023; Nagarajan et al., 2021; Liu et al., 2020, 2022). Detecting OOD inputs as well as devising models that are aware of the distance from inputs to the training distribution are becoming key challenges in uncertainty quantification (Murphy, 2023).

One classical approach to detect OOD data is to fit a generative probability density to the In-Distribution data (IND) (since OOD points are often rare) and use it to detect OOD points as points with low probability (Murphy, 2023). This works well for normal data with a single mode, but becomes computationally prohibitive when the data has very complex modes requiring fitting large mixtures of simpler parametric models. On the other hand, standard deep generative models are easier to train and in theory able to express very complex modes. However, they have been shown to have some difficulty to distinguish IND from OOD even in the simpler case of distinguishing between MNIST and FashionMNIST (Nalisnick et al., 2019). Non-generative deep learning approaches like the one class deep Support Vector Data Description (SVDD) of (Ruff et al., 2018) have yielded better results on that front.

Another approach is to leverage the supervised information from a classifier to obtain finer-grained OOD detectors. For instance, (Lee et al., 2018; Ren et al., 2021) fit a multivariate Gaussian to the classifier embeddings for each of the labels, yielding a squared distance, the Mahalanobis distance, from the Gaussian modes, creating a powerful OOD detector. The Spectral Normalized Gaussian Process (SNGP) in (Liu et al., 2020, 2022) on the other hand fits an approximate Gaussian process to the classifier embeddings producing an input-dependent temperature that is used to scale the classifier logits so that it becomes distance-aware (i.e. its uncertainty grows as we move away from the training set distribution). Finally, the entropic OOD detector from (Macedo et al.,
2021) learns a classifier whose logits are distance-aware by construction and uses the entropy score of that classifier as an OOD detector. For further reference, we refer the reader to (Salehi et al., 2022; Bulusu et al., 2020; Ruff et al., 2018) for comprehensive surveys of the literature.

We propose a new deep generative model, the Morse network, which unifies a number of the separate supervised and unsupervised techniques mentioned. This model produces a join (unnormalized) density \(\mu(x,y)\) by taking the kernel similarity \(K(\phi_{\theta}(x),T(y))\) between the image of the feature \(x\) and the one-hot-encoded \(T(y)\) version of the label \(y\), which we set to a fixed value \(a\) in the unsupervised case (see sections 2 and 4 for details). The unsupervised formulation comprises mixtures of (unnormalized) Gaussian densities (and more generally exponential family densities) along with more flexible densities whose modes are submanifolds rather than discrete points. The unsupervised Morse network with a Gaussian kernel degenerates to the deep one class SVDD proposed in (Ruff et al., 2018), except for a built-in regularizer in the loss somewhat reminiscent of the mixup regularizer (Pinto et al., 2022). For the Cauchy kernel, it has a temperature whose form coincides with that of SNGP, except that the variance is learned and not computed by large matrix inversion. The supervised Morse network with a Laplace kernel produces a distance-aware classifier that coincides with the entropy OOD classifier from (Macedo et al., 2021). Although this is not the focus of this work, we explain how the Morse network yields a sample generator by following certain gradient flows from random initial points, very much in the spirit of the Poisson generative model (Xu et al., 2022).

The focus of this work is to expose this unifying idea, and to explore the relationships with known approaches. Comprehensive evaluation of the approach and its extensions is the topic of future work.

## 2 The Unsupervised Morse Neural Network

We now introduce the (unsupervised) Morse neural networks. These networks produce (unnormalized) generative densities \(\mu_{\theta}(x)\in[0,1]\) directly on a feature space \(X=\mathbb{R}^{d}\) or on a space of embeddings \(\mu_{\theta}(h(x))\in[0,1]\) of the original features obtained from a pre-trained network \(h(x)\). Morse neural networks are very expressive in terms of the modes they can produce (see examples below). Recall that the modes of an (unormalized) density \(\mu:X\rightarrow[0,1]\) is the subset modes\((\mu)\subset X\) where the density achieves its highest possible value, namely 1. This set can be reduced to a single point (e.g., the mean of a Gaussian) or it can be more complex such as a smooth subset of \(X\) of higher dimension, like a curve, or a surface, or, more generally a \(k\)-dimensional submanifold of \(X\), as is the case for the Morse neural network densities. Intuitively, these generative densities are uniformly \(1\) on their mode submanifolds and decrease to \(0\) as we move away from these modes at a speed controlled by a special type of kernels, which we call Morse kernels:

**Definition 2.1**.: A _Morse kernel_\(K\) on a space \(Z=\mathbb{R}^{k}\) is a positive kernel \(K(z_{1},z_{2})\) taking its values in the interval \([0,1]\) and such that \(K(z_{1},z_{2})=1\) if and only if \(z_{1}=z_{2}\).

Many common kernels are Morse kernels. Namely, all kernels of the form \(K(z_{1},z_{2})=\exp(-\lambda D(z_{1},z_{2}))\) where \(D\) is a divergence in the sense of Amari (2016) are Morse kernels (since \(D(z_{1},z_{2})=0\) if and only if \(z_{1}=z_{2}\)). The Gaussian kernel and the Laplace Kernel are Morse Kernel, as well as the Cauchy kernel \(K(z_{1},z_{2})=\frac{1}{1+\lambda\|z_{1}-z_{2}\|^{2}}\).

We are now ready to define the Morse neural network:

**Definition 2.2**.: A _Morse neural network_ is defined by the data of 1) a neural network \(\phi_{\theta}:X\to Z\) (with parameters \(\theta\)) from the feature space \(X=\mathbb{R}^{d}\) to a space \(Z=\mathbb{R}^{k}\), and 2) a Morse kernel \(K\) on \(Z\). The (unnormalized) density of a point \(x\in X\) is given by

\[\mu_{\theta}(x)=K(\phi_{\theta}(x),a) \tag{1}\]

where \(a\) is treated as an hyper-parameter of the model.

From the properties of Morse kernels, it is easy to see that \(\mu_{\theta}(x)\in[0,1]\) and that the modes of \(\mu_{\theta}(x)\) (i.e., the points where \(\mu_{\theta}(x)\) reaches 1, its highest possible value) coincide with the level set of \(\phi_{\theta}\) (Sec. A):

\[\text{modes}(\mu_{\theta})=\{x\in X:\;\phi_{\theta}(x)=a\}. \tag{2}\]

Fitting the Morse network to a dataset so that its modes approximate the modes of the data distribution is explained in section 3. Applications of a fitted unsupervised Morse network are detailed in section 2.1, which we briefly summarize here:

First, the input dependent temperature \(T_{\theta}(x)=1/\mu_{\theta}(x)\), which is 1 on the density modes and grows away from them can be used to scale the logit of a classifier to make it distance-aware in the spirit of (Liu et al., 2022). Second, the classifier \(s_{\theta}(x)=1-\mu_{\theta}(x)\) is a epistemic uncertainty score measuring our uncertainty in whether the point \(x\) comes from the training distribution. Hence it can be used as an OOD detector. Third, the function \(V_{\theta}(x)=-\log\mu_{\theta}(x)\) is a form of square distance from the density modes (see section A for details and relationship with Morse-Bott theory). In consequence, its negative gradient field flow \(-\nabla_{x}V_{\theta}(x)\) converges to the mode submanifolds, giving us a way to generate new samples from random initial points very much in the spirit of the generative Poisson flow from (Xu et al., 2022).

We highlight several examples to showcase the flexibility of this model.
Location/Scale densities: \(k=d\).All the standard location/scale densities of the form \(f((x-\mu)^{T}\Sigma^{-1}(x-\mu))\) can be recovered using a linear neural network with one layer with invertible weight matrix. This encompasses the multivariate Gaussian, Student-\(t\), Cauchy, and Laplace densities. For all these densities we can take the same neural network \(\phi(x)=\Sigma^{-\frac{1}{2}}(\mu-x)\) and \(a=0\), but we change the kernel: The Gaussian kernel \(K(x,x^{\prime})=\exp(-\frac{1}{2}\|x-x^{\prime}\|^{2})\) produces the (unnormalized) multivariate Gaussian; the Laplace kernel \(K(x,x^{\prime})=\exp(-\|x-x^{\prime}\|)\) produces the (unnormalized) multivariate Laplace density; the Student-\(t\) kernel with \(\nu\) degrees of freedom \(K(x,x^{\prime})=(1+\frac{1}{\nu}\|x-x^{\prime}\|^{2})^{-\frac{d+\nu}{2}}\) produces the multivariate (unnormalized) Student-\(t\) density, of which the Cauchy density is a particular case.

Distributions with mode submanifold: \(k<d\).To showcase that the Morse network can produce densities with mode submanifolds, we device here an example where the density modes consist in a sphere whose radius is controlled by the hyper-parameter \(a\). For that, consider the map \(\phi(x)=\|x\|\) and the Gaussian kernel \(K_{\sigma}(z,z^{\prime})=\exp(-\frac{1}{2\sigma^{2}}(z-z^{\prime})^{2})\) on \(\mathbb{R}\) with bandwidth \(\sigma^{2}\). We obtain the density \(\mu_{a,\sigma^{2}}(x)=\exp(-\frac{1}{2\sigma^{2}}(\|x\|-a)^{2})\), which has for modes the sphere of radius \(a\). More generally, any regular value \(a\in Z=\mathbb{R}^{k}\) of a differentiable map \(\phi:\mathbb{R}^{d}\rightarrow\mathbb{R}^{k}\) will produce a density with mode submanifold of dimension \(d-k\). In the kernel bandwidth limit \(\sigma^{2}\to 0\), the resulting limiting density is the uniform density on the level sets of \(\phi\).

Mixture models: \(k>d\)We illustrate how the Morse neural network can produce density mixtures. For instance, a mixture of \(l\) Gaussian densities on \(X\) is captured by taking the neural network \(\phi:X\to X^{l}\), where \(\phi_{i}(x)=\Sigma^{-\frac{1}{2}}(\mu_{i}-x)\) is the map for the \(i^{th}\) multivariate Gaussian on \(X\) and \(a\) is the zero vector. The kernel on \(Z=X^{l}\) is given by a convex sum of the Gaussian kernels on the components of \(Z\): \(K(z,z^{\prime})=\sum_{i=1}^{l}\alpha_{i}\exp(-\frac{1}{2}\|x_{i}-x_{i}^{\prime }\|^{2})\), where \(z=(x_{1},\ldots,x_{l})\). Since in this case, the dimension of \(Z\) is larger than that of \(X\) the pre-image of zero is the empty set. However, the zero-level sets of the components of \(\phi\) give back the modes of the mixture of Gaussian densities. One can obtain mixtures of different densities by changing the component kernels.

Relation to divergences and the exponential family.Given a divergence \(D(z,z^{\prime})\) in the sense of Amari (2016), then \(K(z,z^{\prime})=e^{-\lambda D(z,z^{\prime})}\) is a Morse kernel. A convex function \(A\) on \(Z\) produces a Bregman divergence \(D_{A}(z,z^{\prime})=A(z)+A^{*}(\eta^{\prime})-z\eta^{\prime}\) where \(A^{*}\) is the dual convex function and \(\eta^{\prime}=\nabla A(z^{\prime})\) is the Legendre transform (see Amari (2016) for details). The Morse network for this kernel is then \(\mu_{\theta}(x)=e^{\lambda(\phi_{\theta}(x)a-A^{*}(a)-A(\phi_{\theta}(x)))}\) which an unnormalized version of the exponential family with canonical parameter \(a\), cumulant generating function \(A^{*}\), and dispersion \(\lambda\), when \(\phi_{\theta}(x)\) is the identity. One recovers the Gaussian case with \(A(z)=\frac{1}{2}z^{2}\).

Relation to Energy-based models.Energy-based models (Goodfellow et al., 2016) have been commonly used in generative modeling (Zhao et al., 2017; Gao et al., 2021; Arbel et al., 2021; Che et al., 2020). In fact, the Morse neural network can be rewritten as an unnormalized energy-based model as follows

\[\mu_{\theta}(x)=e^{-V_{\theta}(x)} \tag{3}\]

where the model energy is parameterized using the Morse kernel \(V_{\theta}(x)=-\log K(\phi_{\theta}(x),a)\). In section A, we show that this positive function satisfies the Morse-Bott condition on the density modes and thus can be interpreted as a sort of squared distance from the modes. Note that normalized energy models have been used in the context of uncertainty quantification (Wang et al., 2021), where a classifier uncertainty is added as extra dimension and learned as joint energy model. Also note that the Morse neural network with exponential family Morse kernel resembles the conjugate energy models from Wu et al. (2021).

### Applications and experimental results

OOD detection:Since \(\mu_{\theta}(x)\in[0,1]\) yields an density which is 1 on the modes and goes to zero as the distance from the mode increases, \(s_{\theta}(x):=1-\mu_{\theta}(x)\) provides a measure of how uncertain we are about \(x\) being drawn from the training distribution (epistemic uncertainty). Hence \(s_{\theta}(x)\) can be used as an OOD detector. Figure 1 (bottom row, middle right) shows that \(s_{\theta}(x)\) classifies points as OOD (value 1) as the distance from the two-moons dataset increases. As Table 1 shows, the unsupervised generative Morse network produces a detector capable of distinguishing MNIST images from the FashionMNIST training images in contrast to other deep generative models (Nalisnick et al., 2019). It is also able to distinguish between CIFAR10 and CIFAR100 when trained on a vision transformer embeddings as reported in Table 1. Note that for the Gaussian kernel, \(V_{\theta}(x)=-\log\mu_{\theta}(x)\) coincides with the anomaly score introduced in (Ruff et al., 2018) and with the Mahalanobis distance when the network is further linear and invertible.

Distance-aware calibration:We can use the Morse network to calibrate a classifier \(f(x)=\text{softmax}(h(x))\) trained on a supervised dataset \(D=\{(x_{i},y_{i})\}\) so that it becomes less confident on points far way from the data distribution in the same spirit as SNGP (Liu et al., 2022). The idea is to fit a Morse network on the input data. Then we can scale the classifier logits \(h(x)\) by the inverse of the Morse temperature \(T_{\theta}(x):=1/\mu_{\theta}(x)\). This produces a classifier that becomes uncertain ou
it has been trained (the more so as the Kernel bandwidth decreases), as demonstrated in Figure 1 (top row). Note that when using the kernel \(K_{\lambda}(z,z^{\prime})=1/\sqrt{(1+\lambda\|z-z^{\prime}\|^{2})}\), the temperature becomes \(T_{\theta}(x)=\sqrt{1+\lambda V_{\theta}(x)}\), where \(V_{\theta}(x)=-\log\mu_{\theta}(x)\), which has the same form as the SNGP temperature of (Liu et al., 2022, 2020), except that the squared distance function \(V_{\theta}(x)\) replaces the SNGP approximate variance.

Sample generation:The Morse gradient field \(F(x)=-\nabla_{x}V_{\theta}(x)\) is attracted to the density modes, which are the global minima (zeros) of the positive function \(V_{\theta}(x)=-\log K(\phi_{\theta}(x),a)\). If we follow its gradient flow from a random point, we obtain a new sample close to the data distribution mode. This approach resembles the Poisson generative flow from (Xu et al., 2022), where the \(V_{\theta}(x)\) is called a potential and is derived by solving the Poisson PDE. We illustrate this approach with the two-moons dataset in Figure 1 (bottom, right) where random points following the Morse flow converge toward the two-moons dataset modes using gradient descent on \(V_{\theta}(X)\).

## 3 Fitting the Morse neural network

Consider a dataset \(D=\{x_{1},\ldots,x_{n}\}\) sampled from a data distribution with density \(p(x)\). Theoretically, we want to find the neural network parameters \(\theta\) that minimize the KL divergence \(\text{KL}(p(x)\|\,\mu_{\theta}(x))\) for unnormalized densities (i.e., positive measures; see (Amari, 2016)) between the probability density \(p(x)\) generating the sample and the Morse network density \(\mu_{\theta}(x)\), that is,

\[\mathbb{E}_{x\sim p(x)}\left(\log\frac{p(x)}{\mu_{\theta}(x)}\right)\,+\int \mu_{\theta}(x)dx-\int p(x)dx \tag{4}\]

which amounts to minimizing w.r.t. \(\theta\) the following quantity

\[\mathbb{E}_{x\sim p(x)}\left(-\log K(\phi_{\theta}(x),a)\right)+\mathbb{E}_{x \sim\text{uni}}\left(K(\phi_{\theta}(x),a)\right)\]

The corresponding empirical loss is then

\[L(\theta)=-\frac{1}{n}\sum_{x\in D}\log K(\phi_{\theta}(x),a)+\frac{1}{n}\sum _{x\in D_{\text{uni}}}K(\phi_{\theta}(x),a)\]

which can be optimized with any iterative gradient-based optimizer. Note that \(D_{\text{uni}}\) are points uniformly sampled in \(X=\mathbb{R}^{d}\), and that the second term in the Morse loss can be interpreted as a form of regularization in the spirit of the mixup regularizer term (Pinto et al., 2022). For the Gaussian kernel the first term of this loss (with an added L2 penalty) coincides with the one-class Deep SVDD objective proposed in (Ruff et al., 2018) for anomaly detection as a way to simplify the loss of a deep one-class SVM in the case of normal data.

## 4 The supervised Morse neural network

The Morse neural network architecture offers a natural way to incorporate supervised labels to obtain a finer grained generative model. There are two ways to do it:

Separate Morse networks:For each label \(y\in\{1,\ldots,C\}\), we can fit a separate unsupervised Morse network to the subset of the data with label \(y=i\) producing a separate density \(\mu(x|i)=K_{i}(\phi_{\theta_{i}}(x),a_{i})\) for each label. The overall density is then given by the average \(\mu(x)=\frac{1}{C}\sum_{i}\mu(x|i)\). The corresponding OOD detector is \(s(x)=1-\mu(x)\). We can also create a classifier from this data by interpreting \(V_{i}(x)=-\log\mu(x|i)\) as a squared

\begin{table}
\begin{tabular}{c|c|c} IND/OOD & Method & AUROC \\ \hline FashionMNIST / MNIST & Morse & 0.998 \\  & **DoSE\({}_{KDE}\)** & 0.998 \\ CIFAR10 / CIFAR100 & Morse & 0.955 \\  & **DoSE\({}_{SVM}\)** & 0.571 \\ \end{tabular}
\end{table}
Table 1: **OOD detection with unsupervised Morse:** As a proof of concept, we report unsupervised Morse AUROC along with best baselines from (Morningstar et al., 2021). For CIFAR10/CIFAR100, we report Morse AUROC trained on embeddings from a pre-trained vision transformer to demonstrate the benefit of this approach. If trained on CIFAR10 directly, Morse AUROC is 0.569, corresponding to the second best performance (**DoSE\({}_{KDE}\)**) from (Morningstar et al., 2021). (See Appendix B.)

Figure 1: **Top row (distance-aware calibration): Left:** Probability output of a ResNet trained to separate the noisy two-moons dataset. **Middle and Right:** Probability plots of the same ResNet but with its logits calibrated using the unsupervised Gaussian Morse temperature at decreasing kernel bandwidth. The classifier becomes uncertain away from the training data. **Bottom row (noiseless two-moons): Left (density plot):** The Morse density concentrates on the two-moons modes. **Middle Left (mode plot):** The modes learned by the Morse network approximate well the two-moons modes. **Middle Right (OOD detection):** the Morse detector classifies points away from the modes as OOD. **Right (sample generation):** Random points following the Morse flow converge to the modes. See Appendix B for experiment details.
distance between \(x\) and the modes of the data with label \(i\); the classifier simply associates to a point \(x\) the label \(i\) which has the smallest \(V_{i}(x)\). When the kernels are taken to be all Gaussian, the resulting classifier coincides with the ILOC classifier (Sun et al., 2021) (except for the regularizing terms in the loss) used in the context of continual learning to avoid catastrophic forgetting since it allows to learn new tasks in complete isolation from the previous ones. (See also (Hu et al., 2021) for similar ideas in the same continual learning context).

Shared Morse network:The previous approach can be computationally intensive in the case of a large number of labels. The Morse network offers us a more efficient way to proceed by taking as model for the distribution join density

\[\mu(x,y)=K(\phi_{\theta}(x),T(y)). \tag{5}\]

where \(T(y)\) is the one-hot-encoded version of the label \(y\in\{1,\dots,C\}\) and \(\phi_{\theta}:X\rightarrow\mathbb{R}^{C}\) is a shared neural network for all the labels. (For simplicity, we identify \(y\) and \(T(y)\) and will use at time \(e_{i}\) to denote the \(i^{th}\) basis vector.) We can use the same KL divergence minimization principle (between unnormalized densities) as in the unsupervised case exposed in section 3 but using now the joint density \(p(x,y)\) as the density generating the data, yielding the following empirical Loss for the supervised network:

\[L(\theta)=-\frac{1}{n}\sum_{(x,y)}\log K(\phi_{\theta}(x),y)+\frac{1}{n}\sum_ {(x^{\prime},y^{\prime})}K(\phi_{\theta}(x^{\prime}),y^{\prime})\]

where \((x,y)\) range over the supervised dataset, and the \((x^{\prime},y^{\prime})\)'s are obtained by uniform sampling on the join feature and label space. In this case, the generative density can be obtained by marginalization

\[\mu(x)=\sum_{y}K(\phi_{\theta}(x),y), \tag{6}\]

producing the OOD detector \(s(x)=1-\mu(x)\).

Experimental results:Using this supervised Morse detector trained on the embeddings \(h(x)\) of a vision transformer fined-tuned to CIFAR10 we observe an improvement in AUROC from 0.955 (for the unsupervised Morse detector) to 0.969 in the same setting for CIFAR100 detection. Figure 2 also illustrates visually how the supervised Morse network can learn disconnected mode submanifolds better than the unsupervised version. (See section B for details).

Note that we also obtain a classifier that is distance-aware in the sense of (Liu et al., 2022) by construction:

\[\mu(y|x)=\frac{K(\phi_{\theta}(x),y)}{\sum_{y^{\prime}}K(\phi_{\theta}(x),y^{ \prime})}=\frac{\exp(-V_{y}(x))}{\sum_{y^{\prime}}\exp(-V_{y^{\prime}}(x))} \tag{7}\]

In the case of the Laplace kernel, this classifier coincides with the classifier used for entropic OOD detection (Macedo et al., 2021) where the entropy value of the classifier output is used as an OOD score. The main difference between the Morse supervised classifier with Laplace kernel and the entropic classifier from (Macedo et al., 2021) is that the \(y\)'s are learned for each class and the maximum likelihoods loss is used on \(\mu(y|x)\) rather than the Morse loss, making the second term of both loss related but different.

## Acknowledgements

We would like to thank Jasper Snoek, Sharat Chikkerur, Mihaela Rosca, James Allingham, Alan Weinstein, and the reviewers for helpful discussions and feedback as well as Patrick Cole his support.
