Robust One-Class Classification with Signed Distance Function using
  1-Lipschitz Neural Networks
We propose a new method, dubbed One Class Signed Distance Function (OCSDF),
to perform One Class Classification (OCC) by provably learning the Signed
Distance Function (SDF) to the boundary of the support of any distribution. The
distance to the support can be interpreted as a normality score, and its
approximation using 1-Lipschitz neural networks provides robustness bounds
against $l2$ adversarial attacks, an under-explored weakness of deep
learning-based OCC algorithms. As a result, OCSDF comes with a new metric,
certified AUROC, that can be computed at the same cost as any classical AUROC.
We show that OCSDF is competitive against concurrent methods on tabular and
image data while being way more robust to adversarial attacks, illustrating its
theoretical properties. Finally, as exploratory research perspectives, we
theoretically and empirically show how OCSDF connects OCC with image generation
and implicit neural surface parametrization. Our code is available at
https://github.com/Algue-Rythme/OneClassMetricLearning
Louis Bethune, Paul Novello, Thibaut Boissin, Guillaume Coiffier, Mathieu Serrurier, Quentin Vincenot, Andres Troya-Galvis
# Robust One-Class Classification with Signed Distance Function

###### Abstract

We propose a new method, dubbed One Class Signed Distance Function (OSCDF), to perform One Class Classification (OCC) by provably learning the Signed Distance Function (SDF) to the boundary of the support of any distribution. The distance to the support can be interpreted as a normality score, and its approximation using 1-Lipschitz neural networks provides robustness bounds against \(l2\) adversarial attacks, an under-explored weakness of deep learning-based OCC algorithms. As a result, OCSDF comes with a new metric, certified AUROC, that can be computed at the same cost as any classical AUROC. We show that OCSDF is competitive against concurrent methods on tabular and image data while being way more robust to adversarial attacks, illustrating its theoretical properties. Finally, as exploratory research perspectives, we theoretically and empirically show how OCSDF connects OCC with image generation and implicit neural surface parametrization. Our code is available at [https://anonymous.4open.science/r/CSDFL](https://anonymous.4open.science/r/CSDFL)

Machine Learning, ICML

## 1 Introduction

One class classification (OCC) is an instance of binary classification where all the points of the dataset at hand belong to the same (positive) class. The challenge of this task is to construct a decision boundary without using points from the other (negative) class. It has various safety-critical applications in anomaly detection, for instance to detect banking fraud, cyber-intrusion or industrial defect, in out-of-distribution detection, to prevent wrong decisions of Machine Learning models, or in Open-Set-Recognition. However, OCC algorithms suffer from limitations such as the **lack of negative data**, and **robustness issues**(Azizmalayeri et al., 2022), the latter being an under-explored topic in the OCC spectrum. Even though some algorithms do not use negative examples, many work cope with the lack of negative data with Negative Sampling, either artificially (Sipple, 2020) or using outlier exposure (Hendrycks and Dietterich, 2019; Fort et al., 2021). However, such samplings are often biased or heuristic. As for robustness, although some works design robust algorithms (Goyal et al., 2020; Lo et al., 2022), it is always only empirically demonstrated (Hendrycks and Dietterich, 2019).

In this paper, we introduce a new framework to perform OCC based on the Signed Distance Function (SDF), a function traditionally used in computer graphics. Assume the positive samples are independently and identically obtained from a distribution \(\mathbb{P}_{X}\) with compact support \(\mathcal{X}\subset\mathbb{R}^{d}\). Let \(\partial\mathcal{X}=\overline{\mathcal{X}}/\dot{\mathcal{X}}\) be the boundary of the distribution. The Signed Distance Function is the function \(\mathcal{S}:\mathbb{R}^{d}\rightarrow\mathbb{R}\):

\[\mathcal{S}(x)=\begin{cases}d(x,\partial\mathcal{X})&\text{if }x\in \mathcal{X},\\ -d(x,\partial\mathcal{X})&\text{otherwise},\end{cases} \tag{1}\]

where \(d(x,\partial\mathcal{X})=\inf_{z\in\partial\mathcal{X}}\|x-z\|_{2}\). The idea of our algorithm, which we call One Class Signed Distance Function (OCSDF) is to learn the SDF to the boundary of the positive data distribution and use it as a normality score. We show that the Hinge Kantorovich-Rubinstein (HKR) loss introduced by (Serrurier et al., 2021) allows provably learning the SDF with a 1-Lipschitz network.

SDF exhibits desirable properties. First, by implicitly parametrizing the domain \(\mathcal{X}\), it allows efficiently sampling points outside of \(\mathcal{X}\) and performing principled Negative Sampling. Second, the SDF fulfils the Eikonal equation: \(\|\nabla_{x}\mathcal{S}(x)\|=1\). In particular, \(\mathcal{S}\) is 1-Lipschitz with respect to \(l2\text{-nom : }\forall x,z\in\mathbb{R}^{d},\|\mathcal{S}(x)- \mathcal{S}(z)\|_{2}\leq\|x-z\|_{2}\). This property provides exact robustness certificates for OCSDF in the form of a certified AUROC that can be computed at the same cost as AUROC. This regularity translates into solid empirical robustness as compared to other OCC baselines. In other words, OCSDF alleviates the **lack of negative data** and the **robustness issue**. We go further and highlight in
teresting research perspectives regarding OCSDF. Indeed, we show that learning the SDF with a 1-Lipschitz network enables a generative procedure that allows visualizing points at the boundary of \(\mathcal{X}\). Moreover, It implicitly parametrizes the shape of \(\mathcal{X}\), which connects One-Class Classification with implicit surface parametrization, intensively used in computer graphics for shape reconstruction.

Our contributions are as follows. **(1)** We introduce a new OCC framework based on the Signed Distance Function to the boundary of the data distribution. We theoretically demonstrate that the SDF can be learned with a 1-Lipschitz neural net using the Hinge Kantorovich-Rubinstein (HKR) loss and Negative Sampling; **(2)** We evaluate the performances of OCSDF on several benchmarks and show its benefits for theoretical and empirical robustness; and **(3)** we demonstrate how OCSDF extends the applications of One Class Classification from traditional OOD detection to generative visualization and implicit surface parametrization for shape reconstruction from point clouds.

## 2 Related Work

One Class Classification (OCC)OCC is an instance of binary classification where all the points of the dataset at hand belong to the same (positive) class. The challenge of this task is to construct a decision boundary without using points from the other (negative) class. OCC amounts to finding a domain containing the support of the data distribution. That is why OCC is mainly used in Out Of Distribution (OOD), anomaly or novelty detection, with positive samples considered In Distribution (ID) and negative ones as OOD, anomalies or novelties. This task dates back to (Sager, 1979; Hartigan, 1987) and was popularized for anomaly detection with One-class Support Vector Machines (OC-SVM)(Scholkopf et al., 1999). Since then, the field of OCC has flourished with many well-established algorithms such as Local Outlier Factors (Breunig et al., 2000), Isolation Forests (Liu et al., 2008) and their variants (see (Han et al., 2022) for a thorough benchmark). More recently, since Deep-SVDD (Ruff et al., 2018) - followed by several works such as (Bergman and Hoshen, 2019; Golan and El-Yaniv, 2018; Goyal et al., 2020; Zenati et al., 2018; Sabokrou et al., 2018) - Deep Learning has emerged as a relevant alternative to perform OCC due to its capacities to handle large dimensional data. However, methods of this field suffer from their lack of **robustness and certifications**, which makes them vulnerable to adversarial attacks. In addition, they always struggle to cope with the **lack of OOD data**. In this paper, we tackle these problems with an OCC algorithm based on approximating the SDF using **1-Lipschitz neural nets**. In addition, the SDF being intensively used in Computer Graphics, our algorithm establishes a new link between OCC and **implicit surface** parametrization.

SDF for neural implicit surfacesHistorically, signed distance functions have been used in computer graphics to parametrize a surface as the level set of some function (Novello et al., 2022). Given an incomplete or unstructured representation of a geometrical object (like a 3D point cloud or a triangle soup), recent methods aim at representing a smooth shape either as vectors in the latent space of a generative model (Achlioptas et al., 2018; Ben-Hamu et al., 2018; Groueix et al., 2018; Chou et al., 2022) or directly as parameters of a neural net (Park et al., 2019; Atzmon and Lipman, 2020). The first method allows for easy shape interpolation, while the latter proved to be a more robust approach (Davies et al., 2021). Those neural implicit surfaces alleviate both the problems related to memory requirements of voxel-based representations and the combinatorial nature of meshes, making them ideally suited for rendering using ray marching (Hart, 1995) and constructive solid geometry. In those contexts, the constraint \(\|\nabla_{x}f(x)\|\leq 1\) is necessary to guarantee the validity of the geometrical query while having \(\|\nabla_{x}f(x)\|\) as close as possible to 1 allows for greedier queries and faster computation times. In practice,

Figure 1: Summary of One Class Signed Distance Function (OCSDF). We start with an uniform negative sampling, then we fit a 1-Lipschitz classifier \(f_{\theta}\) using the Hinge Kantorovich-Rubinstein loss. We apply the Adapted Newton Raphson algorithm 1 to attract the points towards the boundary of the domain \(\partial\mathcal{X}\) thanks to the smoothness of \(f_{\theta}\), which in addition allows providing robustness certificates.
training an SDF requires a dataset \((p,d)\) of points \(p\in\mathbb{R}^{3}\) with their corresponding signed distance \(d\) to the desired surface. Computing those distances requires the existence and availability of a ground truth, which is not always the case. Moreover, training tends to be unstable in general, and special care is needed for most computer graphics applications (Sharp and Jacobson, 2022). Our method can instead be trained to approximate a surface without prior knowledge of the distances and is provably robust.

1-Lipschitz neural netsAs noticed in (Bethune et al., 2022; Brau et al., 2023) 1-Lipschitz neural nets (Stasiak and Yatsymirskyy, 2006; Li et al., 2019; Su et al., 2022) are naturally linked to the signed distance function. In particular, they are 1-Lipschitz, i.e. they fulfil \(\|\nabla_{x}f(x)\|\leq 1\) on the whole input space. They boast a rich literature, especially for convolutional neural nets (Gayer and Sheshkus, 2020; Wang et al., 2020; Liu et al., 2021; Achour et al., 2021; Li et al., 2019; Trockman and Kolter, 2021; Singla and Feizi, 2021). These networks benefit from several appealing properties: they are not subject to exploding nor vanishing gradients (Li et al., 2019), they generalize well (Bartlett et al., 2019; Bethune et al., 2022), and they are elegantly connected to optimal transport theory (Arjovsky et al., 2017; Serrurier et al., 2021). 1-Lipschitz neural nets also benefit from certificates against \(l2\)-attacks (Li et al., 2019; Tsuzuku et al., 2018); hence the approximation of \(\mathcal{S}\) is robust against \(l2\)-adversarial attacks _by design_.

Robustness and certificationWhile robustness comes with many aspects, this work focuses mainly on adversarial attacks (Szegedy et al., 2014). Extensive literature explores the construction of efficient attacks (Goodfellow et al., 2014) (Brendel et al., 2018) (Carlini and Wagner, 2017). As nearly any deep learning architecture is vulnerable, defences have also been developed with notably adversarial training (Madry et al., 2018) (Zhang et al., 2019)(Shafahi et al., 2019), or randomized smoothing (Cohen et al., 2019)(Carlini et al., 2022). Since early works pointed out the link between the Lipschitz constant of a network and its robustness, Lipschitz-constrained networks have also been studied (Anil et al., 2019) (Serrurier et al., 2021). Similarly to classifiers, OCC Algorithms based on deep neural nets suffer from their natural weakness to adversarial attacks (Azizmalayeri et al., 2022). Although some works design robust algorithms (Goyal et al., 2020; Lo et al., 2022), the robustness achieved is always only empirically demonstrated (Hendrycks and Dietterich, 2019). Few works provide theoretical certifications (we only found (Bitterwolf et al., 2020) based on interval bounds propagation). In this work, we leverage the properties of 1-Lipschitz networks to provide certifications.

Tackling the lack of OOD dataThe previously mentioned OCC and OOD algorithms, as well as many others (Hendrycks and Gimpel, 2018; Hsu et al., 2020) are designed to avoid the need for OOD data. However, some works aim at falling back to classical binary classification by artificially generating negative samples. The idea of Negative Sampling is not recent and appeared in (Forrest et al., 1994) for detecting computer viruses or to emulate the distinction made by antibodies between pathogens and body cells (Gonzalez et al., 2002). It has been introduced in anomaly detection by (Ayara et al., 2002) and studied by several works summarized in (Jinyin and Dongyong, 2011), but has lost popularity due to its practical inefficiency (e.g. compared to One-Class Support Vector Machines (OCSVM) (Stibor et al., 2005)). Recently, some works revived the idea of using OOD data, either by artificial negative sampling (Lee et al., 2018; Sipple, 2020; Goyal et al., 2020; Pourreza et al., 2021), or by using OOD data from other sources, a procedure called outlier exposure (Fort et al., 2021; Hendrycks et al., 2019). However, outlier exposure suffers from bias since OOD data does not come from the same data space. Therefore, we follow the first idea and sample negative data points close to the domain \(\mathcal{X}\), thanks to the orthogonal neural nets-based estimation of the SDF.

## 3 Method

The method aims to learn the Signed Distance Function (SDF) by reformulating the one-class classification of \(\mathbb{P}_{X}\) as a binary classification of \(\mathbb{P}_{X}\) against a carefully chosen distribution \(Q(\mathbb{P}_{X})\). We show that this formulation yields desirable properties, especially when the chosen classifier is a 1-Lipschitz neural net trained with the Hinge Kantorovich-Rubinstein (HKR) loss.

### SDF learning formulated as binary classification

We formulate SDF learning as a binary classification that consists of classifying samples from \(\mathbb{P}_{X}\) against samples from a complementary distribution, as defined below.

**Definition 1** (\(\overset{B,\epsilon}{\sim}\) Complementary Distribution (informal)): _Let \(Q\) be a distribution of compact support included in \(B\), with disjoint support from that of \(\mathbb{P}_{X}\) that "fills" the remaining space, with \(2\epsilon\) gap between \(\mathcal{X}\) and \(\text{supp}\ Q\). Then we write \(Q\overset{B,\epsilon}{\sim}\mathbb{P}_{X}\)._

A formal definition is given in Appendix A. Binary classification between \(\mathbb{P}_{X}\) and any \(Q\overset{B,\epsilon}{\sim}\mathbb{P}_{X}\) allows the construction of the optimal signed distance function, using the Kantorovich-Rubinstein (HKR) Hinge loss (Serrurier et al., 2021), thanks to the following theorem.

**Theorem 1**: **SDF Learning with HKR loss.** _Let \(\mathcal{L}_{m,\lambda}^{\text{hkr}}(yf(x))=\lambda\max\left(0,m-yf(x)\right)-yf(x)\) be the Hinge Kantorovich Rubinstein loss, with margin \(m=\epsilon\), regularization \(\lambda>0\), prediction \(f(x)\) and label \(y\in\{-1,1\}\). Let \(Q\) be a probability distribution on \(B\). Let \(\mathcal{E}^{\text{hkr}}(f)\) be the population
risk:_

\[\begin{split}\mathcal{E}^{\text{h}kr}(f,\mathbb{P}_{X},Q):=& \mathbb{E}_{x\sim\mathbb{P}_{X}}[\mathcal{L}^{\text{h}kr}_{m,\lambda}(f(x))]\\ &+\mathbb{E}_{z\sim Q}[\mathcal{L}^{\text{h}kr}_{m,\lambda}(-f(z)) ].\end{split} \tag{2}\]

_Let \(f^{*}\) be the minimizer of population risk, whose existence is guaranteed with Arzela-Ascoli theorem (Bethune et al., 2022):_

\[f^{*}\in\operatorname*{arg\,inf}_{f\in\mathcal{L}\text{ip}_{1}(\mathbb{R}^{d}, \mathbb{R})}\mathcal{E}^{\text{h}kr}(f,\mathbb{P}_{X},Q), \tag{3}\]

_where \(\text{Lip}_{1}(\mathbb{R}^{d},\mathbb{R})\) is the set of Lipschitz functions \(\mathbb{R}^{d}\to\mathbb{R}\) of constant \(1\). Assume that \(Q\overset{B,\epsilon}{\sim}\mathbb{P}_{X}\). **Then**, \(f^{*}\) approximates the signed distance function over \(B\):_

\[\begin{split}\forall x\in\mathcal{X},&\mathcal{S}(x )=f^{*}(x)-m,\\ \forall z\in\text{supp }Q,&\mathcal{S}(z)=f^{*}(z)-m.\end{split} \tag{4}\]

_Moreover, for all \(x\in\text{supp }Q\cup\mathcal{X}\):_

\[\text{sign}(f(x))=\text{sign}(\mathcal{S}(x)).\]

Note that if \(m=\epsilon\ll 1\), then we have \(f^{*}(x)\approx\mathcal{S}(x)\). In this work, we parametrize \(f\) as a 1-Lipschitz neural network, as defined below, because they fulfil \(f\in\text{Lip}_{1}(\mathbb{R}^{d},\mathbb{R})\) by construction.

**Definition 2** (1-Lipschitz neural network (informal)):

_Neural network with Groupsort activation function and orthogonal transformation in affine layers parameterized like in (Anil et al., 2019)._

Details about the implementation can be found in Appendix D. Theorem 1 tells us that if we characterize the complementary distribution \(Q\), we can approximate the SDF with a 1-Lipschitz neural classifier trained with HKR loss. We now need to find the complementary distribution \(Q\).

### Finding the complementary distribution by targeting the boundary

We propose to seek \(Q\) through an alternating optimization process: at every iteration \(t\), a proposal distribution \(Q_{t}\) is used to train a 1-Lipschitz neural net classifier \(f_{t}\) against \(\mathbb{P}_{X}\) by minimizing empirical HKR loss. Then, the proposal distribution is updated in \(Q_{t+1}\) based on the loss induced by \(f_{t}\), and the procedure is repeated.

We suggest starting from the uniform distribution: \(Q_{0}=\mathcal{U}(B)\). Observe that in high dimension, due to the _curse of dimensionality_, a sample \(z\sim Q_{0}\) is unlikely to satisfy \(z\in\mathcal{X}\). Indeed the data lies on a low dimensional manifold \(\mathcal{X}\) for which the Lebesgue measure is negligible compared to \(B\). Hence, in the limit of small sample size \(n\ll\infty\), a sample \(Z_{n}\sim Q_{0}^{\otimes n}\) fulfills \(Z_{n}\overset{B,\epsilon}{\sim}\mathbb{P}_{X}\). This phenomenon is called the Concentration Phenomenon and has already been leveraged in anomaly detection in (Sipple, 2020). However, the _curse_ works both ways and yields a high variance in samples \(Z_{n}\). Consequently, the variance of the associated minimizers \(f_{0}\) of equation 3 will also exhibit a high variance, which may impede the generalization and convergence speed. Instead, the distribution \(Q_{t}\) must be chosen to produce higher density in the neighborhood of the boundary \(\partial\mathcal{X}\). The true boundary is unknown, but the level set \(\mathbb{L}_{t}=f_{t}^{-1}(\{-\epsilon\})\) of the classifier can be used as a proxy to improve the initial proposal \(Q_{0}\). We start from \(z_{0}\sim Q_{0}\), and then look for a displacement \(\delta\in\mathbb{R}^{d}\) such that \(z+\delta\in\mathbb{L}_{t}\). To this end, we take inspiration from the multidimensional Newton-Raphson method and consider a linearization of \(f_{t}\):

\[f_{t}(z_{0}+\delta)\approx f_{t}(z_{0})+\langle\nabla_{x}f_{t}(z_{0}),\delta\rangle. \tag{5}\]

Since 1-Lipschitz neural nets with GroupSort activation function are piecewise _affines_(Tanielian and Biau, 2021), the linearization is locally exact, hence the following property.

**Property 1**.: _Let \(f_{t}\) be a 1-Lipschitz neural net with GroupSort activation function. Almost everywhere on \(z_{0}\in\mathbb{R}^{d}\), there exists \(\delta_{0}>0\) such that for every \(\|\delta\|\leq\delta_{0}\), we have:_

\[f_{t}(z_{0}+\delta)=f_{t}(z_{0})+\langle\nabla_{x}f_{t}(z_{0}),\delta\rangle. \tag{6}\]

Since \(f_{t}(z_{0}+\delta)\in\mathbb{L}_{t}\) translates into \(f_{t}(z_{0}+\delta)=-\epsilon\),

\[\delta=-\frac{f_{t}(z_{0})+\epsilon}{\|\nabla_{x}f_{t}(z_{0})\|^{2}}\nabla_{x}f _{t}(z_{0}). \tag{7}\]

Properties of \(\mathcal{L}^{\text{h}kr}_{m,\lambda}\) ensure that the optimal displacement follows the direction of the gradient \(\nabla_{x}f_{t}(z_{0})\), which coincides with the direction of an optimal transportation plan (Serrurier et al., 2021). The term \(\|\nabla_{x}f_{t}(z_{0})\|\) enjoys an interpretation as a Local Lipschitz Constant (see (Jordan and Dimakis, 2020)) of \(f_{t}\) around \(z_{0}\), which we know fulfills \(\|\nabla_{x}f_{t}(z_{0})\|\leq 1\) when parametrized with an 1-Lipschitz neural net. When \(f_{t}\) is trained to perfection, the expression for \(\delta\) simplifies to \(\delta=-f_{t}(z_{0})\nabla_{x}f_{t}(z_{0})\) thanks to Property 2.

**Property 2** (Minimizers of \(\mathcal{L}^{\text{h}kr}_{m,\lambda}\) are Gradient Norm Preserving (from (Serrurier et al., 2021))).: _Let \(f_{t}^{*}\) be the solution of Equation 3. Then for almost every \(z\in B\) we have \(\|\nabla_{x}f_{t}^{*}(z)\|=1\)._

In practice, the exact minimizer \(f_{t}^{*}\) is not always retrieved, but equation 7 still applies to imperfectly fitted classifiers. The final sample \(z^{\prime}\sim Q_{t}\) is obtained by generating a sequence of \(T\) small steps to smooth the generation. The procedure is summarized in algorithm 1. In practice, \(T\) can be chosen very low (below \(16\)) without significantly hurting the quality of generated samples. Finally, we pick a random "learning rate" \(\eta\sim\mathcal{U}([0,1])\) for each negative example in
the batch to ensure they distribute evenly on the path toward the boundary. The procedure also benefits from Property 4, which ensures that the distribution \(Q_{t+1}\) obtained from \(Q_{t}\) across several iterative applications of Algorithm 1 still fulfils \(Q^{t+1}\stackrel{{ B,\epsilon}}{{\sim}}\mathbb{P}_{X}\). The proof is given in Appendix B.

**Property 3** (Complementary distributions are fix points).: _Let \(Q^{t}\) be such that \(Q^{t}\stackrel{{ B,\epsilon}}{{\sim}}\mathbb{P}_{X}\). Assume that \(Q^{t+1}\) is obtained with algorithm 2. Then we have \(Q^{t+1}\stackrel{{ B,\epsilon}}{{\sim}}\mathbb{P}_{X}\)._

```
0: 1-Lipschitz neural net \(f_{t}\)
0: number of steps \(T\)
0: sample \(z^{\prime}\sim Q_{t}(f)\)
1: sample learning rate \(\eta\sim\mathcal{U}([0,1])\)
2:\(z_{0}\sim\mathcal{U}(B)\) { Initial approximation.}
3:for each step \(t=1\) to \(T\)do
4:\(z_{t+1}\gets z_{t}-\frac{\nabla_{x}f(z^{t})}{\|\nabla_{x}f(z^{t})\|_{2}^{ 2}}(f(z_{t})+\epsilon)\){Refining.}
5:\(z_{t+1}\leftarrow\Pi_{B}(z_{t+1})\){ Stay in feasible set.}
6:endfor
7:return\(z_{T}\)
```

**Algorithm 1** Adapted Newton-Raphson for Complementary Distribution Generation

**Remark**.: _In high dimension \(d\gg 1\), when \(\|\nabla_{x}f_{t}(z)\|=1\) and \(\text{Vol}(B)\gg\text{Vol}(\mathcal{X})\) the samples obtained with algorithm 1 are approximately uniformly distributed in the levels of \(f_{t}\). It implies that the density of \(Q\) increases exponentially fast (with factor \(d\)) with respect to the value of \(-|f_{t}(\cdot)|\). This mitigates the adverse effects of the curse of dimensionality._

This scheme of "generating samples by following gradient in input space" reminds diffusion models (Ho et al., 2020), feature visualization tools (Olah et al., 2017), or recent advances in VAE (Kuzina et al., 2022). However, no elaborated scheme is required for the training of \(f_{t}\): 1-Lipschitz networks exhibit smooth and interpretable gradients (Serrurier et al., 2022) which allows sampling from \(\mathcal{X}\) "for free" as illustrated in figure 4.

**Remark**.: _A more precise characterization of \(Q_{t}\) built with algorithm 1 can be sketched below. Our approach bares some similarities with the spirit of Metropolis-adjusted Langevin algorithm (Grenander and Miller, 1994). In this method, the samples of \(p(x)\) are generated by taking the stationary distribution \(x_{t\to\infty}\) of a continuous Markov chain obtained from the stochastic gradient step iterates_

\[x_{t+1}\gets x_{t}+\zeta\nabla_{x}\log p(x)+\sqrt{2\zeta}Z \tag{8}\]

_for some distribution \(p(x)\) and \(Z\sim\mathcal{N}(\mathbf{0},\mathbf{1})\). By choosing the level set \(\epsilon=0\), and \(p(x)\propto\mathds{1}\{f(x)\leq 0\}\exp{(-\eta f^{2}(x))}\) the score function \(\zeta\nabla_{x}\log p(x)\) is transformed into \(\nabla_{x}f(x)|f(x)|\) with \(\zeta=\frac{\eta}{T}\). Therefore, we see that the density decreases exponentially faster with the squared distance to the boundary \(\partial\mathcal{X}\) when there are enough steps \(T\gg 1\). In particular when \(\mathcal{X}=\{0\}\) we recover \(p(x)\) as the pdf of a standard Gaussian \(\mathcal{N}(\mathbf{0},\mathbf{1})\). Although the similarity is not exact (e.g., the diffusion term \(\sqrt{2\eta}Z\) is lacking, \(T\) is low, \(\eta\sim\mathcal{U}([0,1])\) is a r. v.), it provides interesting complementary insights on the algorithm._

### Alternating minimization for SDF learning

Each classifier \(f_{t}\) does not need to be trained from scratch. Instead, the same architecture is kept throughout training, and the algorithm produces a sequence of parameters \(\theta_{t}\) such that \(f_{t}=f_{\theta_{t}}\). Each set of parameters \(\theta_{t}\) is used as initialization for the next one \(\theta_{t+1}\). Moreover, we only perform a low fixed number of parameter updates for each \(t\) in a GAN fashion. The final procedure of OCSDF is shown in Figure 1 and detailed in algorithm 3 of Appendix B.

## 4 Properties

### Certificates against adversarial attacks

The most prominent advantage of 1-Lipschitz neural nets is their ability to produce certificates against adversarial attacks (Szegedy et al., 2014). Indeed, by definition we have \(f(x+\delta)\in[f(x)-\|\delta\|,f(x)+\|\delta\|]\) for every example \(x\in\mathcal{X}\) and every adversarial attack \(\delta\in\mathbb{R}^{d}\). This allows bounding the changes in AUROC score of the classifier for every possible radius \(\epsilon>0\) of adversarial attacks.

**Proposition 1** (certifiable AUROC).: _Let \(F_{0}\) be the cumulative distribution function associated with the negative classifier's prediction (when \(f(x)\leq 0\)), and \(p_{1}\) the probability density function of the positive classifier's prediction (when \(f(x)>0\)). Then, for any attack of radius \(\epsilon>0\), the AUROC of the attacked classifier \(f_{\epsilon}\) can be bounded by_

\[\text{AUROC}_{\epsilon}(f)=\int_{-\infty}^{\infty}F_{0}(t)p_{1}(t-2\epsilon)dt. \tag{9}\]

The proof is left in Appendix C. The certified AUROC score can be computed analytically without performing the attacks empirically, solely from score predictions \(f_{1}(\tau-2\epsilon)\). More importantly, the certificates hold against _any_ adversarial attack whose \(l2\)-norm is bounded by \(\epsilon\), regardless of the algorithm used to perform such attacks. We emphasize that producing certificates is more challenging than traditional defence mechanisms (e.g, adversarial training, see (Bai et al., 2021) and references therein) since they do not target defence against a specific attack method.

### Rank normal and anomalous examples

Beyond the raw certifiable AUROC score, l2-based Lipschitz robustness enjoys another desirable property: the farther from the boundary the adversary is, the more important the attack budget required to change the score. In practice, it means that an attacker can only dissimulate anomalies already close to being "normal". Aberrant and hugely anomalous examples will require a larger (possibly impracticable) attack budget. This ensures that the _normality score_ is ac
tually meaningful: it is high when the point is central in its local cloud, and low when it is far away.

## 5 Experiments

In this section, we evaluate the performances and properties of OCSDF on tabular and image data. All the experiments are conducted with tensorflow, and the 1-Lipschitz neural nets are implemented using the library Deel-Lip1.

Footnote 1: [https://github.com/deel-ai/deel-lip](https://github.com/deel-ai/deel-lip)

### Toy examples from Scikit-Learn

We use two-dimensional toy examples from the Scikit-Learn library (Pedregosa et al., 2011). Results are shown in figure 2. The contour of the decision function are plotted in resolution \(300\times 300\) pixels. The level sets of the classifier are compared against those of One Class SVM (Scholkopf et al., 2001) and Isolation Forest (Liu et al., 2008). We also train a conventional network with Binary Cross Entropy against complementary distribution \(Q_{t}\), and we show it struggles to learn a meaningful decision boundary. Moreover, its Local Lipschitz Constant (Jordan and Dimakis, 2020) increases uncontrollably, as shown in table 7, which makes it prone to adversarial attacks. Finally, there is no natural interpretation of the prediction of the conventional network in terms of distance: the magnitude \(|f(\cdot)|\) of the predictions quickly grows above \(1e-3\), whereas for 1-Lipschitz neural nets, it is approximately equal to the signed distance function \(\mathcal{S}\). We refer to Appendix E for visualizations.

### Anomaly Detection on Tabular datasets

We tested our algorithm on some of the most prominent anomaly detection benchmarks of ODDS library (Rayana, 2016). In this unsupervised setting (like ADBench (Han et al., 2022)) all the examples (normal examples and anomalies) are seen during training, but their true label is unknown. To apply our method, the only hyperparameter needed is the margin \(m\) that we select in the range \([0.01,0.05,0.2,1.]\). For each value, the results are averaged over \(20\) independent runs train/test splits. Following ADBench guidelines and best practices from the AD community, we only compute the AUROC, since this metric is symmetric under label flip. We report the best average in table 2 along baselines from ADBench (Han et al., 2022). As observed by (Han et al., 2022), none of the algorithms clearly dominates the others, because what is considered an anomaly (or not) depends on the context. Among 14 other methods tested, our algorithm ranks \(7.1\pm 3.6/15\), while the best (Isolation Forests) ranks \(4.5\pm 3.2/15\). The experiment shows that our algorithm is competitive with respect to other broadly used baselines. Nonetheless, it brings several additional advantages. First, our algorithm can be seen as a parametric version of kNN for the euclidean distance, which leverages deep learning to avoid the costly construction of structures like a KDTree (Maneewongvatana and Mount, 1999) and the quadratic cost of nearest neighbor search, thereby enabling its application in high dimensions. Second, it provides robustness certificates. We illustrate those two advantages more thoroughly in the next section.

### One Class learning on images

We evaluate the performances of OCSDF for OCC, where only samples of the normal class are supposed to be available. To emulate this setting, we train a classifier on each of the classes of MNIST and Cifar10, and evaluate it on an independent test set in a _one-versus-all_ fashion. We compare our method against DeepSVDD (Ruff et al., 2018), OCSVM (Scholkopf et al., 1999), and Isolation Forests (Liu et al., 2008). Details on the implementation of the baselines can be found in Appendix G. The mean AUROC score is reported in table 3 and averaged over \(20\) runs. It is computed between the \(1,000\) test examples of the target class and the remaining \(9,000\) examples from other classes of the train set (both unseen during training). The histograms of normality scores are given in Appendix G. OCSDF is competitive with respect to other baselines. In addition, it comes with several advantages described in the following.

#### 5.3.1 Certifiable and empirical robustness

For each class, we compute the certified AUROC score of each digit for \(12\) attacks of radii \(\epsilon\in\{8/255,16/255,36/255\}\). None of the concurrent methods can provide certificates against \(12\) attacks: in the work of (Goyal et al., 2020) the attacks are performed empirically (no certificates) with l-\(\infty\) radii. In table 3, we report our certifiable AUROC with various radii \(\epsilon\in\{0,8/25,16/255,36/255,72/255\}\). In figure 3 we report

\begin{table}
\begin{tabular}{|c|c c c c c|} \hline \multirow{2}{*}{Local Lipschitz Constant} & One & Two & Two & Blob & Two \\  & Cloud & Clouds & Blobs & Cloud & Moons \\ \cline{2-6}  & 26.66 & 122.84 & 1421.41 & 53.90 & 258.73 \\ \hline \end{tabular}
\end{table}
Table 1: Lower bound on the Local Lipschitz Constant (LLC) of conventional network after \(10,000\) training steps for each toy example. It is the maximum of \(\|\nabla_{x_{i}}f(x_{i})\|\) over the train set.

Figure 2: Contour plots of our method with 1-Lipschitz (LIP) network and \(\mathcal{L}_{m,\lambda}^{\text{thr}}\) (HKR) loss on toy examples of Scikit-learn.
\begin{table}
\begin{tabular}{c c c c|c c c c c c c} \hline \hline \multirow{2}{*}{Dataset} & \multirow{2}{*}{\(d\)} & \multirow{2}{*}{\#no.+an.} & \multirow{2}{*}{perc.} & \multicolumn{2}{c}{OSCDF} & \multicolumn{2}{c}{Deep} & \multicolumn{2}{c}{OC} & \multicolumn{2}{c}{IF} & PCA & \multicolumn{1}{c}{kNN} & \multicolumn{1}{c}{SOTA} \\  & & & & & SVDD & & SVM & & & & \\ \hline \hline \multicolumn{1}{c|}{Average Rank} & \multicolumn{1}{c}{\(7.1\pm 3.6\)} & \(11.2\) & \(7.5\) & \(4.5\) & \(5.7\) & \(7.8\) & \(4.5\pm 3.2\) (IF) \\ \hline breastw & 9 & 444+239 & \(35\%\) & \((\#10)\)\(82.6\pm 5.9\) & \(65.7\) & \(80.3\) & \(98.3\) & \(95.1\) & \(97.0\) & \(99.7\) (COPD) \\ cardio & 21 & 1,655+176 & \(9.6\%\) & \((\#2)\)\(95.0\pm 0.1\) & \(59.0\) & \(93.9\) & \(93.2\) & \(95.5\) & \(76.6\) & \(95.5\) (PCA) \\ glass & 9 & 205+9 & \(4.2\%\) & \((\#7)\)\(73.9\pm 4.1\) & \(47.5\) & \(35.4\) & \(77.1\) & \(66.3\) & \(82.3\) & \(82.9\) (CBLOF) \\ http (KDDCup99) & 3 & 565,287+2,211 & \(0.4\%\) & \((\#11)\)\(67.5\pm 37\) & \(69.0\) & \(99.6\) & \(99.6\) & \(99.7\) & \(03.4\) & \(99.96\) (IF) \\ Ionosphere & 33 & 225+126 & \(36\%\) & \((\#7)\)\(80.2\pm 0.1\) & \(50.9\) & \(75.9\) & \(84.5\) & \(79.2\) & \(88.3\) & \(90.7\) (CBLOF) \\ Lymphography & 18 & 142+6 & \(4.1\%\) & \((\#8)\)\(96.1\pm 4.9\) & \(32.3\) & \(99.5\) & \(99.8\) & \(99.8\) & \(55.9\) & \(99.8\) (CBLOF) \\ mammography & 6 & 10,923+260 & \(2.32\%\) & \((\#6)\)\(86.0\pm 2.5\) & \(57.0\) & \(84.9\) & \(86.4\) & \(88.7\) & \(84.5\) & \(90.7\) (ECOD) \\ musk & 166 & 2,965+97 & \(3.2\%\) & \((\#8)\)\(92.6\pm 20.\) & \(43.4\) & \(80.6\) & \(99.99\) & \(100.0\) & \(69.9\) & \(100.0\) (PCA) \\ Optdigits & 64 & 5,066+150 & \(3\%\) & \((\#12)\)\(51.0\pm 0.9\) & \(38.9\) & \(54.0\) & \(70.9\) & \(51.7\) & \(41.7\) & \(87.5\) (CBLOF) \\ Pima & 8 & 500+268 & \(35\%\) & \((\#12)\)\(60.7\pm 1.0\) & \(51.0\) & \(66.9\) & \(72.9\) & \(70.8\) & \(73.4\) & \(73.4\) (kNN) \\ satimage-2 & 36 & 5,732+71 & \(1.2\%\) & \((\#3)\)\(97.9\pm 0.4\) & \(53.1\) & \(97.3\) & \(99.2\) & \(97.6\) & \(92.6\) & \(99.8\) (CBLOF) \\ Shuttle & 9 & 45,586+3,511 & \(7\%\) & \((\#4)\)\(99.1\pm 0.3\) & \(52.1\) & \(97.4\) & \(99.6\) & \(98.6\) & \(69.6\) & \(99.6\) (IF) \\ smtp (KDDCup99) & 3 & 95,126+30 & \(0.03\%\) & \((\#4)\)\(87.1\pm 3.5\) & \(78.2\) & \(80.7\) & \(89.7\) & \(88.4\) & \(89.6\) & \(89.7\) (IF) \\ speech & 400 & 3,625+61 & \(1.65\%\) & \((\#15)\)\(46.0\pm 0.2\) & \(53.4\) & \(50.2\) & \(50.7\) & \(50.8\) & \(51.0\) & \(56.0\) (COF) \\ thyroid & 6 & 3,679+93 & \(2.5\%\) & \((\#5)\)\(95.9\pm 0.0\) & \(49.6\) & \(87.9\) & \(98.3\) & \(96.3\) & \(95.9\) & \(98.3\) (IF) \\ vertebral & 6 & 210+30 & \(12.5\%\) & \((\#4)\)\(48.6\pm 2.6\) & \(36.7\) & \(38.0\) & \(36.7\) & \(37.0\) & \(33.8\) & \(53.2\) (DAGMM) \\ vowels & 12 & 1,406+50 & \(3.4\%\) & \((\#2)\)\(94.7\pm 0.7\) & \(52.5\) & \(61.6\) & \(73.9\) & \(65.3\) & \(97.3\) & \(97.3\) (kNN) \\ WBC & 30 & 357+21 & \(5.6\%\) & \((\#10)\)\(93.6\pm 0.1\) & \(55.5\) & \(99.0\) & \(99.0\) & \(98.2\) & \(90.6\) & \(99.5\) (CBLOF) \\ Wine & 13 & 119+10 & \(7.7\%\) & \((\#5)\)\(81.5\pm 0.9\) & \(59.5\) & \(73.1\) & \(80.4\) & \(84.4\) & \(45.0\) & \(91.4\) (HBOS) \\ \hline \hline \end{tabular}
\end{table}
Table 2: AUROC score for tabular data, averaged over \(20\) runs. The dimension of the dataset is denoted by \(d\). In the **Anomaly Detection protocol (AD)** we use all the data (normal class and anomalies) for training, in an unsupervised fashion. The “#no.+an.” column indicates part of normal (no.) and anomalous (an.) data used during training for each protocol. SOTA denominates the best sore ever reported on the dataset, obtained by crawling relevant literature, or ADBench (Han et al., 2022) results (table D4 page 37). We report the rank as (#rank) among 14 other methods.

\begin{table}
\begin{tabular}{c|c c c c c|c c c} \hline \hline MNIST & OCSDF & OCSDF & OCSDF & OCSDF & OC & Deep & IF \\ Certificates & \(\epsilon=0\) & \(\epsilon=8/255\) & \(\epsilon=16/255\) & \(\epsilon=36/255\) & \(\epsilon=72/255\) & \(\epsilon=0\) & \(\epsilon=0\) & \(\epsilon=0\) \\ \hline mAUROC & \(\mathbf{95.5\pm 0.4}\) & \(93.2\pm 2.1\) & \(89.9\pm 3.5\) & \(78.4\pm 6.4\) & \(57.5\pm 7.5\) & \(91.3\pm 0.0\) & \(\mathbf{94.8\pm 0.9}\) & \(92.3\pm 0.5\) \\ \hline digit 0 & \(\mathbf{99.7\pm 0.1}\) & \(99.6\pm 0.2\) & \(99.5\pm 0.2\) & \(99.0\pm 0.6\) & \(96.2\pm 3.0\) & \(98.6\pm 0.0\) & \(98.0\pm 0.7\) & \(98.0\pm 0.3\) \\ digit 1 & \(\mathbf{99.8\pm 0.0}\) & \(99.7\pm 0.0\) & \(99.6\pm 0.1\) & \(99.2\pm 0.3\) & \(96.2\pm
the empirical AUROC against l2-PGD attacks with three random restarts, using stepsize \(\zeta=0.25\)\(\epsilon\) like Foolbox (Rauber et al., 2020). These results illustrate our method's benefits: not only does it come with robustness certificates that are verified empirically, but the empirical robustness is also way better than DeepSVD, especially for Cifar10.

#### 5.3.2 Visualization of the support

OCSDF can be seen as a parametric version of kNN, which enables this approach in high dimensions. As a result, the decision boundary learned by the classifier can be materialized by generating adversarial examples with algorithm 1. The forward computation graph is a classifier based on optimal transport, and the backward computation graph is an image generator. Indeed, the back-propagation through a convolution is a transposed convolution, a popular layer in the generator of GANs. Overall, the algorithm behaves like a WGAN (Arjovsky et al., 2017) with a single network fulfilling both roles. This unexpected feature opens a path to the explainability of the One Class classifier: the support learned can be visualized without complex feature visualization tools. In particular, it helps identify failure modes.

#### 5.3.3 Limitations

We also tested our algorithm on the Cats versus Dogs dataset by training on Cats as the One Class and using Dogs as OOD examples. On this high-dimensional dataset, the AUROC barely exceeds \(55.0\%\). This suggests that the SDF is relevant for tabular and simple image datasets (e.g MNIST) but fails to generalize in a meaningful way in higher dimensions. This is expected: the euclidean norm is notoriously a bad measure of similarity in pixel space. A more thorough discussion of the limitations is left in Appendix H.

## 6 OCSDF for implicit shape parametrization

Our approach to learning the SDF contrasts with the computer graphics literature, where SDF is used to obtain the distance of a point to a surface (here defined as \(\partial\mathcal{X}\)). Indeed, SDFs are usually learned in a supervised fashion, requiring the ground truth of \(l2\) distance. This is classically achieved using Nearest-Neighbor algorithms, which can be cumbersome, especially when the number of points is high. Efficient data structures (e.g., using K-dtrees (Maneewong-vatana and Mount, 1999) or Octrees (Meagher, 1980)) can mitigate this effect but do not scale well to high dimensions. Instead, OCSDF learns the SDF solely based on points contained in the support.

To illustrate this, we use models from Princeton's ModelNet10 dataset (Wu et al., 2015). We sample \(n=2048\) within each mesh to obtain a 3d point cloud. We fit the SDF on the point cloud with the same hyperparameters as the tabular experiment. We use Lewiner marching algorithm (Lewiner et al., 2003) from scikit-image (Van der Walt et al., 2014), on a \(200\times 200\times 200\) voxelization of the input. We plot the mesh reconstructed with Trimesh (Dawson-Haggerty et al., 2019). The results are highlighted in figure 5. We chose the first percentile of \(\mathbb{E}_{x\sim\mathbb{P}_{X}}[f(x)]\) as the level set of the iso-surface we plot. We compare our results visually against a baseline from (Boltcheva and Levy, 2017) implemented in Graphite (Levy, 2022) that rebuilds the mesh solely from the point cloud (without extrapolation). We highlight that \(n=2048\) is considered low resolution; hence many details are expected to be lost. Nonetheless, our method recovers the global aspect of the shape more faithfully, smoothly, and consistently than the other baseline.

## 7 Conclusion

We proposed a new approach to One-Class Classification, OCSDF, based on the parametrization of the Signed Distance Function to the boundary of the known distribution using a 1-Lipschitz neural net. OCSDF comes with robustness guarantees and naturally tackles the lack of negative data inherent to OCC. Finally, this new method extends OCC beyond Out-of-Distribution detection and allows ap

Figure 4: Examples from algorithm 1 with \(T=64\) and \(\eta=1\).

Figure 5: Visualization of the SDF (3rd row) from sparse point clouds of size \(2048\) (2nd row) sampled from ground truth meshes (1st row) with _Trimesh_ library, against a baseline from _Graphite_ tool that attempts to reproduce the meshes solely from a point cloud (4th row). The SDF exhibits better extrapolation properties and provides smooth surfaces.

Figure 3: Empirical Mean AUROC on all classes against adversarial attacks of various radii in One Class setting, using default parameters of FoolBox (Rauber et al., 2020).
plying it to surface parametrization from point clouds and generative visualization.
