Fully Hyperbolic Convolutional Neural Networks for Computer Vision
Real-world visual data exhibit intrinsic hierarchical structures that can be
represented effectively in hyperbolic spaces. Hyperbolic neural networks (HNNs)
are a promising approach for learning feature representations in such spaces.
However, current HNNs in computer vision rely on Euclidean backbones and only
project features to the hyperbolic space in the task heads, limiting their
ability to fully leverage the benefits of hyperbolic geometry. To address this,
we present HCNN, a fully hyperbolic convolutional neural network (CNN) designed
for computer vision tasks. Based on the Lorentz model, we generalize
fundamental components of CNNs and propose novel formulations of the
convolutional layer, batch normalization, and multinomial logistic regression.
{Experiments on standard vision tasks demonstrate the promising performance of
our HCNN framework in both hybrid and fully hyperbolic settings.} Overall, we
believe our contributions provide a foundation for developing more powerful
HNNs that can better represent complex structures found in image data. Our code
is publicly available at https://github.com/kschwethelm/HyperbolicCV.
Ahmad Bdeir, Kristian Schwethelm, Niels Landwehr
# Hyperbolic Geometry in Computer Vision: A Novel Framework for Convolutional Neural Networks

###### Abstract

Real-world visual data exhibit intrinsic hierarchical structures that can be represented effectively in hyperbolic spaces. Hyperbolic neural networks (HNNs) are a promising approach for learning feature representations in such spaces. However, current HNNs in computer vision rely on Euclidean backbones and only project features to the hyperbolic space in the task heads, limiting their ability to fully leverage the benefits of hyperbolic geometry. To address this, we present HCNN, the first fully hyperbolic convolutional neural network (CNN) designed for computer vision tasks. Based on the Lorentz model, we generalize fundamental components of CNNs and propose novel formulations of the convolutional layer, batch normalization, and multinomial logistic regression. Experimentation on standard vision tasks demonstrates the superiority of our HCNN framework and the Lorentz model in both hybrid and fully hyperbolic settings. Overall, we believe our contributions provide a foundation for developing more powerful HNNs that can better represent complex structures found in image data. Our code is publicly available at [https://github.com/kschwethelm/HyperbolicCV](https://github.com/kschwethelm/HyperbolicCV).

## 1 Introduction

Representation learning is a fundamental aspect of deep neural networks, as obtaining an optimal representation of the input data is crucial. While Euclidean geometry has been the traditional choice for representing data due to its intuitive properties and well-defined vector space, recent research has highlighted the advantages of hyperbolic geometry in describing hierarchical structures found in many datasets. Specifically, hyperbolic manifolds allow for distances to scale exponentially with respect to the radius, which matches the exponential scaling of tree distances between hierarchy nodes, preventing spatial distortion and information loss [47]. In addition, research has found that even the natural spatial representations in the human brain exhibit a hyperbolic geometry [53].

Leveraging this better representative capacity, hyperbolic neural networks (HNNs) have demonstrated superior performance compared to Euclidean models in many natural language processing (NLP) and graph embedding tasks [42]. However, hierarchical structures are not limited to textual and graph-structured data but can also be found in images. The notion of hierarchy in images is particularly established through the concept of part-whole relationships within object representations and classes. In addition, Khrulkov et al. [22] have found high hyperbolicity in image datasets, where the hyperbolicity measure represents the degree of innate hierarchy between semantic embeddings and can highlight the potential benefits of using HNNs for representation learning.

In light of these findings, recent works have begun integrating hyperbolic geometry into vision architectures. Specifically, they rely on the Poincare ball and the Lorentz model as descriptors of hyperbolic space and formalize hyperbolic translations of neural network layers. This is challenging due to ill-defined hyperbolic analogs of, e.g., addition, multiplication, and statistical measures.
Currently, most HNN components are only available in the Poincare ball as it supports the gyrovector space with basic vector operations. However, due to its hard numerical constraint, the Poincare ball is more susceptible to numerical instability than the Lorentz model [36], which motivates introducing the missing layers for the Lorentz model. Moreover, HNNs in computer vision have been limited to hybrid architectures that can not fully leverage the advantages of hyperbolic geometry as they rely on Euclidean backbones to learn hyperbolic representations. Until now, fully hyperbolic architectures are missing in computer vision, although prevalent in NLP and graph applications [42].

In this work, we present HCNN, the first fully hyperbolic framework for vision tasks. We generalize the ubiquitous convolutional neural network (CNN) architecture to the Lorentz model and present novel hyperbolic formulations of the convolutional layer, batch normalization, and multinomial logistic regression. Our methodology is general, and we show that our components can be easily integrated into existing architectures. This is done through experiments based on direct translations of Euclidean architectures. Our contributions then become three-fold:

1. We propose the first fully hyperbolic CNN for image data, dubbed HCNN, introducing the fully hyperbolic setting in computer vision.
2. We provide missing Lorentzian formulations of the convolutional layer, batch normalization, and multinomial logistic regression, further filling the gap between HNNs in the Lorentz model and the Poincare ball.
3. We empirically demonstrate the superior performance of HCNNs in experiments on standard vision tasks, including image classification and generation.

## 2 Related work

Hyperbolic image embeddingsPrevious research on HNNs in computer vision has mainly focused on combining Euclidean backbones and hyperbolic embeddings. This approach involves projecting Euclidean embeddings onto the hyperbolic space in the task heads and designing task-related objective functions based on hyperbolic geometry. Such simple hybrid architectures have been proven effective in various vision tasks like recognition [16; 22; 31], segmentation [1; 20], reconstruction/generation [35; 38; 40; 44], and metric learning [10; 51]. However, Guo et al. [16] have shown that learning a mixture of Euclidean and hyperbolic features can exacerbate gradient vanishing, and it remains unclear whether these hybrid models can fully exploit the properties of hyperbolic geometry. In contrast, our HCNN naturally learns latent hyperbolic feature representations in every layer, mitigating these issues. We also forgo the typically used Poincare ball in favor of the Lorentz model, as it offers better stability and optimization properties [36].

Fully hyperbolic neural networksDesigning fully hyperbolic HNNs requires generalizing all Euclidean network components to hyperbolic geometry. Notably, Ganea et al. [12] and Shimizu et al. [49] utilized the Poincare ball and the gyrovector space to generalize various layers, including fully-connected, convolutional, and attention layers, as well as operations like split, concatenation, and multinomial logistic regression (MLR). Researchers have also designed components in the Lorentz model [5; 11; 39; 44], but crucial components for vision, like the standard convolutional layer and the MLR classifier, are still missing. Among the hyperbolic layer definitions, fully hyperbolic neural networks have been built for various tasks in NLP and graph applications [42]. However, no fully

Figure 1: In contrast to hybrid HNNs that use a Euclidean CNN for feature extraction, our HCNN learns features in hyperbolic spaces in every layer, fully leveraging the benefits of hyperbolic geometry. This leads to better image representations and performance.
hyperbolic architecture has yet been utilized in computer vision. Our work provides formulations for missing components in the Lorentz model, allowing for the first fully hyperbolic vision CNNs.

Normalization in HNNsThere are few attempts at translating standard normalization layers to the hyperbolic setting. To the best of our knowledge, there is only a single viable normalization layer for HNNs, i.e., the general Riemannian batch normalization [33]. However, this method is not ideal due to the slow iterative computation of the Frechet mean and the arbitrary re-scaling operation that is not based on hyperbolic geometry. In this work, we propose an efficient batch normalization algorithm founded in the Lorentz model, which utilizes the Lorentzian centroid [27] and a mathematically motivated re-scaling operation.

Numerical stability of HNNsThe exponential growth of the Lorentz model's volume with respect to the radius can introduce numerical instability and rounding errors in floating-point arithmetic. This requires many works to rely on 64-bit precision at the cost of higher memory and runtime requirements. To mitigate this, researchers have introduced feature clipping and Euclidean reparameterizations [16; 35; 36]. We adopt these approaches in HCNN, allowing us to run under 32-bit floating point arithmetic, reducing the computational cost.

## 3 Background

This section summarizes the mathematical background of hyperbolic geometry [3; 45]. The \(n\)-dimensional hyperbolic space \(\mathbb{H}_{K}^{n}\) is a smooth Riemannian manifold \((\mathcal{M}^{n},\mathfrak{g}_{\boldsymbol{x}}^{K})\) with constant negative curvature \(K<0\), where \(\mathcal{M}^{n}\) and \(\mathfrak{g}_{\boldsymbol{x}}^{K}\) represent the manifold and the Riemannian metric, respectively. There are multiple isometrically equivalent models of hyperbolic geometry. We employ the Lorentz model because of its numerical stability and its simple exponential/logarithmic maps and distance functions. Additionally, we use the Poincare ball for baseline implementations. Both hyperbolic models provide closed-form formulae for manifold operations, including distance measures, exponential/logarithmic maps, and parallel transportation. They are detailed in Appendix A.

Lorentz modelThe \(n\)-dimensional Lorentz model \(\mathbb{L}_{K}^{n}=(\mathcal{L}^{n},\mathfrak{g}_{\boldsymbol{x}}^{K})\) models hyperbolic geometry on the upper sheet of a two-sheeted hyperboloid \(\mathcal{L}^{n}\), with origin \(\overline{\boldsymbol{0}}=[\sqrt{-1/K},0,\cdots,0]^{T}\) and embedded in \((n+1)\)-dimensional Minkowski space (see Figure 2). Based on the Riemannian metric \(\mathfrak{g}_{\boldsymbol{x}}^{K}=\mathrm{diag}(-1,1,\ldots,1)\), the manifold is defined as

\[\mathcal{L}^{n}:=\{\boldsymbol{x}\in\mathbb{R}^{n+1}\mid\langle\boldsymbol{x}, \boldsymbol{x}\rangle_{\mathcal{L}}=\frac{1}{K},\;x_{t}>0\}, \tag{1}\]

with the Lorentzian inner product

\[\langle\boldsymbol{x},\boldsymbol{y}\rangle_{\mathcal{L}}:=-x_{t}y_{t}+ \boldsymbol{x}_{s}^{T}\boldsymbol{y}_{s}=\boldsymbol{x}^{T}\mathrm{diag}(-1, 1,\cdots,1)\boldsymbol{y}. \tag{2}\]

When describing points in the Lorentz model, we inherit the terminology of special relativity and call the first dimension the _time component_\(x_{t}\) and the remaining dimensions the _space component_\(\boldsymbol{x}_{s}\), such that \(\boldsymbol{x}\in\mathbb{L}_{K}^{n}=[x_{t},\boldsymbol{x}_{s}]^{T}\) and \(x_{t}=\sqrt{||\boldsymbol{x}_{s}||^{2}-1/K}\).

Poincare ballThe n-dimensional Poincare ball \(\mathbb{B}_{K}^{n}=(\mathcal{B}^{n},\mathfrak{g}_{\boldsymbol{x}}^{K})\) is defined by \(\mathcal{B}^{n}=\{\boldsymbol{x}\in\mathbb{R}^{n}\mid-K||\boldsymbol{x}||^{2} <1\}\) and the Riemannian metric \(\mathfrak{g}_{\boldsymbol{x}}^{K}=(\lambda_{\boldsymbol{x}}^{K})^{2}\mathbf{ I}_{n}\), where \(\lambda_{\boldsymbol{x}}^{K}=2(1+K||\boldsymbol{x}||^{2})^{-1}\). It describes the hyperbolic space by an open ball of radius \(\sqrt{-1/K}\), see Figure 2.

## 4 Fully hyperbolic CNN (HCNN)

Our HCNN framework aims to give way to building vision models that can fully leverage the advantages of hyperbolic geometry by learning features in hyperbolic spaces. For this, we generalize Euclidean CNN components to the Lorentz model, yielding one-to-one replacements that can be integrated into existing architectures. In the following, we first define the cornerstone of HCNNs, i.e., the Lorentz convolutional layer, including its transposed variant. Then, we introduce the Lorentz batch normalization algorithm and the MLR classifier. Finally, we generalize the residual connection and non-linear activation.

Figure 2: Comparison of Lorentz and Poincaré model.
### Lorentz convolutional layer

Hyperbolic feature mapsThe convolutional layer applies vector operations to an input feature map containing the activations of the previous layer. In Euclidean space, arbitrary numerical values can be combined to form a vector. Therefore, it is not required to strictly determine which feature map dimension holds feature vectors. However, in the Lorentz model, not all possible value combinations represent a point that can be processed with hyperbolic operations (\(\mathbb{L}_{K}^{n}\subset\mathbb{R}^{n+1}\)).

We propose using channel-last feature map representations throughout HCNNs and adding the Lorentz model's time component as an additional channel dimension. This defines a hyperbolic feature map as an ordered set of \(n\)-dimensional hyperbolic vectors, where every spatial position contains a vector that can be combined with its neighbors. Additionally, it offers a nice interpretation where an image is an ordered set of color vectors, each describing a pixel.

Formalization of the convolutional layerWe define the convolutional layer as an affine transformation between a linearized kernel and a concatenation of the values in its receptive field, following Shimizu et al. [49]. Then, we generalize this definition by replacing the Euclidean operators with their hyperbolic counterparts in the Lorentz model.

Given a hyperbolic input feature map \(\mathbf{x}=\{\mathbf{x}_{h,w}\in\mathbb{L}_{K}^{n}\}_{h,w=1}^{H,W}\) as an ordered set of \(n\)-dimensional hyperbolic feature vectors, each describing image pixels, the features within the receptive field of the kernel \(\mathbf{K}\in\mathbb{R}^{m\times n\times\hat{H}\times\hat{W}}\) are \(\{\mathbf{x}_{h^{\prime}+\delta\hat{h},w^{\prime}+\delta\hat{w}}\in\mathbb{L}_{K} ^{n}\}_{\hat{h},\hat{w}=1}^{\hat{H},\hat{W}}\), where \((h^{\prime},w^{\prime})\) denotes the starting position and \(\delta\) is the stride parameter. Now, we define the Lorentz convolutional layer as

\[\mathbf{y}_{h,w}=\text{LFC}(\text{HCat}(\{\mathbf{x}_{h^{\prime}+\delta\hat{h},w^{ \prime}+\delta\hat{w}}\in\mathbb{L}_{K}^{n}\}_{\hat{h},\hat{w}=1}^{\hat{H}, \hat{W}})), \tag{3}\]

where HCat denotes the concatenation of hyperbolic vectors, and LFC denotes a Lorentz fully-connected layer performing the affine transformation and parameterizing the kernel and bias, respectively. Additionally, we implement padding using origin vectors, the analog of zero vectors in hyperbolic space.

Finally, our formulation generalizes arbitrary dimensional convolutional layers with little modification to the 2-dimensional case presented here. However, it remains essential to predetermine the structure of hyperbolic feature maps, e.g., using our channel-last format.

Extension to the transposed settingThe transposed convolutional layer is usually used in encoder-decoder architectures for up-sampling. Given a hyperbolic convolutional layer, it is straightforward to generalize to hyperbolic space. A convolutional layer carries out a transposed convolution when the correct local connectivity is established by inserting zeros at certain positions. Specifically, when stride \(s>1\), then \(s-1\) zero vectors are inserted between the features. In addition, the input feature map is always implicitly padded on each side. We refer to Dumoulin and Visin [9] for illustrations. Under this relationship, the Lorentz transposed convolutional layer is a Lorentz convolutional layer with changed connectivity through origin padding.

### Lorentz batch normalization

Given a batch \(\mathcal{B}\) of \(m\) features \(\mathbf{x}_{i}\), the traditional batch normalization algorithm [21] calculates the mean \(\mathbf{\mu}_{\mathcal{B}}=\frac{1}{m}\sum_{i=1}^{m}\mathbf{x}_{i}\) and variance \(\mathbf{\sigma}_{\mathcal{B}}^{2}=\frac{1}{m}\sum_{i=1}^{m}(\mathbf{x}_{i}-\mathbf{\mu}_{ \mathcal{B}})^{2}\) across the batch dimension. Then, the features are _re-scaled_ and _re-centered_ using a parameterized variance \(\mathbf{\gamma}\) and mean \(\mathbf{\beta}\) as follows

\[\text{BN}(\mathbf{x}_{i})=\mathbf{\gamma}\odot\frac{\mathbf{x}_{i}-\mathbf{\mu}_{\mathcal{B}}} {\sqrt{\mathbf{\sigma}_{\mathcal{B}}^{2}+\epsilon}}+\mathbf{\beta}. \tag{4}\]

At test time, running estimates approximate the batch statistics. They are calculated iteratively during training: \(\mathbf{\mu}_{t}=(1-\eta)\mathbf{\mu}_{t-1}+\eta\mathbf{\mu}_{\mathcal{B}}\) and \(\mathbf{\sigma}_{t}^{2}=(1-\eta)\mathbf{\sigma}_{t-1}^{2}+\eta\mathbf{\sigma}_{\mathcal{B}} ^{2}\), with \(\eta\) and \(t\) denoting momentum and the current iteration, respectively. We generalize batch normalization to the Lorentz model using the Lorentzian centroid and the parallel transport operation for re-centering, and the Frechet variance and straight geodesics at the origin's tangent space for re-scaling.
Re-centeringTo re-center hyperbolic features, it is necessary to compute a notion of mean. Usually, the Frechet mean is used [33], which minimizes the expected squared distance between a set of points in a metric space [43]. Generally, the Frechet mean must be solved iteratively, massively slowing down training. To this end, we propose to use the centroid with respect to the squared Lorentzian distance, which can be calculated efficiently in closed form [27]. The weighted Lorentzian centroid, which solves \(\min_{\mathbf{\mu}\in\mathbb{L}_{K}^{n}}\sum_{i=1}^{m}\nu_{i}d_{\mathcal{L}}^{2}( \mathbf{x}_{i},\mathbf{\mu})\), with \(\mathbf{x}_{i}\in\mathbb{L}_{K}^{n}\) and \(\nu_{i}\geq 0,\sum_{i=1}^{m}\nu_{i}>0\), is given by

\[\mathbf{\mu}=\frac{\sum_{i=1}^{m}\nu_{i}\mathbf{x}_{i}}{\sqrt{-K}\,||\, \sum_{i=1}^{m}\nu_{i}\mathbf{x}_{i}||_{\mathcal{L}}}. \tag{5}\]

In batch normalization, the mean is not weighted, which gives \(\nu_{i}=\frac{1}{m}\). Now, we shift the features from the batch's mean \(\mathbf{\mu}_{\mathbf{B}}\) to the parameterized mean \(\mathbf{\beta}\) using the parallel transport operation \(\mathrm{PT}_{\mathbf{\mu}_{\mathbf{\beta}}\rightarrow\mathbf{\beta}}^{K}(\mathbf{x})\). Parallel transport does not change the variance, as it is defined to preserve the distance between all points. Finally, the running estimate is updated iteratively using the weighted centroid with \(\nu_{1}=(1-\eta)\) and \(\nu_{2}=\eta\).

Re-scalingFor re-scaling, we rely on the Frechet variance \(\sigma^{2}\in\mathbb{R}^{+}\), defined as the expected squared Lorentzian distance between a point \(\mathbf{x}_{i}\) and the mean \(\mathbf{\mu}\), and given by \(\sigma^{2}=\frac{1}{m}\sum_{i=1}^{m}d_{\mathcal{L}}^{2}(\mathbf{x}_{i},\mathbf{\mu})\)[24]. In order to re-scale the batch, features must be moved along the geodesics connecting them to their centroid, which is generally infeasible to compute. However, geodesics intersecting the origin are very simple, as they can be represented by straight lines in tangent space \(\mathcal{T}_{\overline{\mathbf{0}}}\mathbb{L}_{K}^{n}\). This is reflected by the equality between the distance of a point to the origin and the length of its corresponding tangent vector (\(d_{\mathcal{L}}(\mathbf{x},\overline{\mathbf{0}})=||\log^{K}_{\overline{\mathbf{0}}}(\mathbf{ x})||\)). Using this property, we propose to re-scale features by first parallel transporting them towards the origin \(\mathrm{PT}_{\mu_{\mathbf{\beta}}\rightarrow\overline{\mathbf{0}}}^{K}(\mathbf{x})\), making the origin the new centroid and straightening the relevant geodesics. Then, a simple multiplication re-scales the features in tangent space. Finally, parallel transporting to \(\mathbf{\beta}\in\mathbb{L}_{K}^{n}\) completes the algorithm and yields the normalized features. The final algorithm of Lorentz batch normalization can be formalized as

\[\text{LBN}(\mathbf{x})=\exp_{\mathbf{\beta}}^{K}\left(\mathrm{PT}_{\mathbf{ \overline{\mathbf{0}}}\rightarrow\mathbf{\beta}}^{K}\left(\gamma\cdot\frac{\mathrm{ PT}_{\mathbf{\mu}_{\mathbf{\beta}}\rightarrow\overline{\mathbf{0}}}^{K}\left(\log^{K}_{\mathbf{\mu}_{ \mathbf{\beta}}}(\mathbf{x})\right)}{\sqrt{\sigma^{2}_{\mathcal{B}}}+\epsilon}\right) \right). \tag{6}\]

In our definition of hyperbolic feature maps, the centroid must be computed across multiple dimensions. For example, given a feature map \(\{\mathbf{x}_{h,w}\in\mathbb{L}_{K}^{n}\}_{h,w=1}^{H,W}\) for all \(m\) instances, we first compute the centroid of all hyperbolic vectors per instance and then the centroid across the batch dimension, i.e., the centroid of instance centroids. Similarly, the Frechet variance is computed for separate instances and then averaged across the batch dimension as it is in \(\mathbb{R}^{+}\). Furthermore, the running estimate of the Frechet variance is computed using a standard Euclidean running average.

### Lorentz MLR classifier

In this section, we consider the problem of classifying instances that are represented in the Lorentz model. A standard method for multi-class classification is multinomial logistic regression (MLR). Inspired by the generalization of MLR to the Poincare ball [12; 49] based on the distance to margin hyperplanes, we derive a formulation in the Lorentz model.

Hyperplane in the Lorentz modelAnalogous to Euclidean space, hyperbolic hyperplanes split the manifold into two half-spaces, which can then be used to separate instances into classes. The hyperplane in the Lorentz model is defined by a geodesic that results from the intersection of an \(n\)-dimensional hyperplane with the hyperboloid in the ambient space \(\mathbb{R}^{n+1}\)[6]. Specifically, for \(\mathbf{p}\in\mathbb{L}_{K}^{n}\) and \(\mathbf{w}\in\mathcal{T}_{\mathbf{p}}\mathbb{L}_{K}^{n}\), the hyperplane passing through \(\mathbf{p}\) and perpendicular to \(\mathbf{w}\) is given by

\[H_{\mathbf{w},\mathbf{p}}=\{\mathbf{x}\in\mathbb{L}_{K}^{n}\mid\langle\mathbf{w},\mathbf{x} \rangle_{\mathcal{L}}=0\}. \tag{7}\]

This formulation comes with the non-convex optimization condition \(\langle\mathbf{w},\mathbf{w}\rangle_{\mathcal{L}}>0\), which is undesirable in machine learning. To eliminate this condition, we use the Euclidean reparameterization
of Mishne et al. [36], which we extend to include the curvature parameter \(K\) in Appendix B.1. In short, \(\mathbf{w}\) is parameterized by a vector \(\mathbf{\overline{z}}\in\mathcal{T}_{\mathbf{\overline{0}}}\mathbb{L}_{K}^{n}=[0,a\mathbf{ z}/||\mathbf{z}||]\), where \(a\in\mathbb{R}\) and \(\mathbf{z}\in\mathbb{R}^{n}\). As \(\mathbf{w}\in\mathcal{T}_{\mathbf{p}}\mathbb{L}_{K}^{n}\), \(\mathbf{\overline{z}}\) is parallel transported to \(\mathbf{p}\), which gives

\[\mathbf{w}:=\mathrm{PT}_{\mathbf{\overline{0}}\rightarrow\mathbf{p}}^{K}(\mathbf{\overline{z} })=[\sinh(\sqrt{-K}a)||\mathbf{z}||,\cosh(\sqrt{-K}a)\mathbf{z}]. \tag{8}\]

Inserting Eq. 8 into Eq. 7, the formula of the Lorentz hyperplane becomes

\[\tilde{H}_{\mathbf{z},a}=\{\mathbf{x}\in\mathbb{L}_{K}^{n}\ |\ \cosh(\sqrt{-K}a) \langle\mathbf{z},\mathbf{x}_{s}\rangle-\sinh(\sqrt{-K}a)\ ||\mathbf{z}||\ x_{t}=0\}, \tag{9}\]

where \(a\) and \(\mathbf{z}\) represent the distance and orientation to the origin, respectively.

Finally, we need the distance to the hyperplane to quantify the model's confidence. It is formulated by the following theorem, proven in Appendix B.2.

**Theorem 1**: _Given \(a\in\mathbb{R}\) and \(\mathbf{z}\in\mathbb{R}^{n}\), the minimum hyperbolic distance from a point \(\mathbf{x}\in\mathbb{L}_{K}^{n}\) to the hyperplane \(\tilde{H}_{\mathbf{z},a}\) defined in Eq. 9 is given by_

\[d_{\mathcal{L}}(\mathbf{x},\tilde{H}_{\mathbf{z},a})=\frac{1}{\sqrt{-K}}\left|\sinh^{ -1}\left(\sqrt{-K}\frac{\cosh(\sqrt{-K}a)\langle\mathbf{z},\mathbf{x}_{s}\rangle- \sinh(\sqrt{-K}a)\ ||\mathbf{z}||\ x_{t}}{\sqrt{||\cosh(\sqrt{-K}a)\mathbf{z}||^{2}-(\sinh(\sqrt{-K}a) ||\mathbf{z}||)^{2}}}\right)\right|. \tag{10}\]

MLR in the Lorentz modelLebanon and Lafferty [29] formulated the logits of the Euclidean MLR classifier using the distance from instances to hyperplanes describing the class regions. Specifically, given input \(\mathbf{x}\in\mathbb{R}^{n}\) and \(C\) classes, the output probability of class \(c\in\{1,...,C\}\) can be expressed as

\[p(y=c\ |\ \mathbf{x})\propto\exp(v_{\mathbf{w}_{c}}(\mathbf{x})),\ \ v_{\mathbf{w}_{c}}(\mathbf{x})= \text{sign}(\langle\mathbf{w}_{c},\mathbf{x}\rangle)||\mathbf{w}_{c}||d(\mathbf{x},H_{\mathbf{w}_{ c}}),\ \ \mathbf{w}_{c}\in\mathbb{R}^{n}, \tag{11}\]

where \(H_{\mathbf{w}_{c}}\) is the decision hyperplane of class \(c\).

We define the Lorentz MLR without loss of generality by inserting the Lorentzian counterparts into Eq. 11. This yields logits given by the following theorem, proven in Appendix B.3.

**Theorem 2**: _Given parameters \(a_{c}\in\mathbb{R}\) and \(\mathbf{z}_{c}\in\mathbb{R}^{n}\), the Lorentz MLR's output logit corresponding to class \(c\) and input \(\mathbf{x}\in\mathbb{L}_{K}^{n}\) is given by_

\[v_{\mathbf{z}_{c},a_{c}}(\mathbf{x})=\frac{1}{\sqrt{-K}}\ \mathrm{sign}(\alpha)\beta \left|\sinh^{-1}\left(\sqrt{-K}\frac{\alpha}{\beta}\right)\right|, \tag{12}\]

_with_

\[\alpha=\cosh(\sqrt{-K}a)\langle\mathbf{z},\mathbf{x}_{s}\rangle-\sinh( \sqrt{-K}a),\] \[\beta=\sqrt{||\cosh(\sqrt{-K}a)\mathbf{z}||^{2}-(\sinh(\sqrt{-K}a)|| \mathbf{z}||)^{2}}.\]

### Lorentz residual connection and activation

Residual connectionThe residual connection is a crucial component when designing deep CNNs. As vector addition is ill-defined in the Lorentz model, we add the vector's space components and concatenate a corresponding time component. This is possible as a point \(\mathbf{x}\in\mathbb{L}_{K}^{n}\) can be defined by an arbitrary space component \(\mathbf{x}_{s}\in\mathbb{R}^{n}\) and a time component \(x_{t}=\sqrt{||\mathbf{x}_{s}||^{2}-1/K}\). Our method is straightforward and provides the best empirical performance compared to other viable methods for addition we implemented, i.e., tangent space addition [39], parallel transport addition [4], Mobius addition (after projecting to the Poincare ball) [12], and fully-connected layer addition [5].
Non-linear activationPrior works use non-linear activation in tangent space [11], which weakens the model's stability due to frequent logarithmic and exponential maps. We propose to simplify the operation for the Lorentz model by applying the activation function to the space component and concatenating a time component. For example, the Lorentz ReLU activation is given by

\[\mathbf{y}=\left[\begin{array}{c}\sqrt{||\text{ReLU}(\mathbf{x}_{s})||^{2}-1/K}\\ \text{ReLU}(\mathbf{x}_{s})\end{array}\right]. \tag{13}\]

This operation can be interpreted similarly to Euclidean activations that break linearity with heuristic mathematical projections.

## 5 Experiments

We evaluate HCNN models on image classification and generation tasks and compare them against Euclidean and hybrid HNN counterparts. To ensure a fair comparison, in every task, we directly translate a Euclidean baseline to the hyperbolic setting by using hyperbolic modules as one-to-one replacements. All experiments are implemented in PyTorch [41], and we optimize hyperbolic models using adaptive Riemannian optimizers [2] provided by Geopot [25], with floating-point precision set to 32 bits. We provide detailed experimental configurations in Appendix C and many ablation experiments in Appendix D. To facilitate reproducibility and further exploration, we make the code of our experiments publicly available at [https://github.com/kschwethelm/HyperbolicCV](https://github.com/kschwethelm/HyperbolicCV).

### Image classification

Experimental setupIn this experiment, we evaluate the performance of HCNN on standard image classification tasks using ResNet-18 [18] and three benchmark datasets: CIFAR-10 [26], CIFAR-100 [26], and Tiny-ImageNet [28]. We compare against the Euclidean network by replacing all components in the ResNet architecture with our proposed Lorentzian modules. Establishing hyperbolic baselines is difficult, as there are currently no fully hyperbolic models for vision tasks, and only hybrid models with task-specific output layers that largely do not apply to standard classification [10; 22; 31; 50]. Specifically, current vision HNNs use Euclidean CNNs for embedding images, project the final embeddings onto hyperbolic space, and only apply a hyperbolic output layer. This leaves us with a single viable hyperbolic baseline from the literature: Following Aigh et al. [1] and Guo et al. [16], we implement the hybrid approach with the Poincare MLR classifier [49]. In addition, however, we utilize our proposed Lorentz MLR to obtain a novel hybrid baseline that uses the Lorentz model instead of the Poincare ball. Finally, we apply feature clipping on embeddings of both hybrid models to prevent gradient issues arising from the hybrid architecture [16].

For all models, we adopt DeVries and Taylor's [8] training procedure and hyperparameters, which have been optimized for Euclidean CNNs and yield a strong Euclidean ResNet baseline. For example, this method increases CIFAR-100 performance from 74.84% for the original ResNet-110 [52] to 77.72% we obtained for the much smaller ResNet-18. To make a rigorous one-to-one comparison, we do not optimize the training procedure and hyperparameters for the hyperbolic models but keep the settings of [8]. This suggests that HNN performance can still be improved.

Main resultsThe results in Table 1 show that our fully hyperbolic ResNet achieves the highest accuracy on all datasets, outperforming both Euclidean and hybrid baselines. Additionally, our novel hybrid HNN based on the Lorentz model performs well, outperforming the Euclidean model on two datasets. In contrast, the hybrid Poincare HNN is inferior to the Euclidean baseline, which is consistent with the results reported by Guo et al. [16]. This suggests that the Lorentz model is better suited for HNNs than the Poincare ball. Overall, our results highlight the potential of our HCNN components and the Lorentz model in improving image classification tasks.

Adversarial robustnessPrior works have indicated that HNNs can obtain better adversarial robustness than Euclidean networks. To study this, we employ the models trained on CIFAR-100 and attack them using FGSM [14] and PGD [34] with a maximal perturbation of \(\epsilon=0.8/255,1.6/255,3.2/255\). The results in Table 2 show that our HCNN is much more robust than the other models, getting up to 5% higher accuracy. In addition, contrary to Guo et al. [16], we observe that hybrid HNNs are more susceptible to adversarial attacks than Euclidean models.
Low embedding dimensionalityHNNs have shown to be most effective in lower-dimensional spaces. To this end, we reduce the dimensionality of the final ResNet block and the embeddings and evaluate classification accuracy on CIFAR-100.

The results in Figure 3 verify the effectiveness of hyperbolic spaces with low dimensions, where all HNNs outperform the Euclidean models. However, our HCNN can leverage this advantage best, suggesting that HCNNs offer great opportunities for dimensionality reduction and designing smaller models with fewer parameters. This can be useful, e.g., in mobile deployment.

### Image generation

Experimental setupVariational autoencoders (VAEs) [23; 46] have been widely adopted in HNN research to model latent embeddings in hyperbolic spaces [20; 35; 38; 40]. However, these works rely on hybrid architectures and mainly focus on learning strong embeddings rather than targeting image generation. In this experiment, we extend the hyperbolic VAE to the fully hyperbolic setting using our proposed HCNN framework and, for the first time, evaluate its performance on image generation using the standard Frechet Inception Distance (FID) metric [19].

Building on the experimental setting of Ghosh et al. [13], we test vanilla VAEs and assess generative performance on CIFAR-10 [26], CIFAR-100 [26], and CelebA [32] datasets. We compare our HCNN-VAE against the Euclidean and two hybrid models. Following prior works, the hybrid models only include a latent hyperbolic distribution and no hyperbolic layers. Specifically, we employ the wrapped normal distributions in the Lorentz model [38] and the Poincare ball [35], respectively.

Main resultsThe results in Table 3 show that our HCNN-VAE outperforms all baselines. Likewise, the hybrid models improve performance over the Euclidean model, indicating that learning the latent embeddings in hyperbolic spaces is beneficial. However, our HCNN is better at leveraging the advantages of hyperbolic geometry due to its fully hyperbolic architecture. These results suggest that our method is a promising approach for generation and for modeling latent structures in image data.

\begin{table}
\begin{tabular}{l c c c} \hline \hline  & CIFAR-10 & CIFAR-100 & Tiny-ImageNet \\ \hline Euclidean [18] & \(95.14_{\pm 0.12}\) & \(77.72_{\pm 0.15}\) & \(65.19_{\pm 0.12}\) \\ \hline Hybrid Poincaré [1; 16] & \(95.04_{\pm 0.13}\) & \(77.19_{\pm 0.50}\) & \(64.93_{\pm 0.38}\) \\ Hybrid Lorentz (Ours) & \(94.98_{\pm 0.12}\) & \(78.03_{\pm 0.21}\) & \(65.63_{\pm 0.10}\) \\ \hline HCNN Lorentz (Ours) & \(\mathbf{95.14_{\pm 0.08}}\) & \(\mathbf{78.07_{\pm 0.17}}\) & \(\mathbf{65.71_{\pm 0.13}}\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Classification accuracy (%) of ResNet-18 models. We estimate the mean and standard deviation from five runs. The best performance is highlighted in bold (higher is better).

\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & \multicolumn{4}{c}{FGSM} & \multicolumn{4}{c}{PGD} \\ \cline{2-7} Max. perturbation \(\epsilon\) & 0.8/255 & 1.6/255 & 3.2/255 & 0.8/255 & 1.6/255 & 3.2/255 \\ \hline Euclidean [18] & \(65.70_{\pm 0.28}\) & \(54.98_{\pm 0.39}\) & \(39.97_{\pm 0.43}\) & \(64.43_{\pm 0.29}\) & \(49.76_{\pm 0.42}\) & \(26.30_{\pm 0.40}\) \\ \hline Hybrid Poincaré [1; 16] & \(64.68_{\pm 0.40}\) & \(53.32_{\pm 0.60}\) & \(37.52_{\pm 0.50}\) & \(63.43_{\pm 0.44}\) & \(48.41_{\pm 0.60}\) & \(23.78_{\pm 0.75}\) \\ Hybrid Lorentz (Ours) & \(65.27_{\pm 0.52}\) & \(53.82_{\pm 0.49}\) & \(40.53_{\pm 0.31}\) & \(64.15_{\pm 0.53}\) & \(49.05_{\pm 0.68}\) & \(27.17_{\pm 0.40}\) \\ \hline HCNN Lorentz (Ours) & \(\mathbf{66.47_{\pm 0.27}}\) & \(\mathbf{57.14_{\pm 0.30}}\) & \(\mathbf{43.51_{\pm 0.35}}\) & \(\mathbf{65.04_{\pm 0.28}}\) & \(\mathbf{52.25_{\pm 0.34}}\) & \(\mathbf{31.77_{\pm 0.55}}\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Classification accuracy (%) after performing FGSM and PGD attacks on CIFAR-100. We estimate the mean and standard deviation from attacking five trained models. The best performance is highlighted in bold (higher is better).

Figure 3: CIFAR-100 accuracy obtained with lower dimensionalities in the final ResNet block. The standard embedding dimension is \(d_{E}=512\).
Analysis of latent embeddingsThe latent embedding space is a crucial component of VAEs as it influences how the data's features are encoded and used for generating the output. We visually analyze the distribution of latent embeddings inferred by the VAEs. For this, the models are retrained on the MNIST [30] dataset with an embedding dimension \(d_{E}=2\). Then, the images of the training dataset are passed through the encoder and visualized as shown in Figure 4.

We observe the formation of differently shaped clusters that correlate with the ground truth labels. While the embeddings of the Euclidean and hybrid models form many clusters that direct towards the origin, the HCNN-VAE obtains rather curved clusters that keep a similar distance to the origin. The structures within the HCNN's latent space can be interpreted as hierarchies where the distance to the origin represents hierarchical levels. As these structures cannot be found for the hybrid model, our results suggest that hybrid HNNs behave more like Euclidean models than hyperbolic models and can not fully leverage the properties of the hyperbolic space, resulting in inferior performance.

## 6 Conclusion

In this work, we proposed HCNN, a generalization of the convolutional neural network that learns latent feature representations in hyperbolic spaces. To this end, we formalized the necessary modules in the Lorentz model, deriving novel formulations of the convolutional layer, batch normalization, and multinomial logistic regression. We empirically demonstrated that ResNet and VAE models based on our HCNN framework achieve better performance on standard vision tasks than Euclidean and hybrid baselines, especially in adversarial and lower dimensional settings. Additionally, we showed that using the Lorentz model in HNNs leads to better stability and performance than the Poincare ball.

However, HCNNs are still in their early stages, and it remains to be seen to which extent they can replace Euclidean networks as they introduce mathematical complexity and computational overhead. Moreover, our HCNN framework relies on generalizations of neural network layers that were designed for Euclidean geometry and might not fully capture the unique properties of hyperbolic geometry. Further research is needed to fully understand the properties of HCNNs and address open questions such as optimization, scalability, and performance on other deep learning problems. We hope our work will inspire future research and development in this exciting and rapidly evolving field.

Figure 4: Embeddings of MNIST training data in 2D latent space of VAEs. Each embedding is represented by a point colored by the input’s ground truth label. The embeddings of Lorentz HNNs are projected onto the Poincaré ball for better visualization. Additionally, we give the generation FID.

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & \multicolumn{2}{c}{CIFAR-10} & \multicolumn{2}{c}{CIFAR-100} & \multicolumn{2}{c}{CelebA} \\ \cline{2-6}  & Rec. FID & Gen. FID & Rec. FID & Gen. FID & Rec. FID & Gen. FID \\ \hline Euclidean & \(61.21_{\pm 0.72}\) & \(92.40_{\pm 0.80}\) & \(63.81_{\pm 0.47}\) & \(103.54_{\pm 0.84}\) & \(54.80_{\pm 0.29}\) & \(79.25_{\pm 0.89}\) \\ \hline Hybrid Poincaré [35] & \(59.85_{\pm 0.50}\) & \(90.13_{\pm 0.77}\) & \(62.64_{\pm 0.43}\) & \(\mathbf{98.19_{\pm 0.57}}\) & \(54.62_{\pm 0.61}\) & \(81.30_{\pm 0.56}\) \\ Hybrid Lorentz [38] & \(59.29_{\pm 0.47}\) & \(90.91_{\pm 0.84}\) & \(62.14_{\pm 0.35}\) & \(98.34_{\pm 0.62}\) & \(54.64_{\pm 0.34}\) & \(82.78_{\pm 0.93}\) \\ \hline HCNN Lorentz (Ours) & \(\mathbf{57.78_{\pm 0.56}}\) & \(\mathbf{89.20_{\pm 0.85}}\) & \(\mathbf{61.44_{\pm 0.64}}\) & \(100.27_{\pm 0.84}\) & \(\mathbf{54.17_{\pm 0.66}}\) & \(\mathbf{78.11_{\pm 0.95}}\) \\ \hline \hline \end{tabular}
\end{table}
Table 3: Reconstruction and generation FID of manifold VAEs. We estimate the mean and standard deviation from five runs. The best performance is highlighted in bold (lower is better).
## Acknowledgments and Disclosure of Funding

This work was performed on the HoreKa supercomputer funded by the Ministry of Science, Research and the Arts Baden-Wurttemberg and by the Federal Ministry of Education and Research. Ahmad Bdeir was funded by the European Union's Horizon 2020 research and innovation programme under the SustInAfrica grant agreement No 861924. Kristian Schwethelm was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - project number 225197905.
