Group convolutional neural networks (G-CNNs) can be used to improve classical CNNs by equipping them with the geometric structure of groups. Central in the success of G-CNNs is the lifting of feature maps to higher dimensional disentangled representations, in which data characteristics are effectively learned, geometric data-augmentations are made obsolete, and predictable behavior under geometric transformations (equivariance) is guaranteed via group theory. Currently, however, the practical implementations of G-CNNs are limited to either discrete groups (that leave the grid intact) or continuous compact groups such as rotations (that enable the use of Fourier theory). In this paper we lift these limitations and propose a modular framework for the design and implementation of G-CNNs for arbitrary Lie groups. In our approach the differential structure of Lie groups is used to expand convolution kernels in a generic basis of B-splines that is defined on the Lie algebra. This leads to a flexible framework that enables localized, atrous, and deformable convolutions in G-CNNs by means of respectively localized, sparse and non-uniform B-spline expansions. The impact and potential of our approach is studied on two benchmark datasets: cancer detection in histopathology slides (PCam dataset) in which rotation equivariance plays a key role and facial landmark localization (CelebA dataset) in which scale equivariance is important. In both cases, G-CNN architectures outperform their classical 2D counterparts and the added value of atrous and localized group convolutions is studied in detail. Group convolutional neural networks (G-CNNs) are a class of neural networks that are equipped with the geometry of groups. This enables them to profit from the structure and symmetries in signal data such as images (Cohen & Welling, 2016) . A key feature of G-CNNs is that they are equivariant with respect to transformations described by the group, i.e., they guarantee predictable behavior under such transformations and are insensitive to both local and global transformations on the input data. Classical CNNs are a special case of G-CNNs that are equivariant to translations and, in contrast to unconstrained NNs, they make advantage of (and preserve) the basic structure of signal data throughout the network (LeCun et al., 1990) . By considering larger groups (i.e. considering not just translation equivariance) additional geometric structure can be utilized in order to improve performance and data efficiency (see G-CNN literature in Sec. 2). Part of the success of G-CNNs can be attributed to the lifting of feature maps to higher dimensional objects that are generated by matching kernels under a range of poses (transformations in the group). This leads to a disentanglement with respect to the pose and together with the group structure this enables a flexible way of learning high level representations in terms of low-level activated neurons observed in specific configurations, which we conceptually illustrate in Fig. 1 . From a neuro-psychological viewpoint, this resembles a hierarchical composition from low-to high-level features akin to the recognition-by-components model by Biederman (1987) , a viewpoint which is also adopted in work on capsule networks (Hinton et al., 2011; Sabour et al., 2017) . In particular in ) the group theoretical connection is made explicit with equivariant capsules that provide a sparse index/value representation of feature maps on groups (Gens & Domingos, 2014) . In G-CNNs feature maps are lifted to the high-dimensional domain of the group G in which features are disentangled with respect to pose/transformation parameters. G-convolution kernels then learn to recognize high-level features in terms of patterns of relative transformations, described by the group structure. This is conceptually illustrated for the detection of faces, which in the SE(2) case are considered as a pattern of lines in relative positions and orientations, or in the R 2 R + case as blobs/circles in relative positions and scales. Representing low-level features via features maps on groups, as is done in G-CNNs, is also motivated by the findings of Hubel & Wiesel (1959) and Bosking et al. (1997) on the organization of orientation sensitive simple cells in the primary visual cortex V1. These findings are mathematically modeled by sub-Riemannian geometry on Lie groups (Petitot, 2003; Citti & Sarti, 2006; Duits et al., 2014) and led to effective algorithms in image analysis (Franken & Duits, 2009; Bekkers et al., 2015b; Favali et al., 2016; Duits et al., 2018; Baspinar, 2018) . In recent work Montobbio et al. (2019) show that such advanced V1 modeling geometries emerge in specific CNN architectures and in Ecker et al. (2019) the relation between group structure and the organization of V1 is explicitly employed to effectively recover actual V1 neuronal activities from stimuli by means of G-CNNs. Figure 2: The Log-map allows us to map elements from curved manifolds such as the 2-sphere to a flat Euclidean tangent space. For Lie groups the Logmap is analytic, globally defined, and it provides us with a flexible tool to define group convolution kernels via Bsplines. In our Lie group context the 2-sphere is treated as a quotient group SO(3)/SO(2). Technical details are given in Sec. 3 and App. B. that are the semi-direct product of the translation group with a Lie group H. As such, only a few core definitions about the Lie group H (group product, inverse, Log, and action on R d ) need to be implemented in order to build full G-CNNs that are locally equivariant to the transformations in H. The impact and potential of our approach is studied on two datasets in which respectively rotation and scale equivariance plays a key role: cancer detection in histopathology slides (PCam dataset) and facial landmark localization (CelebA dataset). In both cases G-CNNs out-perform their classical 2D counterparts and the added value of atrous and localized G-convolutions is studied in detail. This paper presents a flexible framework for building G-CNNs for arbitrary Lie groups. The proposed B-spline basis functions, which are used to represent convolution kernels, have unique properties that cannot be achieved by classical Fourier based basis functions. Such properties include the construction of localized, atrous, and deformable convolution kernels. We experimentally demonstrated the added value of localized and atrous group convolutions on two different applications, considering two different groups. In particular in experiments with scale-translation G-CNNs, kernel localization was important. The B-spline basis functions can be considered as smooth pixels on Lie groups and they enable us to design G-CNNs using familiar notions from classical CNN design (localized, atrous, and deformable convolutions). Future work will focus on exploring these options further in new applications that could benefit from equivariance constraints, for which the tools now are available for a large class of transformation groups via the proposed Lie group B-splines. (Kantorovich & Akilov, 1982 , Ch 9, Thm 5), or (Duits, 2005, Thm 1) , that if K is linear and bounded it is an integral operator.
The paper describes a flexible framework for building CNNs that are equivariant to a large class of transformations groups. A framework for building group CNN with an arbitrary Lie group G, which shows superiority over a CNN in tumor classification and landmark localization. 