# Note About Null Dimensional Reduction of M5-Brane

J. Klusoň
Department of Theoretical Physics and Astrophysics
Faculty of Science, Masaryk University
Kotlářská 2, 611 37, Brno
Czech Republic
E-mail<EMAIL_ADDRESS>
###### Abstract:

In this short note we study null dimensional reduction of M5-brane covariant
action. We analyse longitudinal dimensional reduction that leads to non-
relativistic D4-brane and transverse reduction that leads to NS5-brane in non-
relativistic string theory.

##  1 Introduction and Summary

It has been known from the late nineties that string theories have origin in
eleven dimensional theory- M-theory [1]. Since all string theories can be
considered as limits of M-theory it is clear that M-theory has to contain
extended objects that are parent objects of fundamental strings, D-branes and
NS5-branes [11], for review see for example [12]. Fundamental object of
M-theory is M2-brane as string is fundamental for string theory since M2-brane
electrically couples to three form $C^{(3)}$ in the same way as string couples
to two form $B^{(2)}$. On the other hand more interesting object in M-theory
is M5-brane [4, 5, 6, 7, 8, 9, 10]. This is six dimensional object with an
exceptional property that on its world-volume self-dual three form field
propagates. This fact makes the analysis of this theory rather complicated. On
the other hand covariant action for M5-brane was proposed and extensively
studied in [4, 5, 6]. In particular, it was shown in [4] that this M5-brane
dimensionally reduces into D4-brane under double dimensional reduction and it
was also shown in [14] that D3-brane action arises through double dimensional
reduction of M5-brane action on two torus despite of the fact that the
analysis is rather complicated [14].

It would be certainly interesting to check properties of covariant M5-brane
action in another situation. Such an interesting example could be null
dimensional reduction of M-theory. It is well known that null dimensional
reduction is rather subtle at the level of gravity action [2, 3] however it
can be performed at the level of equations of motion. Further, it is also well
known that null dimensional reduction of gravity leads to torsional Newton-
Cartan gravity [3, 20] in lower dimensional space-time. Clearly we can deduce
that null dimensional reduction of M-theory would lead to torsional Newton-
Cartan string theory even if the precise study of this problem has not been
performed yet 111See also [16]. For another study of non-relativistic limits
of M-theory, see [18, 19].. On the other hand we can ask the question how
extended objects in M-theory map under null dimensional reductions. In our
previous paper [17] we studied null dimensional reduction of M2-brane. We
showed that when we have M2-brane wrapped null direction we get fundamental
string action in torsional Newton-Cartan string theory. Further, when we
consider M2-brane transverse to this null direction we get D2-brane action in
the same background. Then we can ask the question how this procedure can be
applied in case of M5-brane. Exactly the goal of this paper is to study this
problem. To begin with we firstly review dimensional reduction along space-
like circle that should lead to D4-brane in Type IIA theory. This procedure
was firstly discussed in [4] but we analyse this problem in more details and
in slightly different way. First of all we perform spatial dimensional
reduction of M5-brane that leads to five dimensional action with propagating
two form on its world-volume. Since two form in five dimensions is dual to one
form it is natural to perform duality transformations following [8, 13]. We
show that dual action for dimensional reduced M5-brane action is D4-brane
action in Type IIA theory. We would like to stress that this analysis could be
considered as dual to the analysis performed in [8] where the starting point
was D4-brane and then it was shown that dual action corresponds to M5-brane
action dimensionally reduced along spatial dimension.

We extend this idea to the case of null dimensional reduction of M5-brane. We
start with the discussion of the background with null isometry following [2,
3]. Then we insert this background metric into covariant M5-brane action and
we obtain its dimensionally reduced form. Again since this action still
depends on two form field it is natural to perform its duality transformation.
Now we should be more careful than in case of relativistic M5-brane since the
induced metric corresponds to Newton-Cartan metric where there is no Lorentz
invariance in the target space-time. For that reason we consider more general
situation with diagonal components of spatial metric $h^{\alpha\beta}$, non-
zero component of time one form $\tau_{\alpha}$ together with non-zero
components of $m_{\alpha}$ where meaning of these symbols will be given below.
Then we will be able to perform duality transformation as in case of the
spatial dimensional reduction and we argue that resulting action corresponds
to D4-brane in torsional Newton-Cartan background. This is main result of this
paper that together with the paper [17] show that extended objects in M-theory
map under null dimensional reductions to fundamental strings an D-branes in
torsional Newton-Cartan string theory. It is very interesting that these
actions have functionally the same form as their relativistic counterparts
since they depend on the pull-back of the Newton-Cartan volume form [20]. This
is also especially interesting when we compare these actions with the actions
of non-relativistic strings and D-branes that are derived either by specific
limiting procedure [21, 22, 23, 24], see also [25, 26, 27], or by null
reduction of relativistic string [28, 29, 30, 31, 32, 33]. It would be
certainly very interesting to understand relations between these string
theories in more details.

This paper is organized as follows. In the next section (2) we review main
properties of M5-brane action and perform its dimensional reduction along
spatial dimension. In section (3) we perform dimensional reduction when
M5-brane wraps null direction. Finally in section (4) we briefly mention
dimensional reduction of M5-brane when it is transverse to null direction. The
resulting action should corresponds to NS5-brane in torsional Newton-Cartan
string theory.

##  2 Covariant M5-Brane Action and Spatial Dimensional Reduction

In this section we introduce basic notation and study spatial dimensional
reduction of M5-brane action. Let us consider covariant form of M5-brane
action that was introduced in [4, 5, 6, 7], see also [8, 9]. Explicitly, we
consider M5-brane action in the form 222We work with the metric of the
signature $(-,+,\dots,+)$.

$S=-T_{M5}\int
d^{6}\xi\left[\sqrt{-\det\left(\hat{g}_{mn}+i\frac{\hat{H}^{*}_{mn}}{\sqrt{\widehat{\partial
a\partial a}}}\right)}-\frac{\sqrt{-\hat{g}}}{4\widehat{\partial a\partial
a}}\hat{H}^{*mn}H_{mnr}\partial^{r}a\right]\ ,$ (1)

where

$\widehat{\partial a\partial a}=\partial_{m}a\hat{g}^{mn}\ \partial_{n}a\ ,$
(2)

and where

$m,n=0,1,\dots,5\ ,$ (3)

are vector indices of $d=6$ world-volume coordinates $\xi^{m}$,

$M,N=0,1,\dots,10$ (4)

are vector indices of $D=11$ target space-time coordinates $X^{M}$

$\hat{g}_{mn}=\partial_{m}X^{M}\hat{g}_{MN}\partial_{n}X^{N}$ (5)

is the world-volume metric that is induced by embedding the five-brane into
$D=11$ background with metric $\hat{g}_{MN}$. Further, we have

$\hat{H}^{*}_{mn}=\frac{1}{6\sqrt{-\hat{g}}}\hat{g}_{mr}\hat{g}_{ns}\epsilon^{rstuvw}H_{tuv}\partial_{w}a\
,$ (6)

where $H_{mnl}$ is the field strength for the word-volume antisymmetric tensor
$B_{mn}$

$H_{mnl}=\partial_{l}B_{mn}+\partial_{m}B_{nl}+\partial_{n}B_{lm}\ .$ (7)

Finally $a$ is world-volume scalar field that ensures the covariance of the
model.

The M5-brane action is invariant under following gauge symmetries [4]. The
first one is usual gauge symmetry

$\delta a=0\ ,\quad\delta B_{mn}=\partial_{m}\phi_{n}-\partial_{n}\phi_{m}\ .$
(8)

There is another non-trivial gauge symmetry in the form [4, 5, 6]

$\delta B_{mn}=\phi_{m}\partial_{n}a-\partial_{m}a\phi_{n}\ ,\quad\delta a=0\
.$ (9)

Finally the action is invariant under transformation [4, 5, 6]

$\delta a=\phi\ ,\quad\delta B_{mn}=\frac{\delta a}{\sqrt{\hat{\partial
a\partial a}}}\left[\hat{H}^{*}_{\
mn}-H_{mnp}\hat{g}^{ps}\frac{\partial_{s}a}{\sqrt{\widehat{\partial a\partial
a}}}\right]\ ,$ (10)

where

$\displaystyle\hat{\mathcal{H}}^{*}_{mn}=-\frac{2}{\sqrt{-\hat{g}}}\frac{\delta\mathcal{L}_{DBI}}{\delta\hat{H}^{*mn}}\
,\quad\mathcal{L}_{DBI}=\sqrt{-\det\left(\hat{g}_{mn}+i\frac{\hat{H}^{*}_{mn}}{\sqrt{\widehat{\partial
a\partial a}}}\right)}\ .$ (11)

The symmetries (9) and (10) are fundamental properties of M5-brane action (1).
They ensure that $B_{mn}$ equations of motion reduce to self-duality
conditions, see [4, 5, 6] for more details. As was shown in [4] the equation
of motion for $a(\xi)$ is consequence of the equations of motion for $B_{mn}$
and hence it is not a new solution. As a result $a$ can be eliminated by
fixing the gauge transformations (9). Following [4] we fix $a$ by imposing
condition

$\partial_{m}a=\delta_{m}^{5}\ .$ (12)

Then using (9) we get

$\delta B_{5m}=\frac{1}{2}\phi_{m}$ (13)

and hence we can fix components of $B_{5m}$ to be equal to zero

$B_{5m}=0\ .$ (14)

We impose this condition in case of spatial and null dimensional reduction of
M5-brane.

Before we proceed to the case of null dimensional reduction we review spatial
dimensional reduction of M5-brane. To do this we consider eleven dimensional
line element in the form

$ds^{2}=e^{-\frac{2}{3}\phi}g_{\mu\nu}dx^{\mu}dx^{\nu}+e^{\frac{4}{3}\phi}(dy-
C_{\mu}dx^{\mu})^{2}\ ,$ (15)

where $\mu,\nu=0,1,\dots,9$. We impose spatial dimensional reduction when we
presume that $y$ coincides with $\xi^{5}$ and when all world-volume fields do
not depend on $\xi^{5}$ except $a$ that obeys the condition (12). Let us
denote remaining world-volume coordinates as
$\xi^{\alpha},\alpha,\beta,\gamma,...=0,1,\dots,4$. Then using (14) and also
the fact that all world-volume fields do not depend on $\xi^{5}$ we find that
there are non-zero components of $H_{\alpha\beta\gamma}$ only.

Then with the help of (15) we find that components of the induced metric are
equal to

$\hat{g}_{\alpha\beta}=\partial_{\alpha}x^{\mu}\hat{g}_{\mu\nu}\partial_{\beta}x^{\nu}=e^{-\frac{2}{3}\phi}g_{\alpha\beta}+e^{\frac{4}{3}\phi}C_{\alpha}C_{\beta}\
,\quad\hat{g}_{\alpha y}=-e^{\frac{4}{3}\phi}C_{\alpha}\
,\quad\hat{g}_{yy}=e^{\frac{4}{3}\phi}\ $ (16)

with corresponding components of the inverse metric $\hat{g}^{mn}$

$\hat{g}^{\alpha\beta}=e^{\frac{2}{3}\phi}g^{\alpha\beta}\
,\quad\hat{g}^{y\alpha}=e^{\frac{2}{3}\phi}C_{\gamma}g^{\gamma\beta}\
,\quad\hat{g}^{yy}=e^{-\frac{4}{3}\phi}+e^{\frac{2}{3}\phi}C_{\alpha}g^{\alpha\beta}C_{\beta}\
,$ (17)

where $g_{\alpha\beta}$ is induced five dimensional matrix and hence
$g^{\alpha\beta}$ is its inverse metric that obeys the equation

$g_{\alpha\beta}g^{\beta\gamma}=\delta_{\alpha}^{\gamma}\ .$ (18)

Further, using (16) we obtain

$\det\hat{g}_{mn}=e^{-2\phi}\det g_{\alpha\beta}\ .$ (19)

Let us now turn our attention to $\hat{H}_{mn}^{*}$ that using (12) has the
form

$\displaystyle\hat{H}^{*}_{mn}=\frac{1}{6\sqrt{-\hat{g}}}\hat{g}_{mr}\hat{g}_{ns}\epsilon^{rstuv5}H_{tuv}\
.$ (20)

Now since $\epsilon^{mnrstu}$ is totally antisymmetric symbol we find that all
non-zero components have to the form $\epsilon^{\alpha\beta\gamma\delta\omega
5}\equiv\epsilon^{\alpha\beta\gamma\delta\omega}$ so that

$\hat{H}^{*}_{mn}=\frac{e^{-2\phi}}{6\sqrt{-g}}\hat{g}_{m\alpha}\hat{g}_{n\beta}\epsilon^{\alpha\beta\gamma\delta\omega}H_{\gamma\delta\omega}\
.$ (21)

To proceed further we use the fact that we can write

$\displaystyle\sqrt{-\det\left(\hat{g}_{mn}+i\frac{\hat{H}^{*}_{mn}}{\sqrt{\widehat{\partial
a\partial a}}}\right)}=$
$\displaystyle=\sqrt{-\det\hat{g}_{mk}}\sqrt{-\det\left(\hat{g}^{kl}+i\frac{\hat{g}^{kr}\hat{H}_{rs}^{*}\hat{g}^{sl}}{\sqrt{\widehat{\partial
a\partial a}}}\right)}\sqrt{-\det\hat{g}_{ln}}\ ,$

where

$\hat{g}^{kl}+i\frac{\hat{g}^{kr}\hat{H}_{rs}^{*}\hat{g}^{sl}}{\sqrt{\widehat{\partial
a\partial a}}}=\hat{g}^{kl}+i\frac{1}{6\sqrt{-\hat{g}}\sqrt{\widehat{\partial
a\partial a}}}\epsilon^{kltuvw}H_{tuv}\partial_{w}a\ .$ (23)

Now in the gauge $\partial_{m}a=\delta_{m}^{5}$ we obtain

$\displaystyle\det(\hat{g}^{kl}+i\frac{1}{6\sqrt{-\hat{g}}\sqrt{\widehat{\partial
a\partial a}}}\epsilon^{kltuvw}H_{tuv}\partial_{w}a)=$
$\displaystyle\left|\begin{array}[]{cc}e^{\frac{2}{3}\phi}g^{\alpha\beta}+i\frac{1}{6\sqrt{-\hat{g}}\sqrt{\widehat{\partial
a\partial
a}}}\epsilon^{\alpha\beta\gamma\delta\omega}H_{\gamma\delta\omega}&g^{\alpha\gamma}C_{\gamma}e^{\frac{2}{3}\phi}\\\
C_{\gamma}g^{\gamma\beta}e^{\frac{2}{3}\phi}&e^{-\frac{4}{3}\phi}+e^{\frac{2}{3}\phi}C_{\alpha}g^{\alpha\beta}C_{\beta}\\\
\end{array}\right|=$ (26)
$\displaystyle=\det(e^{\frac{2}{3}\phi}g^{\alpha\beta}-\frac{e^{\frac{4}{3}\phi}g^{\alpha\gamma}C_{\gamma}C_{\delta}g^{\delta\beta}}{e^{-\frac{4}{3}\phi}+e^{\frac{2}{3}\phi}C_{\alpha}g^{\alpha\beta}C_{\beta}}+i\frac{1}{6e^{-\frac{5}{3}\phi}\sqrt{-\det
g}\sqrt{1+e^{2\phi}C_{\alpha}g^{\alpha\beta}C_{\beta}}}\epsilon^{\alpha\beta\gamma\delta\omega}H_{\gamma\delta\omega})\times$
$\displaystyle\times(e^{-\frac{4}{3}\phi}+e^{\frac{2}{3}\phi}C_{\alpha}g^{\alpha\beta}C_{\beta})\
.$

Let us now analyse the second term in the action (1) that using (12) takes the
form

$\displaystyle-\frac{\sqrt{-\hat{g}}}{4\widehat{\partial a\partial
a}}\hat{H}^{*mn}H_{mnr}\partial^{r}a=-\frac{\sqrt{-\hat{g}}}{4\hat{g}^{yy}}\hat{H}^{*\alpha\beta}H_{\alpha\beta\gamma}\hat{g}^{\gamma
5}=$
$\displaystyle=-\frac{e^{\frac{2}{3}\phi}}{24(e^{-\frac{4}{3}\phi}+e^{\frac{2}{3}\phi}C_{\alpha}C^{\alpha})}\epsilon^{\alpha\beta\gamma\delta\omega}H_{\gamma\delta\omega}H_{\alpha\beta\rho}g^{\rho\sigma}C_{\sigma}\
.$

Collecting these terms together we finally obtain dimensionally reduced form
of the action

$\displaystyle S=-T_{D4}\int d^{5}\xi
e^{-\phi}\sqrt{-\det(g_{\alpha\beta}-\frac{C_{\alpha}C_{\beta}}{1+e^{2\phi}C_{\gamma}g^{\gamma\delta}C_{\delta}}+i\frac{e^{\phi}g_{\alpha\alpha^{\prime}}g_{\beta\beta^{\prime}}}{6\sqrt{-\det
g}\sqrt{1+e^{2\phi}C_{\gamma}g^{\gamma\delta}C_{\delta}}}\epsilon^{\alpha^{\prime}\beta^{\prime}\gamma\delta\omega}H_{\gamma\delta\omega})}\times$
$\displaystyle\sqrt{1+e^{2\phi}C_{\gamma}g^{\gamma\delta}C_{\delta}}$
$\displaystyle-\frac{e^{2\phi}}{24(1+e^{2\phi}C_{\gamma}g^{\gamma\delta}C_{\delta})}\epsilon^{\alpha\beta\gamma\delta\omega}H_{\gamma\delta\omega}H_{\alpha\beta\rho}g^{\rho\sigma}C_{\sigma}+$
$\displaystyle+\frac{1}{6}\int d^{5}\xi
V^{\alpha\beta\gamma}(H_{\alpha\beta\gamma}-(\partial_{\alpha}B_{\beta\gamma}+\partial_{\beta}B_{\gamma\alpha}+\partial_{\gamma}B_{\alpha\beta}))\
,$

where we identified $T_{D4}\equiv T_{M5}\int dy$ since we anticipate that the
action above corresponds to D4-brane action in Type IIA theory. However this
correspondence is not quite correct since the action (2) still depends on the
field strength of two form field $B_{\alpha\beta}$ while DBI action is defined
using gauge field which is one form. In order to find such an action we should
perform duality transformation following [13, 8]. The first step is to extend
the action as in (2) that allows us to treat $B_{\beta\gamma}$ and
$H_{\alpha\beta\gamma}$ as independent variables. Since now $B_{\alpha\beta}$
is independent it can be integrated out that implies

$\partial_{\alpha}V^{\alpha\beta\gamma}=0\ .$ (30)

This equation has solution in five dimensions in the form

$V^{\alpha\beta\gamma}=\epsilon^{\alpha\beta\gamma\delta\omega}\partial_{\delta}A_{\omega}\
.$ (31)

In order to find dual action we should integrate out three form
$H_{\alpha\beta\gamma}$. To do this we will perform inverse procedure to the
analysis presented in [13, 8]. Explicitly, we start with the action (2) and
find its dual formulation. Following [13, 8] we use five dimensional Lorentz
invariance and switch to local inertial frame where
$g_{\alpha\beta}=\eta_{\alpha\beta}$ and where we also presume that
$C_{\alpha}$ has non-zero component $C_{0}$ only. We further presume that
there are non-zero components $H_{340}$ and $H_{120}$ so that the matrix

$\mathbf{M}_{\alpha\beta}\equiv
g_{\alpha\beta}-\frac{e^{2\phi}C_{\alpha}C_{\beta}}{1+e^{2\phi}C_{\gamma}C^{\gamma}}+i\frac{e^{\phi}g_{\alpha\alpha^{\prime}}g_{\beta\beta^{\prime}}}{6\sqrt{-\det
g}\sqrt{1+e^{2\phi}C_{\gamma}C^{\gamma}}}\epsilon^{\alpha^{\prime}\beta^{\prime}\gamma\delta\omega}H_{\gamma\delta\omega}$
(32)

has the form

$\mathbf{M}_{\alpha\beta}=\left(\begin{array}[]{ccccc}-1-\frac{e^{2\phi}C_{0}^{2}}{1-e^{2\phi}C_{0}^{2}}&0&0&0&0\\\
0&1&i\frac{e^{\phi}}{\sqrt{1-e^{2\phi}C_{0}^{2}}}H_{340}&0&0\\\
0&-i\frac{e^{\phi}}{\sqrt{1-e^{2\phi}C_{0}^{2}}}H_{340}&1&0&0\\\
0&0&0&1&i\frac{e^{\phi}}{\sqrt{1-e^{2\phi}C_{0}^{2}}}H_{120}\\\
0&0&0&-i\frac{e^{\phi}}{\sqrt{1-e^{2\phi}C_{0}^{2}}}H_{120}&1\\\
\end{array}\right)$ (33)

so that $\det\mathbf{M}_{\alpha\beta}$ is equal to

$\det\mathbf{M}_{\alpha\beta}=-\left(1+\frac{e^{2\phi}C_{0}^{2}}{1-e^{2\phi}C_{0}^{2}}\right)\left(1-\frac{e^{2\phi}}{1-e^{2\phi}C_{0}^{2}}H_{2}^{2}\right)\left(1-\frac{e^{2\phi}}{1-e^{2\phi}C_{0}^{2}}H_{1}^{2}\right)\
,$ (34)

where $H_{340}=H_{2}\ ,H_{120}=H_{1}$. We also introduce notation
$V^{120}=V_{1},V^{340}=V_{2}$. Then the action has the form

$\displaystyle S=-T_{D4}\int d^{5}\xi
e^{-\phi}\sqrt{\left(1-\frac{e^{2\phi}}{1-e^{2\phi}C_{0}^{2}}H_{2}^{2}\right)\left(1-\frac{e^{2\phi}}{1-e^{2\phi}C_{0}^{2}}H_{1}^{2}\right)}+$
$\displaystyle+T_{D4}\int
d^{5}\xi\left(\frac{e^{2\phi}C_{0}}{1-e^{2\phi}C_{0}^{2}}H_{1}H_{2}+V_{1}H_{1}+V_{2}H_{2}\right)\
.$

Let us solve equations of motion for $H_{1}$ and $H_{2}$ that follow from (2)

$\displaystyle\triangle e^{-\phi}\sqrt{\frac{1-\triangle
H_{2}^{2}}{1-\triangle H_{1}^{2}}}H_{1}+\triangle C_{0}H_{2}+V_{1}=0\ ,$
$\displaystyle\triangle e^{-\phi}\sqrt{\frac{1-\triangle
H_{1}^{2}}{1-\triangle H_{2}^{2}}}H_{2}+\triangle C_{0}H_{1}+V_{2}=0\ ,$

where

$\triangle=\frac{e^{2\phi}}{1-e^{2\phi}C_{0}^{2}}\ .$ (37)

The equations (2) have solutions

$\displaystyle H_{1}=V_{2}C_{0}\mp
e^{-\phi}\sqrt{\frac{1+V_{2}^{2}}{1+V_{1}^{2}}}V_{1}\ ,\quad
H_{2}=V_{1}C_{0}\mp e^{-\phi}\sqrt{\frac{1+V_{1}^{2}}{1+V_{2}^{2}}}V_{2}\ .$

Finally inserting these solutions into (2) we obtain

$\displaystyle S=-T_{D4}\int
d^{5}\xi[e^{-\phi}\sqrt{(1+V_{1}^{2})(1+V_{2}^{2})}-C_{0}V_{1}V_{2}]$

using that on-shell we have

$-e^{-\phi}\sqrt{(1-\triangle H_{1}^{2})(1-\triangle
H_{2}^{2})}=\frac{1}{\triangle H_{1}}(\triangle C_{0}H_{2}+V_{1})(1-\triangle
H_{1}^{2})\\\ $

and we have chosen $-$ sign in (2) to have an action with correct sign. Then
we finally introduce
$F_{\alpha\beta}=\partial_{\alpha}A_{\beta}-\partial_{\beta}A_{\alpha}$ and
restore world-volume covariance so that the action (2) has final form

$S=-T_{D4}\int
d^{5}\xi\left[e^{-\phi}\sqrt{-\det(g_{\alpha\beta}+F_{\alpha\beta})}-\frac{1}{8}\epsilon^{\alpha\beta\gamma\delta\omega}C_{\alpha}F_{\beta\gamma}F_{\delta\omega}\right]\
.$ (40)

This is action for D4-brane in the background with metric $g_{\mu\nu}$ and
Ramond-Ramond form $C_{\mu}$. Observe that we proceeded in the inverse
direction to the duality transformation studied in [8] where the starting
point was D4-brane in the background with non-trivial metric and Ramond-Ramond
one form. In the next section we apply similar idea to the case of null
dimensional reduction of M5-brane.

##  3 Null Dimensional Reduction of M5-brane

We begin this section with the review of basic facts about null dimensional
reduction of gravity.

Let us consider gravity background with null isometry and perform its
dimensional reduction, following [2, 3]. We presume that we have vierbein
$\hat{E}_{M}^{\ A}$ and inverse $\hat{E}^{M}_{\ A}$ where $M=0,1,\dots,10$ and
tangent space-indices $A,B=0,1,\dots,10$ that obey

$\hat{E}_{M}^{\ A}\hat{E}^{M}_{\ B}=\delta^{A}_{B}\ ,\quad\hat{E}_{M}^{\
A}\hat{E}^{N}_{\ A}=\delta_{M}^{N}\ .$ (41)

If we try to dimensionally reduce along null direction we assume an existence
of null Killing vector $\xi=\xi^{M}\partial_{M}$ that obeys

$\mathcal{L}_{\xi}\hat{g}_{MN}=0\ ,\quad\xi^{M}\hat{g}_{MN}\xi^{N}=0\ .$ (42)

Without loss of generality we choose adapted coordinates $x^{M}=(x^{\mu},y)$
where $\mu=0,1,\dots,9$ and where $x^{10}=y$ so that
$\xi=\xi^{y}\partial_{y}$. Then $\mathcal{L}_{\xi}\hat{g}=0$ implies that
$\hat{g}$ is independent on $y$ and null condition on $\xi$ implies

$\hat{g}_{yy}=0\ .$ (43)

To proceed further we introduce suitable ansatz for vierbein, following [2,
3]. We split $11$ tangent space indices into $a=1,\dots,9$ and remaining as
$+,-$ so that $\eta_{ab}=\delta_{ab}\ ,\eta_{+-}=\eta_{-+}=1$. Then we have
following ansatz for $\hat{E}_{M}^{\ A}$

$\displaystyle\hat{E}_{\mu}^{\ a}=e_{\mu}^{\ a}\ ,\quad\hat{E}_{\mu}^{\
-}=S^{-1}\tau_{\mu}\ ,\quad\hat{E}_{\mu}^{\ +}=-Sm_{\mu}\ ,$
$\displaystyle\hat{E}_{y}^{\ a}=0\ ,\quad\hat{E}_{y}^{\ -}=0\
,\quad\hat{E}_{y}^{\ +}=S\ $

and hence we have following components of metric

$\displaystyle\hat{g}_{\mu\nu}=h_{\mu\nu}-m_{\mu}\tau_{\nu}-\tau_{\mu}m_{\nu}\
,\quad\hat{g}_{\mu y}=\tau_{\mu}=\hat{g}_{y\mu}\ ,\quad\hat{g}_{yy}=0\ .$

Let us now consider induced metric on the world-volume of M5-brane with world-
volume coordinates $m,n=0,1,\dots,5$ when we presume that M5-brane is extended
along $y-$direction. Explicitly

$\xi^{5}=y\ $ (46)

so that the induced metric $\hat{g}_{mn}$ has the form

$\displaystyle\hat{g}_{\alpha\beta}=h_{\alpha\beta}-m_{\alpha}\tau_{\beta}-\tau_{\alpha}m_{\beta}\equiv\bar{h}_{\alpha\beta}\
,\quad\hat{g}_{yy}=0\ ,\quad\hat{g}_{y\alpha}=\tau_{\alpha}\ ,$

where
$h_{\alpha\beta}=h_{\mu\nu}\partial_{\alpha}X^{\mu}\partial_{\beta}X^{\nu}\
,\quad m_{\alpha}=m_{\mu}\partial_{\alpha}X^{\mu}\
,\quad\tau_{\alpha}=\tau_{\mu}\partial_{\alpha}X^{\mu}$. For further purposes
we also write elements of the world-volume inverse metric in the form

$\hat{g}^{yy}=2\Phi\ ,\quad\hat{g}^{\alpha y}=-\hat{v}^{\alpha}\
,\quad\hat{g}^{y\alpha}=-\hat{v}^{\alpha},\quad\hat{g}^{\alpha\beta}=h^{\alpha\beta}\
,$ (48)

where we defined

$\hat{v}^{\alpha}=v^{\alpha}-h^{\alpha\beta}m_{\beta}\
,\quad\Phi=-v^{\alpha}m_{\alpha}+\frac{1}{2}m_{\alpha}h^{\alpha\beta}m_{\beta}\
,$ (49)

and where we also introduced $v^{\alpha},h^{\alpha\beta}$ through the
relations

$\tau_{\alpha}v^{\alpha}=-1\ ,\quad\tau_{\alpha}h^{\alpha\beta}=0\ ,\quad
v^{\alpha}h_{\alpha\beta}=0\ ,\quad
h_{\alpha\beta}h^{\beta\gamma}-\tau_{\alpha}v^{\gamma}-=\delta_{\alpha}^{\gamma}\
.$ (50)

It is important to stress that $v^{\alpha}$ and $h^{\alpha\beta}$ are five
dimensional vector and tensor respectively.

Let us now proceed to the null dimensional reduction of M5-brane action (1).
First of all we again use the fact that generally we have

$\displaystyle-\det\left(\hat{g}_{mn}+i\frac{\hat{H}^{*}_{mn}}{\sqrt{\widehat{\partial
a\partial a}}}\right)$
$\displaystyle=-\det\hat{g}_{mr}\left(-\det(\hat{g}^{rs}+i\frac{1}{6\sqrt{\widehat{\partial
a\partial
a}}\sqrt{-\hat{g}}}\epsilon^{rstuvw}H_{tuv}\partial_{w}a)\right)(-\det\hat{g}_{sn})\
.$

As in previous section we find that non-zero components of $H_{mnr}$ are
$H_{\alpha\beta\gamma}$ when we also imposed the gauge fixing condition (12).
Using these facts we find that the second bracket in (3) is equal to

$\displaystyle-\det(\hat{g}^{rs}+i\frac{1}{6\sqrt{\widehat{\partial a\partial
a}}\sqrt{-\hat{g}}}\epsilon^{\alpha\beta\gamma\delta\omega}H_{\gamma\delta\omega})=-\left|\begin{array}[]{cc}h^{\alpha\beta}+i\frac{1}{6\sqrt{2\Phi}\sqrt{-\hat{g}}}\epsilon^{\alpha\beta\gamma\delta\omega}H_{\gamma\delta\omega}&-\hat{v}^{\beta}\\\
-\hat{v}^{\alpha}&2\Phi\\\ \end{array}\right|=$ (54)
$\displaystyle=-2\Phi\det(h^{\alpha\beta}-\frac{1}{2\Phi}\hat{v}^{\alpha}\hat{v}^{\beta}+i\frac{1}{6\sqrt{-\det\bar{h}_{\alpha\beta}}}\epsilon^{\alpha\beta\gamma\delta\omega}H_{\gamma\delta\omega})\
,$

where in the final step we used the fact that

$\det\hat{g}_{mn}=\left|\begin{array}[]{cc}0&-\tau_{\beta}\\\
-\tau_{\alpha}&\bar{h}_{\alpha\beta}\\\
\end{array}\right|=\left|\begin{array}[]{cc}-\tau_{\gamma}\bar{h}^{\gamma\delta}\tau_{\delta}&0\\\
-\tau_{\alpha}&\bar{h}_{\alpha\beta}\\\
\end{array}\right|=\frac{1}{2\Phi}\det\bar{h}_{\alpha\beta}\ $ (56)

together with the fact that the matrix

$\bar{h}^{\alpha\beta}=h^{\alpha\beta}-\frac{1}{2\Phi}\hat{v}^{\alpha}\hat{v}^{\beta}\
$ (57)

is inverse to
$\bar{h}_{\alpha\beta}=h_{\alpha\beta}-m_{\alpha}\tau_{\beta}-\tau_{\alpha}m_{\beta}$

$\displaystyle\bar{h}_{\alpha\beta}\bar{h}^{\beta\gamma}=\delta_{\alpha}^{\gamma}\
.$ (58)

This can be easily proved using the fact that

$\bar{h}_{\alpha\beta}\hat{v}^{\beta}=2\tau_{\alpha}\Phi\
,\quad\tau_{\alpha}\bar{h}^{\alpha\beta}\tau_{\beta}=-\frac{1}{2\Phi}\ .$ (59)

With the help of these results we obtain that the gauge fixed form of the
expression (54) is equal to

$\displaystyle-\det\left(\hat{g}_{mn}+i\frac{\hat{H}^{*}_{mn}}{\sqrt{\widehat{\partial
a\partial
a}}}\right)=-\frac{1}{2\Phi}\det\left(\bar{h}_{\alpha\beta}+i\frac{1}{6\sqrt{-\det\bar{h}_{\alpha\beta}}}\bar{h}_{\alpha\gamma}\bar{h}_{\beta\delta}\epsilon^{\gamma\delta\sigma\omega\rho}H_{\sigma\omega\rho}\right)\
.$

Further we have

$\displaystyle-\frac{\sqrt{-\hat{g}}}{4\widehat{\partial a\partial
a}}\hat{H}^{*mn}H_{mnr}\partial^{r}a=\frac{1}{48\Phi}\epsilon^{\alpha\beta\gamma\delta\omega}H_{\gamma\delta\omega}H_{\alpha\beta\sigma}\hat{v}^{\sigma}$

so that we obtain M5-brane action dimensionally reduced along null direction
in the form

$\displaystyle S=-T_{M5}\int dy\int
d^{5}\xi\left[\sqrt{-\frac{1}{2\Phi}\det(\bar{h}_{\alpha\beta}+i\frac{1}{6\sqrt{-\det\bar{h}_{\alpha\beta}}}\bar{h}_{\alpha\gamma}\bar{h}_{\beta\delta}\epsilon^{\gamma\delta\sigma\omega\rho}H_{\sigma\omega\rho})}\right.$
$\displaystyle\left.+\frac{1}{48\Phi}\epsilon^{\alpha\beta\gamma\delta\omega}H_{\gamma\delta\omega}H_{\alpha\beta\sigma}\hat{v}^{\sigma}\right]\
.$

It is important to stress that $\Phi$ is equal to
$-\frac{1}{2}\hat{v}^{\alpha}\bar{h}_{\alpha\beta}\hat{v}^{\beta}$. However it
is convenient to treat it as independent when we add $\int d^{5}\xi
B(2\Phi+\hat{v}^{\alpha}\bar{h}_{\alpha\beta}\hat{v}^{\beta})$ into action.
Further, this action still depend on two form $B_{\alpha\beta}$ however it is
more natural to find its dual formulation in the same way as in case of the
spatial dimensional reduction. In order to find such formulation we again
extend the action as in previous section

$\displaystyle S=-T_{ND4}\int
d^{5}\xi\left[\sqrt{-\frac{1}{2\Phi}\det(\bar{h}_{\alpha\beta}+i\frac{1}{6\sqrt{-\det\bar{h}_{\alpha\beta}}}\bar{h}_{\alpha\gamma}\bar{h}_{\beta\delta}\epsilon^{\gamma\delta\sigma\omega\rho}H_{\sigma\omega\rho})}\right.$
$\displaystyle\left.+\frac{1}{48\Phi}\epsilon^{\alpha\beta\gamma\delta\omega}H_{\gamma\delta\omega}H_{\alpha\beta\sigma}\hat{v}^{\sigma}+B(2\Phi+\hat{v}^{\alpha}\bar{h}_{\alpha\beta}\hat{v}^{\beta})\right]+$
$\displaystyle+\frac{1}{6}\int d^{5}\xi
V^{\alpha\beta\gamma}(H_{\alpha\beta\gamma}-(\partial_{\alpha}B_{\beta\gamma}+\partial_{\beta}B_{\gamma\alpha}+\partial_{\gamma}B_{\alpha\beta}))\
,$

where we introduced effective non-relativistic D4-brane tension through the
relation

$T_{ND4}=T_{M5}\int dy\ .$ (64)

In (3) we treat $H_{\alpha\beta\gamma}$ and $B_{\alpha\beta}$ as independent
fields. Then the equations of motion for $B_{\alpha\beta}$ implies that
$V^{\alpha\beta\gamma}$ is equal to

$V^{\alpha\beta\gamma}=\epsilon^{\alpha\beta\gamma\delta\omega}\partial_{\delta}A_{\omega}\
.$ (65)

Finally we should solve equations of motion for $H_{\alpha\beta\gamma}$. Since
it is difficult to solve them in the full generality we proceed in the similar
way as in previous section even if we have to be more careful now. First of
all we can still presume that the space metric $h_{\alpha\beta}$ has the form

$h_{\alpha\beta}=\delta_{\alpha}^{i}\delta_{\beta}^{j}\delta_{ij}\ ,\quad
i,j=1,\dots,4\ .$ (66)

On the other hand in case of time one form we presume that it is constant and
equal to

$\tau_{\alpha}=\tau\delta_{\alpha}^{0}\ .$ (67)

We further presume that $m$ has non-zero components $m_{0},m_{1}$ so that
$\bar{h}_{00}=-2\tau m_{0},\bar{h}_{01}=-\tau m_{1}=\bar{h}_{10}$. Then since
$v^{\alpha}$ is defined as $\tau_{\alpha}v^{\alpha}=-1$ we have
$v^{0}=-\frac{1}{\tau}$. Finally, as in previous section we presume that non-
zero components of $H_{\alpha\beta\gamma}$ are $H_{340}\equiv
H_{2},H_{120}\equiv H_{1}$. Then the matrix
$M_{\alpha\beta}=\bar{h}_{\alpha\beta}+i\frac{1}{6\sqrt{-\det\bar{h}_{\alpha\beta}}}\bar{h}_{\alpha\gamma}\bar{h}_{\beta\delta}\epsilon^{\gamma\delta\sigma\omega\rho}H_{\sigma\omega\rho}$
has the form

$\left(\begin{array}[]{ccccc}\bar{h}_{00}&\bar{h}_{01}&i\frac{\bar{h}_{01}}{\sqrt{-\bar{h}}}H_{2}&0&0\\\
\bar{h}_{10}&1&i\frac{1}{\sqrt{-\bar{h}}}H_{2}&0&0\\\
-i\frac{\bar{h}_{01}}{\sqrt{-\bar{h}}}H_{2}&-i\frac{1}{\sqrt{-\bar{h}}}H_{2}&1&0&0\\\
0&0&0&1&i\frac{1}{\sqrt{-\bar{h}}}H_{1}\\\
0&0&0&-i\frac{1}{\sqrt{-\bar{h}}}H_{1}&1\\\ \\\ \end{array}\right)$ (68)

where we defined
$\bar{h}=\det\bar{h}_{\alpha\beta}=\bar{h}_{00}-\bar{h}_{01}^{2}$. Then the
extended action (3) has the form

$\displaystyle S=-T_{ND4}\int
d^{5}\xi\sqrt{-\frac{\bar{h}}{2\Phi}}\left[\sqrt{(1-\tilde{H}_{1}^{2})(1-\tilde{H}_{2}^{2})}+\sqrt{-\frac{\bar{h}(v^{0})^{2}}{2\Phi}}\tilde{H}_{1}\tilde{H}_{2}-\tilde{V}_{1}\tilde{H}_{1}-\tilde{V}_{2}\tilde{H}_{2}\right]+$
$\displaystyle+T_{ND4}\int d^{5}\xi B(2\Phi+2\frac{m_{0}}{\tau}+m_{1}^{2})\ ,$

where $V_{1}\equiv V^{120}\ ,V_{2}\equiv V^{340}$ and where we performed
rescaling
$H_{1,2}=\sqrt{-\bar{h}}\tilde{H}_{1,2},V_{1,2}=\frac{1}{\sqrt{2\Phi}}\tilde{V}_{1,2}$.
From (3) we obtain following equations of motion

$\displaystyle-\tilde{H}_{1}\sqrt{\frac{1-\tilde{H}_{2}^{2}}{1-\tilde{H}_{1}^{2}}}+\triangle\tilde{H}_{2}-\tilde{V}_{1}=0\
,$
$\displaystyle-\tilde{H}_{2}\sqrt{\frac{1-\tilde{H}_{1}^{2}}{1-\tilde{H}_{2}^{2}}}+\triangle\tilde{H}_{1}-\tilde{V}_{2}=0\
,$

where $\triangle=\sqrt{-\frac{\bar{h}(v^{0})^{2}}{2\Phi}}$. If we multiply two
equations in (3) together we obtain

$\tilde{H}_{1}\tilde{H}_{2}=(\tilde{V}_{1}-\triangle\tilde{H}_{2})(\tilde{V}_{2}-\triangle\tilde{H}_{1})$
(71)

that allows us to express $\tilde{H}_{2}$ as

$\tilde{H}_{2}=\frac{\tilde{V}_{1}\tilde{V}_{2}-\triangle\tilde{H}_{1}\tilde{V}_{1}}{(1-\triangle^{2})\tilde{H}_{1}+\triangle\tilde{V}_{2}}\
.$ (72)

Inserting this result into the first equation in (3) we obtain quadratic
equation for $\tilde{H}_{1}$ that has solution

$\displaystyle\tilde{H}_{1}=\frac{1}{1-\triangle^{2}}\left[-\triangle\tilde{V}_{2}\pm\tilde{V}_{1}\sqrt{\frac{(1-\triangle^{2})+\tilde{V}_{2}^{2}}{(1-\triangle^{2})+\tilde{V}_{1}^{2}}}\right]\
.$

Then inserting this result into (72) we also find $\tilde{H}_{2}$ to be equal
to

$\tilde{H}_{2}=\frac{1}{1-\triangle^{2}}\left[-\triangle\tilde{V}_{1}\pm\tilde{V}_{2}\sqrt{\frac{(1-\triangle^{2})+\tilde{V}_{1}^{2}}{(1-\triangle^{2})+\tilde{V}_{2}^{2}}}\right]\
.$ (74)

Finally inserting (3),(74) into the action (3) we obtain

$S=-T_{ND4}\int
d^{5}\xi\left[\sqrt{-\frac{1}{2\Phi}(\bar{h}+\bar{h}\hat{V}_{1}^{2})(1+\hat{V}_{2}^{2})}-\frac{1}{2\Phi}\bar{h}v^{0}\hat{V}_{1}\hat{V}_{2}+B(2\Phi+2\frac{m_{0}}{\tau}+m_{1}^{2})\right]\
,$ (75)

where we performed rescalling
$\tilde{V}_{1.2}=\sqrt{1-\triangle^{2}}\hat{V}_{1,2}$ To proceed further let
us observe that we can write

$\bar{h}(1+\hat{V}_{1}^{2})=\left|\begin{array}[]{ccc}\bar{h}_{00}-\bar{h}_{01}\bar{h}_{10}&0&0\\\
0&1&\hat{V}_{1}\\\ 0&-\hat{V}_{1}&1\\\ \end{array}\right|$ (76)

so that it is natural to introduce redefined form of the metric
$\bar{h}^{\prime}_{00}=\bar{h}_{00}-\bar{h}_{01}\bar{h}_{10}$. Further, we
have $\bar{h}v^{0}=m_{0}+m_{1}m_{0}\equiv m_{0}^{\prime}$ introducing
redefined $m_{0}^{\prime}$. Using these results we can restore general form of
the action for D4-brane in Newton-Cartan background in the form

$S=-T_{ND4}\int
d^{5}\xi\left[\sqrt{-\frac{1}{2\Phi}(\bar{h}_{\alpha\beta}+F_{\alpha\beta})}-\frac{1}{16\Phi}\epsilon^{\alpha\beta\gamma\delta\omega}m_{\alpha}F_{\beta\gamma}F_{\delta\omega}+B(2\Phi+\hat{v}^{\alpha}\bar{h}_{\alpha\beta}\hat{v}^{\beta})\right]\
.$ (77)

Finally we can integrate $B$ to replace $2\Phi$ with
$-\hat{v}^{\alpha}\bar{h}_{\alpha\beta}\hat{v}^{\beta}$. Then we can rewrite
the action into alternative form with the help of following manipulation

$\displaystyle\frac{1}{2\Phi}\det(\bar{h}_{\alpha\beta}+F_{\alpha\beta})=\frac{1}{2\Phi}\det\bar{h}_{\alpha\gamma}\det(\delta^{\gamma}_{\beta}+\bar{h}^{\gamma\omega}F_{\omega\beta})=$
$\displaystyle\det(H_{\alpha\gamma})\det(\delta^{\gamma}_{\beta}+\bar{h}^{\gamma\omega}F_{\omega\beta})=\det(H_{\alpha\beta}+H_{\alpha\gamma}\bar{h}^{\gamma\omega}F_{\omega\beta})=$
$\displaystyle=\det(H_{\alpha\beta}+\tilde{F}_{\alpha\beta})\ ,$

where we introduced $H_{\alpha\beta}$ and its inverse $H^{\alpha\beta}$ as

$H_{\alpha\beta}=-\tau_{\alpha}\tau_{\beta}+h_{\alpha\beta}\ ,\quad
H^{\alpha\beta}=h^{\alpha\beta}-v^{\alpha}v^{\beta}$ (79)

and also $\tilde{F}_{\alpha\beta}$ through the relation

$\tilde{F}_{\alpha\beta}=H_{\alpha\gamma}\bar{h}^{\gamma\delta}F_{\delta\beta}\
.$ (80)

Finally we used the fact that

$\displaystyle\frac{1}{2\Phi}\det\bar{h}_{\alpha\beta}=\det\left(\begin{array}[]{cc}0&-\tau_{\beta}\\\
-\tau_{\alpha}&\bar{h}_{\alpha\beta}\\\
\end{array}\right)=\det\left(\begin{array}[]{cc}0&-\tau_{\beta}\\\
-\tau_{\alpha}&-\tau_{\alpha}\tau_{\beta}+h_{\alpha\beta}\\\
\end{array}\right)$ (85)
$\displaystyle=\det\left(\begin{array}[]{cc}-\tau_{\gamma}H^{\gamma\delta}\tau_{\delta}&0\\\
-\tau_{\alpha}&H_{\gamma\delta}\\\ \end{array}\right)=\det(H_{\alpha\beta})\
.$ (88)

Then we obtain that the action (77) can be rewritten into the form

$\displaystyle S=-T_{ND4}\int
d^{5}\xi\left[\sqrt{-(H_{\alpha\beta}+\tilde{F}_{\alpha\beta})}-\frac{1}{16\Phi}\epsilon^{\alpha\beta\gamma\delta\omega}m_{\alpha}\tilde{F}_{\beta\gamma}\tilde{F}_{\delta\omega}\right.$
$\displaystyle\left.-\frac{1}{8\Phi}\epsilon^{\alpha\beta\gamma\delta\omega}m_{\alpha}\tilde{F}_{\beta\gamma}\tau_{\delta}(-v^{\sigma}\tilde{F}_{\sigma\omega}+\frac{1}{2\Phi}(1+v^{\delta}m_{\delta})\hat{v}^{\sigma}\tilde{F}_{\sigma\omega})\right]\
,$

The action (77) or its alternative form (3) is final form of non-relativistic
D4-brane action in the background that arises by null dimensional reduction of
M-theory. The kinetic part of the action has standard DBI like form and has
functionally the same form as D2-brane action in torsional Newton-Cartan
background that was found in [17]. There is also an additional WZ like term
that has an exceptional property that it depends on the world-volume fields
$v^{\alpha},h^{\alpha\beta}$.

##  4 M5-brane Transverse to Null Direction

In this last section we briefly discuss situation when we presume that
M5-brane is transverse to null direction. In this case the induced metric has
the form

$\hat{g}_{mn}=\bar{h}_{mn}+\tau_{m}\partial_{n}y+\partial_{m}y\tau_{n}\
,\quad\tau_{m}=\tau_{\mu}\partial_{m}X^{\mu}\
,\quad\bar{h}_{mn}=\bar{h}_{\mu\nu}\partial_{m}X^{\mu}\partial_{n}X^{\nu}\ ,$
(91)

where

$\bar{h}_{mn}=h_{mn}-\tau_{m}m_{n}-\tau_{n}m_{m}\ .$ (92)

Note that this metric is non-singular with inverse metric

$\bar{h}^{mn}=h^{mn}-\frac{1}{2\Phi}\hat{v}^{m}\hat{v}^{n}\ ,$ (93)

where again

$\hat{v}^{m}=v^{m}-h^{mn}m_{n}\
,\quad\Phi=-v^{m}m_{m}+\frac{1}{2}m_{m}h^{mn}m_{n}\ ,$ (94)

where $v^{m}$ is six dimensional vector that obeys $v^{m}\tau_{m}=-1\
,v^{m}h_{mn}=0$. Finally $h^{mn}$ is six dimensional singular matrix that
obeys

$h_{mn}h^{nk}-\tau_{m}v^{k}=\delta_{m}^{k}\ ,\tau_{m}h^{mn}=0\ .$ (95)

In order to find NS5-brane in torsional Newton-Cartan background it is
appropriate to express all components of world-volume metric $\hat{g}_{mn}$
and its inverse $\hat{g}^{mn}$ with the help of $\bar{h}_{mn},\tau_{m},m_{n}$,
derivatives of $y$ together with inverse fields $\bar{h}^{mn},\hat{v}^{m}$. In
case of $\hat{g}_{mn}$ these relations are given in (91). In case of
$\hat{g}^{mn}$ the situation is more complicated and after some calculations
we find

$\displaystyle\hat{g}^{kl}=\bar{h}^{kl}+\frac{1}{1+\frac{1}{\Phi}\hat{v}^{m}\partial_{m}y+\frac{1}{2\Phi}\partial_{m}yh^{mn}\partial_{n}y}(-\frac{1}{4\Phi^{2}}\hat{v}^{k}\hat{v}^{l}\partial_{m}y\bar{h}^{mn}\partial_{n}y$
$\displaystyle+\frac{1}{2\Phi}\bar{h}^{kp}\partial_{p}y(1+\frac{1}{2\Phi}\hat{v}^{m}\partial_{m}y)\hat{v}^{l}+\frac{1}{2\Phi}\hat{v}^{k}(1+\frac{1}{2\Phi}\hat{v}^{m}\partial_{m}y)\partial_{p}y\bar{h}^{pl}-\frac{1}{2\Phi}\bar{h}^{km}\partial_{m}y\partial_{n}y\bar{h}^{nl})\
.$

Then we obtain an action for non-relativistic NS5-brane when we insert (91)
and (4) into (1). Note that there are two scalar fields $y(\xi),a(\xi)$ that
propagate on the world-volume of NS5-brane. In fact this result is similar to
transverse dimensional reduction of M5-brane as was studied in [7]. However
due to the form of the metric $\hat{g}_{mn},\hat{g}^{mn}$ it is impossible to
simplify M5-brane action further so that we will not write explicit form of
the action for NS5-brane in torsional Newton-Cartan background here.

###### Acknowledgments.

This work is supported by the grant “Integrable Deformations” (GA20-04800S)
from the Czech Science Foundation (GACR).

## References

  * [1] E. Witten, _“String theory dynamics in various dimensions,”_ Nucl. Phys. B 443 (1995), 85-126 doi:10.1016/0550-3213(95)00158-O [arXiv:hep-th/9503124 [hep-th]].
  * [2] B. Julia and H. Nicolai, _“Null Killing vector dimensional reduction and Galilean geometrodynamics,”_ Nucl. Phys. B 439 (1995), 291-326 doi:10.1016/0550-3213(94)00584-2 [arXiv:hep-th/9412002 [hep-th]].
  * [3] E. Bergshoeff, A. Chatzistavrakidis, L. Romano and J. Rosseel, _“Newton-Cartan Gravity and Torsion,”_ JHEP 10 (2017), 194 doi:10.1007/JHEP10(2017)194 [arXiv:1708.05414 [hep-th]].
  * [4] P. Pasti, D. P. Sorokin and M. Tonin, _“Covariant action for a D = 11 five-brane with the chiral field,”_ Phys. Lett. B 398 (1997), 41-46 doi:10.1016/S0370-2693(97)00188-3 [arXiv:hep-th/9701037 [hep-th]].
  * [5] I. A. Bandos, K. Lechner, A. Nurmagambetov, P. Pasti, D. P. Sorokin and M. Tonin, _“On the equivalence of different formulations of the M theory five-brane,”_ Phys. Lett. B 408 (1997), 135-141 doi:10.1016/S0370-2693(97)00784-3 [arXiv:hep-th/9703127 [hep-th]].
  * [6] I. A. Bandos, K. Lechner, A. Nurmagambetov, P. Pasti, D. P. Sorokin and M. Tonin, _“Covariant action for the superfive-brane of M theory,”_ Phys. Rev. Lett. 78 (1997), 4332-4334 doi:10.1103/PhysRevLett.78.4332 [arXiv:hep-th/9701149 [hep-th]].
  * [7] I. A. Bandos, A. Nurmagambetov and D. P. Sorokin, _“The Type IIA NS5-brane,”_ Nucl. Phys. B 586 (2000), 315-330 doi:10.1016/S0550-3213(00)00398-9 [arXiv:hep-th/0003169 [hep-th]].
  * [8] M. Aganagic, J. Park, C. Popescu and J. H. Schwarz, _“Dual D-brane actions,”_ Nucl. Phys. B 496 (1997), 215-230 doi:10.1016/S0550-3213(97)00257-5 [arXiv:hep-th/9702133 [hep-th]].
  * [9] M. Aganagic, J. Park, C. Popescu and J. H. Schwarz, _“World volume action of the M theory five-brane,”_ Nucl. Phys. B 496 (1997), 191-214 doi:10.1016/S0550-3213(97)00227-7 [arXiv:hep-th/9701166 [hep-th]].
  * [10] E. Witten, _“Five-brane effective action in M theory,”_ J. Geom. Phys. 22 (1997), 103-133 doi:10.1016/S0393-0440(97)80160-X [arXiv:hep-th/9610234 [hep-th]].
  * [11] P. K. Townsend, _“D-branes from M-branes,”_ Phys. Lett. B 373 (1996), 68-75 doi:10.1016/0370-2693(96)00104-9 [arXiv:hep-th/9512062 [hep-th]].
  * [12] P. K. Townsend, _“Four lectures on M theory,”_ [arXiv:hep-th/9612121 [hep-th]].
  * [13] A. A. Tseytlin, _“Selfduality of Born-Infeld action and Dirichlet three-brane of type IIB superstring theory,”_ Nucl. Phys. B 469 (1996), 51-67 doi:10.1016/0550-3213(96)00173-3 [arXiv:hep-th/9602064 [hep-th]].
  * [14] D. Berman, _“M5 on a torus and the three-brane,”_ Nucl. Phys. B 533 (1998), 317-332 doi:10.1016/S0550-3213(98)80009-6 [arXiv:hep-th/9804115 [hep-th]].
  * [15] D. Hansen, J. Hartong and N. A. Obers, _“Non-Relativistic Gravity and its Coupling to Matter,”_ JHEP 06 (2020), 145 doi:10.1007/JHEP06(2020)145 [arXiv:2001.10277 [gr-qc]].
  * [16] N. Lambert and T. Orchard, _“Null reductions of the M5-brane,”_ JHEP 12 (2020), 037 doi:10.1007/JHEP12(2020)037 [arXiv:2005.14331 [hep-th]].
  * [17] J. Kluson, _“Null Dimensional Reduction of M2-Brane,”_ [arXiv:2103.00566 [hep-th]].
  * [18] C. D. A. Blair, D. Gallegos and N. Zinnato, _“A non-relativistic limit of M-theory and 11-dimensional membrane Newton-Cartan,”_ [arXiv:2104.07579 [hep-th]].
  * [19] J. Klusoň and P. Novosad, _“Non-Relativistic M2-Brane,”_ JHEP 06 (2019), 072 doi:10.1007/JHEP06(2019)072 [arXiv:1903.12450 [hep-th]].
  * [20] D. Hansen, J. Hartong and N. A. Obers, _“Non-Relativistic Gravity and its Coupling to Matter,”_ JHEP 06 (2020), 145 doi:10.1007/JHEP06(2020)145 [arXiv:2001.10277 [gr-qc]].
  * [21] E. A. Bergshoeff, J. Gomis, J. Rosseel, C. Şimşek and Z. Yan, _“String Theory and String Newton-Cartan Geometry,”_ J. Phys. A 53 (2020) no.1, 014001 doi:10.1088/1751-8121/ab56e9 [arXiv:1907.10668 [hep-th]].
  * [22] E. Bergshoeff, J. Gomis and Z. Yan, _“Nonrelativistic String Theory and T-Duality,”_ JHEP 11 (2018), 133 doi:10.1007/JHEP11(2018)133 [arXiv:1806.06071 [hep-th]].
  * [23] R. Andringa, E. Bergshoeff, J. Gomis and M. de Roo, _“’Stringy’ Newton-Cartan Gravity,”_ Class. Quant. Grav. 29 (2012), 235020 doi:10.1088/0264-9381/29/23/235020 [arXiv:1206.5176 [hep-th]].
  * [24] E. A. Bergshoeff, J. Lahnsteiner, L. Romano, J. Rosseel and C. Şimşek, _“A Non-Relativistic Limit of NS-NS Gravity,”_ [arXiv:2102.06974 [hep-th]].
  * [25] J. Kluson, _“D-Brane Actions in Non-Relativistic String Theory and T-Duality,”_ [arXiv:2011.14323 [hep-th]].
  * [26] J. Kluson, _“Stable and unstable D $p$-branes in $p$-brane Newton–Cartan background,”_ J. Phys. A 54 (2021) no.21, 215401 doi:10.1088/1751-8121/abf768 [arXiv:2003.14037 [hep-th]].
  * [27] J. Klusoň, _“Nonrelativistic String Theory Sigma Model and Its Canonical Formulation,”_ Eur. Phys. J. C 79 (2019) no.2, 108 doi:10.1140/epjc/s10052-019-6623-9 [arXiv:1809.10411 [hep-th]].
  * [28] T. Harmark, J. Hartong, L. Menculini, N. A. Obers and G. Oling, _“Relating non-relativistic string theories,”_ JHEP 11 (2019), 071 doi:10.1007/JHEP11(2019)071 [arXiv:1907.01663 [hep-th]].
  * [29] T. Harmark, J. Hartong, L. Menculini, N. A. Obers and Z. Yan, _“Strings with Non-Relativistic Conformal Symmetry and Limits of the AdS/CFT Correspondence,”_ JHEP 11 (2018), 190 doi:10.1007/JHEP11(2018)190 [arXiv:1810.05560 [hep-th]].
  * [30] T. Harmark, J. Hartong and N. A. Obers, _“Nonrelativistic strings and limits of the AdS/CFT correspondence,”_ Phys. Rev. D 96 (2017) no.8, 086019 doi:10.1103/PhysRevD.96.086019 [arXiv:1705.03535 [hep-th]].
  * [31] J. Klusoň, _“Unstable D-brane in Torsional Newton-Cartan Background,”_ JHEP 09 (2020), 191 doi:10.1007/JHEP09(2020)191 [arXiv:2001.11543 [hep-th]].
  * [32] J. Klusoň, _“T-duality of Non-Relativistic String in Torsional Newton-Cartan Background,”_ JHEP 05 (2020), 024 doi:10.1007/JHEP05(2020)024 [arXiv:1909.13508 [hep-th]].
  * [33] J. Klusoň, _“Remark About Non-Relativistic String in Newton-Cartan Background and Null Reduction,”_ JHEP 05 (2018), 041 doi:10.1007/JHEP05(2018)041 [arXiv:1803.07336 [hep-th]].


eqfloatequation

# High-dimensional Convolutional Networks for Geometric Pattern Recognition

Christopher Choy
NVIDIA  Junha Lee
POSTECH  René Ranftl
Intel Labs  Jaesik Park
POSTECH  Vladlen Koltun
Intel Labs

###### Abstract

Many problems in science and engineering can be formulated in terms of
geometric patterns in high-dimensional spaces. We present high-dimensional
convolutional networks (ConvNets) for pattern recognition problems that arise
in the context of geometric registration. We first study the effectiveness of
convolutional networks in detecting linear subspaces in high-dimensional
spaces with up to 32 dimensions: much higher dimensionality than prior
applications of ConvNets. We then apply high-dimensional ConvNets to 3D
registration under rigid motions and image correspondence estimation.
Experiments indicate that our high-dimensional ConvNets outperform prior
approaches that relied on deep networks based on global pooling operators.

##  1 Introduction

Finding structure in noisy data is a general problem that arises in many
different disciplines. For example, robust linear regression requires finding
a pattern (line, plane) in noisy data. 3D registration of point clouds
requires the identification of veridical correspondences in the presence of
spurious ones [6]. Structure from motion (SfM) pipelines use verification
based on prescribed geometric models to filter spurious image matches [40]. A
variety of such applications can benefit from improved methods for the
detection of geometric structures in noisy data.

Such detection is challenging. Data points that belong to the sought-after
structure often constitute only a small fraction, while the majority are
outliers. Various algorithms have been proposed over the years to cope with
noisy data [3, 11, 16, 21, 22, 34, 38, 41], but they are usually specific to a
subset of problems.

Recent works have advocated for using deep networks [35, 48, 51] to learn
robust models to classify geometric structures in the presence of outliers.
Deep networks offer significant flexibility and the promise to replace hand-
crafted algorithms and heuristics by models learned directly from data.
However, due to the unstructured nature of the data in geometric problems,
existing works have treated such data as unordered sets, and relied on network
architectures based predominantly on global pooling operators and multi-layer
perceptrons (MLPs) [33, 49]. Such network architectures lack the capacity to
model local geometric structures and do not leverage the nature of the data,
which is often embedded in a (high-dimensional) metric space and has a
meaningful geometric structure.

In this work, we introduce a novel type of deep convolutional network that can
operate in high dimensions. Our network takes a sparse tensor as input and
employs high-dimensional convolutions as the fundamental operator. The
distinguishing characteristic of our approach is that it is able to
effectively leverage local neighborhood relations together with global context
even for high-dimensional data. Our network is fully-convolutional,
translation-invariant, and incorporates best practices from the development of
ConvNets for two-dimensional image analysis [18, 23, 36].

To demonstrate the effectiveness and generality of our approach, we tackle
various geometric pattern recognition problems. We begin with the diagnostic
setting of linear subspace detection and show that our construction is
effective in high-dimensional spaces and at low signal-to-noise ratios. We
then apply the presented construction to geometric pattern recognition
problems that arise in computer vision, including registration of three-
dimensional point sets under rigid motion (a 6D problem) and correspondence
estimation between images under epipolar constraints (a 4D problem). In both
settings, the problem is made difficult by the presence of outliers.

Our experiments indicate that the presented construction can reliably detect
geometric patterns in high-dimensional data that is heavily contaminated by
noise. It can operate in regimes where existing algorithms break down. Our
approach significantly improves 3D registration performance when combined with
standard approaches for 3D point cloud alignment [6, 39, 53, 54]. The
presented high-dimensional convolutional network also outperforms state-of-
the-art methods for correspondence estimation between images [48, 51]. All
networks and training scripts are available at
https://github.com/chrischoy/HighDimConvNets.

##  2 Related Work

Robust model fitting. Fitting a geometric model to a set of observations that
are contaminated by outliers is a fundamental problem that frequently arises
in computer vision and related fields. The most widely used approach for
robust geometric model fitting is RANdom SAmple Consensus (RANSAC) [15]. Due
to its fundamental importance, many variants and improvements of RANSAC have
been proposed over the years [3, 11, 25, 34, 38, 41, 44, 43].

Alternatively, algorithms for robust geometric model fitting are frequently
derived using techniques from robust statistics [16, 21, 22, 52], where
outlier rejection is performed by equipping an estimator with a cost function
that is insensitive to gross outliers. While the resulting algorithms are
computationally efficient, they require careful initialization and
optimization procedures to avoid poor local optima [53].

Another line of work proposes to find globally optimal solutions to the
consensus maximization problem [5, 26, 47]. However, these approaches are
currently computationally too demanding for many practical applications.

3D registration. Finding reliable correspondences between a pair of surfaces
is an essential step for 3D reconstruction [6, 12, 14, 31]. The problem has
been conventionally framed as an energy minimization problem which can be
solved using various techniques such as branch and bound [46], Riemannian
optimization [37], mixed-integer programming [24], robust error minimization
[53], semi-definite programming [20, 28], or random sampling [6].

Recent work has begun to leverage deep networks for geometric registration [2,
30, 32]. These works are predominantly based on PointNet and related
architectures, which detect patterns via global pooling operations [33, 49].
In contrast, we develop a high-dimensional convolutional network that operates
at multiple scales and can leverage not just global but also local geometric
structure.

Image correspondences. Yi  _et al_. [48] and Zhang  _et al_. [51] reduce
essential matrix estimation to classification of correspondences into inliers
and outliers. Ranftl and Koltun [35] present a similar formulation for
fundamental matrix estimation. Brachmann and Rother [4] propose to learn a
neural network that guides hypothesis sampling in RANSAC for model fitting
problems. Dang  _et al_. [13] propose a numerically stable loss function for
essential matrix estimation.

All of these works employ variants of PointNets to classify inliers in an
unordered set of putative correspondences. These architectures, based on
pointwise MLPs, lack the capacity to model local geometric structure. In
contrast, we develop convolutional networks that directly leverage
neighborhood relations in the high-dimensional space of correspondences.

##  3 High-Dimensional Convolutional Networks

In this section, we introduce the two main building blocks of our high-
dimensional convolutional network construction: generalized sparse tensors and
generalized convolutions.

###  3.1 Sparse Tensor and Convolution

A tensor is a multi-dimensional array that represents high-order data. A
$D$-th order tensor $\mathcal{T}$ requires $D$ indices to uniquely access its
elements. We denote such indices or coordinates as
$\mathbf{x}=[x^{1},...,x^{D}]$ and the element at the coordinate as
$\mathscr{T}[\mathbf{x}]$ similar to how we access components in a matrix.
Likewise, a sparse tensor is a high-dimensional extension of a sparse matrix
where the majority of the elements are 0. Concretely,

$\mathscr{T}[\mathbf{x}_{i}]=\begin{cases}\mathbf{f}_{i}\;\;&\text{if}\;\mathbf{x}_{i}\in\mathcal{C}\\\
0\;\;&\text{otherwise},\end{cases}$ (1)

where
$\mathcal{C}=\\{\mathbf{x}_{i}\;|\;\mathbf{x}_{i}\in\mathbb{N}^{D},\mathscr{T}[\mathbf{x}_{i}]\neq\mathbf{0}\\}^{N}_{i=1}$
is the set of coordinates with non-zero values, $N$ is the number of non-zero
elements, and $\mathbf{f}_{i}$ is the non-zero value at the $i$-th coordinate.
A sparse tensor feature map is a $(D+1)$-th order tensor with
$\mathbf{f}_{i}\in\mathbb{R}^{N_{D+1}}$ as we use the last dimension to denote
the feature dimension. A sparse tensor has the constraint that
${\mathbf{x}_{i}\in\mathbb{N}^{D}}$. We extend the sparse tensor coordinates
to integer indices ${\mathbf{x}_{i}\in\mathbb{Z}^{D}}$ and define
$\mathscr{T}\in\mathbb{R}^{\aleph_{0}^{D}\times N_{D+1}}$ where $\aleph_{0}$
denotes the cardinality of the integer space $|\mathbb{Z}|$ to define a
generalized sparse tensor.

A convolution on this generalized sparse tensor can then be defined as a
simple extension of the generalized sparse convolution [7]:

$\displaystyle\mathbf{f}^{\text{out}}_{\mathbf{x}}$
$\displaystyle=\sum_{\mathbf{i}\in\mathcal{N}^{D}(\mathbf{x})\cap\mathcal{C}^{\text{in}}}\mathbf{W}_{\mathbf{i}}\mathbf{f}^{\text{in}}_{\mathbf{x}+\mathbf{i}}\quad\text{
for}\quad\mathbf{x}\in\mathcal{C}^{\text{out}},$ (2)

where $\mathcal{C}^{\text{in}}$ and $\mathcal{C}^{\text{out}}$ are the set of
input and output locations which is predefined by the user, $\mathbf{W_{i}}$
is a weight matrix, and $\mathcal{N}^{D}(\mathbf{x})$ defines a set of
neighbors of $\mathbf{x}$ which is defined by the shape of the convolution
kernel. For example, if the convolution kernel is a hypercube of size $K$,
$\mathcal{N}^{D}(\mathbf{x})\cap\mathcal{C}^{\text{in}}$ is a set of all the
non-zero elements of the input sparse tensor centered at $\mathbf{x}$ within
the $L_{\infty}$-ball of extent $K$.

###  3.2 Convolutional Networks

We design a high-dimensional fully-convolutional neural network for sparse
tensors (sparse tensor networks) based on generalized convolution [7, 9]. We
use U-shaped networks [36] to capture large receptive fields while maintaining
the original resolution at the final layer. The network has residual
connections [18] within layers with the same resolution and across the network
to speed up convergence and to recover the lost spatial resolution in the last
layer. The network architecture is illustrated in Fig. 1.

For computational efficiency in high dimensions, we use the cross-shaped
kernel [7] for all convolutions. We denote the kernel size by $K$. The cross-
shaped kernel has non-zero weights only for the $K-1$ nearest neighbors along
each axis, which results in one weight parameter for the center location and
$K-1$ weight parameters for each axis. Note that a cross-shaped kernel is
similar to separable convolution, where a full convolution is approximated by
$D$ one-dimensional convolutions of size $K$. Both types of kernels are rank-1
approximations of the full hyper-cubic kernel $K^{D}$, but separable
convolution requires $KD$ matrix multiplications, whereas the cross-shaped
kernel requires only $(K-1)D+1$ matrix multiplications.

###  3.3 Implementation

We extend the implementation of Choy  _et al_. [7], which supports arbitrary
kernel shapes, to high-dimensional convolutional networks. To implement the
sparse tensor networks, we need an efficient data structure that can generate
a new sparse tensor as well as find neighbors within the sparse tensor. Choy
_et al_. [7] use a hash table that is efficient for both insertion and search.
We replaced the hash table with a faster and more efficient variant [1]. In
addition, as the neighbor search can be run in parallel, we create an iterator
function that can run in parallel with OpenMP [29] by dividing the table into
smaller parallelization blocks.

Lastly, U-shaped networks generate hierarchical feature maps that expand the
receptive field. Choy  _et al_. [7] use stride-$K$ convolutions with kernel
size $\geq K$ to generate lower-resolution hierarchical feature maps. Still,
such implementation requires iterating over at least $K^{D}$ elements within a
hypercubic kernel as the coordinates are stored in a hash table, which results
in $O(NK^{D})$ complexity where $N$ is the cardinality of input. Consequently,
it becomes infeasible to store the weights on the GPU for high-dimensional
spaces. Instead of strided convolutions, we propose an efficient
implementation of stride-$K$ sum pooling layers with kernel size $K$. Instead
of iterating over all possible neighbors, we iterate over all input
coordinates and round them down to multiples of $K$, which requires only
$O(N)$ complexity.

##  4 Geometric Pattern Recognition

Our approach uses convolutional networks to recognize geometric patterns in
high-dimensional spaces. Specifically, we classify each point $\mathbf{x}_{i}$
in a high-dimensional dataset $\mathcal{X}=\\{\mathbf{x}_{i}\\}_{i=1}^{N}$ as
an inlier or an outlier. We start by validating our approach on synthetic
datasets of varying dimensionality and then show results on 3D registration
and essential matrix estimation.

Figure 1: A generic U-shaped high-dimensional convolutional network
architecture. The numbers next to each block indicate kernel size, stride, and
the number of channels. The strided convolutions that reduce the resolution of
activations are shifted upward to indicate different levels of resolution.
Table 1: Line detection in high-dimensional spaces in the presence of extreme
noise. All networks are trained with the cross-entropy loss for 40 epochs.
Inlier ratios are listed on the left. In the 32-dimensional setting, only 7
out of 10,000 data points are inliers. The table reports Mean Squared Error
(MSE), F1 score, and Average Precision (AP) of our approach (a high-
dimensional ConvNet) versus baselines (PointNet variants). MSE: lower is
better. F1 and AP: higher is better.

|  | Qi  _et al_. [33] | Zaheer  _et al_. [49] \+ BN + IN | Yi  _et al_. [48] | Ours
---|---|---|---|---|---
Dim. | Inlier Ratio | MSE | F1 | AP (AUC) | MSE | F1 | AP (AUC) | MSE | F1 | AP (AUC) | MSE | F1 | AP (AUC)
4 | 15.59% | 1.337 | 0.025 | 0.164 | 6.33E-4 | 0.867 | 0.936 | 9.11E-5 | 0.981 | 0.996 | 2.33E-5 | 0.998 | 0.999
8 | 5.54% | 2.369 | 0.022 | 0.065 | 0.001 | 0.891 | 0.955 | 2.45E-4 | 0.946 | 0.989 | 1.64E-5 | 0.999 | 0.999
16 | 2.75% | 3.854 | 0.012 | 0.034 | 4.86E-4 | 0.970 | 0.992 | 0.002 | 0.962 | 0.986 | 3.39E-5 | 0.999 | 0.999
24 | 0.40% | 5.372 | 0.021 | 0.011 | 0.676 | 0.634 | 0.691 | 0.775 | 0.610 | 0.674 | 5.34E-5 | 0.994 | 0.996
32 | 0.07% | 6.715 | 0.012 | 6.71E-5 | - | 0.0 | 0.295 | - | 0.0 | 0.050 | 0.010 | 0.669 | 0.689

Table 2: Plane detection in high-dimensional spaces in the presence of extreme
noise. Inlier ratios are listed on the left. In the 32-dimensional setting,
fewer than 5 out of 100,000 data points are inliers. The table reports the F1
score and Average Precision (AP) of our approach (a high-dimensional ConvNet)
versus baselines (PointNet variants). Higher is better.

|  | Qi  _et al_. [33] | Zaheer  _et al_. [49] \+ BN + IN | Yi  _et al_. [48] | Ours
---|---|---|---|---|---
Dim. | Inlier Ratio | F1 | AP (AUC) | F1 | AP (AUC) | F1 | AP (AUC) | F1 | AP (AUC)
4 | 29.96% | 0.0 | 0.315 | 0.980 | 0.996 | 0.993 | 0.999 | 0.991 | 0.998
8 | 8.07% | 0.0 | 0.088 | 0.985 | 0.999 | 0.990 | 0.999 | 0.998 | 0.999
16 | 0.34% | 0.0 | 0.004 | 0.155 | 0.299 | 0.182 | 0.359 | 0.951 | 0.961
24 | 0.01% | 0.0 | 1.61E-4 | 0.032 | 0.133 | 0.0 | 0.081 | 0.304 | 0.346
32 | 4.64E-3% | 0.0 | 5.56E-5 | 0.0 | 0.221 | 0.0 | 0.023 | 0.138 | 0.240

|  |  |
---|---|---|---
Qi  _et al_. [33] |  Zaheer  _et al_. [49] |  Yi  _et al_. [48] | Ours
Figure 2: 16D line detection projected to a 2D plane for visualization. Black dots are noise and blue dots are samples from a line in the 16D space. The dashed red line is the prediction of the respective method. Samples from the ground-truth line (blue) are enlarged by a factor of 10 for visualization. |  |
---|---|---
Figure 3: 16D line detection, progression of training. We plot the running
mean and standard deviation of precision, recall, and F1 score on the
validation set. Our high-dimensional convolutional network quickly attains
much higher accuracy than the baselines.

For all experiments, we first quantize the input coordinates to create a
sparse tensor of order $D+1$, where the last dimension denotes the feature
channels. The network then predicts a logit score for each non-zero element in
the sparse tensor to indicate if a point is part of the geometric pattern or
if it is an outlier.

###  4.1 Line and Plane Detection

We first test the capabilities of our fully-convolutional networks on simple
high-dimensional pattern recognition problems that involve detecting linear
subspaces amidst noise. Our dataset consists of uniformly sampled noise from
the $D$-dimensional space and a small number of samples from a line under a
Gaussian noise model. The number of outliers increases exponentially in the
dimension ($O(L^{D})$), while the number of inliers increases sublinearly
($O(\sqrt{LD})$), where $L$ is the extent of the domain. Further details are
given in the supplement. The network predicts a likelihood score for each non-
zero element in the input sparse tensor, and we threshold inliers with
probability $\geq 0.5$. We estimate the line equation from the predicted
inliers using unweighted least squares.

We use PointNet variants as the baselines for this experiment [33, 48, 49].
For Zaheer  _et al_. [49], we were not able to get reasonable results with the
network architecture proposed in the paper. We thus augmented the architecture
with batch normalization and instance normalization layers after each linear
transformation similar to Yi  _et al_. [48], which boosted performance
significantly. For all experiments, we use the cross-entropy loss. We used the
same training hyperparameters, including loss, batch size, optimizer, and
learning rate schedule for all approaches.

We use three metrics to analyze the performance of the networks: Mean Squared
Error (MSE), F1 score, and Average Precision (AP). For the MSE, we estimate
the line equation with Least Squares to fit the line to the inliers. The
second metric is the F1 score, the harmonic mean of precision and recall. In
many problems, F1 score is a direct indicator of the performance of a
classifier, and we also found a strong correlation between F1 score and the
mean squared error. The final metric we use is average precision (AP), which
measures the area under the precision-recall curve. We report the results in
Tab. 1 and provide qualitative examples in Fig. 2. Tab. 1 lists the inlier
ratio to indicate the difficulty of each task.

In a second experiment, we create another synthetic dataset where the inlier
pattern is sampled from a plane spanned by two vectors:
$\\{c_{1}\mathbf{v}_{1}+c_{2}\mathbf{v}_{2}+\mathbf{c}\;|\;c_{1},c_{2}\in\mathbb{R}\\}$.
The two basis vectors are sampled uniformly from the $D$-dimensional unit
hypercube. We use the same training procedure for our network and the
baselines, and report the results in Tab. 2.

We found that the convolutional network is more robust to noise in high-
dimensional spaces than the PointNet variants. In addition, the convolutional
network training converges quickly, as shown in Fig. 3, which is a further
indication that the architecture can effectively leverage the structure of the
data.

###  4.2 3D Registration

A typical 3D registration pipeline consists of 1) feature extraction, 2)
feature matching, 3) match filtering, and 4) global registration. In this
section, we show that in the match filtering stage the correct (inlier)
correspondences form a 6-dimensional geometric structure. We then extend our
geometric pattern recognition networks to identify inlier correspondences in
this 6-dimensional space.

Let $\mathcal{X}$ be a set of points sampled from a 3D surface,
$\mathcal{X}=\\{\mathbf{x}_{i}\;|\;\mathbf{x}_{i}\in\mathbb{R}^{3}\\}_{i=1}^{N}$,
and let $\mathcal{X}^{\prime}$ be a subset of $\mathcal{X}$ that went through
a rigid transformation $T$,
$\mathcal{X}^{\prime}=\\{T(\mathbf{x})\;|\;\mathbf{x}\in\mathcal{S},\mathcal{S}\subseteq\mathcal{X}\\}$.
For example, $\mathcal{X}^{\prime}$ could be a 3D scan from a different
perspective that has an overlap with $\mathcal{X}$. We denote a correspondence
between points $\mathbf{x}_{i}\in\mathcal{X}$ and
$\mathbf{x}^{\prime}_{j}\in\mathcal{X}^{\prime}$ as
$\mathbf{x}_{i}\leftrightarrow\mathbf{x}^{\prime}_{j}$. When we form an
ordered pair $(\mathbf{x}_{i},\mathbf{x}^{\prime}_{j})\in\mathbb{R}^{6}$, the
ground truth correspondences satisfy $T(\mathbf{x})=\mathbf{x}^{\prime}$ along
the common 3D geometry $\mathcal{S}$ whereas an incorrect correspondence
implies $T(\mathbf{x})\neq\mathbf{x}^{\prime}$. For example, in Fig. 4, we
visualize the ordered pairs from 1-dimensional sets. Note that the inliers
follow the geometry of the inputs and form a line segment. Similarly, the
geometry $(\mathbf{x},\mathbf{x}^{\prime})\in\mathbb{R}^{6}$ or
$(\mathbf{x},T(\mathbf{x}))$ for $\mathbf{x}\in\mathcal{S}$ forms a surface in
6-dimensional space.

###### Theorem 1

The 6D representation of ground-truth 3D correspondences
$(\mathbf{x},\mathbf{x}^{\prime})$ lies on the intersection of three
hyperplanes where each hyperplane is a row of the following block equation
$\begin{bmatrix}R&-I\end{bmatrix}\begin{bmatrix}\mathbf{x}\\\
\mathbf{x}^{\prime}\end{bmatrix}+\mathbf{t}=\mathbf{0}$.

Proof: A ground-truth 3D correspondence,
$\mathbf{x}\leftrightarrow\mathbf{x}^{\prime}$, must satisfy
$R\mathbf{x}+\mathbf{t}=\mathbf{x}^{\prime}$. We move $\mathbf{x}^{\prime}$ to
the LHS and convert both $\mathbf{x}$ and $\mathbf{x}^{\prime}$ to the 6D
representation to get three hyperplane equations. $\square$

###### Corollary 1.1

The rank of the block matrix, $\begin{bmatrix}R&-I\end{bmatrix}$, is 3 since
$R$ and $I$ are orthogonal matrices. Thus, all hyperplanes defined by the
block matrix intersect each other and the 6D inlier correspondences form a 3D
plane since the solution of the intersection of three hyperplanes in the 6D
space, $\begin{bmatrix}R&-I\end{bmatrix}\begin{bmatrix}\mathbf{x}\\\
\mathbf{x}^{\prime}\end{bmatrix}+\mathbf{t}=\mathbf{0}$, is a 3D plane.

Figure 4: The set $\mathcal{Y}$ is rigid translation of the set
$\mathcal{X}=[1,...,7]$, $\mathcal{Y}=\\{x+4|x\in\mathcal{X}\\}$. The ordered
pairs of correspondences (blue) form a line segment while outliers (red) form
random noise outside the line.

We can thus use our high-dimensional convolutional network construction to
segment the 6-dimensional set of correspondences into inliers and outliers by
estimating the inlier likelihood for each correspondence.

Network. We use a 6-dimensional instantiation of the U-shaped convolutional
network presented in Sec. 3. As the dimensionality is manageable, we use
hypercubic kernels. The network takes an order-6 sparse tensor whose
coordinates are correspondences
$(\mathbf{x}_{i},\mathbf{x}^{\prime}_{j})\in\mathbb{R}^{6}$. We discretize the
coordinates with the voxel size used to extract features. Our baseline is Yi
_et al_. [48], which takes dimensionless mean-centered correspondences without
discretization. We train the networks to predict the inlier probability of
each correspondence with the balanced cross-entropy loss.

Dataset. We use the 3DMatch dataset for this experiment [50]. The 3DMatch
dataset is a composition of various 3D scan datasets [17, 45, 50] and thus
covers a wide range of scenes and different types of 3D cameras. We integrate
RGB-D images to form fragments of the scenes following [50]. During training,
we randomly rotate each scene on the fly to augment the dataset. We use a
popular hand-designed feature descriptor, FPFH [39], to compute
correspondences. Note, however, that our pipeline is agnostic to the choice of
feature and can also be used with learned features [9].

Table 3: Pairwise registration on 3DMatch test scenes with 2.5cm downsampling.
Translation Error (TE), Rotation Error (RE), success rate. Registration is
considered successful if TE $<30$cm and RE $<15{}^{\circ}$.

|  | FPFH + FGR | FPFH + Ours + FGR | FPFH + RANSAC | FPFH + Ours + RANSAC
---|---|---|---|---|---
| Inlier Ratio | TE | RE | Succ. Rate | TE | RE | Succ. Rate | TE | RE | Succ. Rate | TE | RE | Succ. Rate
Kitchen | 1.62% | 10.98 | 4.99 | 37.15 | 5.68 | 2.21 | 65.61 | 6.25 | 2.17 | 44.47 | 5.90 | 1.98 | 69.57
Home 1 | 2.71% | 11.12 | 4.40 | 45.51 | 6.52 | 2.08 | 80.77 | 7.07 | 2.19 | 61.54 | 6.00 | 1.87 | 80.13
Home 2 | 2.83% | 9.61 | 3.83 | 36.54 | 7.13 | 2.56 | 64.42 | 6.47 | 2.40 | 50.00 | 7.86 | 2.56 | 69.71
Hotel 1 | 1.35% | 12.31 | 5.09 | 33.19 | 7.95 | 2.65 | 76.11 | 7.48 | 2.75 | 48.67 | 7.38 | 2.38 | 80.09
Hotel 2 | 1.54% | 12.27 | 5.22 | 25.00 | 7.86 | 2.56 | 69.23 | 9.54 | 3.18 | 47.12 | 6.40 | 2.25 | 70.19
Hotel 3 | 1.59% | 13.52 | 7.04 | 27.78 | 5.39 | 1.99 | 72.22 | 5.91 | 2.46 | 59.26 | 5.85 | 2.36 | 81.48
Study | 0.87% | 16.10 | 6.01 | 16.78 | 9.61 | 2.64 | 53.42 | 10.05 | 3.01 | 30.48 | 8.51 | 2.23 | 56.16
Lab | 1.59% | 10.48 | 4.80 | 42.86 | 7.69 | 2.44 | 61.04 | 8.01 | 2.31 | 45.45 | 6.64 | 2.12 | 68.83
Average |  | 12.05 | 5.17 | 33.10 | 7.23 | 2.39 | 67.85 | 7.60 | 2.56 | 48.37 | 6.82 | 2.22 | 72.02

---
Figure 5: The success rate of baseline methods and ours on the 3DMatch
benchmark [50] with 5cm voxel size. FGR denotes registration with FPFH [39]
and Zhou  _et al_. [53], Yi  _et al_. \+ X denotes FPFH filtering with Yi  _et
al_. [48] and registration with X, and Ours + X denotes our method for
filtering followed by registration with X.

|  |  |
---|---|---|---
Kitchen | Bedroom
|  |  |
Livingroom | Study

Figure 6: Visualization of color-coded correspondences before and after
outlier filtering (Sec. 4.2). For each pair, we visualize 100 random
correspondences from the candidate set on the left and 100 random
correspondences after outlier pruning on the right. Red lines are outlier
correspondences and blue lines are inlier correpondences. On the bottom right
pair, there are two identical chairs. The average inlier ratio is 1.76%.

We follow the standard procedures in the 3D registration literature to
generate candidate correspondences. First, since 3D scans often exhibit
irregular densities, we resample the input point clouds using a voxel grid to
produce a regular point cloud. We use voxel sizes of 2.5cm and 5cm for our
experiments. Next, we compute FPFH features and find the nearest neighbor for
each point in feature space to form correspondences. The correspondences
obtained from this procedure often exhibit a very low inlier ratio, as little
as 0.87% with a 2.5cm voxel size. Among these correspondences, we regard
$\mathbf{x}\leftrightarrow\mathbf{x}^{\prime}$ as an inlier if it satisfies
$\|\mathbf{T}(\mathbf{x})-\mathbf{x}^{\prime}\|_{2}<\tau$ and all others as
outliers. We set $\tau$ to be two times the voxel size.

Finally, we use a registration method to convert the filtered correspondences
into a final registration result. We show results with two different
registration methods. The first is Fast Global Registration [53], which
directly minimizes a robust error metric. The second is a variant of RANSAC
[15] that is specialized to 3D registration [54].

Evaluation. We use three standard metrics to evaluate registration
performance: rotation error, translation error, and success rate. The rotation
error measures the absolute angular deviation from the ground truth rotation
$\hat{\textbf{R}}$,
$\arccos\frac{\text{Tr}(\hat{\textbf{R}}^{T}\textbf{R})-1}{2}$. Similarly, the
translation error measures the deviation of the translation
$\|\hat{\mathbf{t}}-\mathbf{t}\|_{2}$. When we report these metrics, we
exclude alignments that exceed a threshold following [9] since the results of
the registration methods [53, 15] can be arbitrarily bad when registration
fails. Finally, the success rate is the ratio of registrations that were
successful; registration is considered successful if both rotation and
translation errors are within the respective thresholds. For all experiments,
we use a rotation error of 15 degrees and a translation error of 30cm as the
thresholds.

Tab. 3 shows the 3D registration pipelines with and without our network to
filter the outliers. Note that for FGR [53], we observe a considerable
improvement with our network since FGR assumes more accurate correspondences
as inputs. The improvement is smaller with RANSAC, since it is more robust to
high outlier rates. Although the inlier ratio is as low as 1% for many 3D
scene pairs, our network generates very accurate predictions. Similar to the
linear regression experiments in Fig. 3, we find that the network converges
very quickly. We compare to the model of Yi  _et al_. [48] in Fig. 5 with 5cm
voxel size to study the robustness of the 6-dimensional convolutional network
to voxel size and find that the convolutional network improves the
registration success rate significantly even for the higher inlier ratio seen
with a 5cm discretization resolution. Qualitatively, our network accurately
filters out outliers even in the presence of extreme noise (Fig. 6).

###  4.3 Filtering Image Correspondences

In this section, we apply the high-dimensional convolutional network to image
correspondence inlier detection. In the projective space $\mathbb{P}^{2}$, an
inlier correspondence ${\mathbf{u}\leftrightarrow\mathbf{u}^{\prime}}$ must
satisfy $\mathbf{u}^{\prime\top}\mathbf{E}\mathbf{u}=0$, where $\mathbf{E}$ is
an essential matrix, $\mathbf{u}$ denotes a normalized homogeneous coordinate
$\mathbf{u}=\mathbf{K}^{-1}\mathbf{x}$, $\mathbf{x}$ is the corresponding
homogeneous image coordinate, and $\mathbf{K}$ is the camera intrinsic matrix.
When we expand $\mathbf{u}^{\prime\top}\mathbf{E}\mathbf{u}=0$, we get
$u^{\prime 1}Au^{1}+u^{\prime 2}Bu^{1}+u^{\prime 1}Cu^{2}+u^{\prime
2}Du^{2}+Eu^{\prime 1}+Fu^{\prime 2}+Gu^{1}+Hu^{2}+I=0$, which is a
quadrivariate quadratic function. If there is a real-valued solution, there
are infinitely many solutions that form either an ellipse (sphere), a
parabola, or a hyperbola. These are known as conic sections. Thus a set of
ground-truth image correspondences will form a hyper conic section in
4-dimensional space. We use a convolutional network to predict the likelihood
that a correspondence is an inlier.

Dataset: YFCC100M. We use the large-scale photo-tourism dataset YFCC100M [42]
for the experiment. The dataset contains 100M Flicker images of tourist hot-
spots with metadata, which is curated into 72 locations with camera extrinsics
estimated using SfM [19]. We follow Zhang  _et al_. [51] to generate a dataset
and use 68 locations for training and the others for testing. We filtered any
image pairs that have fewer than 100 overlapping 3D points from SfM to
guarantee non-zero overlap between images.

We use SIFT features [27] to create correspondences and label a correspondence
to be a ground-truth inlier if the symmetric epipolar distance of the
correspondence is below a certain threshold using the provided camera
parameters, _i.e_.,

$\Big{(}\frac{r^{2}}{l_{1}^{2}+l_{2}^{2}}+\frac{r^{2}}{l_{1}^{\prime
2}+l_{2}^{\prime 2}}\Big{)}<\tau,$ (3)

where $l=\mathbf{u}^{\prime\top}\mathbf{E}=(l_{1},l_{2},l_{3})$ is a
homogeneous line, $l^{\prime}=\mathbf{E}\mathbf{u}$, and
$r=\mathbf{u}^{\prime\top}\mathbf{E}\mathbf{u}$.

Network. We convert a set of candidate image correspondences into an order-5
sparse tensor with four spatial dimensions and vectorized features. The
coordinates are defined as normalized image coordinates $\mathbf{u}$. We
define integer coordinates by discretizing the normalized image coordinates
with quantization resolution 0.01 and additionally use the normalized
coordinates as features. We use state-of-the-art baselines on the YFCC dataset
and two variants of convolutional networks for this task. The first variant is
a U-shaped convolutional network (Ours); the second network is a ResNet-like
network with spatial correlation modules (Ours + SC). Spatial correlation
modules [51] are blocks of shared MLPs that encode global context from a set
of correspondences.

Note that unlike the FPFH descriptor [39], which extracts features densely,
SIFT features are extracted from only a few keypoints sparsely in the image.
In the 4-dimensional space of correspondences, the sparsity gets even worse as
the volume increases multiplicatively while the number of points
(correspondences) stay the same. Such sparsity leads to fewer neighbors in the
high-dimensional space, which results in the degeneration of the convolution
to a multi-layer perceptron. Our convolutional networks remain effective in
this setting, but their distinctive ability to leverage local geometric
structure is not strongly utilized. To increase the density of the
correspondences in the 4-dimensional space, we use a dense fully convolutional
feature UCN [10]. We train the UCN on the YFCC100M training set for 100 epochs
and we follow the preprocessing stage outlined in the open-source version [8]
to create correspondences. Training and testing on UCN follow the same
standard procedure. We use the balanced cross-entropy loss [48] for all
experiments.

Table 4: Classification scores on the YFCC100M test set. We consider a
correspondence a true positive if the symmetric epipolar distance is below
$10^{-4}$ and the prediction confidence is above 0.5.

| Yi  _et al_. [48] | Zhang  _et al_. [51] | Ours | Ours + SC | Ours + UCN
---|---|---|---|---|---
| Prec. | Recall | F1 | Prec. | Recall | F1 | Prec. | Recall | F1 | Prec. | Recall | F1 | Prec. | Recall | F1
Buckingham | 0.497 | 0.772 | 0.605 | 0.486 | 0.889 | 0.629 | 0.535 | 0.822 | 0.648 | 0.611 | 0.835 | 0.705 | 0.769 | 0.892 | 0.826
Notre dame | 0.581 | 0.894 | 0.705 | 0.629 | 0.951 | 0.757 | 0.647 | 0.915 | 0.758 | 0.721 | 0.929 | 0.812 | 0.844 | 0.954 | 0.896
Reichtag | 0.747 | 0.877 | 0.807 | 0.734 | 0.917 | 0.815 | 0.695 | 0.911 | 0.789 | 0.769 | 0.897 | 0.827 | 0.856 | 0.937 | 0.895
Sacre coeur | 0.658 | 0.871 | 0.750 | 0.662 | 0.948 | 0.780 | 0.632 | 0.917 | 0.748 | 0.718 | 0.932 | 0.811 | 0.855 | 0.948 | 0.899
Average | 0.621 | 0.854 | 0.717 | 0.628 | 0.926 | 0.745 | 0.628 | 0.891 | 0.736 | 0.704 | 0.898 | 0.789 | 0.830 | 0.933 | 0.879

Buckingham | Notre Dame | Sacre Coeur
---|---|---
|  |  |  |  |
|  |  |  |  |
|  |  |  |  |
|  |  |  |  |

Figure 7: Matching results using Yi  _et al_. [48] (first row), Zhang  _et
al_. [51] (second row), Ours + SC (third row) and Ours + UCN (last row). We
visualize correspondences with inlier probability above 0.5. We color a
correspondence green if it is a true positive (symmetric epipolar distance
smaller than $10^{-4}$) and red otherwise.

Evaluation. We use precision, recall, and F1 score to evaluate correspondence
classification accuracy. We use $\tau=10^{-4}$ for the distance threshold to
define ground truth-correspondences. Our network predicts a correspondence to
be accurate if its inlier probability prediction is above 0.5. We provide
quantitative results in Tab. 4 and qualitative results in Fig. 7. Our
approachs outperform the PointNet variants [48, 51] as measured by the F1
score.

##  5 Conclusion

Many interesting problems in computer vision involve geometric patterns in
high-dimensional spaces. We proposed high-dimensional convolutional networks
for geometric pattern recognition. We presented a fully-convolutional network
architecture that is efficient and able to find patterns in high-dimensional
data even in the presence of severe noise. We validated the efficacy of our
approach on tasks such as line and plane detection, 3D registration, and
geometric filtering of image correspondences.

## Acknowledgements

This work was partially supported by a National Research Foundation of Korea
(NRF) grant and funded by the Korean government (MSIT) (No. 2020R1C1C1015260).

## References

  * [1] Martin Ankerl.  Robin hood hashing.  https://github.com/martinus/robin-hood-hashing, 2019.
  * [2] Yasuhiro Aoki, Hunter Goforth, Rangaprasad Arun Srivatsan, and Simon Lucey.  PointNetLK: Robust & efficient point cloud registration using pointnet.  In CVPR, 2019.
  * [3] Eric Brachmann, Alexander Krull, Sebastian Nowozin, Jamie Shotton, Frank Michel, Stefan Gumhold, and Carsten Rother.  DSAC - differentiable RANSAC for camera localization.  In CVPR, 2017.
  * [4] Eric Brachmann and Carsten Rother.  Neural-guided RANSAC: Learning where to sample model hypotheses.  In ICCV, 2019.
  * [5] Tat-Jun Chin, Pulak Purkait, Anders P. Eriksson, and David Suter.  Efficient globally optimal consensus maximisation with tree search.  IEEE Transactions on Pattern Anaysis and Machine Intelligence, 2017.
  * [6] Sungjoon Choi, Qian-Yi Zhou, and Vladlen Koltun.  Robust reconstruction of indoor scenes.  In CVPR, 2015.
  * [7] Christopher Choy, JunYoung Gwak, and Silvio Savarese.  4D spatio-temporal ConvNets: Minkowski convolutional neural networks.  In CVPR, 2019.
  * [8] Christopher Choy and Junha Lee.  Open universal correspondence network.  https://github.com/chrischoy/open-ucn, 2019.
  * [9] Christopher Choy, Jaesik Park, and Vladlen Koltun.  Fully convolutional geometric features.  In ICCV, 2019.
  * [10] Christopher B Choy, JunYoung Gwak, Silvio Savarese, and Manmohan Chandraker.  Universal correspondence network.  In Advances in Neural Information Processing Systems, 2016.
  * [11] Ondrej Chum and Jiri Matas.  Matching with prosac-progressive sample consensus.  In CVPR, 2005.
  * [12] Angela Dai, Matthias Nießner, Michael Zollöfer, Shahram Izadi, and Christian Theobalt.  BundleFusion: Real-time globally consistent 3d reconstruction using on-the-fly surface re-integration.  ACM Transactions on Graphics, 2017.
  * [13] Zheng Dang, Kwang Moo Yi, Yinlin Hu, Fei Wang, Pascal Fua, and Mathieu Salzmann.  Eigendecomposition-free training of deep networks with zero eigenvalue-based losses.  In ECCV, 2018.
  * [14] Wei Dong, Jaesik Park, Yi Yang, and Michael Kaess.  GPU accelerated robust scene reconstruction.  In IROS, 2019.
  * [15] Martin A. Fischler and Robert C. Bolles.  Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography.  Communications of the ACM, 1981.
  * [16] Andrew W. Fitzgibbon.  Robust registration of 2D and 3D point sets.  Image and Vision Computing, 2003.
  * [17] Ben Glocker, Shahram Izadi, Jamie Shotton, and Antonio Criminisi.  Real-time RGB-D camera relocalization.  In ISMAR, 2013.
  * [18] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In CVPR, 2016.
  * [19] Jared Heinly, Johannes L Schonberger, Enrique Dunn, and Jan-Michael Frahm.  Reconstructing the world in six days.  In CVPR, 2015.
  * [20] M. B. Horowitz, N. Matni, and J. W. Burdick.  Convex relaxations of SE(2) and SE(3) for visual pose estimation.  In ICRA, 2014.
  * [21] Reza Hoseinnezhad and Alireza Bab-Hadiashar.  An m-estimator for high breakdown robust estimation in computer vision.  Computer Vision and Image Understanding, 2011.
  * [22] Peter J Huber.  Robust statistics.  Springer, 2011.
  * [23] Sergey Ioffe and Christian Szegedy.  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  In ICML, 2015.
  * [24] Gregory Izatt and Russ Tedrake.  Globally optimal object pose estimation in point clouds with mixed-integer programming.  In International Symposium on Robotics Research, 2017.
  * [25] Karel Lebeda, Jirı Matas, and Ondrej Chum.  Fixing the locally optimized RANSAC–full experimental evaluation.  In British Machine Vision Conference, 2012.
  * [26] Hongdong Li.  Consensus set maximization with guaranteed global optimality for robust geometry estimation.  In ICCV, 2009.
  * [27] David G. Lowe.  Distinctive image features from scale-invariant keypoints.  International Journal of Computer Vision, 2004.
  * [28] Haggai Maron, Nadav Dym, Itay Kezurer, Shahar Kovalsky, and Yaron Lipman.  Point registration via efficient convex relaxation.  ACM Transactions on Graphics, 2016.
  * [29] OpenMP Architecture Review Board.  OpenMP application program interface version 3.0, May 2008.
  * [30] G. Dias Pais, Pedro Miraldo, Srikumar Ramalingam, Venu Madhav Govindu, Jacinto C. Nascimento, and Rama Chellappa.  3DRegNet: A deep neural network for 3D point registration.  arXiv, 2019.
  * [31] Jaesik Park, Qian-Yi Zhou, and Vladlen Koltun.  Colored point cloud registration revisited.  In ICCV, 2017.
  * [32] Thomas Probst, Danda Pani Paudel, Ajad Chhatkuli, and Luc Van Gool.  Unsupervised learning of consensus maximization for 3D vision problems.  In CVPR, 2019.
  * [33] Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas.  Pointnet: Deep learning on point sets for 3D classification and segmentation.  In CVPR, 2017.
  * [34] Rahul Raguram, Ondrej Chum, Marc Pollefeys, Jiri Matas, and Jan-Michael Frahm.  USAC: A universal framework for random sample consensus.  IEEE Transactions on Pattern Anaysis and Machine Intelligence, 2013.
  * [35] René Ranftl and Vladlen Koltun.  Deep fundamental matrix estimation.  In ECCV, 2018.
  * [36] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In MICCAI, 2015.
  * [37] David M Rosen, Luca Carlone, Afonso S Bandeira, and John J Leonard.  Se-sync: A certifiably correct algorithm for synchronization over the special euclidean group.  International Journal of Robotics Research, 2019.
  * [38] Peter J. Rousseeuw.  Least median of squares regression.  Journal of the American Statistical Association, 1984.
  * [39] Radu Bogdan Rusu, Nico Blodow, and Michael Beetz.  Fast point feature histograms (FPFH) for 3d registration.  In ICRA, 2009.
  * [40] Schönberger, Johannes Lutz, and Jan-Michael Frahm.  Structure-from-motion revisited.  In CVPR, 2016.
  * [41] Ruwan B. Tennakoon, Alireza Bab-Hadiashar, Zhenwei Cao, Reza Hoseinnezhad, and David Suter.  Robust model fitting using higher than minimal subset sampling.  IEEE Transactions on Pattern Anaysis and Machine Intelligence, 2016.
  * [42] Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li.  YFCC100M: The new data in multimedia research.  Communications of the ACM, 2016.
  * [43] Philip HS Torr and Andrew Zisserman.  MLESAC: A new robust estimator with application to estimating image geometry.  Computer Vision and Image Understanding, 2000.
  * [44] Philip H. S. Torr.  Bayesian model estimation and selection for epipolar geometry and generic manifold fitting.  International Journal of Computer Vision, 2002.
  * [45] Jianxiong Xiao, Andrew Owens, and Antonio Torralba.  Sun3D: A database of big spaces reconstructed using sfm and object labels.  In ICCV, 2013.
  * [46] Jiaolong Yang, Hongdong Li, and Yunde Jia.  Go-ICP: Solving 3D registration efficiently and globally optimally.  In ICCV, 2013.
  * [47] Jiaolong Yang, Hongdong Li, and Yunde Jia.  Optimal essential matrix estimation via inlier-set maximization.  In ECCV, 2014.
  * [48] Kwang Moo Yi, Eduard Trulls, Yuki Ono, Vincent Lepetit, Mathieu Salzmann, and Pascal Fua.  Learning to find good correspondences.  In CVPR, 2018.
  * [49] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan R Salakhutdinov, and Alexander J Smola.  Deep sets.  In Advances in Neural Information Processing Systems. 2017.
  * [50] Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas Funkhouser.  3DMatch: Learning local geometric descriptors from RGB-D reconstructions.  In CVPR, 2017.
  * [51] Jiahui Zhang, Dawei Sun, Zixin Luo, Anbang Yao, Lei Zhou, Tianwei Shen, Yurong Chen, Long Quan, and Hongen Liao.  Learning two-view correspondences and geometry using order-aware network.  In ICCV, 2019.
  * [52] Zhengyou Zhang.  Determining the epipolar geometry and its uncertainty: A review.  International Journal of Computer Vision, 1998.
  * [53] Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun.  Fast global registration.  In ECCV, 2016.
  * [54] Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun.  Open3D: A modern library for 3D data processing.  arXiv, 2018.