# Characteristic Polynomials of Orthogonal and Symplectic Random Matrices,
Jacobi Ensembles & L-functions

MUSTAFA ALPER GUNES

###### Abstract

Starting from Montgomery’s conjecture, there has been a substantial interest
on the connections of random matrix theory and the theory of L-functions. In
particular, moments of characteristic polynomials of random matrices have been
considered in various works to estimate the asymptotics of moments of
L-function families. In this paper, we first consider joint moments of the
characteristic polynomial of a symplectic random matrix and its second
derivative. We obtain the asymptotics, along with a representation of the
leading order coefficient in terms of the solution of a Painlevé equation.
This gives us the conjectural asymptotics of the corresponding joint moments
over families of Dirichlet L-functions. In doing so, we compute the
asymptotics of a certain additive Jacobi statistic, which could be of
independent interest in random matrix theory. Finally, we consider a slightly
different type of joint moment that is the analogue of an average considered
over $U(N)$ in various works before. We obtain the asymptotics and the leading
order coefficient explicitly.

###### Contents

  1. 1 Motivation
    1. 1.1 Random Matrix Theory & L-functions
    2. 1.2 Main Results
  2. 2 Proof of Theorem
  3. 3 Proof of Theorem

##  1 Motivation

Let $U$ be a matrix in $SO(2N)$ or $Sp(2N)$ sampled according to the Haar
measure with eigenvalues $e^{\pm i\theta_{1}},\ldots,e^{\pm i\theta_{N}}$ and
define the functions,

$\psi_{U}(\theta)=\prod_{j=1}^{N}\left(1-e^{-i\left(\theta-\theta_{j}\right)}\right)\left(1-e^{-i\left(\theta+\theta_{j}\right)}\right),$

and

$\mathcal{Z}_{U}(\theta)=e^{\frac{iN}{2}(\theta+\pi)-i\sum_{k=1}^{N}\frac{\theta_{k}}{2}}\prod_{j=1}^{N}\left(1-e^{-i\left(\theta-\theta_{j}\right)}\right).$

Note that $\psi_{U}$ is the characteristic polynomial of $U$, whereas
$\mathcal{Z}_{U}$ is the so-called $\mathcal{Z}$-function corresponding to the
eigenvalues on the upper part of the unit circle. In this paper, we will
compute the $N\to\infty$ asymptotics of the corresponding quantities:

$\mathfrak{R}_{N}^{G}(s,h)=\mathbb{E}_{N}\left[\left|\mathcal{Z}_{U}(0)\right|^{2s}\left|\frac{d}{d\theta}\log\mathcal{Z}_{U}(\theta)\bigg{\rvert}_{\theta=0}\right|^{h}\right],$
(1)

and

$\mathfrak{L}_{N}^{G}(s,h)=\mathbb{E}_{N}\left[\left|\psi_{U}(0)\right|^{2s-h}\left|\psi^{\prime\prime}_{U}(0)\right|^{h}\right],$
(2)

where $G$ is one of the matrix groups $SO(2N)$ or $Sp(2N)$, and the averages
are taken with respect to the Haar measures on each group.
While $\mathfrak{R}_{N}^{G}(s,h)$ can be seen as an analogue (notwithstanding
certain differences) of a quantity that has been considered as an average over
$U(N)$ in various works before due to its connections with the Riemann
$\zeta$-function, (see [20], [5], [2], [3], [4], [11]), results obtained in
this paper regarding the asymptotics of $\mathfrak{L}_{N}^{G}(s,h)$ give rise
to conjectures about moments of families of L-functions.

###  1.1 Random Matrix Theory & L-functions

Starting from a conjecture due to Montgomery [16], there has been a
substantial amount of work done on the connections between random matrix
theory and analytic number theory. Montgomery’s conjecture, which was based on
both rigorous calculations and certain conjectures on twin primes, stated that
the pair correlation function of the zeros of the Riemann Zeta function, high
up the critical line, could be modeled via the pair correlation function of
eigenangles of a random unitary matrix of large size. Later, it was proved by
Keating and Snaith [15], that the value distribution of $\zeta(s)$ coincided
with the distribution of the characteristic polynomial of a random unitary
matrix in the same limit. In [13], as an attempt to generalize this connection
to other classes of L-functions, Katz and Sarnak considered various statistics
related to Haar-distributed random matrices from classical compact groups,
$U(N)$, $Sp(2N)$ and $SO(2N)$. They showed that in the $N\to\infty$ limit,
certain statistics related to consecutive spacings of eigenangles all converge
to the same limit, that of the Circular Unitary Ensemble (CUE). On the other
hand, they realized that this universality does not hold if one considers the
distribution of the $k$-th eigenvalue over these compact groups. Namely, let
$G$ be one of the groups, $U(N),Sp(2N)$ or $O(N)$ and let the eigenvalues of a
matrix in these groups be ordered around the unit circle (starting from $1$
and going anti-clockwise). It is proved in [13] that the averages

$\nu_{k}(G)[\alpha,\eta]:=\mu_{\textbf{Haar}}\left\\{A\in
G:\alpha\leq\frac{\theta_{k}(A)N}{2\pi}\leq\eta\right\\},$

as $N\to\infty$, converge to different quantities corresponding to the
symmetry type of the matrix group.
To translate this to the language of L-functions, consider a family of
L-functions $\mathcal{L}$ where the L-functions are ordered by their
conductors $c(L)$ (as we will see later on, they may be ordered by various
other types of quantities, for instance cusp form L-functions are ordered by
their weight).
If one assumes the Riemann Hypothesis for these L-functions, then all the
zeros lie on the critical line (we will assume that the critical line is
always $\Re(s)=\frac{1}{2}$) and the ones with non-negative imaginary part can
be written as:

$\left\\{\frac{1}{2}+i\gamma_{n}\right\\}_{n=1}^{\infty},$

where $\gamma_{n}$’s are in increasing order. Katz and Sarnak provide strong
evidence that analogously to the random matrix theory average, as
$N\to\infty$, the quantity

$\frac{\left|\left\\{L\in\mathcal{L},c(L)\leq
N:\alpha\leq\frac{\gamma_{L,j}\log
c(L)}{2\pi}\leq\eta\right\\}\right|}{\left|\left\\{L\in\mathcal{L},c(L)\leq
N\right\\}\right|},$

converges where $\gamma_{L,j}$ denotes the $j$-th zero on the line
$\Re(s)=\frac{1}{2}$. The factor $\log c(L)$ appears here due to the density
of the zeros. The limiting quantity, as they conjecture, depends on the
symmetry type of the L-function family, which, in most cases, is difficult to
determine.
Due to the strong evidence that the zero statistics near $s=\frac{1}{2}$ over
families of L-functions can be modeled by one of the compact random matrix
groups, it becomes natural to wonder whether one can obtain conjectural
asymptotics for a sum of the form

$\frac{1}{|\mathcal{L}_{N}|}\sum_{L\in\mathcal{L}_{N}}L\left(\frac{1}{2}\right)^{m},$

as $N\to\infty$, where $\mathcal{L}$ is an L-function family ordered by some
quantity $N$, and $\mathcal{L}_{N}$ is the subset of the family determined by
$N$. Conrey and Farmer [8] provide evidence that asymptotically these averages
are also determined by the symmetry type of the family. Hence, one would
expect that these asymptotics agree with asymptotics of moments of
characteristic polynomials of random matrices from the corresponding matrix
group. Keating and Snaith give conjectures in this direction in [14]. Thus, we
expect that the asymptotics we will compute give rise to conjectures regarding
families of orthogonal and symplectic L-functions. Here note that we consider
averages over $SO(2N)$ since L-functions with orthogonal symmetry either
exhibit odd or even symmetry and in the odd case, the function vanishes at
$s=\frac{1}{2}$ whereas for even symmetry we expect that the statistics
related to the zeros near $s=\frac{1}{2}$ exhibit behaviour similar to
eigenangles of $SO(2N)$ (see [13] for any details).

###  1.2 Main Results

For technical reasons that appear in the proofs, we are able to prove our main
results for a restricted parameter range (which still covers most of the range
for which these moments exist). Thus, for ease of notation, if
$x\in\mathbb{R}$, we will let $\alpha(x)$ be the greatest integer strictly
less than $x$. In addition, throughout the rest of the paper, we will let
$G(\cdot)$ denote the Barnes $G$-function. We are finally in position to state
our main results.

######  Theorem 1.1.

Let $\mathfrak{L}_{N}^{Sp}(s,h)$ be defined as before. For
$\alpha\left(s+\frac{3}{2}\right)>h\geq 0$, we have the convergence

$\lim_{N\to\infty}\frac{\mathfrak{L}_{N}^{Sp}(s,h)}{N^{\frac{s(s+1)}{2}+2h}}=\frac{2^{\frac{s^{2}}{2}}G(1+s)\sqrt{\Gamma(1+s)}}{\sqrt{G(1+2s)}\Gamma(1+2s)}\mathbb{E}\left[\left(\frac{1}{2}M\left(s+\frac{1}{2},\frac{1}{2}\right)+1\right)^{h}\right].$

where for $a\geq b$, $a,b>0$, $M(a,b)$ is a non-trivial random variable whose
distribution is given by (7). Furthermore, the $\tau$-function corresponding
to $M(a,b)$, which is defined as:

$\tau_{a,b}(t):=t\frac{d}{dt}\log\mathbb{E}\left[e^{-tM(a,b)}\right],$

solves the following form of the $\sigma$-Painlevé-III system:

$\left(t\frac{d^{2}\tau_{a,b}(t)}{dt^{2}}\right)^{2}+4\left(\frac{d\tau_{a,b}}{dt}\right)^{2}\left(t\frac{d\tau_{a,b}}{dt}-\tau_{a,b}(t)\right)-\left(a\frac{d\tau_{a,b}}{dt}+1\right)^{2}=0,$

for all $t>0$, with boundary conditions:

$\displaystyle\begin{cases}\tau_{a,b}(0)=0,&\text{for }\ a>0,\\\ &\\\
\left.\frac{d}{dt}\tau_{a,b}(t)\right|_{t=0}=-\frac{1}{4a},&\text{for}\
a>1.\end{cases}$ (3)

######  Remark 1.2.

We believe that the random variable $M(a,b)$ in fact only depends on $a$, and
is given by the sum of inverse points of the Bessel point process with
parameter $a$ (which we will denote by $Y(a)$). This would not be very
surprising since the correlation kernel of the Jacobi ensemble converges to
the Bessel correlation kernel under the $\frac{1}{N^{2}}$ scaling, and the
Laplace transform of $Y(a)$ is also shown to satisfy the Painlevé system given
above (see [2]). Finally, we also note that this random variable appears in
the ergodic decomposition of inverse Wishart measures (see [1]).

At the centre of our proofs will be the Jacobi ensemble, which we define now.

######  Definition 1.3.

Let $a,b>-1$. Define the probability measure, supported on $[0,1]^{N}$:

$\mu_{N}^{(a,b)}(d\mathbf{x})=\prod_{j=0}^{N-1}\frac{\Gamma(a+b+N+j+1)}{\Gamma(a+j+1)\Gamma(b+j+1)\Gamma(j+2)}\prod_{j=1}^{N}x_{j}^{a}\left(1-x_{j}\right)^{b}\prod_{1\leq
j<k\leq N}\left(x_{k}-x_{j}\right)^{2}d\mathbf{x}.$ (4)

This is called the Jacobi ensemble corresponding to parameters $a,b$.
Similarly, for a measurable function $f:\mathbb{R}^{N}\to\mathbb{R}$, we
denote

$\mathbb{E}_{N}^{(a,b)}[f(x_{1},\ldots,x_{N})]:=\int_{\mathbb{R}^{N}}f(\mathbf{x})\mu_{N}^{(a,b)}(d\mathbf{x}).$

In section 2, we will start by writing $\mathfrak{L}_{N}^{Sp}$ in terms of the
moments of a linear Jacobi statistic. We will show that all derivatives of the
Laplace transform of this statistic converge to those of a limiting random
variable. In essence, the transformation to the Jacobi ensemble should work
for $SO(2N)$ as well, but the Riemann-Hilbert methods used in [6] require an
extra restriction on the parameters of the Jacobi ensemble. That is, it is
crucial for the arguments in this paper that the Laplace transform of
$\\{M_{N}\\}_{N\geq 1}$ converges pointwise to some limiting function.
However, for the Jacobi parameters that correspond to $SO(2N)$, this does not
follow from the results of [6]. Finally, we also note that the analogous
linear statistic corresponding to the Laguerre unitary ensemble has been
studied in various works before (see [7]), [21], [22]).
This result, following the discussion in the previous section, gives rise to
the following conjecture about joint moments of Dirichlet L-functions. These
type of joint moments have been studied in the past both in random matrix
theory and number theory settings (see, for instance, [11], [17]). Here we
denote by $L(s,\chi_{d})$, the L-function corresponding to a Dirichlet
character modulo $d$, $\chi_{d}$.

######  Conjecture 1.4.

Let $s+\frac{1}{2}>h\geq 0$. Then, as $N\to\infty$,

$\frac{1}{N^{*}}\sum\left|L\left(\frac{1}{2},\chi_{d}\right)\right|^{2s-h}\left|L^{\prime\prime}\left(\frac{1}{2},\chi_{d}\right)\right|^{h}\sim\log^{s(s+1)/2+2h}(N^{1/2})\\\
a_{s}\frac{2^{\frac{s^{2}}{2}}G(1+s)\sqrt{\Gamma(1+s)}}{\sqrt{G(1+2s)}\Gamma(1+2s)}\mathbb{E}\left[\left(\frac{1}{2}M\left(s+\frac{1}{2},\frac{1}{2}\right)+1\right)^{h}\right],$

where for the sum is taken over all quadratic Dirichlet characters with
modulus less than or equal to $N$, $N^{*}$ is the number of such characters
and

$a_{s}=\prod_{p\text{ prime
}}\frac{\left(1-\frac{1}{p}\right)^{s(s+1)/2}}{1+\frac{1}{p}}\left(\frac{1}{p}+\frac{1}{2}\left(\left(1+\frac{1}{\sqrt{p}}\right)^{-s}+\left(1-\frac{1}{\sqrt{p}}\right)^{-s}\right)\right).$

Using a similar method, that is switching to the Jacobi ensemble, one can also
obtain the asymptotics of $\mathfrak{R}_{N}(s,h)$, which requires a few extra
technical results that are proved using the theory of orthogonal polynomials.
This is the content of our second main result. Here, $G$ denotes the Barnes
G-function.

######  Theorem 1.5.

For $s>\frac{1}{2}$, $\alpha\left(2s+1\right)>h\geq 0$,

$\lim_{N\to\infty}\frac{\mathfrak{R}_{N}^{SO}(s,h)}{N^{\frac{s(s-1)}{2}+h}\log^{h}(N)}=\frac{1}{\pi^{h}}\frac{2^{\frac{s^{2}}{2}}G(1+s)\sqrt{\Gamma(1+2s)}}{\sqrt{G(1+2s)}\Gamma(1+s)},$

and for $s>\frac{3}{2}$, $\alpha(2s+3)>h\geq 0$,

$\lim_{N\to\infty}\frac{\mathfrak{R}_{N}^{Sp}(s,h)}{N^{\frac{s(s+1)}{2}+h}\log^{h}(N)}=\frac{1}{\pi^{h}}\frac{2^{\frac{s^{2}}{2}}G(1+s)\sqrt{\Gamma(1+s)}}{\sqrt{G(1+2s)}\Gamma(1+2s)}.$

Acknowledgements. We would like to thank Prof. Jon Keating for many useful
suggestions that lead to the improvement of both the results and their
presentation. We would also like to thank anonymous referees both for their
very helpful comments and suggestions that improved the presentation of the
paper, and also for pointing out a number of important references.

##  2 Proof of Theorem 1.1

The relation between the characteristic polynomials of symplectic/orthogonal
matrices and Jacobi ensemble is due to Weyl’s integration formula. In
particular, via a standard application of Weyl’s formula one can compute (see
for instance [9]) the eigenangle densities of these compact matrix groups:

G | Eigenvalues | Joint pdf of $\theta$s
---|---|---
$SO(2N)$ | $e^{\pm i\theta_{1}},\ldots,e^{\pm i\theta_{N}}$ | $\prod\limits_{1\leq k<j\leq N}\left|\cos\left(\theta_{k}\right)-\cos\left(\theta_{j}\right)\right|^{2}d\bm{\theta}$
$Sp(2N)$ | $e^{\pm i\theta_{1}},\ldots,e^{\pm i\theta_{N}}$ | $\prod\limits_{1\leq k<j\leq N}\left|\cos\left(\theta_{k}\right)-\cos\left(\theta_{j}\right)\right|^{2}\prod\limits_{k=1}^{N}\sin^{2}(\theta_{k})d\bm{\theta}$

Using the above table, one can see that the transformation
$x_{j}=\frac{1}{2}(1-\cos\theta_{j})$ transfers these measures to the
corresponding Jacobi ensembles.
Now, focusing again on the average $\mathfrak{L}_{N}^{Sp}(s,h)$, we compute

$\frac{d^{2}\psi_{U}}{d\theta^{2}}=\sum_{j=1}^{N}\left\\{\psi^{\prime\prime}_{j}(\theta)\prod_{\begin{subarray}{c}l=1\\\
l\neq
j\end{subarray}}^{N}\psi_{j}(\theta)+\psi^{\prime}_{j}(\theta)\sum_{\begin{subarray}{c}l=1\\\
l\neq
j\end{subarray}}^{N}\left\\{\psi^{\prime}_{l}(\theta)\prod_{\begin{subarray}{c}k=1\\\
k\neq l,j\end{subarray}}^{N}\psi_{k}(\theta)\right\\}\right\\},$

where

$\psi_{j}(\theta)=(1-e^{i(\theta-\theta_{j}})(1-e^{-i(\theta-\theta_{j})}).$

Dividing by $\psi_{U}$ and evaluating at $\theta=0$, we obtain;

$\frac{\psi^{\prime\prime}_{U}(0)}{\psi_{U}(0)}=\sum_{j=1}^{N}\frac{\psi^{\prime\prime}_{j}(0)}{\psi_{j}(0)}+\sum_{\begin{subarray}{c}j,l=1\\\
j\neq
l\end{subarray}}^{N}\frac{\psi^{\prime}_{j}(0)\psi^{\prime}_{l}(0)}{\psi_{j}(0)\psi_{l}(0)}=\sum_{j=1}^{N}\frac{\cos(\theta_{j})-2}{1-\cos(\theta_{j})}-N(N-1).$

In particular, applying the transformation
$x_{j}=\frac{1}{2}\left(1-\cos(\theta_{j})\right)$:

$\mathfrak{L}_{N}^{Sp}(s,h)=\mathfrak{L}_{N}^{Sp}(s,0)\mathbb{E}_{N}^{\left(s+\frac{1}{2},+\frac{1}{2}\right)}\left[\left(\sum_{j=1}^{N}\frac{1}{2x_{j}}+N^{2}\right)^{h}\right].$
(5)

The $N\to\infty$ asymptotics of $\mathfrak{L}_{N}^{Sp}(s,0)$ is already given
in [14] by

$\mathfrak{L}_{N}^{Sp}(s,0)\sim
N^{s(s+1)/2}\frac{2^{\frac{s^{2}}{2}}G(1+s)\sqrt{\Gamma(1+s)}}{\sqrt{G(1+2s)}\Gamma(1+2s)}.$
(6)

Hence, the following theorem, which is the main result of this section, gives
the asymptotics of $\mathfrak{L}_{N}^{Sp}(s,h)$.

######  Theorem 2.1.

Let $a\geq b$, $a,b>0$. If $\left(x_{1}^{(N)},\ldots,x_{N}^{(N)}\right)$
denote points sampled according to
$\mu_{N}^{\left(a,b\right)}\left(d\bm{x}\right)$, there exists a non-negative
(and non-trivial) random variable $M(a,b)$ such that

$M_{N}:=\sum_{j=1}^{N}\frac{1}{N^{2}x_{j}^{(N)}}\xrightarrow[N\to\infty]{d}M(a,b),$
(7)

and for all $h\in[0,\alpha\left(a+1)\right)$

$\mathbb{E}_{N}^{\left(a,b\right)}\left[\left(\frac{M_{N}}{2}+1\right)^{h}\right]\xrightarrow[]{N\to\infty}\mathbb{E}\left[\left(\frac{M(a,b)}{2}+1\right)^{h}\right].$

Furthermore, the ’tau’ function corresponding to the random variable $M(a,b)$,

$\tau_{a,b}(t):=t\frac{d}{dt}\log\mathbb{E}\left[e^{-tM(a,b)}\right],$

solves the following form of the $\sigma$-Painlevé-III equation:

$\left(t\frac{d^{2}\tau_{a,b}(t)}{dt^{2}}\right)^{2}+4\left(\frac{d\tau_{a,b}}{dt}\right)^{2}\left(t\frac{d\tau_{a,b}}{dt}-\tau_{a,b}(t)\right)-\left(a\frac{d\tau_{a,b}}{dt}+1\right)^{2}=0.$

To prove this result, we will first prove convergence in distribution, and
then show that the $h$-th moments are uniformly integrable so that convergence
of $L^{h}$ norms also takes place. In order to do this, we first focus on
$h\in\mathbb{N}$. Throughout the rest of this section, whenever we say $M_{N}$
sampled according to $\mu_{N}^{(a,b)}$, we will mean $M_{N}$ is defined as in
(7) where $(x_{1}^{(N)},\ldots,x_{N}^{(N)})$ has the law $\mu_{N}^{(a,b)}$.
Now, notice that one may expand:

$\mathbb{E}_{N}^{(a,b)}\left[M_{N}^{h}\right]=\frac{1}{N^{2h}}\sum_{\begin{subarray}{c}\lambda_{1}+\ldots+\lambda_{l}=h\\\
\lambda_{1}\geq\ldots\geq\lambda_{l}>0\end{subarray}}\binom{h}{\lambda_{1},\ldots,\lambda_{l}}\kappa_{N}(\lambda_{1},\ldots,\lambda_{l})\mathbb{E}_{N}^{\left(a,b\right)}\left[\prod_{j=1}^{l}x_{j}^{-\lambda_{j}}\right],$
(8)

where

$\kappa_{N}(\lambda_{1},\ldots,\lambda_{l})=\frac{N!}{(N-l)!n_{1}!\ldots
n_{m}!}.$

Here, $m$ is the number of distinct integers occurring in the partition
$(\lambda_{1},\ldots,\lambda_{l})$ and $n_{k}$ is the number of times $k$-th
integer occurs. Notice that to obtain this sum, we expanded the $h$-th power
as a multinomial and used the symmetry of the Jacobi ensemble under
permutations of indices.
Using standard arguments from random matrix theory (see, for instance, [9]),
one can show that the $l$-point density of the Jacobi ensemble in (4) is given
by:

$\rho_{N}^{(a,b)}\left(x_{1},\ldots,x_{l}\right)=\frac{(N-l)!}{N!}\det\left[K_{N}^{(a,b)}(x_{i},x_{j})\right]_{i,j=1}^{l}.$
(9)

The so-called correlation kernel, $K_{N}(x,y)$, appearing in the determinant
is given by:

$K_{N}^{(a,b)}(x,y)=\sqrt{w^{(a,b)}(x)w^{(a,b)}(y)}\sum_{j=1}^{N}P_{j}^{(a,b)}(x)P_{j}^{(a,b)}(y),$

where

$w^{(a,b)}(x)=x^{a}(1-x)^{b},$

and $\left\\{P_{j}^{(a,b)}\right\\}_{j=1}^{\infty}$ is the sequence of
orthonormal Jacobi polynomials corresponding to parameters $a,b$. To be more
precise, these are polynomials indexed by their degree that satisfy the
orthonormality condition:

$\int_{0}^{1}P_{j}^{(a,b)}(x)P_{k}^{(a,b)}(x)w^{(a,b)}(x)dx=\delta_{jk}.$

In particular, by studying the asymptotics of fixed-dimensional integrals of
the correlation kernel, one should be able to obtain the asymptotics of
moments of $M_{N}(a,b)$.

######  Proposition 2.2.

Let $a\geq b\geq-\frac{1}{2}$, $h\in\mathbb{N}$, $0<h<a+1$, and
$(\lambda_{1},\ldots,\lambda_{l})$ be a partition of $h$. Then;

$\kappa_{N}(\lambda_{1},\ldots,\lambda_{l})\mathbb{E}_{N}^{\left(a,b\right)}\left[\prod_{j=1}^{l}x_{j}^{-\lambda_{j}}\right]=O\left(N^{2h}\right).$

###### Proof.

Using (9), (from here on the constant may change from line to line)

$\kappa_{N}(\lambda_{1},\ldots,\lambda_{l})\mathbb{E}_{N}^{\left(a,b\right)}\left[\prod_{j=1}^{l}x_{j}^{-\lambda_{j}}\right]\leq\text{const
}\int_{[0,1]^{l}}\prod_{j=1}^{l}x_{j}^{-\lambda_{j}}\det\left[K_{N}^{(a,b)}(x_{i},x_{j})\right]_{i,j=1}^{l}dx_{1}\ldots
dx_{l}.$ (10)

Now exapnding the determinant as a finite sum over the $l$-th symmetric group,
and using the Cauchy-Schwarz inequality to note

$\left|K_{N}^{\left(a,b\right)}(x,y)\right|\leq\sqrt{K_{N}^{\left(a,b\right)}(x,x)}\sqrt{K_{N}^{\left(a,b\right)}(y,y)},$
(11)

we can bound the integral in (10) by:

$\text{const}\cdot\prod_{j=1}^{l}\int_{0}^{1}x^{-\lambda_{j}}K_{N}^{\left(a,b\right)}(x,x)dx.$

We claim, for $\lambda\geq 1$, that

$\int_{0}^{1}x^{-\lambda}K_{N}^{\left(a,b\right)}(x,x)dx=O\left(N^{2\lambda}\right).$
(12)

First, note that we may restrict the integration to $(0,\frac{1}{2})$ since

$\int_{\frac{1}{2}}^{1}x^{-\lambda}K_{N}^{\left(a,b\right)}(x,x)dx\leq
2^{h}\int_{\frac{1}{2}}^{1}K_{N}^{(a,b)}(x,x)dx=2^{h}N\mathbb{P}\left(x_{1}\in\left(\frac{1}{2},1\right)\right)=O(N^{2\lambda}).$

Now, making the substitution $x=\frac{u}{N^{2}}$, the integral over
$\left(0,\frac{1}{2}\right)$ becomes:

$N^{2\lambda}\int_{0}^{N^{2}/2}u^{-\lambda}\frac{1}{N^{2}}K_{N}^{\left(a,b\right)}\left(\frac{u}{N^{2}},\frac{u}{N^{2}}\right)du.$

On $(0,1)$, we use Szegö’s well-known bound (see [19]):

$\sup_{x\in[0,1]}P_{j}^{2}(x)\leq\text{const}\cdot j^{2a+1},$

to see that

$\int_{0}^{1}u^{-\lambda}\frac{1}{N^{2}}K_{N}^{\left(a,b\right)}\left(\frac{u}{N^{2}},\frac{u}{N^{2}}\right)du\\\
\leq\frac{\text{const}}{N^{2a+2}}\sum_{j=0}^{N}j^{2a+1}\int_{0}^{1}u^{a-\lambda}du\leq\text{const}\cdot\frac{1}{N^{2a+2}}\sum_{j=0}^{N}j^{2a+1}=O(1).$

On $(1,\frac{N^{2}}{2})$, we use the bound (see [18]):

$w^{\left(a,b\right)}(x)\left(P_{j}^{\left(a,b\right)}(x)\right)^{2}\leq\frac{\text{const}}{\sqrt{x(1-x)}},$

to note that

$\int_{1}^{N^{2}/2}u^{-\lambda}\frac{1}{N^{2}}K_{N}^{\left(a,b\right)}\left(\frac{u}{N^{2}},\frac{u}{N^{2}}\right)du\leq\text{const}\int_{1}^{N^{2}}u^{-\lambda-\frac{1}{2}}du=O(1),$

which gives us the desired result. ∎

######  Proposition 2.3.

Let $a\geq b\geq-\frac{1}{2}$, $a>0$. Then, the sequence
$\left\\{M_{N}\right\\}_{N\geq 1}$sampled according to $\mu_{N}^{(a,b)}$ is
tight.

###### Proof.

Using Markov’s inequality:

$\sup_{N\in\mathbb{N}}\mathbb{P}_{N}^{(a,b)}\left(M_{N}\geq
R\right)\leq\sup_{N\in\mathbb{N}}\frac{\mathbb{E}_{N}^{(a,b)}\left[M_{N}\right]}{R}.$

We claim that

$\sup_{N\in\mathbb{N}}\mathbb{E}_{N}^{(a,b)}\left[M_{N}\right]<\infty.$

Indeed, using symmetry:

$\mathbb{E}_{N}^{(a,b)}\left[M_{N}\right]=\frac{1}{N}\mathbb{E}_{N}^{(a,b)}\left[\frac{1}{x_{1}}\right]=\frac{1}{N^{2}}\int_{0}^{1}\frac{K_{N}^{\left(a,b\right)}(x,x)}{x}dx.$

We first restrict the integral to $[0,\frac{1}{2}]$ by using the boundedness
of the random variable on $\left[\frac{1}{2},1\right]$. Then, arguing as in
the proof of Proposition 2.2, we see that this integral is bounded uniformly
in $N$. Hence,

$\lim_{R\to\infty}\sup_{N\in\mathbb{N}}\mathbb{P}_{N}^{(a,b)}\left(M_{N}\geq
R\right)=0,$

which proves the proposition. ∎

######  Proposition 2.4.

For $a\geq b$; $a,b>0$ and $h\in[0,\alpha\left(a+1)\right)$, the sequence
$\left\\{M_{N}^{h}\right\\}_{N\geq 1}$ where $M_{N}$ is sampled according to
$\mu_{N}^{(a,b)}$ is uniformly integrable.

###### Proof.

It is sufficient to prove that (let $\alpha=\alpha\left(a+1\right)$)

$\sup_{N\in\mathbb{N}}\mathbb{E}_{N}^{(a,b)}\left[M_{N}^{\alpha}\right]<\infty.$

To see this, we may expand as before:

$\mathbb{E}_{N}^{(a,b)}\left[M_{N}^{\alpha}\right]=\frac{1}{N^{2\alpha}}\sum_{\begin{subarray}{c}\lambda_{1}+\ldots+\lambda_{l}=\alpha\\\
\lambda_{1}\geq\ldots\geq\lambda_{l}>0\end{subarray}}\binom{\alpha}{\lambda_{1},\ldots,\lambda_{l}}\kappa_{N}(\lambda_{1},\ldots,\lambda_{l})\mathbb{E}_{N}^{\left(a,b\right)}\left[\prod_{j=1}^{l}x_{j}^{-\lambda_{j}}\right].$
(13)

Using the bounds obtained in Proposition 2.2, this is uniformly bounded in
$N$, which completes the proof, since uniform boundedness of a higher moments
of a random variable gives the uniform integrability of lower moments. ∎

######  Proposition 2.5.

Fix $a\geq b$; $a,b>0$. Then there exists a holomorphic function $f(z)$ on
$S:=\left\\{z\in\mathbb{C}:\Re(z)>0\right\\}$ such that uniformly on compact
subsets of $S$,

$f_{N}(z):=\mathbb{E}_{N}^{(a,b)}\left[e^{-zM_{N}}\right]\xrightarrow[]{\text{as
}N\to\infty}f(z).$

###### Proof.

First, note that $f_{N}(t)$ converges pointwise for all $t\in(0,\infty)$ (see
[6]). Now, if $z\in{S}$,

$\left|\mathbb{E}_{N}^{(a,b)}\left[e^{-zM_{N}}\right]\right|\leq\mathbb{E}_{N}^{(a,b)}\left[\left|e^{-zM_{N}}\right|\right]\leq\mathbb{E}_{N}^{(a,b)}[1]=1,$

and hence by Montel’s theorem, every subsequence of $f_{N}$ has a further
subsequence that converges uniformly on compact subsets of $S$ to a
holomorphic function. Further, these limiting holomorphic functions agree on
$(0,\infty)$ so that by identity theorem, every converging subsequence
converges uniformly on compacts to the same limiting function, $f(z)$. Hence,
the full sequence converges to $f(z)$ uniformly on compacts.

∎

###### Proof of Theorem 1.1.

By Proposition 2.3, the sequence $\\{M_{N}\\}_{N\geq 1}$ is tight so that
there exists a subsequence $\\{M_{N_{k}}\\}_{k\geq 1}$ that converges weakly
to some limiting random variable $M=M(a,b)$. In particular, for all
$t\in(0,\infty)$,

$\lim_{k\to\infty}f_{N_{k}}(t)=f(t)=\mathbb{E}\left[e^{-tM(a,b)}\right].$ (14)

But then by Proposition 2.5, this implies,

$\lim_{N\to\infty}f_{N}(t)=\mathbb{E}\left[e^{-tM(a,b)}\right].$

Thus, using a standard result from probability theory (see for example [12]),
we deduce that

$M_{N}\xrightarrow[d]{\text{as }N\to\infty}M=M(a,b).$

which proves the first claim of the theorem. To prove the second claim, we
simply use Skorokhod’s representation theorem so that we may assume without
loss of generality that the convergence is in fact in almost sure sense.
Combining this with the uniform integrability of $\\{M_{N}^{h}\\}$ gives the
second part of the claim. For the final part, we use Proposition 2.5 to
observe that since $\\{f_{N}\\}_{N\geq 1}$ converges uniformly on compact sets
to $f$, so do all the derivatives of $\\{f_{N}\\}_{N\geq 1}$. Hence, one may
take the limit of the finite-$N$ differential equation (and the corresponding
boundary conditions) proved for $f_{N}(t)$ in [6] to get the desired
differential equation.111We note that the scaling used in [6] has an extra
factor of $2$, which we ommitted here to make the presentation clearer, which
causes a minor difference in the differential equation and boundary
conditions. For the boundary conditions, we note that the extra conditions on
the parameters is due to the fact that for $\tau_{a,b}$ to have derivatives at
$0$, we need certain moments of $M(a,b)$ to be finite. ∎

##  3 Proof of Theorem 1.5

In this section, we will compute the $N\to\infty$ asymptotics of
$\mathfrak{R}_{N}^{G}(s,h)$. We start by observing that using the same
transformation as before, $x_{j}=\frac{1}{2}\left(1-\cos\theta_{j}\right)$,
one may write,

$\mathfrak{R}_{N}^{G}(s,h)=\frac{\mathfrak{R}_{N}^{G}(s,0)}{2^{h}}\cdot\mathbb{E}_{N}^{\left(a,b\right)}\left[\left|\sum_{i=1}^{N}\frac{\sqrt{1-x_{j}}}{\sqrt{x_{j}}}\right|^{h}\right],$
(15)

where $(a,b)=(s-\frac{1}{2},-\frac{1}{2})$ if $G=SO(2N)$; and
$(a,b)=(s+\frac{1}{2},\frac{1}{2})$ if $G=Sp(2N)$. Here, as before,
$\mathbb{E}_{N}^{\left(a,b\right)}$ denotes expectation taken against the
Jacobi ensemble with parameters $a,b$. Now, the asymptotics of
$\mathfrak{R}_{N}^{Sp}(s,0)=\mathfrak{L}_{N}^{Sp}(s,0)$ are given in (6); and
the asymptotics $\mathfrak{R}_{N}^{SO}(s,0)$ are also computed explicitly in
[14] to be:

$\mathfrak{R}_{N}^{SO}(s,0)\sim
N^{\frac{s(s-1)}{2}}\frac{2^{\frac{s^{2}}{2}}G(1+s)\sqrt{\Gamma(1+2s)}}{\sqrt{G(1+2s)}\Gamma(1+s)}.$

Hence, it becomes sufficient to compute the $N\to\infty$ asymptotics of

$\mathbb{E}_{N}^{\left(a,b\right)}\left[\left|\sum_{i=1}^{N}\frac{\sqrt{1-x_{j}}}{\sqrt{x_{j}}}\right|^{h}\right],$

for appropriate choices of $a,b$; which is precisely the content of the main
result of this section.

######  Theorem 3.1.

Let $a\geq b\geq-\frac{1}{2}$ and $a>0$. Then for all $h\in[0,\alpha(2a+2))$,
we have the convergence:

$\mathbb{E}_{N}^{(a,b)}\left[\left(\mathcal{Z}_{N}\right)^{h}\right]=\mathbb{E}_{N}^{(a,b)}\left[\left(\frac{1}{N\log
N}\sum_{j=1}^{N}\frac{\sqrt{1-x_{j}^{(N)}}}{\sqrt{x_{j}^{(N)}}}\right)^{h}\right]\xrightarrow[]{N\to\infty}\left(\frac{2}{\pi}\right)^{h}.$

We will first show the convergence of $\mathcal{Z}_{N}$ to $\frac{2}{\pi}$ in
$L^{2}$, and then combine this with the uniform integrability of the $h$-th
power of $\mathcal{Z}_{N}$. With this goal in mind, let us focus on the case
$h\in\mathbb{N}$ for now. Then, as in equation (8), we can expand:

$\mathbb{E}_{N}^{\left(a,b\right)}\left[\left|\sum_{j=1}^{N}\frac{\sqrt{1-x_{j}}}{\sqrt{x_{j}}}\right|^{h}\right]\\\
=\sum_{\begin{subarray}{c}\lambda_{1}+\ldots+\lambda_{l}=h\\\
\lambda_{1}\geq\ldots\geq\lambda_{l}>0\end{subarray}}\binom{h}{\lambda_{1},\ldots,\lambda_{l}}\kappa_{N}(\lambda_{1},\ldots,\lambda_{l})\mathbb{E}_{N}^{\left(a,b\right)}\left[\prod_{j=1}^{l}x_{j}^{-\lambda_{j}/2}\left(1-x_{j}\right)^{\lambda_{j}/2}\right].$
(16)

The main idea will be to show that the dominating contribution in this finite
sum comes from the partition $\lambda_{1}=\ldots=\lambda_{h}=1$. Then, to
compute the asymptotics of the dominating term for the second moment, we will
write the expectation as a $2$-dimensional integral which will involve a
$2\times 2$ determinant of the correlation kernel. We will see that the
dominating contribution within this integral will come form the product of the
diagonal entries. Thus, in essence, the main contribution will come from the
integral whose asymptotics is computed in the following proposition.

######  Proposition 3.2.

Let $a\geq b\geq-\frac{1}{2}$, $a\geq 0$. We have:

$\mathcal{S}_{N}=\int_{0}^{1}x^{-\frac{1}{2}}(1-x)^{1/2}K_{N}^{\left(a,b\right)}(x,x)dx\sim\frac{2}{\pi}N\log
N.$

###### Proof.

We start by fixing an $\epsilon\in(0,1)$ and dividing the integral into two
contributions: $[0,\epsilon]$ and $[\epsilon,1]$. First, observe that:

$\frac{1}{N\log
N}\int_{\epsilon}^{1}x^{-\frac{1}{2}}(1-x)^{1/2}K_{N}^{\left(a,b\right)}(x,x)dx=\frac{1}{\log
N}\mathbb{E}_{N}^{\left(a,b\right)}\left[\frac{\sqrt{1-x_{1}}}{\sqrt{x_{1}}}\mathbf{1}_{\\{x_{1}\geq\epsilon\\}}\right]\\\
\leq\frac{1}{\log
N}\frac{\sqrt{1-\epsilon}}{\sqrt{\epsilon}}\xrightarrow[]{\text{as
}N\to\infty}0.$ (17)

To compute the asymptotics of the contribution on $[0,\epsilon]$, one first
needs to consider the $l\to\infty$ asymptotics of:

$H(l):=\int_{0}^{\epsilon}x^{-\frac{1}{2}}(1-x)^{\frac{1}{2}}w^{\left(s-\frac{1}{2},-\frac{1}{2}\right)}(x)\left(P_{l}^{\left(a,b\right)}(x)\right)^{2}dx.$
(18)

$\textbf{Step 1}:$ Let

$F(l):=\int_{0}^{\epsilon}{h_{l}}\frac{\arccos\left(1-2x\right)}{x}\frac{J_{a}^{2}\left(\left(l+\frac{a+b+1}{2}\right)\arccos(1-2x)\right)}{\left(l+\frac{a+b+1}{2}\right)^{2a}}dx.$

We claim that there exists a constant $\mathfrak{C}>0$ such that:

$\sup_{l\in\mathbb{N}}\left|H(l)-F(l)\right|\leq\mathfrak{C}.$

To prove this claim, we will use an asymptotic expansion obtained in [10]:

$\sqrt{w^{\left(a,b\right)}(x)}P_{l}^{\left(a,b\right)}(x)\\\
=\sqrt{h_{l}}\left(\frac{\arccos\left(1-2x\right)}{\sqrt{x(1-x)}}\right)^{\frac{1}{2}}\left(\frac{J_{a}\left(\left(l+\frac{a+b+1}{2}\right)\arccos(1-2x)\right)}{\left(l+\frac{a+b+1}{2}\right)^{a}}+O\left(\frac{1}{\left(l+\frac{a+b+1}{2}\right)^{a+1}}\right)\right),$
(19)

where

$h_{l}=\frac{(2l+a+b+1)\Gamma\left(l+a+1\right)\Gamma(l+a+b+1)}{2\Gamma(l+1)\Gamma(l+b+1)},$

and $J_{\nu}$ denotes the Bessel function with parameter $\nu$. Here, error
term is uniform on $x\in[0,\epsilon]$. Now, note that for two sequences of
functions on $[0,\epsilon]$; $\left\\{a_{l}(x)\right\\}_{l=1}^{\infty}$ and
$\left\\{b_{l}(x)\right\\}_{l=1}^{\infty}$, if

$\lim_{l\to\infty}\sup_{x\in[0,\epsilon]}\left|a_{l}(x)-b_{l}(x)\right|=0,$

then for $l$ large enough,

$\left|a_{l}^{2}(x)-b_{l}^{2}(x)\right|\leq\left|a_{l}(x)+b_{l}(x)\right|\leq
2(1+\left|b_{l}(x)\right|).$

Thus, using (19), and the asymptotics of $\Gamma(z)$; we can get the estimate:

$x^{-\frac{1}{2}}(1-x)^{\frac{1}{2}}\biggl{|}w^{\left(a,b\right)}(x)\left(P_{l}^{\left(a,b\right)}(x)\right)^{2}-{h_{l}}\frac{\arccos\left(1-2x\right)}{\sqrt{x(1-x)}}\frac{J_{a}^{2}\left(\left(l+\frac{a+b+1}{2}\right)\arccos(1-2x)\right)}{\left(l+\frac{a+b+1}{2}\right)^{2a}}\biggl{|}\\\
\leq\text{const}\cdot
x^{-\frac{1}{2}}(1-x)^{\frac{1}{2}}\left(1+\sqrt{l}\left(\frac{\arccos\left(1-2x\right)}{\sqrt{x(1-x)}}\right)^{\frac{1}{2}}\left|J_{a}\left(\left(l+\frac{a+b+1}{2}\right)\arccos(1-2x)\right)\right|\right)\\\
\leq\text{const}\cdot x^{-\frac{3}{4}},$

where the constant is independent of $x$, and for the last inequality we use
the bound,

$\left|J_{\alpha}(x)\right|\leq Cx^{-\frac{1}{2}}.$

Since $x^{-3/4}\in L^{1}[0,\epsilon]$, our claim follows.
Now that our claim is established, we want to compute the asymptotics of
$\sum_{l}H(l)$ by comparing with $\sum_{l}F(l)$. In particular, note that:

$\lim_{N\to\infty}\frac{1}{N\log
N}\sum_{l=1}^{N}\left|H(l)-F(l)\right|\leq\lim_{N\to\infty}\frac{1}{N\log
N}\sum_{l=1}^{N}\text{const}=0.$ (20)

Step 2: We compute the asymptotics of $\sum_{l}F(l)$. We begin by making the
change of variables $x=\frac{1}{2}\left(1-\cos\theta\right)$ to obtain:

$F(l)=\frac{h_{l}}{\left(l+\frac{a+b+1}{2}\right)^{2a}}\int_{0}^{\arccos(1-2\epsilon)}\frac{\theta\sin\theta}{1-\cos\theta}J_{a}^{2}\left(\left(l+\frac{a+b+1}{2}\right)\theta\right)d\theta.$

Noting the fact that the function

$g(\theta):\theta\mapsto\frac{\theta\sin\theta}{1-\cos\theta},$ (21)

is decreasing on $[0,\epsilon]$ with $g(0)=2$; and making the change of
variables $u=\left(l+\frac{a+b+1}{2}\right)\theta$, one can infer,

$g(\arccos(1-2\epsilon))\frac{h_{l}}{\left(l+\frac{a+b+1}{2}\right)^{2a+1}}\int_{0}^{\left(l+\frac{a+b+1}{2}\right)\arccos(1-2\epsilon)}J_{a}^{2}(u)du\\\
\leq F(l)\leq
2\frac{h_{l}}{\left(l+\frac{a+b+1}{2}\right)^{2a+1}}\int_{0}^{\left(l+\frac{a+b+1}{2}\right)\arccos(1-2\epsilon)}J_{a}^{2}(u)du.$

Now, on $[0,1]$, the integrand is bounded and hence we want to compute the
growth at infinity. Using the well-known asymptotics of Bessel functions with
large argument:

$J_{a}(x)=\sqrt{\frac{2}{\pi
x}}\cos\left(x-\frac{\pi}{2}\left(a+\frac{1}{2}\right)\right)+O(x^{-3/2}),$
(22)

we obtain (where we let
$l^{\prime}=\left(l+\frac{a+b+1}{2}\right)\arccos(1-2\epsilon)$):

$\int_{1}^{l^{\prime}}J_{a}^{2}(u)du=\int_{1}^{l^{\prime}}\frac{2}{\pi
u}\cos^{2}\left(u-\frac{\pi}{2}\left(a+\frac{1}{2}\right)\right)du+O(1)\\\
=\int_{1}^{l^{\prime}}\frac{1}{\pi
u}\left(1+\cos\left(2u-{\pi}\left(a+\frac{1}{2}\right)\right)\right)du+O(1)=\frac{\log
l}{\pi}+O(1).$

Here, for the last equality we used the fact that

$\sup_{l^{\prime}\geq
1}\int_{1}^{l^{\prime}}\frac{\cos\left(2u-{\pi}\left(a+\frac{1}{2}\right)\right)}{u}du<\infty.$

Using Stirling’s approximation for the Gamma function, one can compute:

$\lim_{l\to\infty}\frac{h_{l}}{\left(l+\frac{a+b+1}{2}\right)^{2a+1}}=1,$

and hence if $\delta>0$, there exists $M\in\mathbb{N}$ such that for every
$l\geq M$,

$\frac{h_{l}}{\left(l+\frac{a+b+1}{2}\right)^{2a+1}}\leq 1+\delta.$

In particular,

$\limsup_{N\to\infty}\frac{1}{N\log
N}\sum_{l=1}^{N}F(l)\leq\limsup_{N\to\infty}\frac{2(1+\delta)\log\left(N!\right)}{\pi
N\log N}+\frac{O(1)}{\log N}=\frac{2(1+\delta)}{\pi},$

so that letting $\delta\to 0$, we get:

$\limsup_{N\to\infty}\frac{1}{N\log N}\sum_{l=1}^{N}F(l)\leq\frac{2}{\pi}.$

Similarly one may obtain the lower bound:

$\liminf_{N\to\infty}\frac{1}{N\log
N}\sum_{l=1}^{N}F(l)\geq\frac{g(\arccos(1-2\epsilon))}{\pi}.$

But then by combining (17) and (20), this implies:

$\frac{g(\arccos(1-2\epsilon))}{\pi}\leq\liminf_{N\to\infty}\mathcal{S}_{N}\leq\limsup_{N\to\infty}\mathcal{S}_{N}\leq\frac{2}{\pi}.$

Since this holds for any $\epsilon>0$ small enough, we get the desired result.
∎

Thus, focusing on $h=2$, we expect the contribution coming from the partition
$\lambda_{1}=\lambda_{2}=1$ to be of order $N^{2}\log^{2}N$. Next, we prove
that the contribution coming from other terms have strictly smaller order than
this.

######  Proposition 3.3.

Let $a\geq b\geq-\frac{1}{2}$, $h\in\mathbb{N}$, $0<l<h<2a+2$, and
$\lambda_{1}+\ldots+\lambda_{l}=h$. Then;

$\kappa_{N}(\lambda_{1},\ldots,\lambda_{l})\mathbb{E}_{N}^{\left(a,b\right)}\left[\prod_{j=1}^{l}x_{j}^{-\lambda_{j}/2}\left(1-x_{j}\right)^{\lambda_{j}/2}\right]=O\left(N^{h}\log^{h-2}N\right).$

###### Proof.

As in the proof of Proposition 2.2, we write:

$\kappa_{N}(\lambda_{1},\ldots,\lambda_{l})\mathbb{E}_{N}^{\left(a,b\right)}\left[\prod_{j=1}^{l}x_{j}^{-\lambda_{j}/2}\left(1-x_{j}\right)^{\lambda_{j}/2}\right]\\\
\leq\text{const
}\int_{[0,1]^{l}}\prod_{j=1}^{l}x_{j}^{-\lambda_{j}/2}\left(1-x_{j}\right)^{\lambda_{j}/2}\det\left[K_{N}^{(a,b)}(x_{i},x_{j})\right]_{i,j=1}^{l}dx_{1}\ldots
dx_{l}.$ (23)

Now, again arguing as in Proposition 2.2 we can bound this integral by:

$\text{const}\cdot\prod_{j=1}^{l}\int_{0}^{1}x^{-\lambda_{j}/2}(1-x)^{\lambda_{j}/2}K_{N}^{\left(a,b\right)}(x,x)dx.$

Further, arguing as in Proposition 2.2, we obtain the bound:

$\int_{0}^{1}x^{-\lambda/2}(1-x)^{\lambda/2}K_{N}^{\left(a,b\right)}(x,x)dx=O\left(N^{\lambda}\right),$
(24)

for $\lambda\geq 2$. This proves the desired result since if $l<h$, there
exists some $\lambda_{j}\geq 2$. ∎

Hence, for $h=2$, upon dividing by $N^{2}\log^{2}N$ and letting $N\to\infty$,
the only (possibly) nonzero limiting contribution will come from the term:

$\mathbb{E}_{N}^{\left(a,b\right)}\left[x_{1}^{-1/2}x_{2}^{-1/2}\left(1-x_{1}\right)^{1/2}\left(1-x_{2}\right)^{1/2}\right]\\\
=\frac{1}{N(N-1)}\int_{0}^{1}\int_{0}^{1}\left[K_{N}^{\left(a,b\right)}(x,x)K_{N}^{\left(a,b\right)}(y,y)-\left(K_{N}^{\left(a,b\right)}(x,y)\right)^{2}\right]\\\
x^{-1/2}y^{-1/2}\left(1-x\right)^{1/2}\left(1-y\right)^{1/2}dxdy.$

We already know from Proposition 3.2 that:

$\int_{0}^{1}\int_{0}^{1}x^{-1/2}y^{-1/2}\left(1-x\right)^{1/2}\left(1-y\right)^{1/2}K_{N}^{\left(a,b\right)}(x,x)K_{N}^{\left(a,b\right)}(y,y)dxdy\sim\frac{4}{\pi^{2}}N^{2}\log^{2}(N).$

Thus, we next prove that the other term gives a contribution of $0$ in the
limit, which will give us the desired asymptotics.

######  Proposition 3.4.

Let $a\geq b\geq-\frac{1}{2}$, $a\geq 0$. Then,

$\lim_{N\to\infty}\frac{1}{N^{2}\log^{2}N}\int_{0}^{1}\int_{0}^{1}x^{-1/2}y^{-1/2}\left(1-x\right)^{1/2}\left(1-y\right)^{1/2}\left(K_{N}^{\left(a,b\right)}(x,y)\right)^{2}dxdy=0.$
(25)

###### Proof.

The main idea, again, will be to estimate this integral using Bessel
functions. First, note that we can instead consider the integral over
$\left[0,\frac{1}{2}\right]\times\left[0,\frac{1}{2}\right]$ since,

$\int_{0}^{1}\int_{1/2}^{1}x^{-1/2}y^{-1/2}\left(1-x\right)^{1/2}\left(1-y\right)^{1/2}\left(K_{N}^{\left(a,b\right)}(x,y)\right)^{2}dxdy\\\
\leq\int_{0}^{1}x^{-1/2}(1-x)^{1/2}K_{N}^{(a,b)}(x,x)dx\int_{1/2}^{1}y^{-1/2}(1-y)^{1/2}K_{N}^{(a,b)}(y,y)dy\\\
\sim\mathbb{E}_{N}^{(a,b)}\left[x_{1}^{-1/2}(1-x_{1})^{1/2}\mathbf{1}_{\\{x_{1}\geq\frac{1}{2}\\}}\right]N^{2}\log
N=O(N^{2}\log N),$

and similarly we get a bound of $O(N^{2})$ for the integral over
$\left[\frac{1}{2},1\right]\times\left[\frac{1}{2},1\right]$. Now, we observe
that,

$\int_{0}^{1/2}\int_{0}^{1/2}x^{-1/2}y^{-1/2}\left(1-x\right)^{1/2}\left(1-y\right)^{1/2}\left(K_{N}^{\left(a,b\right)}(x,y)\right)^{2}dxdy=\sum_{j,k=1}^{N}\left({I}(j,k)\right)^{2},$

where

$I(j,k)=\int_{0}^{1/2}x^{-1/2}\left(1-x\right)^{1/2}w^{\left(a,b\right)}(x)P_{j}^{\left(a,b\right)}(x)P_{k}^{\left(a,b\right)}(x)dx.$

Since $I(j,k)$ is symmetric, we may write:

$\sum_{j,k=1}^{N}I^{2}(j,k)=2\sum_{0\leq k<j\leq
N}I^{2}(j,k)+\sum_{j=1}^{N}I^{2}(j,j).$

Now, we bound the second term using Proposition 3.2:

$\sum_{j=1}^{N}I^{2}(j,j)\leq\text{const
}\sum_{j=1}^{N}\log^{2}(j)+1=O\left(N\log^{2}N\right).$

Thus, it is sufficient to consider only the first sum. With this aim, fix some
small $\delta\in(0,1)$. Divide the first sum into two sums:

$\mathcal{T}_{N,\delta}+\mathcal{B}_{N,\delta}=\sum_{\begin{subarray}{c}j,k=1\\\
\frac{j}{k}\geq
1+\delta\end{subarray}}^{N}I^{2}(j,k)+\sum_{\begin{subarray}{c}j,k=1\\\
1<\frac{j}{k}\leq 1+\delta\end{subarray}}^{N}I^{2}(j,k).$

Claim 1: For fixed $\delta\in(0,1)$,

$\lim_{N\to\infty}\frac{\mathcal{T}_{N,\delta}}{N^{2}\log^{2}N}=0.$

To obtain an upper bound on $I(j,k)$, we will again use the estimate (19).
Note that if $\left\\{a_{l}(x)\right\\}_{l=1}^{\infty}$ and
$\left\\{b_{l}(x)\right\\}_{l=1}^{\infty}$ are two sequences of functions on
$[0,\frac{1}{2}]$ such that:

$\lim_{l\to\infty}\sup_{x\in[0,\frac{1}{2}]}\left|a_{l}(x)-b_{l}(x)\right|=0,$

then there exists an $M>0$ such that,

$\left|a_{j}(x)a_{k}(x)-b_{j}(x)b_{k}(x)\right|\leq\left|a_{j}(x)\right|\left|a_{k}(x)-b_{k}(x)\right|+\left|b_{k}(x)\right|\left|a_{j}(x)-b_{j}(x)\right|\\\
\leq\text{const}\left(\left|a_{j}(x)\right|+\left|b_{k}(x)\right|\right)\leq
M\left(1+\left|b_{j}(x)\right|+\left|b_{k}(x)\right|\right).$

Thus, arguing as before, we obtain that there exists a constant $K>0$ such
that:

$\sup_{j,k}\left|I(j,k)-L(j,k)\right|\leq K,$

where we define:

$L(j,k)=\int_{0}^{1/2}\sqrt{h_{j}h_{k}}\frac{\arccos\left(1-2x\right)}{x}\\\
\frac{J_{a}\left(\left(j+\frac{a+b+1}{2}\right)\arccos(1-2x)\right)J_{a}\left(\left(k+\frac{a+b+1}{2}\right)\arccos(1-2x)\right)}{\left(j+\frac{a+b+1}{2}\right)^{a}\left(k+\frac{a+b+1}{2}\right)^{a}}dx.$

Thus, for all $j,k$, using the trivial bound $|x|\leq x^{2}+1$:

$I^{2}(j,k)\leq CL^{2}(j,k)+\text{const}.$ (26)

Hence, it is enough to show the claim for $I(j,k)$ replaced by $L(j,k)$. Note
that making the substitution $\theta=\arccos(1-2x)$ in the definition of
$L(j,k)$ gives:

$L(j,k)=\int_{0}^{\pi/2}\frac{\sqrt{h_{j}h_{k}}}{\left(j+\frac{a+b+1}{2}\right)^{a}\left(k+\frac{a+b+1}{2}\right)^{a}}\frac{\theta\sin\theta}{1-\cos\theta}\\\
J_{a}\left(\left(j+\frac{a+b+1}{2}\right)\theta\right)J_{a}\left(\left(k+\frac{a+b+1}{2}\right)\theta\right)d\theta.$

Now, we divide the integral into two contributions: $(0,\frac{1}{k})$ and
$(\frac{1}{k},\frac{\pi}{2})$. Note that since Bessel functions are bounded
(by a constant depending on the parameter of the function), the contribution
on $(0,\frac{1}{k})$ is bounded by:

$\text{const}\cdot\frac{1}{k}\frac{\sqrt{h_{j}h_{k}}}{\left(j+\frac{a+b+1}{2}\right)^{a}\left(k+\frac{a+b+1}{2}\right)^{a}}\leq\text{const}\sqrt{\frac{j}{k}},$

Now, to get asymptotics on the other contribution, we use (22). To be more
precise, since on $\theta\in\left(\frac{1}{k},\frac{\pi}{2}\right)$;
$\left(j+\frac{a+b+1}{2}\right)\theta$ and
$\left(k+\frac{a+b+1}{2}\right)\theta$ are bounded away from zero uniformly in
$j,k$; we see that there exists a bounded function $f:[1,\infty)\to\mathbb{R}$
such that:

$J_{a}\left(\left(j+\frac{a+b+1}{2}\right)\theta\right)\\\
=\sqrt{\frac{2}{\pi\left(j+\frac{a+b+1}{2}\right)\theta}}\cos\left(\left(j+\frac{a+b+1}{2}\right)\theta-\frac{\pi(a+\frac{1}{2})}{2}\right)+\frac{f\left(\left(j+\frac{a+b+1}{2}\right)\theta\right)}{\left(j+\frac{a+b+1}{2}\right)^{\frac{3}{2}}\theta^{\frac{3}{2}}},$

and similarly with $j$ replaced by $k$. Using this, we obtain that the
contribution on $(\frac{1}{k},\frac{\pi}{2})$ is given as (since the terms
coming from the errors are integrable):

$M(j,k):=\int_{1/k}^{\pi/2}g(\theta)\frac{2\sqrt{h_{j}h_{k}}}{\pi\left(j+\frac{a+b+1}{2}\right)^{(a+\frac{1}{2})}\left(k+\frac{a+b+1}{2}\right)^{(a+\frac{1}{2})}}\\\
\frac{\cos\left(\left(j+\frac{a+b+1}{2}\right)\theta-\frac{\pi(a+\frac{1}{2})}{2}\right)\cos\left(\left(k+\frac{a+b+1}{2}\right)\theta-\frac{\pi(a+\frac{1}{2})}{2}\right)}{\theta}d\theta+O(1)\\\
=O(1)\int_{1/k}^{\pi/2}g(\theta)\frac{\cos\left(\left(j+\frac{a+b+1}{2}\right)\theta-\frac{\pi(a+\frac{1}{2})}{2}\right)\cos\left(\left(k+\frac{a+b+1}{2}\right)\theta-\frac{\pi(a+\frac{1}{2})}{2}\right)}{\theta}d\theta+O(1).$

where we recall that we denote:

$g(\theta)=\frac{\theta\sin\theta}{1-\cos\theta}.$

Now, we divide the integral into two integrals via the identity

$\cos(a)\cos(b)=\frac{1}{2}\left(\cos(a-b)+\cos(a+b)\right).$ (27)

Then, the first integral becomes (up to multiplication by a bounded quantity):

$\int_{1/k}^{\pi/2}\frac{g(\theta)\cos\left((j-k)\theta\right)}{\theta}d\theta=\int_{1}^{\frac{\pi
k}{2}}\frac{\cos\left(\left(\frac{j}{k}-1\right)u\right)g\left(\frac{u}{k}\right)}{u}du,$

which can be seen to be bounded for all $j,k$ such that
$\frac{j}{k}-1\geq\delta>0$, using integration by parts. As for the second
integral that comes from (27),

$\int_{1/k}^{\pi/2}\frac{g(\theta)\cos\left((j+k+a+b+1)\theta-\pi(a+\frac{1}{2})\right)}{\theta}d\theta\\\
=\int_{1}^{\frac{\pi
k}{2}}\frac{g\left(\frac{u}{k}\right)\cos\left(\left(\frac{j+a+b+1}{k}+1\right)u-\pi(a+\frac{1}{2})\right)}{u}du=O(1),$

which can be seen again via integrating by parts. In particular, using the
trivial identity $(x+y)^{2}\leq 2(x^{2}+y^{2})$, there exists a constant $C>0$
such that for all $j,k$ such that $\frac{j}{k}\geq 1+\delta$:

$L^{2}(j,k)\leq C\frac{j}{k}.$

Noting that

$\sum_{\begin{subarray}{c}j,k=1\\\ \frac{j}{k}\geq
1+\delta\end{subarray}}^{N}\frac{j}{k}\leq\left(\sum_{k=1}^{N}\frac{1}{k}\right)\left(\sum_{j=1}^{N}j\right)=O(N^{2}\log(N)),$

the claim follows.
$\textbf{Claim 2}:$ There exists a constant $C>0$ independent of $\delta$ such
that

$\limsup_{N\to\infty}\frac{\mathcal{B}_{N,\delta}}{N^{2}\log^{2}N}\leq
C\delta.$

Note that once again, it is sufficient to prove the claim for $I(j,k)$
replaced by $L(j,k)$. To prove this bound, we start again by dividing the
integral defining $L(j,k)$ into two contributions:
$\left(0,\frac{1}{k}\right)$ and $\left(\frac{1}{k},\frac{\pi}{2}\right)$.
Exactly as before, we get the bound,

$C\sqrt{\frac{j}{k}}\leq C\sqrt{1+\delta}\leq 2C,$

on the integral over $\left(0,\frac{1}{k}\right)$, where $C$ only depends on
the parameter of the Bessel function and hence does not depend on $\delta$. As
for the other integral, we use the estimate,

$\left|J_{a}(x)\right|\leq\frac{C}{\sqrt{x}},$

to show that the integral over $\left(\frac{1}{k},\frac{\pi}{2}\right)$ is
bounded by

$\text{const}\int_{1/k}^{\pi/2}\frac{1}{u}du=O(\log k).$

Thus, we get a constant $C>0$ independent of $\delta$ such that

$\mathcal{B}_{N,\delta}\leq C\left(\sum_{\begin{subarray}{c}j,k=1\\\
1<\frac{j}{k}\leq 1+\delta\end{subarray}}^{N}\log^{2}(k)+1\right).$

Now, note that for each $k$, $\log^{2}(k)+1$ appears as many times as the
number of integers $j\leq N$ such that $k<j\leq k(1+\delta)$. But this is at
most $\delta k$, and hence,

$\sum_{\begin{subarray}{c}j,k=1\\\ 1<\frac{j}{k}\leq
1+\delta\end{subarray}}^{N}\log^{2}(k)+1\leq\delta\sum_{k=1}^{N}k(\log^{2}(k)+1)\leq\delta
N^{2}\log^{2}N+\delta N^{2},$

which proves the claim. Combining the two claims, we see that:

$\limsup_{N\to\infty}\frac{1}{N^{2}\log^{2}N}\sum_{j,k=1}^{N}I^{2}(j,k)=\limsup_{N\to\infty}\left(\mathcal{T}_{N,\delta}+\mathcal{B}_{N,\delta}\right)\leq\limsup_{N\to\infty}\mathcal{T}_{N,\delta}+\limsup_{N\to\infty}\mathcal{B}_{N,\delta}\leq
C\delta.$

Since $C$ is independent of $\delta$ and $\delta\in(0,1)$ was arbitrary the
result follows. ∎

###### Proof of Theorem 1.5.

Combining Propositions 3.2, 3.3, 25, one can see that for $h=1,2$,

$\lim_{N\to\infty}\mathbb{E}_{N}^{\left(a,b\right)}\left[\left(\mathcal{Z}_{N}\right)^{h}\right]=\left(\frac{2}{\pi}\right)^{h}.$

In particular:

$\lim_{N\to\infty}\mathbb{E}_{N}^{\left(a,b\right)}\left[\left(\mathcal{Z}_{N}-\frac{2}{\pi}\right)^{2}\right]=\lim_{N\to\infty}\left[\mathbb{E}_{N}^{\left(a,b\right)}\left[\left(\mathcal{Z}_{N}\right)^{2}\right]-\frac{4}{\pi}\mathbb{E}_{N}^{\left(a,b\right)}\left[\mathcal{Z}_{N}\right]+\frac{4}{\pi^{2}}\right]=0.$

Hence, we have that $\mathcal{Z}_{N}$ converges, in distribution, to
$\frac{2}{\pi}$, and hence by Skorokhod’s Representation Theorem we may assume
without loss of generality that the convergence is in almost sure sense.
Moreover, expanding as in (16) and applying Propositions
$\ref{prop1},\ref{prop2}$ and 25, one can infer,

$\sup_{N\geq
1}\left\\{\mathbb{E}_{N}^{\left(a,b\right)}\left[\left(\mathcal{Z}_{N}\right)^{\alpha(2a+2)}\right]\right\\}<\infty,$

which means the sequence of random variables

$\left\\{\left(\mathcal{Z}_{N}\right)^{h}\right\\}_{N\geq 1},$

sampled according to $\mu_{N}^{(a,b)}$ are uniformly integrable. Combining the
uniform integrability of this sequence with the assumed almost sure
convergence, we get the desired result. ∎

## References

  * [1] T. Assiotis, Ergodic decomposition for inverse Wishart measures on infinite positive-definite matrices, Symmetry, Integrability and Geometry: Methods and Applications, 15 (2019).
  * [2] T. Assiotis, B. Bedert, M. A. Gunes, and A. Soor, On a distinguished family of random variables and Painlevé equations, Probability & Mathematical Physics, 2-3 (2021), pp. 613–642.
  * [3] T. Assiotis, J. P. Keating, and J. Warren, On the joint moments of the characteristic polynomials of random unitary matrices, arXiv: 2005.13961, to appear IMRN, (2020).
  * [4] E. C. Bailey, S. Bettin, G. Blower, J. B. Conrey, A. Prokhorov, M. O. Rubinstein, and N. C. Snaith, Mixed moments of characteristic polynomials of random unitary matrices, Journal of Mathematical Physics, 60 (2019), p. 083509.
  * [5] E. Basor, P. Bleher, R. Buckingham, T. Grava, A. Its, E. Its, and J. P. Keating, A representation of joint moments of CUE characteristic polynomials in terms of Painlevé functions, Nonlinearity, 32 (2019), p. 4033–4078.
  * [6] M. Chen, Y. Chen, and E.-G. Fan, The Riemann-Hilbert analysis to the Pollaczek-Jacobi type orthogonal polynomials, Stud. Appl. Math., 143 (2019), pp. 42–80.
  * [7] Y. Chen and A. Its, Painlevé III and a singular linear statistics in Hermitian random matrix ensembles. I, J. Approx. Theory, 162 (2010), pp. 270–297.
  * [8] J. B. Conrey and D. W. Farmer, Mean values of $L$-functions and symmetry, Internat. Math. Res. Notices, (2000), pp. 883–908.
  * [9] P. J. Forrester, Log-gases and random matrices, Princeton University Press, (2010).
  * [10] C. L. Frenzen and R. Wong, A uniform asymptotic expansion of the Jacobi polynomials with error bounds, Canad. J. Math., 37 (1985), pp. 979–1007.
  * [11] C. Hughes, On the characteristic polynomial of a random unitary matrix and the Riemann zeta function, PhD Thesis, University of Bristol, (2001).
  * [12] O. Kallenberg, Foundations of modern probability, Probability and its Applications (New York), Springer-Verlag, New York, second ed., 2002.
  * [13] N. M. Katz and P. Sarnak, Random matrices, Frobenius eigenvalues, and monodromy, vol. 45 of American Mathematical Society Colloquium Publications, American Mathematical Society, Providence, RI, 1999.
  * [14] J. P. Keating and N. C. Snaith, Random matrix theory and $L$-functions at $s=1/2$, Comm. Math. Phys., 214 (2000), pp. 91–110.
  * [15] J. P. Keating and N. C. Snaith, Random matrix theory and $\zeta(1/2+it)$, Communications in Mathematical Physics, 214 (2000), pp. 57–89.
  * [16] H. L. Montgomery, The pair correlation of zeros of the zeta function, in Analytic number theory (Proc. Sympos. Pure Math., Vol. XXIV, St. Louis Univ., St. Louis, Mo., 1972), vol. Vol. XXIV of Proc. Sympos. Pure Math., Amer. Math. Soc., Providence, RI, 1973, pp. 181–193.
  * [17] M. Mourtada and V. Murty, Omega theorems for $\frac{L^{\prime}}{L}(1,\chi_{d})$, International Journal of Number Theory, 3 (2013).
  * [18] P. Nevai, T. Erdélyi, and A. P. Magnus, Generalized Jacobi weights, Christoffel functions, and Jacobi polynomials, SIAM J. Math. Anal., 25 (1994), pp. 602–614.
  * [19] G. Szegő, Orthogonal polynomials, vol. Vol. XXIII of American Mathematical Society Colloquium Publications, American Mathematical Society, Providence, RI, fourth ed., 1975.
  * [20] B. Winn, Derivative moments for characteristic polynomials from the CUE, Communications in Mathematical Physics, 315 (2012), p. 531–562.
  * [21] S.-X. Xu, D. Dai, and Y.-Q. Zhao, Critical edge behavior and the Bessel to Airy transition in the singularly perturbed Laguerre unitary ensemble, Comm. Math. Phys., 332 (2014), pp. 1257–1296.
  * [22] S.-X. Xu, D. Dai, and Y.-Q. Zhao, Painlevé III asymptotics of Hankel determinants for a singularly perturbed Laguerre weight, Journal of Approximation Theory, 192 (2015), pp. 1–18.

Fine Hall, 304 Washington Rd, Princeton, NJ 08544, USA.
<EMAIL_ADDRESS>


# Object-based SLAM utilizing unambiguous pose parameters
considering general symmetry types

Taekbeom Lee†, Youngseok Jang†, and H. Jin Kim Taekbeom Lee and H. Jin Kim are
with the Aerospace Engineering Department, Seoul National University, South
Korea<EMAIL_ADDRESS><EMAIL_ADDRESS>Jang is with the
Mechanical and Aerospace Engineering Department, Seoul National University,
South Korea<EMAIL_ADDRESS>Taekbeom Lee and Youngseok Jang are
contributed equally to this manuscript.This research was supported by Unmanned
Vehicles Core Technology Research and Development Program through the National
Research Foundation of Korea(NRF) and Unmanned Vehicle Advanced Research
Center(UVARC) funded by the Ministry of Science and ICT, the Republic of
Korea(NRF-2020M3C1C1A01086411)

###### Abstract

Existence of symmetric objects, whose observation at different viewpoints can
be identical, can deteriorate the performance of simultaneous localization and
mapping (SLAM). This work proposes a system for robustly optimizing the pose
of cameras and objects even in the presence of symmetric objects. We classify
objects into three categories depending on their symmetry characteristics,
which is efficient and effective in that it allows to deal with general
objects and the objects in the same category can be associated with the same
type of ambiguity. Then we extract only the unambiguous parameters
corresponding to each category and use them in data association and joint
optimization of the camera and object pose. The proposed approach provides
significant robustness to the SLAM performance by removing the ambiguous
parameters and utilizing as much useful geometric information as possible.
Comparison with baseline algorithms confirms the superior performance of the
proposed system in terms of object tracking and pose estimation, even in
challenging scenarios where the baseline fails.

##  I INTRODUCTION

Simultaneous localization and mapping (SLAM), one of the core technologies for
autonomous driving, is a technology that reconstructs the environment around
the robot and estimates the robot position in the reconstructed map. Despite a
significant progress in precise localization using geometrical information of
the surrounding environment, it is still difficult for a SLAM system to
achieve advanced tasks based on human interaction and scene understanding. To
overcome the problem, semantic SLAM that uses semantic information in SLAM has
been in using deep learning techniques such as recently developed instance
segmentation [1], [2], [3] and 3d object detection [4], [5], [6].

Object-based SLAM [7], [8] is a branch of semantic SLAM that reconstructs an
object-based map and provides high-level information. Object-based SLAM
simultaneously estimates the location of semantic object inferred from the
network while performing feature-based localization used in existing SLAM
systems, and enables to estimate camera poses even in featureless
environments. In addition, a more expressive map can be constructed by
expressing the pose, type, and shape of the semantic objects in the map.

However, if the shape of the object itself is symmetric or if the observed
shape is symmetric due to occlusion, object-based SLAM can suffer from
significant error in the estimation process of ego motion and pose of the
object. Since symmetric objects may have the same observation at different
viewpoints, data association or motion estimation may fail.

Figure 1:  Illustration of object tracking results using the data obtained
surrounding a rectangular table. Although only a single object is observed,
the symmetry-agnostic approach in (b) may generate multiple map objects and
fail object tracking. On the other hand, the symmetry-aware approach in (a)
recognizes the discrete symmetry type of the object and succeed in object
tracking using all the reliable measurements. Such improvement of (a) makes
the object-based SLAM robust, since a large number of constraints obtained
from long tracking of objects are useful for backend optimization.

We propose a system for robustly optimizing the pose of cameras and objects
even in the presence of symmetric objects. The 3d detection module [4] is
modified to obtain a local observation that predicts multiple poses for each
object, similar to [9]. We classify object types into asymmetry, discrete, and
continuous symmetry based on the local observation of the observed objects. By
extracting a parameter corresponding to the symmetry type from the pose of
object, only the robust parameter is used as a constraint to jointly optimize
the camera pose and object pose. Therefore, the proposed SLAM system has the
advantage of robustness by using as much useful geometric information as
possible even when symmetric objects are observed, while directly using 3d
detection networks with many existing research and dataset. In summary, the
contributions of the paper are as follows:

  * •

We design a symmetry and pose ambiguity aware object SLAM system which fully
utilizes information from multiple pose hypotheses of objects to jointly
optimize camera pose and globally consistent object pose.

  * •

We propose a method to extract reliable information from ambiguous pose of
symmetric objects. We categorize symmetry types of general objects and
distinguish pose parameters into ambiguous one and unambiguous ones. The
extracted unambiguous parameters of each symmetry type are used in the
proposed object association and optimization modules.

  * •

We use multiple hypotheses 3d detection network as observation module of our
system, which can be easily edited from existing networks and changed to
better networks in the future.

The remainder of this paper is organized as follows. Section II reviews
related literature. The overview and core concepts of the system are provided
in Section III. Detailed methods for applying core concepts to systems are
described in Sections IV and V. In Section VI, experiments in simulation and
public dataset are presented. Finally, conclusions are provided in Section
VII.

Figure 2:  The overview of the proposed object-based SLAM system.

##  II RELATED WORK

This section reviews related studies on object-based SLAM and symmetric-aware
object pose estimation.

###  II-A Object-based SLAM

Object-based SLAM attempts to build a robust SLAM system by simultaneously
optimizing camera poses, feature positions, and poses of objects from semantic
information. To express the shape and pose of an object, [8], [7], and [10]
use prior object model, cuboid, and ellipsoid, respectively. And [11], [12]
represent the shape and pose of an object using category specific embedding
method. Although they showed that joint optimization could increase the
robustness of both camera and object pose estimation, they did not consider
the presence of ambiguous detection due to symmetric objects or occlusion.

###  II-B Symmetry-aware object pose estimation

####  II-B1 Single view

[13] proposes a network that can determine whether a shape has symmetry using
an object’s CAD model, and [14] predicts multiple candidate poses for the
detected object and analyzes ambiguity that may be caused by occlusion as well
as ambiguity by the shape of an object. However, they estimated object pose
only from a single view and did not treat multi-view cases such as SLAM.

####  II-B2 Multiple view

Recently, studies considering the pose-ambiguity of objects have been proposed
in object-based SLAM. PoseRBPF [15] estimates the pose distribution of
asymmetric and symmetric objects using the rao-blackwellized particle filter.
However, it requires a predefined codebook for the pose of the object model.
[16] is the object-based SLAM system that simultaneously optimizes camera
poses and object poses using the projection of reconstructed 3d keypoints as a
prior. However, every time they encounter on a new dataset, they have to label
the prior. [17] expresses geometrical primitives in an unified, decomposed
quadric form and explicitly deals with the degenerative case due to a
symmetric shape. However, the degenerative cases considered in [17] are
limited to a few quadric types, so error can be induced when fitting arbitrary
shapes to quadric. [9] uses a single neural network to predict poses as multi-
hypotheses and optimizes them through a max-mixture[18] model. Since the
symmetric object has ambiguous pose parameters, such as the rotation angle for
an axis of symmetry, pose estimation should be performed by considering only
the unambiguous elements. However, error can occur because [9] directly uses
hypotheses containing even ambiguous parameters for pose estimation.

We propose a robust SLAM system using existing well-developed 3d detection
modules while extracting only available geometric elements from symmetric
objects and using them as optimization constraints.

##  III Proposed system

###  III-A Symmetry types

The key idea of the proposed system is to propose a criterion that can
effectively classify general objects using only three symmetry types and to
define different pose representations suitable for each symmetry type in order
to fully utilize the geometry information available in the symmetric object.

In general, objects can be asymmetric or symmetric. The asymmetric object can
be inferred to have a consistent object pose in the 3d detection module, no
matter which viewpoint it is observed from, but the symmetric object cannot.
In addition, we classify symmetry as discrete and continuous types. Discrete
symmetric objects refer to reflection symmetric objects that can have a finite
number of poses as the observed viewpoints change, and continuous symmetric
objects refer to objects with an infinite number of poses based on an axis of
symmetry, such as a circular table. Objects which are classified into the
following three types are represented in different ways to reflect all
possible unambiguous pose parameters:

  * •

Asymmetry: All detection results of a asymmetric object can be used to
optimize a unique pose since the unique pose can be determined when viewpoint
changes. Accordingly, the pose can be represented by 6 degrees of freedom
(DoF) which is commonly used.

  * •

Discrete symmetry: Objects with multiple planes of symmetry have as many poses
supporting the same shape as the number of planes of symmetry. We express the
pose assuming that the symmetrical planes of most discrete symmetric objects
present have one intersection line. The intersection of symmetric planes is
defined as the axis of symmetry, and the rotation angle based on the axis is
defined as a symmetric angle. Accordingly, the position and the axis of
symmetry are shared with the reflected poses of a discrete symmetric object,
and only the symmetric angle can be expressed differently. In other words, the
pose of a discrete symmetric object is defined as five shared parameters for
position and axis of symmetry and unshared parameters for all the symmetry
angles.

  * •

Continuous symmetry: For objects such as round tables, there is an infinite
number of symmetry planes, so there is an infinite number of object poses that
support the same shape. For continuous symmetric objects, it is assumed that
planes of symmetry have a single intersection, as in the case of discrete
symmetry. Therefore, we express the pose of continuous symmetric objects only
by position and axis of symmetry after removing the symmetric angle by
classifying it as an ambiguous parameter.

###  III-B Pipeline

The entire pipeline is described in Fig. 2, and the tracking and object
mapping modules are designed based on DSP-SLAM [12]. When a new keyframe is
selected, the multi-hypothesis detection network uses an rgb-d image to detect
objects, and the detected objects infer observable multiple pose hypotheses
(Section IV-A). The categorization module determines the symmetry type of the
detected object based on the distribution of multiple pose hypotheses. We
extract the axis of symmetry for symmetric objects and cluster the poses with
similar symmetry angles for discrete symmetry objects (Section IV-B). We
associate the categorized detection with map objects using the class and pose
except for ambiguous parameters. For discrete symmetric objects where a new
symmetry angle is observed, we add a non-shared parameter (Section IV-C). The
camera pose, map point, and map object are jointly optimized at the backend.
The constraint between the camera pose and map object pose is formed
differently for each map object type during optimization (Section V).

##  IV CATEGORIZED DETECTION AND OBJECTS

###  IV-A Multi-hypothesis 3d object detection

We modify the 3d object detection module so that it can infer multiple pose
hypotheses. Since the distribution of inferred multiple hypotheses influences
the determination of the object’s symmetry type, it should be modified so that
the multi-hypothesis 3d object detection module can cover all the poses that
the object can have. [19] extends the single-loss single-output system to have
multiple outputs, calculating loss [20] using only the hypothesis that
succeeded in the most accurate inference among multiple hypotheses in the
training process. Unlike the method of using the average loss of multiple
outputs, this can increase multi-hypothesis diversity because each hypothesis
is randomly selected and learned individually. We also modified the 3d object
detection module [4] in the same way as [19], allowing a source for the
object’s symmetry type to be inherent in multiple hypotheses.

###  IV-B Symmetry type categorization of detection

As shown in Fig. 3, since the multi-hypothesis detection implies the ambiguity
of the object’s pose, we can classify the object’s symmetry type and extract
the type-specific pose parameters only from the detection result without prior
information.

Unlike the symmetric object, the multiple pose hypotheses of the asymmetric
object are similar to single pose. In other words, the distribution of pose
hypotheses of the asymmetric object is very close to unimodal, and the
variance is much smaller than that of the symmetrical object.

For discrete and continuous symmetry types, we perform an additional
classification process because the normalized singular values of multiple
hypotheses are large in both cases. As mentioned in Section III-A, both types
have certain axis of symmetry, and the axis of symmetry $l$ is obtained as
follows:

$\displaystyle
l^{*}=\underset{{l\in\mathbb{R}^{3}}}{\text{argmax}}\left\|[\overline{\omega}_{o_{1}o_{2}},\overline{\omega}_{o_{1}o_{3}},\cdots,\overline{\omega}_{o_{1}o_{N}}]^{T}\cdot
l\right\|_{2}~{},$ (1)
$\displaystyle\text{where}\quad\overline{\omega}_{o_{1}o_{i}}=\frac{\omega_{o_{1}o_{i}}}{\left\|\omega_{o_{1}o_{i}}\right\|_{2}},\;\omega_{o_{1}o_{i}}=\log_{\text{so(3)}}(R_{co_{1}}^{T}R_{co_{i}})\in\mathbb{R}^{3}.$

$N$ and $\overline{\omega}$ are the number of hypotheses and rotation axis,
respectively. And, symmetry angles are computed by
$\theta_{o_{1}o_{i}}=\left\|\omega_{o_{1}o_{i}}\right\|_{2}$. After clustering
the multiple hypotheses using the DBSCAN algorithm [21], the variance between
the representative $\theta$ of each cluster is compared. As illustrated in
Fig. 3, compared to the continuous symmetric object which corresponds to a
continuous set of $\theta$, the variance among clusters of discrete symmetric
object is very large. Representative $\theta$ values of rectangular table form
two clusters with about 180$\degree$ difference, while the round table has
many similar representative $\theta$ clusters.

Then, reliable parameters of pose can be extracted for each classified
symmetry type. First, the asymmetric object can use 6 DoF from detection
results, and the continuous symmetric object can use position and axis of
symmetry. In discrete symmetric objects, parameters are position, axis of
symmetry, and representative symmetry angles of the clusters.

Figure 3:  Multi-hypothesis results for each symmetry type. Objects of three
types are observed from different viewpoints (a), (b), and (c).

###  IV-C Data association of objects

The categorized detection results are associated with the previously generated
map objects. First, the detection results, which represent relative
transformation between the camera and the object, are warped in world
coordinate using tracked camera pose $T_{wc}$. Then, object matching is
performed by comparing the unambiguous parameters of the detected object and
map objects considering the symmetry type. Due to inaccurate detection or
partial observation, the same object can be categorized by different symmetry
types in different views. We address the misclassification by following
association strategies. If there is a discrete or continuous symmetry type in
the two comparison groups, the position and axis of symmetry (5 DoF)
parameters are compared; otherwise, the distance between the 6 DoF parameters
is computed. If the distance is small enough and the class is the same,
matching between the objects is successful.

After object matching is performed, the symmetric angles of the detected
discrete symmetric object are associated with the nearest symmetric angle of
the matched map object. The unmatched angles are added to the map object as a
new symmetric angle.

Discrete symmetric object can be occasionally classified into asymmetry type
at a specific viewpoint, as can be seen from Fig. 3 (a) and (c) of the the
discrete case. Therefore, even if 6 DoF matching between asymmetry objects
fails, if 5 DoF matching is successful, we change the map object to the
discrete symmetry type to express the pose with position, an axis of symmetry,
and symmetric angles. Object association using unambiguous parameters for each
symmetry type alleviates mismatching caused by ambiguous parameters.
Therefore, the proposed system can track objects for a long time, which acts
as an important source in joint optimization.

The detection result that failed to match is registered as a new map object
after the shape reconstruction using deep-sdf [22] similar to DSP-SLAM.

##  V JOINT OPTIMIZATION

We perform joint optimization by modifying the existing SLAM optimization
problem using the categorized map objects and associated detection results.

$\displaystyle
C^{*},O^{*},P^{*}=\underset{{C,O,P}}{\text{argmin}}~{}E_{reproj}(C,P)+E_{obj}(C,O)~{},$
(2)

where C, O, and P refer to the camera and object pose and map point included
in the optimized window size, respectively. $E_{reproj}$ denotes the
reprojection error, and $E_{obj}$ denotes the pose error between the map
object and the camera. The joint optimization is solved by the Levenberg-
Marquardt method through g2o[23] solver.

$E_{obj}$ is designed only with unambiguous parameters in consideration of the
symmetry type of map object, which can produce a more robust solution than
existing methods that utilize all pose parameters with an ambiguous parameter
as a constraint.

####  V-1 Asymmetry

The results of asymmetry map object and associated multiple hypotheses are not
linked by multiple edges but by a single edge using a max-mixture model, which
selects the hypothesis with the lowest error at each iteration of
optimization, similar to [9].

$\displaystyle
e_{\text{asym}}(T_{wc},T_{wo})=\min_{j\in{[1,N]}}\log_{\text{se(3)}}{({T_{co}^{j}}\cdot
T_{wo}^{-1}\cdot T_{wc})}~{},$ (3)

where $T_{wc}$ and $T_{wo}$, which mean the pose of the camera and object on
the world, are optimization variables, $T_{co}^{j}$ represents the $j$-th
hypothesis, and $N$ is the total number of hypotheses.

####  V-2 Discrete symmetry

Discrete symmetric map objects have position and axis of symmetry as shared
parameters and $M$ symmetric angles as non-shared parameters. Each hypothesis
is associated with one of symmetry angles of the matched map object, and edges
are formed as many as the number of associated symmetric angles, $m$. Each
edge is modeled with the max-mixture like the asymmetry case.

$\displaystyle
e_{\text{disc}}(T_{wc},T_{wo_{i}})=\min_{j_{i}\in{[1,N_{i}]}}\log_{\text{SE(3)}}{({T_{co}^{j_{i}}}\cdot
T_{wo_{i}}^{-1}\cdot T_{wc})}~{},$ (4)
$\displaystyle\text{where}~{}T_{wo_{i}}=\begin{bmatrix}\exp{(\theta_{wo_{i}}\cdot\overline{\omega}_{wo})}&&t_{wo}\\\
\text{0}_{1\times 3}&1\end{bmatrix},~{}i\in{[1,m]}~{}.$

$N_{i}$ refers to the number of hypotheses associated with the $i$-th
symmetric angle, and $T_{wo_{i}}$ means the $i$-th symmetric pose constructed
by position and axis of symmetry which are shared parameters and the $i$-th
symmetric angle. Furthermore, for unconstrained parameterization in
optimization, the axis of symmetry is used by the following expression:
$\overline{\omega}_{wo}=\text{f}(\phi_{wo},\psi_{wo})$, where $\phi_{wo}$,
$\psi_{wo}$ and $\text{f}(\cdot)$ are polar angle, azimuth in spherical
coordinates, and the transformation function from $(\phi,\psi)$ to
$\overline{\omega}$, respectively.

####  V-3 Continuous symmetry

The continuous symmetric map object has only position and axis of symmetry as
unambiguous parameters. The axis of symmetry ($\overline{\omega}_{co}$)
extracted from the rotation part and individual position parts of multiple
hypotheses are used to formulate the following:

$\displaystyle e_{\text{cts}}(T_{wc},T_{wo})=e_{\text{trans}}+\gamma\cdot
e_{\text{axis}}~{},$ (5)
$\displaystyle\begin{split}\text{where}~{}e_{\text{trans}}&=\min_{j\in{[1,N]}}\|t_{co}^{j}-R_{wc}^{\text{T}}(t_{wc}-t_{wo})\|_{2}~{},\\\
e_{\text{axis}}&=\|\text{f}^{-1}(R_{wc}\cdot\overline{\omega}_{co})-[\phi_{wo},\psi_{wo}]^{\text{T}}\|_{2}~{}.\end{split}$

$e_{\text{trans}}$ and $e_{\text{axis}}$ are max-mixture based translation
error and error related to axis of symmetry, respectively. $\gamma$ is the
constant weight for balancing two error terms.

Figure 4:  Simulation environment.

##  VI EXPERIMENTAL RESULTS

This section presents the performance of the proposed symmetry-aware object
SLAM system compared with the baseline. Simulation and public datasets are
used to evaluate the proposed system.

###  VI-A Setup

When a camera looks at the feature-rich scene, ego-motion can be estimated
accurately by the feature-based SLAM backbone [24] in object-based SLAM
systems. On the other hand, object-camera constraints are dominant estimation
sources if the camera observes a featureless scene. Therefore, in order to
effectively evaluate the proposed object-based SLAM system, we construct a
simulation environment such that one side of the environment has rich feature
points and the other side has a few feature points, as shown in Fig. 4. Then
we obtain the simulation dataset using Unreal Engine and AirSim [25]. The
camera moves surrounding a center object in the environment so that camera
observes feature-rich and featureless scenes alternately for each specific
region, as shown in Fig. 4 (a) and (b). As shown in Fig. 4, the center object
can be replaced with discrete and continuous symmetric objects in the
simulation environment. These cases are called sim(disc) and sim(cts). In
addition, we evaluate the proposed system in the general indoor scenes using
the popular scanNet dataset [26].

For training of the 3D detection network used in our system, pre-training is
performed using SUN RGB-D dataset [27]. After that, to reduce the domain gap
between actual and simulation data, additional data are acquired in the
simulation, and fine-tuning is performed. We only fine-tune the detection
network for simulation dataset since ScanNet dataset only provide axis-aligned
bounding box. Both pre-training and fine-tuning is done in the same way as
[19]. The SUN RGB-D dataset assumes that the input point clouds are
represented in the coordinates aligned with the direction of gravity, and
expresses the object’s orientation using the rotation angle with respect to
the axis of gravity (i.e. yaw). However, ScanNet dataset has no gravity
direction, so we calculated in advance the rotation matrix that aligns the
y-axis of camera with the normal vector of the ground plane for each frame. An
additional sensor such as an inertial measurement unit or ground detection
module may help to change the preprocessing for online implementation. We used
the number of hypotheses as 30, which was selected empirically to categorize
symmetry types of detection well.

We test using Intel i7-10700 (2.9GHz) and NVIDIA RTX 2060 GPU. DSP-SLAM using
single hypothesis detection results is used as the first baseline system (SH),
and the second baseline (MH) is the modified DSP-SLAM to integrate with key
idea on [9] which uses multi-hypothesis detection with no consideration of
symmetry types.

Figure 5:  The results of object tracking and map object reconstruction in
sim(disc). Figure 6:  The translation error of the estimated map object’s pose
in case study. Figure 7:  The qualitative results of the proposed and baseline
algorithms (SH, MH) in sim(disc). Figure 8:  The qualitative results of the
proposed and baseline (SH) algorithms in Scene0022_00 in ScanNet.

###  VI-B Case study

To understand how the proposed system enhances the overall performance, we
test object tracking and pose estimation under the presence of symmetric
objects with pose ambiguity, using the environment in Fig. 4. For fair
comparison by isolating the other sources that affect the performance except
the pose ambiguity, we construct a pose graph using edges from the camera to
objects obtained from the proposed categorization and association module and
true camera nodes and estimated the object pose by optimization. The same
setup is employed to test the baseline algorithms.

Fig. 5 shows the result of object tracking and the reconstructed map object
using the sim(disc) dataset. The proposed method continuously tracks the
center object, and a single map object is reconstructed accordingly. On the
other hand, the baseline algorithms (MH, SH) fail to track, and incorrectly
recognize a single object as two or more map objects.

The translation error of the estimated object is reported in Fig. 6. MH and SH
show large error when establishing the object again after the tracking
failure, whereas the proposed method maintains small error through successful
object tracking.

TABLE I:  The translation error of the proposed and baseline systems using ScanNet and simulation dataset. The bold indicates the best performance and the symbol ‘-’ means failure of pose estimation. dataset | RMSE of translation [m]
---|---
OURS | SH | MH
ScanNet | Scene0022_00 | 0.161 | 0.237 | 0.179
Scene0049_00 | 0.137 | 0.137 | 0.152
Scene0091_00 | 0.079 | 0.101 | 0.091
Scene0289_00 | 0.125 | 0.144 | 0.153
sim | disc | 0.086 | - | -
cts | 0.265 | 0.341 | 0.284

###  VI-C System evaluation

The performance of the entire system was compared and evaluated with the
baseline system using the simulation environment and ScanNet. First, Fig. 7
shows the quantitative results compared with the baseline system using the
simulation environment sim(disc). The baseline and camera trajectory are
plotted on the map built by the proposed system. In the beginning, a feature-
rich scene is observed as shown in Fig. 4 (a), so all three systems are good
at estimating pose. Such trend changes as the features that can be used for
localization disappear, since the camera location must be estimated using only
the tracked object. Based on the good object tracking performance, the
proposed system also demonstrates good performance even in the featureless
region using an unambiguous pose parameter that fits the symmetry type.
However, both MH and SH systems fail to estimate the pose in the featureless
region. The baseline algorithms associate the detection with the map object on
the map using all 6 DoF parameters, and there are cases when none of the
multiple hypotheses fits the previous detection result. The data association
may fail due to such mismatch in yaw angle. Quantitative results for the
simulation environment are shown in Table I. For sim(cts), the baseline
algorithms also had no failure in pose estimation, but we can see that the
proposed algorithm has the highest performance.

Fig. 8 shows qualitative results on a ScanNet Scene0022_00. This sequence
includes not only the symmetrical objects but also the objects that do not
fully enter the camera field of view, so the uncertainty of detection is high.
This is revealed in the result of SH. As seen in simulation setting, the
baseline system fails data association and many objects are registered in same
position of the map. However, the proposed system robustly recognizes as a
single object and optimizes the pose even in these cases. The quantitative
results are shown in table I. We evaluate the system performance using root
mean squared error (RMSE) error of each keyframe position. The proposed system
exhibits similar or better path estimation performance in most sequences.

##  VII CONCLUSIONS

Symmetric objects present in the scene can cause the performance degradation
or even failure of SLAM, since their observation at different viewpoints can
be identical and cause obscurity. We proposed a method for robustly optimizing
the pose of cameras and objects even in the presence of symmetric objects.

The proposed classification of objects into three categories depending on
their symmetry characteristics was successfully applied to various objects.
Under the proposed method, the objects in the same category can be associated
with the same type of ambiguity, which contributes to the efficiency in data
association. By extracting only the unambiguous parameters corresponding to
each category and using them in data association and joint optimization of the
camera and object pose, the proposed approach provides significant robustness
to the SLAM performance. Proposed system showed better performance than
baseline systems in environments with many symmetric objects.

## References

  * [1] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask r-cnn,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 2961–2969.
  * [2] L. Du, J. Tan, X. Xue, L. Chen, H. Wen, J. Feng, J. Li, and X. Zhang, “3dcfs: Fast and robust joint 3d semantic-instance segmentation via coupled feature selection,” in _2020 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2020, pp. 6868–6875.
  * [3] B. Xu, W. Li, D. Tzoumanikas, M. Bloesch, A. Davison, and S. Leutenegger, “Mid-fusion: Octree-based object-level multi-instance dynamic slam,” in _2019 International Conference on Robotics and Automation (ICRA)_. IEEE, 2019, pp. 5231–5237.
  * [4] D. Rukhovich, A. Vorontsova, and A. Konushin, “Fcaf3d: Fully convolutional anchor-free 3d object detection,” _arXiv preprint arXiv:2112.00322_ , 2021\.
  * [5] Z. Ding, X. Han, and M. Niethammer, “Votenet: A deep learning label fusion method for multi-atlas segmentation,” in _International Conference on Medical Image Computing and Computer-Assisted Intervention_. Springer, 2019, pp. 202–210.
  * [6] A. D. Pon, J. Ku, C. Li, and S. L. Waslander, “Object-centric stereo matching for 3d object detection,” in _2020 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2020, pp. 8383–8389.
  * [7] S. Yang and S. Scherer, “Cubeslam: Monocular 3-d object slam,” _IEEE Transactions on Robotics_ , vol. 35, no. 4, pp. 925–938, 2019.
  * [8] R. F. Salas-Moreno, R. A. Newcombe, H. Strasdat, P. H. Kelly, and A. J. Davison, “Slam++: Simultaneous localisation and mapping at the level of objects,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2013, pp. 1352–1359.
  * [9] J. Fu, Q. Huang, K. Doherty, Y. Wang, and J. J. Leonard, “A multi-hypothesis approach to pose ambiguity in object-based slam,” in _2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2021, pp. 7639–7646.
  * [10] L. Nicholson, M. Milford, and N. Sünderhauf, “Quadricslam: Dual quadrics from object detections as landmarks in object-oriented slam,” _IEEE Robotics and Automation Letters_ , vol. 4, no. 1, pp. 1–8, 2018.
  * [11] E. Sucar, K. Wada, and A. Davison, “Nodeslam: Neural object descriptors for multi-view shape reconstruction,” in _2020 International Conference on 3D Vision (3DV)_. IEEE, 2020, pp. 949–958.
  * [12] J. Wang, M. Rünz, and L. Agapito, “Dsp-slam: Object oriented slam with deep shape priors,” in _2021 International Conference on 3D Vision (3DV)_. IEEE, 2021, pp. 1362–1371.
  * [13] E. Corona, K. Kundu, and S. Fidler, “Pose estimation for objects with rotational symmetry,” in _2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2018, pp. 7215–7222.
  * [14] F. Manhardt, D. M. Arroyo, C. Rupprecht, B. Busam, T. Birdal, N. Navab, and F. Tombari, “Explaining the ambiguity of object detection and 6d pose from visual data,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 6841–6850.
  * [15] X. Deng, A. Mousavian, Y. Xiang, F. Xia, T. Bretl, and D. Fox, “Poserbpf: A rao–blackwellized particle filter for 6-d object pose tracking,” _IEEE Transactions on Robotics_ , vol. 37, no. 5, pp. 1328–1342, 2021.
  * [16] N. Merrill, Y. Guo, X. Zuo, X. Huang, S. Leutenegger, X. Peng, L. Ren, and G. Huang, “Symmetry and uncertainty-aware object slam for 6dof object pose estimation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 14 901–14 910.
  * [17] W. Zhen, H. Yu, Y. Hu, and S. Scherer, “Unified representation of geometric primitives for graph-slam optimization using decomposed quadrics,” in _2022 International Conference on Robotics and Automation (ICRA)_. IEEE, 2022, pp. 5636–5642.
  * [18] E. Olson and P. Agarwal, “Inference on networks of mixtures for robust robot mapping,” _The International Journal of Robotics Research_ , vol. 32, no. 7, pp. 826–840, 2013.
  * [19] C. Rupprecht, I. Laina, R. DiPietro, M. Baust, F. Tombari, N. Navab, and G. D. Hager, “Learning in an uncertain world: Representing ambiguity through multiple hypotheses,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 3591–3600.
  * [20] A. Guzman-Rivera, D. Batra, and P. Kohli, “Multiple choice learning: Learning to produce multiple structured outputs,” _Advances in neural information processing systems_ , vol. 25, 2012.
  * [21] M. Ester, H.-P. Kriegel, J. Sander, X. Xu, _et al._ , “A density-based algorithm for discovering clusters in large spatial databases with noise.” in _kdd_ , vol. 96, no. 34, 1996, pp. 226–231.
  * [22] J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove, “Deepsdf: Learning continuous signed distance functions for shape representation,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2019, pp. 165–174.
  * [23] G. Grisetti, R. Kümmerle, H. Strasdat, and K. Konolige, “g2o: A general framework for (hyper) graph optimization,” in _Proceedings of the IEEE international conference on robotics and automation (ICRA), Shanghai, China_ , 2011, pp. 9–13.
  * [24] R. Mur-Artal and J. D. Tardós, “Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras,” _IEEE transactions on robotics_ , vol. 33, no. 5, pp. 1255–1262, 2017.
  * [25] S. Shah, D. Dey, C. Lovett, and A. Kapoor, “Airsim: High-fidelity visual and physical simulation for autonomous vehicles,” in _Field and service robotics_. Springer, 2018, pp. 621–635.
  * [26] A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner, “Scannet: Richly-annotated 3d reconstructions of indoor scenes,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 5828–5839.
  * [27] S. Song, S. P. Lichtenberg, and J. Xiao, “Sun rgb-d: A rgb-d scene understanding benchmark suite,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2015, pp. 567–576.