# Natural density of rank-$2$ Drinfeld modules with big Galois image

Chien-Hua Chen Electronic address<EMAIL_ADDRESS>ORCID:
0000-0003-3267-5603 ; Corresponding author Mathematics Division,
National Center for Theoretical Sciences,
Taipei, Taiwan

###### Abstract

In this paper, we compute the natural density of rank-$1$ Drinfeld module over
$\mathbb{F}_{q}[T]$ with surjective adelic Galois representation; and the
natural density of rank-$2$ Drinfeld modules over $\mathbb{F}_{q}[T]$ whose
$\mathfrak{l}$-adic Galois image containing the special linear subgroup for
finitely many prime ideal $\mathfrak{l}$.

##  1 Introduction

In [Jon10], Jones studied the natural density of elliptic curves over
$\mathbb{Q}$ whose adelic Galois representation is an index-$2$ subgroup of
$\mathrm{GL}_{2}(\hat{Z})$. This result was generalized later on by Zywina
[Zyw10] to elliptic curves over arbitrary number fields. It turns out that the
natural density of elliptic curves over a number field with maximal adelic
Galois image is equal to one.

Besides, in [LSTX19] they generalized the natural density problem to Abelian
varieties with big monodromy. The techniques in these results all involved in
classification of maximal subgroups in $\mathrm{GL}_{2}(\mathbb{Z}/\ell)$ ( or
${\rm GSP}_{2n}(\mathbb{Z}/\ell)$ for abelian varieties), Serre’s large sieve
method, and Messer-Wüstholz bound for irreducibility of mod-$\ell$ Galois
representations.

In the function field analogy, Pink and Rütsche [PR09] proved the open image
theorem for Drinfeld modules of arbitrary rank without complex multiplication.
And [Zyw11] Zywina proved the existence of rank-$2$ Drinfeld modules with
surjective adelic Galois representation. The result is later on generalized by
the author to Drinfeld modules of rank $3$ case [Che22] and prime rank case
[Che21]. Therefore, it is reasonable to ask the natural density of Drinfeld
modules of prime rank with surjective adelic Galois representation. Due to Van
der Heiden’s result [vdH04] on determinant of adelic Galois representation for
Drinfeld modules, the natural density estimation can be split into two cases:

  1. Case 1:

Natural density of rank-$1$ Drinfeld modules with surjective adelic Galois
representation.

  2. Case 2:

Natural density of rank-$r$ Drinfeld modules whose adelic Galois image
containing $\mathrm{SL}_{r}(\widehat{\mathbb{F}_{q}[T]})$.

In this paper, we compute and give a brief estimation for case 1.

######  Theorem 1 (Theorem 3.3 & Corollary 3.4).

For each $x\in\mathbb{R}_{>0}$, let $||\cdot||$ to be the absolute value
$|\cdot|_{\infty}$ of $F_{\infty}$. Define

$B_{1}(x)=\\{\Delta\in A\mid\Delta\neq 0,||\Delta||<x\\}.$

The natural density of rank-$1$ Drinfeld modules with surjective adelic Galois
representation is equal to

$\displaystyle 1-\lim_{x\rightarrow\infty}\frac{\\#\\{\Delta\in
B_{1}(x)\mid{\rm Im}\rho_{\phi^{\Delta}}\neq\hat{A}^{*}\\}}{\\#B_{1}(x)}$
$\displaystyle=1-\sum_{2\leqslant t\leqslant q-2,t\mid
q-1}\sum_{i=0}^{\lfloor\frac{\lfloor{\rm
log}_{q}x\rfloor-1}{t}\rfloor}\left(q^{ti+1}\cdot\sum_{j\mid
i}\mu(j)\lfloor\frac{q-1}{tj}\rfloor\right)$ $\displaystyle\leqslant
1-\sum_{2\leqslant t\leqslant q-2,{t\mid q-1}}\frac{q-1}{(q^{t}-1)\cdot t}$

The computation is based on Gekeler’s [Gek16] characterization of adelic
Galois image for rank-$1$ Drinfeld modules, which he called twisted Carlitz
modules.

On the other hand, for case 2 we only compute the natural density of rank-$2$
Drinfeld modules with $\mathfrak{l}$-adic Galois image containing
$\mathrm{SL}_{2}(\mathbb{F}_{q}[T]_{\mathfrak{l}})$ for finitely many prime
ideal $\mathfrak{l}$.

######  Theorem 2 (Corollary 4.12).

Let $q\geqslant 4$ be a prime power, and $S$ be a finite set of prime ideals
of $A=\mathbb{F}_{q}[T]$. Then we have

$\displaystyle L_{2}^{S}$
$\displaystyle:=\lim_{x\rightarrow\infty}\frac{\\#\\{(g_{1},g_{2})\in
B_{2}(x)\mid{\rm Im}\rho_{\phi^{(g_{1},g_{2})},\mathfrak{l}}\supset{\rm
SL}_{2}(A_{\mathfrak{l}})\textrm{ for }\mathfrak{l}\in S\\}}{\\#B_{2}(x)}$
$\displaystyle\geqslant
1+\sum_{i=1}^{|S|}(-1)^{i}\cdot\left(\sum_{\mathfrak{l}_{1},\cdots,\mathfrak{l}_{i}\in
S\textrm{ distinct
}}\frac{1}{q^{\deg_{T}\mathfrak{l}_{1}+\cdots+\mathfrak{l}_{i}}}\right).$

The reason why we can only estimate the natural density for finitely many
$\mathfrak{l}$-adic Galois representations is because of the followings:

First, the technical difficulty on the criterion to determine whether a
subgroup of $\mathrm{GL}_{2}(\mathbb{F}_{q}[T]/\mathfrak{l})$ contains
$\mathrm{SL}_{2}(\mathbb{F}_{q}[T]/\mathfrak{l})$, one can compare Proposition
4.2 with the elliptic curve situation ([Zyw10], Lemma A.8). Second, there is
no function field analogue of Messer-Wüstholz bound for surjective mod
$\mathfrak{l}$ Galois representations (see [Zyw11], Theorem 5.8 for the
elliptic curve case). Currently, in [Che23] we have derived Drinfeld module
analogue of Messer-Wüstholz bound for irreducible mod $\mathfrak{l}$ Galois
representations, but there is no such a bound for a Drinfeld module $\phi$
whose mod $\mathfrak{l}$ Galois image lies in a normalizer of Cartan subgroup
of $\mathrm{GL}_{2}(\mathbb{F}_{q}[T]/\mathfrak{l})$. The current existed
bound needs to assume that $\phi$ has good reduction at $\mathfrak{l}$, see
[CL19] Theorem 2.1 (2).

In the end of the introduction, we mention that Ray [Ray23] proved
independently that the natural density of rank-$2$ Drinfeld modules with
surjective $(T)$-adic Galois representation is positive. Apart from his
estimation of natural density (density $\geqslant q^{-7}$), we give a stronger
estimation (see Corollary 4.13):

$\displaystyle\lim_{x\rightarrow\infty}\frac{\\#\\{(g_{1},g_{2})\in
B_{2}(x)\mid{\rm Im}\rho_{\phi^{(g_{1},g_{2})},(T)}={\rm
GL}_{2}(A_{(T)})\\}}{\\#B_{2}(x)}\geqslant(q-1)\cdot(\frac{1}{q}-\sum_{2\leqslant
t\leqslant q-2,{t\mid q-1}}\frac{1}{(q^{t}-1)}).$

##  2 Preliminary

###  2.1 Notation

  * •

$q=p^{e}$ is a prime power

  * •

$A=\mathbb{F}_{q}[T]$

  * •

$F=\mathbb{F}_{q}(T)$

  * •

$F^{{\rm{sep}}}$= separable closure of $F$

  * •

$F^{{\rm{alg}}}$= algebraic closure of $F$

  * •

$G_{F}={\rm{Gal}}(F^{{\rm{sep}}}/F)$

  * •

$\mathfrak{l}=(l)$ a nonzero prime ideal of $A$ with monic generator $l$, and
define $\deg_{T}(\mathfrak{l})=\deg_{T}(l)$

  * •

$A_{\mathfrak{p}}=$ completion of A at the nonzero prime ideal
$\mathfrak{p}\triangleleft A$

  * •

$\mathbb{F}_{\mathfrak{p}}=A/\mathfrak{p}$

###  2.2 Drinfeld modules and Galois representations

Let $A=\mathbb{F}_{q}[T]$ be the polynomial ring over finite field with
$q=p^{e}$ an odd prime power, $F=\mathbb{F}_{q}(T)$ be the fractional field of
$A$, and $K$ be a finite extension over $F$. Set $F\\{\tau\\}$ to be the
twisted polynomial ring with usual addition rule, and the multiplication rule
is defined to be $\tau\alpha=\alpha^{q}\tau\text{ for any }\alpha\in K$.

######  Definition 2.1.

A Drinfeld $A$-module of rank $r$ over $F$ is a ring homomorphism

$\phi:A\rightarrow F\\{\tau\\}$

such that

  1. (i)

$\phi(a):=\phi_{a}$ satisfies ${\rm deg}_{\tau}\phi_{a}=r\cdot{\rm deg}_{T}a$

  2. (ii)

Denote $\partial:F\\{\tau\\}\rightarrow F$ by $\partial(\sum
a_{i}\tau^{i})=a_{0}$, then $\phi$ satisfies $\gamma=\partial\circ\phi$.

From the definition of Drinfeld $A$-module, we can characterize a Drinfeld
module $\phi$ by writing down

$\phi_{T}=T+g_{1}\tau+\cdots+g_{r-1}\tau^{r-1}+g_{r}\tau^{r},\textrm{ where
}g_{i}\in F\textrm{ and }g_{r}\in F^{*}.$

######  Definition 2.2.

Let $\phi$ and $\psi$ be two rank-$r$ Drinfeld $A$-modules over $K$. A
morphism $u:\phi\rightarrow\psi$ over $K$ is a twisted polynomial $u\in
K\\{\tau\\}$ such that

$u\phi_{a}=\psi_{a}u\text{ \rm for all }a\in A.$

A non-zero morphism $u:\phi\rightarrow\psi$ is called an isogeny. A morphism
$u:\phi\rightarrow\psi$ is called an isomorphism if its inverse exists.

######  Definition 2.3.

  1. (i)

Let $\phi$ be a Drinfeld $A$-module over $K$. The Drinfeld module $\phi$ gives
$K^{{\rm{alg}}}$ an $A$-module structure, where $a\in A$ acts on
$K^{{\rm{alg}}}$ via $\phi_{a}$.

  2. (ii)

Let $\mathfrak{a}$ be an ideal of $A$ with monic generator $a$.The
$\mathfrak{a}$-torsion of $\phi$ is defined to be the set

$\phi[\mathfrak{a}]=\phi[a]:=\\{{\rm{zeros}}\ {\rm{of}}\
\phi_{a}(x)=\gamma(a)x+\sum^{r\cdot{\rm{deg}}(a)}_{i=1}g_{i}(a)x^{q^{i}}\\}\subseteq
K^{alg}.$

The action $b\cdot\alpha=\phi_{b}(\alpha)\ \forall\ b\in
A,\forall\alpha\in\phi[\mathfrak{a}]$ also gives $\phi[\mathfrak{a}]$ an
$A$-module structure.

######  Proposition 2.4.

Let $\phi$ be a rank $r$ Drinfeld module over $K$ and $\mathfrak{a}$ a non-
zero ideal of $A$, , then the $A/\mathfrak{a}$-module $\phi[\mathfrak{a}]$ is
free of rank $r$.

###### Proof.

See [Gos96] Proposition 4.5.7. ∎

Let $\phi$ be a rank $r$ Drinfeld module over $F$ of generic characteristic,
then $\phi_{a}(x)$ is separable, so we have $\phi[a]\subseteq F^{{\rm{sep}}}$.
This implies that $\phi[a]$ has a $G_{F}$-module structure. Given a nonzero
prime ideal $\mathfrak{l}$ of $A$, we can consider the $G_{F}$-module
$\phi[\mathfrak{l}]$. We obtain the so-called mod $\mathfrak{l}$ Galois
representation

$\bar{\rho}_{\phi,\mathfrak{l}}:G_{F}\longrightarrow{\rm{Aut}}(\phi[\mathfrak{l}])\cong
GL_{r}(\mathbb{F}_{\mathfrak{l}}).$

For a prime power $\mathfrak{l}^{i}$, we may construct the residual
representation

$\bar{\rho}_{\phi,\mathfrak{l}^{i}}:G_{F}\longrightarrow{\rm{Aut}}(\phi[\mathfrak{l}^{i}])\cong
GL_{r}(A/{\mathfrak{l}}^{i}).$

By taking inverse limit with respect to $\mathfrak{l}^{i}$, we have the
$\mathfrak{l}$-adic Galois representation

${\rho}_{\phi,\mathfrak{l}}:G_{F}\longrightarrow\varprojlim_{i}{\rm{Aut}}(\phi[\mathfrak{l^{i}}])\cong{\rm{GL_{r}}}(A_{\mathfrak{l}}).$

###  2.3 Reduction of Drinfeld modules

######  Definition 2.5.

Let $K$ be a local field with uniformizer $\pi$, valuation ring $R$, unique
maximal ideal $\mathfrak{p}:=(\pi)$, normalized valuation $v$ and residue
field $\mathbb{F}_{\mathfrak{p}}$. Let $\phi:A\longrightarrow K\\{\tau\\}$ be
a Drinfeld module of rank $r$.

  1. (i)

We say that $\phi$ has stable reduction if there is a Drinfeld module
$\phi^{\prime}:A\longrightarrow R\\{\tau\\}$ such that

    1. 1.

$\phi^{\prime}$ is isomorphic to $\phi$ over $K$;

    2. 2.

$\phi^{\prime}$ mod $\mathfrak{p}$ is still a Drinfeld module (i.e.
$\phi^{\prime}_{T}$ mod $\mathfrak{p}$ has ${\rm{deg}}_{\tau}\geqslant 1$ ).

  2. (ii)

$\phi$ is said to have stable bad reduction of rank $r_{1}<r$ if $\phi$ has
stable reduction and $\phi$ mod $\mathfrak{p}$ has rank $r_{1}$.

  3. (iii)

$\phi$ is said to have good reduction if $\phi$ has stable reduction and
$\phi$ mod $\mathfrak{p}$ has rank $r$.

######  Remark 2.6.

We denote $\phi\mod\mathfrak{p}$ by $\phi\otimes\mathbb{F}_{\mathfrak{p}}$.

##  3 Density of rank-$1$ Drinfeld modules

In this section, we apply the following result from Gekeler to compute the
natural density of rank-$1$ Drinfeld modules whose adelic Galois
representation is surjective.

######  Theorem 3.1 ([Gek16], Corollary 3.14).

Let $\phi^{\Delta}$ be the rank-$1$ Drinfeld $A$-module determined by

$\phi^{\Delta}_{T}=T+\Delta\tau,$

where $\Delta\in A-\\{0\\}$. Then the image of adelic Galois representation of
$\phi^{\Delta}$ is a subgroup of $\hat{A}^{*}$ of index

${\rm def}(\rho_{\phi^{\Delta}})=g.c.d.(d-1,q-1,k_{0}^{*}).$

Here we write $\Delta=c^{k_{0}}P_{1}^{e_{1}}\cdots P_{s}^{e_{s}}$ to be the
prime factorization of $\Delta$ in $A$, and $c$ is a generator of
$\mathbb{F}_{q}^{*}$. The notation $d:=\deg_{T}\Delta$, and

$k_{0}^{*}=\begin{cases}k_{0},&\text{if $q$ or $d=\deg\Delta$ is even}\\\ \\\
\text{the unique $k\equiv k_{0}+(q-1)/2\mod q-1$ with $0\leqslant
k<q-1$},&\text{otherwise}\end{cases}$

Now we define the natural density of rank-$1$ Drinfeld module with surjective
adelic representation:

######  Definition 3.2.

  1. 1.

For each $x\in\mathbb{R}_{>0}$, let $||\cdot||$ to be the absolute value
$|\cdot|_{\infty}$ of $F_{\infty}$. Define

$B_{1}(x)=\\{\Delta\in A\mid\Delta\neq 0,||\Delta||<x\\}.$

  2. 2.

The natural density of rank-$1$ Drinfeld module with surjective adelic
representation is defined to be

$L_{1}=\lim_{x\rightarrow\infty}\frac{\\#\\{\Delta\in B_{1}(x)\mid{\rm
Im}\rho_{\phi^{\Delta}}=\hat{A}^{*}\\}}{\\#B_{1}(x)}$

######  Theorem 3.3.

The number of $\Delta\in B_{1}(x)$ with ${\rm
Im}\rho_{\phi^{\Delta}}\neq\hat{A}^{*}$ is equal to

$\sum_{2\leqslant t\leqslant q-2,t\mid
q-1}\sum_{i=0}^{\lfloor\frac{\lfloor{\rm
log}_{q}x\rfloor-1}{t}\rfloor}\left(q^{ti+1}\cdot\sum_{j\mid
i}\mu(j)\lfloor\frac{q-1}{tj}\rfloor\right)$

###### Proof.

From Theorem 3.1, those $\Delta\in A$ such that ${\rm
Im}\rho_{\phi^{\Delta}}\neq\hat{A}^{*}$ have the following property:
$\deg_{T}\Delta-1$, $q-1$ and $k_{0}^{*}$ have a nontrivial common divisor.
Thus we know that the possible common divisor can only be those $2\leqslant
t\leqslant q-2$ with $t\mid q-1$. Now we count the number of $\Delta\in
B_{1}(x)$ with ${\rm def}(\rho_{\phi^{\Delta}})=g.c.d.(d-1,q-1,k_{0}^{*})=t.$

We write $\Delta=c^{k_{0}}P_{1}^{e_{1}}\cdots P_{s}^{e_{s}}$. For each $t$, we
consider $0\leqslant i\leqslant\lfloor\frac{\lfloor{\rm
log}_{q}x\rfloor-1}{t}\rfloor$, the number $q^{ti+1}$ represents the number of
monic $\Delta\in B_{1}(x)$ whose degree is equal to $ti+1$, and the summation

$\sum_{j\mid i}\mu(j)\lfloor\frac{q-1}{tj}\rfloor$

represents the choices of $c^{k_{0}}$ such that $k_{0}^{*}$ is a multiple of
$t$ but $k_{0}^{*}/t$ is coprime to $i$. Hence the result follows

∎

######  Corollary 3.4.

$\sum_{2\leqslant t\leqslant q-2,\\\ {t\mid
q-1}}\frac{q-1}{(q^{t}-1)}\geqslant\lim_{x\rightarrow\infty}\frac{\\#\\{\Delta\in
B_{1}(x)\mid{\rm
Im}\rho_{\phi^{\Delta}}\neq\hat{A}^{*}\\}}{\\#B_{1}(x)}\geqslant\sum_{2\leqslant
t\leqslant q-2,\\\ {t\mid q-1}}\frac{q-1}{(q^{t}-1)\cdot t}$

###### Proof.

The inequality is obtained by direct computation and the fact that

$q-1\geqslant\sum_{j\mid
i}\mu(j)\lfloor\frac{q-1}{tj}\rfloor\geqslant\frac{q-1}{t}.$

Therefore, on the right inequality we have

$\lim_{x\rightarrow\infty}\frac{\\#\\{\Delta\in B_{1}(x)\mid{\rm
Im}\rho_{\phi^{\Delta}}\neq\hat{A}^{*}\\}}{\\#B_{1}(x)}\geqslant\sum_{2\leqslant
t\leqslant q-2,\\\ {t\mid
q-1}}\frac{q-1}{t}\cdot\left(\lim_{x\rightarrow\infty}\frac{\sum_{i=0}^{\lfloor\frac{\lfloor{\rm
log}_{q}x\rfloor-1}{t}\rfloor}q^{ti+1}}{q^{\lfloor{\rm
log}_{q}x\rfloor}-1}\right).$

Now we compute the limit

$\lim_{x\rightarrow\infty}\frac{\sum_{i=0}^{\lfloor\frac{\lfloor{\rm
log}_{q}x\rfloor-1}{t}\rfloor}q^{ti+1}}{q^{\lfloor
x\rfloor}-1}=\lim_{x\rightarrow\infty}\frac{q\cdot(q^{t\cdot\lfloor\frac{\lfloor{\rm
log}_{q}x\rfloor-1}{t}\rfloor}-1)}{(q^{t}-1)(q^{\lfloor{\rm
log}_{q}x\rfloor}-1)}=\frac{1}{q^{t}-1},$

hence the right inequality in the statement follows. The left inequality is
derived in the same way so we omit the proof.

∎

######  Remark 3.5.

It is worth to mention that apart from the natural density of elliptic curves
with maximal adelic Galois representation, the above corollary shows that

$L_{1}=1-\lim_{x\rightarrow\infty}\frac{\\#\\{\Delta\in B_{1}(x)\mid{\rm
Im}\rho_{\phi^{\Delta}}\neq\hat{A}^{*}\\}}{\\#B_{1}(x)}\leqslant
1-\sum_{2\leqslant t\leqslant q-2,\\\ {t\mid q-1}}\frac{q-1}{(q^{t}-1)\cdot
t}.$

This implies that NOT almost all rank-$1$ Drinfeld modules over $A$ have
maximal adelic Galois representation.

Combining with the following proposition, we can say that NOT almost all
rank-$r$ Drinfeld modules over $A$ has surjective adelic Galois
representation.

######  Proposition 3.6 ([vdH04], Proposition 7.1).

Let $\phi$ be a Drinfeld $A$-module over $F$ of rank $r$ defined by
$\phi_{T}=T+g_{1}\tau+g_{2}\tau^{2}+\cdots+g_{r}\tau^{r},\ \text{where
$g_{r}\in F^{*}$.}$ Then
$\det\circ\bar{\rho}_{\phi,\mathfrak{a}}=\bar{\rho}_{\psi,\mathfrak{a}}$ where
$\psi$ is the Drinfeld module over $F$ of rank $1$ defined by

$\psi_{T}=T+(-1)^{r-1}g_{r}\tau.$

##  4 Density of rank-$2$ Drinfeld modules

From now on, we make a further assumption that

  1. $\bullet$

$q\geqslant 4$

In this section, let $S$ be a finite set of prime ideals in $A$. We aim to
estimate the natural density of rank-$2$ Drinfeld modules whose
$\mathfrak{l}$-adic Galois representation is large enough.

######  Definition 4.1.

  1. 1.

For each $x\in\mathbb{R}_{>0}$, let $||\cdot||$ to be the absolute value
$|\cdot|_{\infty}$ of $F_{\infty}$. Define

$B_{2}(x)=\\{(g_{1},g_{2})\in A^{2}\mid g_{2}\neq 0,||g_{i}||<x\\}.$

  2. 2.

The natural density of rank-$2$ Drinfeld module with large finite
$\mathfrak{l}$-adic representation is defined to be

$L_{2}^{S}=\lim_{x\rightarrow\infty}\frac{\\#\\{(g_{1},g_{2})\in
B_{2}(x)\mid{\rm Im}\rho_{\phi^{(g_{1},g_{2})},\mathfrak{l}}\supset{\rm
SL}_{2}(A_{\mathfrak{l}})\textrm{ for }\mathfrak{l}\in S\\}}{\\#B_{2}(x)}$

###  4.1 Some group theory

######  Proposition 4.2.

Let $\mathfrak{l}$ be a prime ideal of $A$, and
$H\subset\mathrm{GL}_{2}(A/\mathfrak{l})$. If $H\cap C\neq\emptyset$ for any
$C\subset\mathrm{SL}_{2}(A/\mathfrak{l})$ stable under
$\mathrm{GL}_{2}(\mathbb{F}_{p})$-conjugation. Then
$H\supset\mathrm{SL}_{2}(A/\mathfrak{l}).$

###### Proof.

It is clear that
$H\cap\mathrm{SL}_{2}(\mathbb{F}_{p})\supset\mathrm{SL}_{2}(\mathbb{F}_{p})$.
Hence we know that $H$ is irreducible and stable under
$\mathrm{SL}_{2}(\mathbb{F}_{p})$-conjugation. Moreover, for each $b\in
A/\mathfrak{l}$, we have that

$\left(\begin{array}[]{cc}1&b\\\ 0&1\end{array}\right)\in H.$

Indeed, the statement holds because $H\supset\mathrm{SL}_{2}(\mathbb{F}_{p})$
and the fact that $H\cap C_{b}\neq\emptyset$ where $C_{b}$ is the
$\mathrm{GL}_{2}(\mathbb{F}_{p})$-orbit of $\left(\begin{array}[]{cc}1&b\\\
0&1\end{array}\right).$

Therefore, we know that $H$ is an irreducible subgroup of
$\mathrm{GL}_{2}(A/\mathfrak{l})$ that contains a Sylow-$p$\- subgroup of
$\mathrm{GL}_{2}(A/\mathfrak{l})$. This implies that
$H\supset\mathrm{SL}_{2}(A/\mathfrak{l}).$ ∎

######  Definition 4.3.

For convenience, we denote $\mathfrak{C}$ to be the set of all
$C\subset\mathrm{SL}_{2}(A/\mathfrak{l})$ stable under
$\mathrm{GL}_{2}(\mathbb{F}_{p})$-conjugation. And denote $\mathcal{C}$ to be
the set of all conjugacy classes $C$ in $\mathrm{GL}_{2}(A/\mathfrak{l})$ with
$\det C=\\{1\\}$.

The following result gives a criteria so that we can deduce from
mod-$\mathfrak{l}$ representation toward $\mathfrak{l}$-adic representation.
Here we restrict ourselves to the rank-$2$ case.

######  Theorem 4.4 ([PR09]).

Let $\phi$ be a rank-$2$ Drinfeld module over $A$. Suppose that

  1. 1.

${\rm
Im}\bar{\rho}_{\phi,\mathfrak{l}}\supset\mathrm{SL}_{2}(A/\mathfrak{l})$,

  2. 2.

${\rm Im}\bar{\rho}_{\phi,\mathfrak{l}^{2}}$ contains a non-scalar matrix that
becomes trivial after modulo $\mathfrak{l}$ ,

then the $\mathfrak{l}$-adic Galois representation of $\phi$ satisfies

${\rm Im}\rho_{\phi,\mathfrak{l}}\supset\mathrm{SL}_{2}(A_{\mathfrak{l}}).$

######  Note 4.5.

The assumption $q\geqslant 4$ is required for the Theorem.

###### Proof.

See Proposition 4.1 in [PR09]. Note that the statements in Proposition 4.1
needs a condition on $\det\circ{\rm Im}\bar{\rho}_{\phi,\mathfrak{l}}$ to make
the image of $\mathfrak{l}$-adic representation become
$\mathrm{GL}_{2}(A_{\mathfrak{l}})$. At here we only focus on the relationship
between ${\rm Im}\bar{\rho}_{\phi,\mathfrak{l}}$ and
$\mathrm{SL}_{2}(A_{\mathfrak{l}})$, hence we drop the requirement on
determinant. ∎

Now we fix a prime $\mathfrak{l}\in S$, and compute the natural density

$\bar{L}_{2,\mathfrak{l}}=\lim_{x\rightarrow\infty}\frac{\\#\\{(g_{1},g_{2})\in
B_{2}(x)\mid{\rm
Im}\bar{\rho}_{\phi^{(g_{1},g_{2})},\mathfrak{l}}\not\supset{\rm
SL}_{2}(A/\mathfrak{l})\textrm{ for }\mathfrak{l}\in S\\}}{\\#B_{2}(x)}$

From Proposition 4.2, we have

$\begin{array}[]{lll}\bar{L}_{2,\mathfrak{l}}&\leqslant\lim_{x\rightarrow\infty}\sum_{C_{i}\in\mathfrak{C}}\frac{\\#\\{(g_{1},g_{2})\in
B_{2}(x)\mid{\rm Im}\bar{\rho}_{\phi^{(g_{1},g_{2})}}\cap
C_{i}=\emptyset\\}}{\\#B_{2}(x)}\\\ \\\
&\leqslant\lim_{x\rightarrow\infty}\sum_{C\in\mathcal{C}}M_{\mathfrak{l},C}\cdot\frac{\\#\\{(g_{1},g_{2})\in
B_{2}(x)\mid{\rm Im}\bar{\rho}_{\phi^{(g_{1},g_{2})}}\cap
C=\emptyset\\}}{\\#B_{2}(x)}\end{array}.$

Here $M_{\mathfrak{l},C}$ represents the number of $C_{i}\in\mathfrak{C}$ lies
in a conjugacy class $C$. Therefore, $M_{\mathfrak{l},C}$ is upper bounded by
a constant depending only on $q$ and $\mathfrak{l}$.

###  4.2 Large sieve inequality

In this subsection, we apply function field analogue of large sieve method to
estimate the number $Y_{C}(x)=\\#\\{(g_{1},g_{2})\in B_{2}(x)\mid{\rm
Im}\bar{\rho}_{\phi^{(g_{1},g_{2})}}\cap C=\emptyset\\}$ for $C\in\mathcal{C}$
when $x$ is large.

######  Theorem 4.6 ([Hsu96], Theorem 3.2).

Let $K$ be a positive integer. For each monic irreducible polynomial
$\mathbf{p}\in A$ with $\deg\mathbf{p}\geqslant 1$, let $a_{\mathbf{p}}$ be a
real number such that $0<a_{\mathbf{p}}\leqslant 1$. Set $X\subset A^{2}$ to
be a subset satisfies the following property:

$\\#X_{\mathbf{p}}\leqslant a_{\mathbf{p}}\cdot
q^{2\deg_{T}\mathbf{p}}=a_{\mathbf{p}}\cdot\\#({A^{2}/\mathbf{p}\cdot
A^{2}}),$

where $X_{\mathbf{p}}\subset A^{2}/\mathbf{p}A^{2}$ denotes the canonical
image of $X$ in the quotient group $A^{2}/\mathbf{p}A^{2}$.

Then

$\\#(X\cap B_{2}(x))\cdot L(K)\leqslant q^{2(m_{0}+1)},$

where

$\begin{array}[]{lll}L(K)=1+\sum_{M\in MS,1\leqslant\deg_{T}M\leqslant
K}\prod_{\mathbf{p}\in MI,\mathbf{p}\mid
M}\left(\frac{1-a_{\mathbf{p}}}{a_{\mathbf{p}}}\right),\\\ \\\ MS=\textrm{ all
monic square-free polynomials in }\mathbb{F}_{q}[T],\\\ \\\ MI=\textrm{ all
monic irreducible polynomials with positive degree in}A,\\\ \\\ m_{0}={\rm
sup}\\{{\rm log}\ x,2K-1\\}\end{array}$

In order to apply the large sieve method, set $K=\frac{1}{2}{\rm log}x$ and
$X\cap B_{2}(x)=Y_{C}(x)$. Let $\sum{(\frac{1}{2}{\rm
log}x)}=\\{\mathbf{p}\neq\mathfrak{l}\textrm{ prime ideal of
}A\mid\deg_{T}\mathbf{p}\leqslant\frac{1}{2}{\rm log}x\\}$. For
$\mathbf{p}\in\sum(\frac{1}{2}{\rm log}x)$, define

$\begin{array}[]{lll}\Omega_{\mathbf{p},C}=\\{(\bar{g}_{1},\bar{g}_{2})\in\mathbb{F}_{\mathbf{p}}^{2}\mid\bar{g}_{2}\neq
0,\bar{\rho}_{\phi^{(\bar{g}_{1},\bar{g}_{2})},\mathfrak{l}}({\rm
Frob}_{\mathbf{p}})\in C\\}\\\ \\\
\alpha_{\mathbf{p}}=\frac{\\#\Omega_{\mathbf{p},C}}{|\mathbb{F}_{\mathbf{p}}|^{2}}\end{array}.$

For $\mathbf{p}\not\in\sum(\frac{1}{2}{\rm log}x)$, define
$\alpha_{\mathbf{p}}=0$. Now we examine that $Y_{C}(x)$ satisfies the
assumption of the above theorem. Let $Y_{\mathbf{p},C}$ be the canonical image
of $Y_{C}(x)$ modulo $\mathbf{p}$, then

$Y_{\mathbf{p}}\subset\mathbb{F}_{\mathbf{p}}^{2}-\Omega_{\mathbf{p},C}\Rightarrow\\#Y_{\mathbf{p}}\leqslant(1-\alpha_{\mathbf{p}})|\mathbb{F}_{\mathbf{p}}|^{2}.$

Therefore, Theorem 4.6 implies

$Y_{C}(x)\cdot L(\frac{1}{2}{\rm log}x)\leqslant q^{2({\rm log}x+1)},$

and $L(\frac{1}{2}{\rm log}x)=1+\sum_{M\in
MS,1\leqslant\deg_{T}M\leqslant\frac{1}{2}{\rm log}x}\prod_{\mathbf{p}\in
MI,\mathbf{p}\mid
M}\left(\frac{\alpha_{\mathbf{p}}}{1-\alpha_{\mathbf{p}}}\right)$.

Note that

$L(\frac{1}{2}{\rm log}x)\geqslant\sum_{\mathbf{p}\in\sum(\frac{1}{2}{\rm
log}x)}\frac{\alpha_{\mathbf{p}}}{1-\alpha_{\mathbf{p}}}\geqslant\sum_{\mathbf{p}\in\sum(\frac{1}{2}{\rm
log}x)}\alpha_{\mathbf{p}}.$

The next step is to estimate the number

$\alpha_{\mathbf{p}}=\frac{\\#\Omega_{\mathbf{p},C}}{|\mathbb{F}_{\mathbf{p}}|^{2}}.$

By Corollary of Proposition 7 in [Yu95], we have the following property:

######  Proposition 4.7.

The number of isomorphism classes of rank-$2$ Drinfeld module
$\phi\otimes\mathbb{F}_{\mathbf{p}}$ such that
$\bar{\rho}_{\phi\otimes\mathbb{F}_{\mathbf{p}},\mathfrak{l}}({\rm
Frob}_{\mathbf{p}})\in C$ is equal to $h(\mathcal{O})=\\#{\rm
Pic}(\mathcal{O})$, where $\mathcal{O}$ is an imaginary quadratic order over
$A$ depending only on $C$.

As a corollary, we immediately have

$\\#\Omega_{\mathbf{p},C}=\frac{|\mathbb{F}_{\mathbf{p}}^{*}|}{|\mathcal{O}^{*}|}h(\mathcal{O})$

######  Theorem 4.8.

$\lim_{x\rightarrow\infty}\frac{Y_{C}(x)}{B_{2}(x)}=0.$

###### Proof.

Since $L(\frac{1}{2}{\rm
log}x)\geqslant\sum_{\mathbf{p}\in\sum(\frac{1}{2}{\rm
log}x)}\alpha_{\mathbf{p}}$ and
$\\#\Omega_{\mathbf{p},C}=\frac{|\mathbb{F}_{\mathbf{p}}^{*}|}{|\mathcal{O}^{*}|}h(\mathcal{O})$,
we have

$L(\frac{1}{2}{\rm
log}x)\geqslant\frac{h(\mathcal{O})}{|\mathcal{O^{*}}|}\cdot\sum_{\mathbf{p}\in\sum(\frac{1}{2}{\rm
log}x)}\frac{|\mathbb{F}_{\mathbf{p}}^{*}|}{|\mathbb{F}_{\mathbf{p}}|^{2}}\geqslant\frac{h(\mathcal{O})}{2|\mathcal{O^{*}}|}\cdot\sum_{\mathbf{p}\in\sum(\frac{1}{2}{\rm
log}x)}\frac{1}{|\mathbb{F}_{\mathbf{p}}|}.$

Therefore, we get

$\frac{1}{L(\frac{1}{2}{\rm
log}x)}\leqslant\frac{2|\mathcal{O}^{*}|}{h(\mathcal{O})}\cdot\frac{1}{\sum_{\mathbf{p}\in\sum(\frac{1}{2}{\rm
log}x)}\frac{1}{|\mathbb{F}_{\mathbf{p}}|}}.$

This implies

$Y_{C}(x)\leqslant\frac{x^{2}q^{2}\cdot
2|\mathcal{O}^{*}|}{h(\mathcal{O})}\cdot\frac{1}{\sum_{\mathbf{p}\in\sum(\frac{1}{2}{\rm
log}x)}\frac{1}{|\mathbb{F}_{\mathbf{p}}|}}.$

On the other hand, $\\#B_{2}(x)=(q^{{\rm log}x})^{2}=x^{2}$. Thus

$\lim_{x\rightarrow\infty}\frac{Y_{C}(x)}{B_{2}(x)}\leqslant\lim_{x\rightarrow\infty}\frac{q^{2}\cdot
2|\mathcal{O}^{*}|}{h(\mathcal{O})}\cdot\frac{1}{\sum_{\mathbf{p}\in\sum(\frac{1}{2}{\rm
log}x)}\frac{1}{|\mathbb{F}_{\mathbf{p}}|}}=\frac{q^{2}\cdot
2|\mathcal{O}^{*}|}{h(\mathcal{O})}\cdot\frac{1}{\sum_{\mathbf{p}\neq\mathfrak{l}\textrm{
prime of }A}\frac{1}{|\mathbb{F}_{\mathbf{p}}|}}=0$

∎

As a corollary, we can compute the limit $\bar{L}_{2,\mathfrak{l}}$.

######  Corollary 4.9.

$\bar{L}_{2,\mathfrak{l}}=0.$

###### Proof.

$\bar{L}_{2,\mathfrak{l}}\leqslant\lim_{x\rightarrow\infty}\sum_{C\in\mathcal{C}}M_{\mathfrak{l},C}\cdot\frac{Y_{C}(x)}{B_{2}(x)}\leqslant{\rm
sup}_{C\in\mathcal{C}}\\{M_{\mathfrak{l},C}\\}\cdot\sum_{C\in\mathcal{C}}\lim_{x\rightarrow\infty}\frac{Y_{C}(x)}{B_{2}(x)}=0.$

∎

###  4.3 Toward density of $\mathfrak{l}$-adic representations

In this subsection, we aim at computing the natural density

$L_{2}^{S}=\lim_{x\rightarrow\infty}\frac{\\#\\{(g_{1},g_{2})\in
B_{2}(x)\mid{\rm Im}\rho_{\phi^{(g_{1},g_{2})},\mathfrak{l}}\supset{\rm
SL}_{2}(A_{\mathfrak{l}})\textrm{ for }\mathfrak{l}\in S\\}}{\\#B_{2}(x)}.$

######  Theorem 4.10.

Let $\phi^{(g_{1},g_{2})}$ be a rank-$2$ Drinfeld $A$-module such that ${\rm
Im}\bar{\rho}_{\phi^{(g_{1},g_{2})},\mathfrak{l}}\supset\mathrm{SL}_{2}(\mathbb{F}_{\mathfrak{l}}).$
Suppose further that $g_{2}\not\in\mathfrak{l}$, then the image of mod
$\mathfrak{l}^{2}$ Galois representation
$\bar{\rho}_{\phi^{(g_{1},g_{2})},\mathfrak{l}^{2}}$ contains a non-scalar
matrix which becomes identity after modulo $\mathfrak{l}$.

###### Proof.

The proof is based on Lemma 4.2 and Proposition 4.4 of [PR09].

∎

######  Corollary 4.11.

$\lim_{x\rightarrow\infty}\frac{\\#\\{(g_{1},g_{2})\in B_{2}(x)\mid{\rm
Im}\rho_{\phi^{(g_{1},g_{2})},\mathfrak{l}}\supset{\rm
SL}_{2}(A_{\mathfrak{l}})\\}}{\\#B_{2}(x)}\geqslant
1-\frac{1}{q^{\deg_{T}\mathfrak{l}}}$

###### Proof.

By Theorem 4.10, we have

$\\{(g_{1},g_{2})\in B_{2}(x)\mid{\rm
Im}\rho_{\phi^{(g_{1},g_{2})},\mathfrak{l}}\supset{\rm
SL}_{2}(A_{\mathfrak{l}})\\}\supset\\{(g_{1},g_{2})\in B_{2}(x)\mid{\rm
Im}\bar{\rho}_{\phi^{(g_{1},g_{2})},\mathfrak{l}}\supset{\rm
SL}_{2}(A/\mathfrak{l}),\textrm{ and }g_{2}\not\in\mathfrak{l}\\}.$

Therefore,

$\begin{array}[]{ll}\lim_{x\rightarrow\infty}\frac{\\#\\{(g_{1},g_{2})\in
B_{2}(x)\mid{\rm Im}\rho_{\phi^{(g_{1},g_{2})},\mathfrak{l}}\supset{\rm
SL}_{2}(A_{\mathfrak{l}})\\}}{\\#B_{2}(x)}\\\ \\\ \\\
\geqslant\lim_{x\rightarrow\infty}\frac{\\#\\{(g_{1},g_{2})\in
B_{2}(x)\mid{\rm Im}\bar{\rho}_{\phi^{(g_{1},g_{2})},\mathfrak{l}}\supset{\rm
SL}_{2}(A/\mathfrak{l}),\textrm{ and
}g_{2}\not\in\mathfrak{l}\\}}{\\#B_{2}(x)}\\\ \\\ \\\
\geqslant\lim_{x\rightarrow\infty}\frac{\\#\\{(g_{1},g_{2})\in
B_{2}(x)\mid{\rm Im}\bar{\rho}_{\phi^{(g_{1},g_{2})},\mathfrak{l}}\supset{\rm
SL}_{2}(A/\mathfrak{l})\\}-\\#\\{(g_{1},g_{2})\in B_{2}(x)\mid
g_{2}\not\in\mathfrak{l}\\}}{\\#B_{2}(x)}\\\ \\\ \\\
=1-\lim_{x\rightarrow\infty}\frac{\\#\\{(g_{1},g_{2})\in B_{2}(x)\mid
g_{2}\not\in\mathfrak{l}\\}}{\\#B_{2}(x)}=1-\frac{1}{q^{\deg_{T}\mathfrak{l}}}\end{array}$

∎

As an immediate result, we get

######  Corollary 4.12.

$\displaystyle L_{2}^{S}\geqslant
1+\sum_{i=1}^{|S|}(-1)^{i}\cdot\left(\sum_{\mathfrak{l}_{1},\cdots,\mathfrak{l}_{i}\in
S\textrm{ distinct
}}\frac{1}{q^{\deg_{T}\mathfrak{l}_{1}+\cdots+\mathfrak{l}_{i}}}\right).$

On the other hand, notice that by Proposition 3.6 we have

$\begin{array}[]{cc}\left\\{(g_{1},g_{2})\in B_{2}(x)\mid{\rm
Im}\rho_{\phi^{(g_{1},g_{2})},\mathfrak{l}}\not\subset{\rm
GL}_{2}(A_{\mathfrak{l}})\textrm{ for }\mathfrak{l}\in S\right\\}\\\ \\\
\subset\left\\{(g_{1},g_{2})\in B_{2}(x)\mid{\rm
Im}\rho_{\phi^{(g_{1},g_{2})},\mathfrak{l}}\not\subset{\rm
SL}_{2}(A_{\mathfrak{l}})\textrm{ for }\mathfrak{l}\in
S\right\\}\cup\left\\{(g_{1},-\Delta)\in B_{2}(x)\mid{\rm
Im}\rho_{\phi^{\Delta}}\neq\hat{A}^{*}\right\\}.\end{array}$

Hence Corollary 3.4 and corollary 4.12 imply the following:

######  Corollary 4.13.

$\displaystyle\lim_{x\rightarrow\infty}\frac{\\#\\{(g_{1},g_{2})\in
B_{2}(x)\mid{\rm Im}\rho_{\phi^{(g_{1},g_{2})},\mathfrak{l}}={\rm
GL}_{2}(A_{\mathfrak{l}})\textrm{ for }\mathfrak{l}\in S\\}}{\\#B_{2}(x)}$
$\displaystyle\geqslant
1+\sum_{i=1}^{|S|}(-1)^{i}\cdot\left(\sum_{\mathfrak{l}_{1},\cdots,\mathfrak{l}_{i}\in
S\textrm{ distinct
}}\frac{1}{q^{\deg_{T}\mathfrak{l}_{1}+\cdots+\mathfrak{l}_{i}}}\right)-\sum_{2\leqslant
t\leqslant q-2,{t\mid q-1}}\frac{q-1}{(q^{t}-1)}$

In particular, the natural density of rank-$2$ Drinfeld modules with
surjective $(T)$-adic Galois representation is bounded below by

$\displaystyle\lim_{x\rightarrow\infty}\frac{\\#\\{(g_{1},g_{2})\in
B_{2}(x)\mid{\rm Im}\rho_{\phi^{(g_{1},g_{2})},(T)}={\rm
GL}_{2}(A_{(T)})\\}}{\\#B_{2}(x)}$ $\displaystyle\geqslant
1-\frac{1}{q}-\sum_{2\leqslant t\leqslant q-2,{t\mid
q-1}}\frac{q-1}{(q^{t}-1)}=(q-1)\cdot(\frac{1}{q}-\sum_{2\leqslant t\leqslant
q-2,{t\mid q-1}}\frac{1}{(q^{t}-1)})$

## Acknowledgement

The author would like to thank Professor Fu-Tsun Wei for helpful and inspiring
discussions to carry out this paper.

## References

  * [Che21] Chien-Hua Chen.  Surjectivity of the adelic galois representation associated to a drinfeld module of prime rank, 2021.
  * [Che22] Chien-Hua Chen.  Surjectivity of the adelic Galois representation associated to a Drinfeld module of rank 3.  J. Number Theory, 237:99–123, 2022.
  * [Che23] Chien-Hua Chen.  Masser-wüstholz bound for reducibility of galois representations for drinfeld modules of arbitrary rank, 2023.
  * [CL19] Imin Chen and Yoonjin Lee.  Explicit surjectivity results for Drinfeld modules of rank 2.  Nagoya Math. J., 234:17–45, 2019.
  * [Gek16] Ernst-Ulrich Gekeler.  The Galois image of twisted Carlitz modules.  J. Number Theory, 163:316–330, 2016.
  * [Gos96] David Goss.  Basic structures of function field arithmetic, volume 35 of Ergebnisse der Mathematik und ihrer Grenzgebiete (3) [Results in Mathematics and Related Areas (3)].  Springer-Verlag, Berlin, 1996.
  * [Hsu96] Chih-Nung Hsu.  A large sieve inequality for rational function fields.  J. Number Theory, 58(2):267–287, 1996.
  * [Jon10] Nathan Jones.  Almost all elliptic curves are Serre curves.  Trans. Amer. Math. Soc., 362(3):1547–1570, 2010.
  * [LSTX19] Aaron Landesman, Ashvin Swaminathan, James Tao, and Yujie Xu.  Surjectivity of Galois representations in rational families of abelian varieties.  Algebra Number Theory, 13(5):995–1038, 2019.  With an appendix by Davide Lombardo.
  * [PR09] Richard Pink and Egon Rütsche.  Adelic openness for Drinfeld modules in generic characteristic.  J. Number Theory, 129(4):882–907, 2009.
  * [Ray23] Anwesh Ray.  The $t$-adic galois representation is surjective for a positive density of drinfeld modules, 2023.
  * [vdH04] Gert-Jan van der Heiden.  Weil pairing for Drinfeld modules.  Monatsh. Math., 143(2):115–143, 2004.
  * [Yu95] Jiu-Kang Yu.  Isogenies of Drinfeld modules over finite fields.  J. Number Theory, 54(1):161–171, 1995.
  * [Zyw10] David Zywina.  Elliptic curves with maximal Galois action on their torsion points.  Bull. Lond. Math. Soc., 42(5):811–826, 2010.
  * [Zyw11] David Zywina.  Drinfeld modules with maximal galois action on their torsion points, 2011\.


# On Efficient Uncertainty Estimation for Resource-Constrained Mobile
Applications

Johanna Rock
SPSC Lab
Graz, University of Technology
<EMAIL_ADDRESS>
Tiago Azevedo
Arm ML Research Lab
<EMAIL_ADDRESS>
René de Jong
Arm ML Research Lab
<EMAIL_ADDRESS>
Daniel Ruiz-Muñoz
Arm ML Research Lab
<EMAIL_ADDRESS>
Partha Maji
Arm ML Research Lab
<EMAIL_ADDRESS>
Work performed while the author was at Arm ML Research Lab

###### Abstract

Deep neural networks have shown great success in prediction quality while
reliable and robust uncertainty estimation remains a challenge. Predictive
uncertainty supplements model predictions and enables improved functionality
of downstream tasks including embedded and mobile applications, such as
virtual reality, augmented reality, sensor fusion, and perception. These
applications often require a compromise in complexity to obtain uncertainty
estimates due to very limited memory and compute resources. We tackle this
problem by building upon Monte Carlo Dropout (MCDO) models using the Axolotl
framework; specifically, we diversify sampled subnetworks, leverage dropout
patterns, and use a branching technique to improve predictive performance
while maintaining fast computations. We conduct experiments on (1) a multi-
class classification task using the CIFAR10 dataset, and (2) a more complex
human body segmentation task. Our results show the effectiveness of our
approach by reaching close to Deep Ensemble prediction quality and uncertainty
estimation, while still achieving faster inference on resource-limited mobile
platforms.

##  1 Introduction

Uncertainty estimation and out-of-distribution robustness are vital aspects in
modern deep learning. Predictive uncertainty supplements model predictions and
enables improved functionality of downstream tasks including various resource-
constrained embedded and mobile applications. Popular examples are virtual
reality (VR), augmented reality (AR), sensor fusion, perception, and health
applications including fitness indicators, arrhythmia detection, and skin
lesion detection. Robust and reliable predictions with uncertainty estimates
are increasingly important when operating on noisy in-the-wild data from
sensory inputs. A large variety of deep learning architectures have been
applied to various tasks with great success in terms of prediction quality,
however, producing reliable and robust uncertainty without additional
computational and memory overhead remains a challenge [1]. This issue is
further aggravated due to the limited computational and memory budget
available in typical battery-powered mobile devices.

There exist many approaches towards uncertainty estimation, however, many of
them are complex to train, lack good predictive performance, or are very
resource-intensive [2]. One of the best performing approaches is Deep Ensemble
[3], where multiple models are trained independently and used at inference,
thus making it not viable on mobile platforms with low-latency requirements. A
more resource-efficient approach is Monte Carlo Dropout (MCDO) [4], where
dropout is used during training and inference on a single model, yielding an
approximate distribution of predictions due to dropout-based sampling. MCDO
has been shown to produce less diverse predictions than Deep Ensemble [5] and
therefore achieve poor predictive performance in terms of prediction quality
as well as uncertainty estimates. There are multiple other interesting
approaches [6, 7] for efficient uncertainty estimation, however, in this work
we focus on improving MCDO due to its simplicity and low-memory footprint.

We consider Convolutional Neural Network (CNN) based architectures and aim to
(a) improve MCDO performance through diversification of sampled subnetworks,
and (b) improve latency by exploitation of dropout patterns as well as by
enabling batched computation on branched multi-head models. Diversification of
MCDO subnetworks essentially means diversifying the considered feature
subsets, which results in more diverse predictions that are, in combination,
more robust [5]. Structured dropout in CNNs enables computational optimization
by fusing with convolution layers. Additionally, batched computation can be
applied when using branched partial MCDO, i.e. dropout is applied only to
deeper convolutional layers rather than all layers. In branched partial MCDO,
as depicted in Figure 1, the common backbone is computed once and cached. The
subsequent stochastic model paths are computed individually and in parallel by
feeding in the cached output from the backbone. Note, that all MCDO samples
(subnetworks) are based on the same super network and thus share weights also
in their stochastic paths.

Our experiments were conducted using the Axolotl framework
111https://github.com/ARM-software/fast-probabilistic-models. This paper’s
main contribution is the introduction of a novel approach that (1) improves
predictive performance by diversification of MCDO subnetworks using spatial
dropout and contrastive dropout rates during training and inference, (2)
exploits spatial convolutions and batched computation on branched models
achieving low-latency inference on mobile platforms, and (3) is demonstrated
to benefit uncertainty-aware human body segmentation which is a representative
task for VR/AR on mobile platforms.

Figure 1: A high-level diagram of an Axolotl model with three branches and
shared parameters. During inference, the common and deterministic backbone is
computed and cached; its outputs are connected to each branch, i.e. one
replica of the stochastic portion of the network, which are processed in
parallel.

##  2 Methods

##### Spatial Monte Carlo Dropout (SMCDO)

Structured dropout patterns have been used for regularization of CNNs [8, 9,
10, 11]. The removal of entire features [8] (spatial dropout) or slices of
multiple features [9] for different members (samples/branches) of MCDO during
inference results in a diversification of considered features by definition
and also prevents co-adaptation during training. See Figure 2 for an
illustration of spatial dropout in comparison to randomly dropping
activations. Spatial dropout enables optimized computation by fusing dropout
and convolution operations: instead of setting feature maps to zero (standard
computation), the dropped feature maps are removed completely together with
the corresponding channels in the convolution kernel. The computation is
performed only on the non-dropped features which reduces the number of
operations and thus the latency when used with high dropout rates.

Figure 2: Spatial dropout (right) removes whole feature maps instead of
randomly dropping activations (left).

##### Contrastive dropout rates during training and inference

Dropout during training is typically used for regularization purposes [12].
With large dropout rates during training, there is a large regularization
effect. Thus, the model focuses on the most important rather than the less
informative features, and therefore may potentially replicate important
features while actually preventing feature diversity. SMCDO, however, requires
models that are robust towards random feature removal which can be achieved
when using large dropout rates during training. Therefore, there is a trade-
off between feature diversity and robustness towards random feature removal
for SMCDO that directly relates to the dropout rate during training.

Furthermore, higher dropout rates during inference than during training are
beneficial for SMCDO predictive uncertainty, since this increases the
diversity between SMCDO samples (members/subnetworks).

Therefore, we propose to use contrastive dropout rates where the dropout is
slightly higher during inference than it is during training. In combination
with a good trade-off solution for the train-time dropout rate, contrastive
dropout is beneficial for SMCDO member diversity and yields improved
uncertainty estimation capabilities.

The trade-off relation between train-time and inference-time dropout rates
including their effects on the sample capacity, sample diversity, robustness
against feature removal, and feature diversity are illustrated in Figure 3.

Figure 3: Illustration of the configuration space for train-time and
inference-time dropout including the effects on four aspects that influence
the quality of uncertainty estimates, namely sample capacity, sample
diversity, robustness against feature removal, and feature diversity.

##### Speedup through Branched MCDO

To further improve latency we propose Branched MCDO, depicted in Figure 1 as a
high-level diagram, where a specified number of dropout layers is added to the
Vanilla model starting from the last layer (output). After training, the
network is reshaped such that the deterministic backbone is shared and the
subsequent stochastic portion of the network is replicated once per MCDO
sample (branches). This way, the deterministic computation is performed only
once and intermediate activations are cached. The branches are computed using
batched inference meaning that there is only one forward pass required which
improves latency considerably. Note, that Branched MCDO computation per
definition does not affect predictive performance but instead reduces
computational effort and thus latency.

##  3 Experiments

###  3.1 Robust uncertainty estimation on CIFAR-10

Appendix A details the experimental setup and Appendix C contains additional
experiments showing that higher dropout rates for SMCDO require an increased
model capacity to retain high predictive performance. Based on that, we chose
Wide-ResNet20 [13] with a widening factor $k=3$ for experiments on CIFAR-10.

####  3.1.1 Improved predictive performance using contrastive dropout rates

Figure 4 shows the accuracy and expected calibration error (ECE) for different
train-time dropout rates $\mathrm{DO_{train}}\in\\{10\%,30\%,50\%,70\%\\}$ and
inference-time dropout rates $\mathrm{DO_{inf}}\in\\{0\%,10\%,...,90\%\\}$.
Metrics are computed as the mean over all corruption types for corruption
level 5 (strong corruptions).

With larger dropout rates during training $\mathrm{DO_{train}}$, the accuracy
decreases slightly. With increasing dropout rates during inference, the
accuracy decreases fast with a small $\mathrm{DO_{train}}$, or it decreases
slowly or stays the same with a larger $\mathrm{DO_{train}}$. Also, the ECE
worsens (increases) slowly with larger $\mathrm{DO_{train}}$. However, the ECE
improves (decreases) rapidly with increasing dropout rates during inference.
In particular, the ECE improves considerably, when $\mathrm{DO_{inf}}$ exceeds
$\mathrm{DO_{train}}$ by $10\%$ to $50\%$. Therefore we conclude, that a delta
in train-time and inference-time dropout rates improves the predictive
uncertainty considerably while retaining the accuracy at a high level.

Generally, high dropout rates during inference improve uncertainty estimation
capabilities given that the model is robust towards random feature removal.
High dropout rates during training provide such a robustness towards random
feature removals, however, too large dropout rates during training over-
regularize model parameters and may hurt performance and feature diversity.

Figure 4: Accuracy and ECE for different dropout rates during training and
inference. Results are computed as the mean over all corruption types for
corruption level 5 (strong corruptions).

####  3.1.2 Deep Ensemble performance

Figure 5 shows accuracy and ECE for two selected configurations of SMCDO. The
best performance is achieved by SMCDO with $\mathrm{DO_{train}}=10\%$ and
$\mathrm{DO_{inf}}=30\%$, which has similar or better predictive performance
than Deep Ensemble in terms of both accuracy and ECE. Particularly for inputs
with strong corruption SMCDO yields even better results. SMCDO with
$\mathrm{DO_{train}}=50\%$ and $\mathrm{DO_{inf}}=75\%$ yields slightly worse
results in terms of accuracy and ECE. Note, however, that high dropout rates
enable an optimized computation of fused dropout and convolution operations.
For a dropout rate of 75% our preliminary results indicate a speedup of up to
$3.8\times$ in comparison to standard computation! This configuration is
therefore promising in terms of predictive performance to latency trade-off.

Figure 5: Accuracy and ECE comparison of Vanilla, Deep Ensemble, SMCDO with
$\mathrm{DO_{train}}=10\%$ and $\mathrm{DO_{inference}}=30\%$, and SMCDO with
$\mathrm{DO_{train}}=50\%$ and $\mathrm{DO_{inference}}=75\%$. X-axis shows
different corruption levels and boxplot results contain metrics over all
corruption types.

###  3.2 Uncertainty estimation in human body segmentation on mobile platforms

In this experiment, we use our approach, namely Branched SMCDO with
contrastive dropout rates during training and inference, and demonstrate its
usefulness for human body segmentation. This task (a) benefits from accurate
uncertainty estimates, (b) is relevant for mobile platforms and (c) requires
resource-intensive models such that efficient methods for uncertainty
estimation are required and Deep Ensemble is not a viable alternative due to
limited power and low-latency constraints. See Appendix B for details about
the experimental setup.

##### Predictive performance

Figure 6 shows the dice score versus pixel-wise ECE for different
configurations of Branched SMCDO. Vanilla (Square) and Deep Ensemble (Star)
are included as reference. Circular markers represent our approach, where
color encodes train-time dropout and size corresponds to inference-time
dropout. Good configurations include train-time dropout of $10\%$ and $30\%$,
and inference-time dropout between $10\%$ and $50\%$. The red border marks the
configuration with $10\%$ train-time and $30\%$ inference-time dropout, which
achieves a good trade-off between dice score and ECE. It is considerably
better than Vanilla and close to Deep Ensemble predictive performance while
requiring less memory and latency than the latter. Figure 7 shows exemplary
predictions and their per-pixel uncertainty comparing Vanilla, Deep Ensemble
and our approach.

Figure 6: Dice score versus pixel-wise ECE for human body segmentation.
Circular markers represent SMCDO configurations; color and size correspond to
train-time and inference-time dropout rates, respectively.

Table 1: Latency in seconds and overhead relative to the Vanilla model.
Measurements were taken on an NVIDIA Jetson Nano, executing on a quad-core Arm
Cortex-A57 CPU.

Model | Latency | Overhead | Dice | ECE
---|---|---|---|---
Vanilla | 0.9 | 1 | 0.7905 | 0.0348
Deep Ensemble | 2.7 | 3 | 0.8891 | 0.0073
SMCDO | 2.8 | 3.1 | 0.8551 | 0.013
Branched SMCDO | 1.4 | 1.6 | 0.8551 | 0.013

Figure 7: Exemplary segmentation predictions and uncertainties using Vanilla,
Deep Ensemble and Branched SMCDO.

##### Speedup using Branched SMCDO

Table 1 lists the latency of standard SMCDO in comparison with Branched SMCDO,
where the shared backbone of SMCDO samples is computed only once and used as
input for all branches (deterministic portion of network). Vanilla and Deep
Ensemble latency is included for reference. Branched SMCDO requires only halve
the latency in comparison to standard SMCDO computation or Deep Ensemble!
Note, that additional speedup can be achieved by exploiting sparsity due to
spatial dropout with high dropout rates as mentioned in Section 2.

##  4 Conclusion

We propose an extension of MCDO, namely (a) the use of spatial dropout, (b)
contrastive dropout, and (c) branching. We show, that this combination yields
improved predictive performance and enables speedup due to optimization. We
obtain close to Deep Ensemble performance in terms of accuracy and ECE, but
with lower latency. Experiments on the uncertainty-aware human body
segmentation task show the applicability to a wider range of applications and
low-latency capabilities on mobile platforms.

## References

  * [1] Y. Ovadia, E. Fertig, J. Ren, Z. Nado, D. Sculley, S. Nowozin, J. V. Dillon, B. Lakshminarayanan, and J. Snoek, “Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift,” arXiv preprint arXiv:1906.02530, 2019.
  * [2] M. P. Vadera, A. D. Cobb, B. Jalaian, and B. M. Marlin, “Ursabench: Comprehensive benchmarking of approximate bayesian inference methods for deep neural networks,” arXiv preprint arXiv:2007.04466, 2020.
  * [3] B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and scalable predictive uncertainty estimation using deep ensembles,” arXiv preprint arXiv:1612.01474, 2016.
  * [4] Y. Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” in international conference on machine learning, pp. 1050–1059, PMLR, 2016.
  * [5] S. Fort, H. Hu, and B. Lakshminarayanan, “Deep ensembles: A loss landscape perspective,” arXiv preprint arXiv:1912.02757, 2019.
  * [6] G. Huang, Y. Li, G. Pleiss, Z. Liu, J. E. Hopcroft, and K. Q. Weinberger, “Snapshot ensembles: Train 1, get m for free,” arXiv preprint arXiv:1704.00109, 2017.
  * [7] M. Havasi, R. Jenatton, S. Fort, J. Z. Liu, J. Snoek, B. Lakshminarayanan, A. M. Dai, and D. Tran, “Training independent subnetworks for robust prediction,” arXiv preprint arXiv:2010.06610, 2020.
  * [8] J. Tompson, R. Goroshin, A. Jain, Y. LeCun, and C. Bregler, “Efficient object localization using convolutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 648–656, 2015.
  * [9] P. Notin, A. N. Gomez, J. Yoo, and Y. Gal, “Improving compute efficacy frontiers with sliceout,” arXiv preprint arXiv:2007.10909, 2020.
  * [10] G. Ghiasi, T.-Y. Lin, and Q. V. Le, “Dropblock: A regularization method for convolutional networks,” arXiv preprint arXiv:1810.12890, 2018.
  * [11] H. Pham and Q. V. Le, “Autodropout: Learning dropout patterns to regularize deep networks,” arXiv preprint arXiv:2101.01761, vol. 1, no. 2, p. 3, 2021\.
  * [12] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research, vol. 15, no. 56, pp. 1929–1958, 2014.
  * [13] S. Zagoruyko and N. Komodakis, “Wide residual networks,” arXiv preprint arXiv:1605.07146, 2016.
  * [14] T. Azevedo, R. de Jong, M. Mattina, and P. Maji, “Stochastic-yolo: Efficient probabilistic object detection under dataset shifts,” arXiv preprint arXiv:2009.02967, 2020.
  * [15] A. Paszke, A. Chaurasia, S. Kim, and E. Culurciello, “Enet: A deep neural network architecture for real-time semantic segmentation,” 2016.

##  Appendix A Experimental setup for CIFAR-10

We use the CIFAR-10 data set for training, and additionally the corrupted
CIFAR-10 test set for evaluation, including 19 corruption types and 5
corruption levels. We use SGD for 200 epochs of training and standard data
augmentation. We use a standard step-wise learning rate scheduler, i.e. at
epochs 1, 80, 120, 160, 180 with learning rates 0.1, 0.01, 0.001, 0.0001,
0.0005. For SMCDO we use Wide-ResNet-20-k with a widening factor
$\mathrm{k}=3$ and $M=3$ samples. Dropout is added to approximately the second
halve of the model (Convolution layer 13 and deeper), since previous
experiments have shown that this configuration yields a good performance to
latency trade-off [14]. Dropout layer are always positioned before a
convolution layer. As baselines we use Vanilla ResNet-20 and Deep Ensemble
with ResNet-20 and $M=3$ members. We used the Axolotl framework
222https://github.com/ARM-software/fast-probabilistic-models and Keras for
implementation.

##  Appendix B Experimental setup for human body segmentation

We use a human body segmentation data set
333https://github.com/VikramShenoy97/Human-Segmentation-Dataset. and ENet [15]
model architecture. For training we use the Adam optimizer for 300 epochs and
resize inputs to $640\times 640$ pixels. Dropout is added to the layers from
Bottleneck40 and above, where dropout is always added before a convolution
layer. We use $M=3$ members for SMCDO. As baselines we use Vanilla ENet and
Deep Ensemble with ENet and $M=3$ members. Models were evaluated on NVIDIA
Jetson Nano executing on a quad-core Arm Cortex-A57 CPU for latency
benchmarks. We used the Axolotl framework 2 and Keras.

##  Appendix C Increasing model capacity for higher dropout rates

We analyze the relation between model capacity, hence the widening factor k,
the SMCDO dropout rate and the predictive performance. Figure 8 shows the
accuracy and ECE for different widening factors $\mathrm{k}=\\{1,2,3\\}$ and
dropout rates $\mathrm{DO}=\\{5\%,10\%,30\%,50\%\\}$. Metrics are computed as
the mean over all corruption types for corruption level 5 (strong
corruptions). The results show that higher dropout rates require larger model
capacity to retain accuracy, while slightly higher dropout rates have positive
effects on the ECE (e.g. $\mathrm{DO}=10\%$). Therefore, we chose a widening
factor $k=3$ for all SMCDO experiments using ResNet20 and CIFAR-10. Note that
a widening factor $k=3$ increases the number of channels and thus requires
more memory; the latency may or may not be affected, depending on the target
device.

Figure 8: Accuracy and ECE for different widening factors k and dropout rates
DO. Results are computed as the mean over all corruption types for corruption
level 5 (strong corruptions).