# Invariant differential operators and the generalized symmetric group

Ibrahim Nonkané, Latévi M. Lawson  Departement d’économie et de mathématiques
appliquées, IUFIC, Université Thomas Sankara, Burkina faso
<EMAIL_ADDRESS>African Institute for Mathematical Sciences (AIMS),
Summerhill Estates, East Legon Hills, Santoe, Accra, Ghana P.O. Box LG DTD
20046, Legon, Accra, Ghana<EMAIL_ADDRESS>
###### Abstract.

In this paper we study the decomposition of the direct image of $\pi_{+}({\cal
O}_{X})$ the polynomial ring ${\cal O}_{X}$ as a $\mathcal{D}$-module, under
the map $\pi:\operatorname{spec}{\cal O}_{X}\to\operatorname{spec}{\cal
O}_{X}^{G(r,n)}$, where ${\cal O}_{X}^{G(r,n)}$ is the ring of invariant
polynomial under the action of the wreath product
$G(r,p):={\mathbb{Z}}/r{\mathbb{Z}}\wr{{\cal S}}_{n}$. We first describe the
generators of the simple components of $\pi_{+}({\cal O}_{X})$ and give their
multiplicities. Using an equivalence of categories and the higher Specht
polynomials, we describe a $\mathcal{D}$-module decomposition of the of the
polynomial ring localized at the discriminant of $\pi$. Furthermore, we study
the action invariants differential operators on the higher Specht polynomials.

keywords: Direct image, Differential structure, complex reflection groups,
Representation theory, wreath product, Higher Specht polynomials, Primitive
idempotents, Reflection groups, Young diagram.

Mathematics Subject Classification: Primary 13N10, Secondary 20C30.

##  1\. Introduction

A fundamental problem in representation theory is the description of all
irreducible representations. We are interested in the polynomial
representation the invariant differential operators with respect to the
generalized symmetric group. We know that the direct image of a simple module
for a proper map in semi-simple by the Decomposition Theorem [5]. The simplest
case is when the map $\pi:X=\operatorname{spec}B\to Y=\operatorname{spec}A$ is
finite, in which case it is easy to give an elementary and wholly algebraic
proof, using essentially the (generic) correspondence with the differential
Galois group, which equals the ordinary Galois group $G$. The irreducible
submodule of the direct image are in one-to-one correspondence with the
irreducible representations of $G$ (see [9]). In this paper we make the
differential structure more explicit in the case of the invariants of the
complex reflection group $G(r,n),\ B=\mathbb{C}[x_{1},\ldots,x_{n}]\subset
A=\mathbb{C}[x_{1},\ldots,x_{n}]^{G(r,n)}.$ We explicitly study the simple
component of the direct image $\pi_{+}({\cal O}_{X})$ of the polynomial ring
${\cal O}_{X}$ as a $\mathcal{D}$-module under the map
$\pi:\operatorname{spec}{\cal O}_{X}\to\operatorname{spec}{\cal O}_{Y}$ where
${\cal O}_{Y}={\cal O}_{X}^{G(r,n)};$ the ring of invariant polynomials under
the action of ${G(r,n)}.$ We describe the generators of the simple components
of $\pi_{+}({\cal O}_{X})$ and their multiplicities as in [9]. We thus
establish the decomposition structure of $\pi_{+}({\cal O}_{X})$ by means of
the higher Specht polynomials. This proof uses the fact that the irreducible
$\mathcal{D}$-submodules of $\pi_{+}({\cal O}_{X})$ are in one-to-one
correspondence with irreducible representations of $G(r,n)$.
Secondly we elaborate a $\mathcal{D}$-module decomposition of the polynomial
ring localized at the discriminant of $\pi.$
Finally we describe the action of invariant differential operators on higher
Specht polynomials. The higher Specht polynomials (introduced combinatorially
[1]), are adapted to the $\mathcal{D}$-module structure.
This paper generalizes results on modules over the Weyl algebra appeared in
[9] and [11]. The case $r=2$ have been presented at 10th International
Conference on Mathematical Modeling in Physical Sciences in order to describe
the action of the rational Olshanetsky-Perelomov operator Hamiltonian on
polynomials [12].

##  2\. Preliminaries

###  2.1. Direct image

We briefly recall the definition of the direct image of a $\mathcal{D}$-module
[4].
Let $K$ be a field of characteristic zero, put $X=K^{n}$. The polynomial ring
$K[x_{1},\ldots,x_{n}]$ will be denoted by $K[X];$ and the Weyl algebra
generated by $x_{i}$’s and $\frac{\partial}{\partial x_{i}}$’s by
$\mathcal{D}_{X}$. The $n$-tuple $(x_{1},\ldots,x_{n})$ will be denoted by
$X$. Similar conventions will holds for $Y=K^{m}$, with polynomial ring $K[Y]$
and Weyl algebra $\mathcal{D}_{Y}$.

Let $\pi:X\to Y$ be a polynomial map, with $\pi=(\pi_{1},\ldots,\pi_{m})$. Let
$M$ be a left $\mathcal{D}_{Y}$-module. The inverse image of $M$ under the map
$\pi$ is $\pi^{+}(M)=K[X]\otimes_{K[Y]}M.$ This is a $K[X]$-module. It becomes
a $\mathcal{D}_{X}$-module with $\partial_{x_{i}}$ acting according to the
formula

$\frac{\partial}{\partial x_{i}}(h\otimes u)=\frac{\partial
h}{\partial{x_{i}}}\otimes u+\sum_{j=1}^{m}\frac{\partial\pi_{j}}{\partial
x_{i}}\otimes\frac{\partial}{\partial y_{j}}u,\ h\in K[X],u\in M.$

Since $\mathcal{D}_{Y}\otimes_{\mathcal{D}_{Y}}M\cong M,$ we have that

$\pi^{+}(M)\cong
K[X]\otimes_{K[Y]}\mathcal{D}_{Y}\otimes_{\mathcal{D}_{Y}}M=\pi^{+}(K[Y])\otimes_{\mathcal{D}_{Y}}M.$

Writing $D_{X\to Y}$ for $\pi^{+}(K[Y])$, on has that
$\pi^{+}(M)=\mathcal{D}_{X\to Y}\otimes_{\mathcal{D}_{Y}}M.$ Note that
$\mathcal{D}_{X\to Y}$ is $\mathcal{D}_{X}$\- $\mathcal{D}_{Y}$-bimodule.

Let $N$ be a right $D_{X}$-module. The tensor product

$\pi_{+}(N)=N\otimes_{\mathcal{D}_{X}}\mathcal{D}_{X\to Y}$

is a right $\mathcal{D}_{Y}$-module, which is called the direct image of $N$
under the polynomial map $\pi$. Let us consider the standard transposition
$\tau:\mathcal{D}_{X}\to\mathcal{D}_{X}$ defined by
$\tau(h\partial^{\alpha})=(-1)^{|\alpha|}\partial^{\alpha}h,$ where $h\in
K[X]$ and $\alpha\in\mathbb{N}^{n}.$ If $N$ is a right
$\mathcal{D}_{X}$-module then we define a left $\mathcal{D}_{X}$-module
$N^{t}$ as follows. As an abelian group, $N^{t}=N.$ If $a\in\mathcal{D}_{X}$
and $u\in N^{t}$ then the left action of $a$ on $u$ is defined by $a\star
u=u\tau(a).$ Using the standard transposition for $\mathcal{D}_{Y}$ and
$\mathcal{D}_{X}$, put $D_{Y\leftarrow X}=(D_{X\to Y})^{t}$, this is a
$\mathcal{D}_{Y}$-$\mathcal{D}_{X}$-bimodule. Let $M$ be a left
$\mathcal{D}_{X}$-module. The direct image of $M$ under $\pi$ is defined by
the formula

$\pi_{+}(M)=D_{Y\leftarrow X}\otimes_{\mathcal{D}_{X}}M.$

It is clear that $\pi_{+}(M)$ is a $\mathcal{D}_{Y}$-module.
The following is the Kashiwara decomposition theorem

######  Theorem 2.1.

[5] Let $\pi:X\to Y$ be a polynomial map. If $M$ is a a simple (holonomic)
module over $\mathcal{D}_{X}$. Then $\pi_{+}(M)$ is a semisimple
$\mathcal{D}_{Y}$-module. we have

$\pi_{+}(M)=\oplus M_{i}^{\alpha_{i}},$

where the $M_{i}$ are inequivalent irreducible $\mathcal{D}_{Y}$-submodules.

###  2.2. Higher Specht Polynomials for Reflections group

In this subsection we recall some facts about the representation of the
generalized symmetric group $G(r,n)$ [1]. Let ${{\cal S}}_{n}$ be the group of
permutations of the set $\\{1,\ldots,n\\}$ and ${\mathbb{Z}}/r{\mathbb{Z}}$ be
the cyclic group of order $r$.
The generalized symmetric group $G(r,n)$ is the semi-direct product of
$({\mathbb{Z}}/r{\mathbb{Z}})^{n}$ with ${{\cal S}}_{n}$, written as
$({\mathbb{Z}}/r{\mathbb{Z}})^{n}\rtimes{{\cal S}}_{n}$, where
$({\mathbb{Z}}/r{\mathbb{Z}})^{n}$ is the direct product of $n$ copies of
${\mathbb{Z}}/r{\mathbb{Z}}$. Let $\xi$ be a primitive $r$-th root of 1.
$({\mathbb{Z}}/r{\mathbb{Z}})^{n}\rtimes{{\cal
S}}_{n}=\\{(\xi^{i_{1}},\ldots,\xi^{i_{n}};\sigma)|\ i_{k}\in\mathbb{N},\
\sigma\in{{\cal S}}_{n}\\}$, whose product is given by

$(\xi^{i_{1}},\ldots,\xi^{i_{n}};\sigma)(\xi^{j_{1}},\ldots,\xi^{j_{n}};\pi)=(\xi^{i_{1}+j_{\sigma^{-1}(1)}},\ldots,\xi^{i_{n}+j_{\sigma^{-1}(n)}},;\sigma\pi).$

Let ${\cal O}_{X}=\mathbb{C}[x_{1},\ldots,x_{n}]$ be the ring of polynomials
in $n$ indeterminates on which the group $G(r,n)$ acts as follows:

$(\xi^{i_{1}},\ldots,\xi^{i_{n}};\sigma)f=f(\xi^{i_{\sigma(1)}}x_{\sigma(1)},\ldots,\xi^{i_{\sigma(n)}}x_{\sigma(n)};\sigma),$

where $f\in{\cal O}_{X}$ and $(\xi^{i_{1}},\ldots,\xi^{i_{n}};\sigma)\in
G(r,n)$. It is known that the fundamental invariants under this action are
given by the elementary symmetric functions
$e_{j}(x_{1}^{r},\ldots,x_{n}^{r}),\ 1\leq j\leq n.$ Let $J_{+}$ be the ideal
of ${\cal O}_{X}$ generated by these fundamental invariants and $\Lambda={\cal
O}_{X}/J_{+}$ be the quotient ring. It is also known that the $G(r,n)$-module
$\Lambda$ is isomorphic to the group ring $\mathbb{C}[G(r,n)]$, namely the
left regular representation. A description of all irreducible components of
$\Lambda$ is known in [1], in terms of what is called ”higher Specht
polynomials”. The irreducible representation of $G(r,n)$ are parametrized by
the $r$-tuple of Young diagrams $(\lambda^{1},\ldots,\lambda^{r})$ with
$|\lambda^{1}|+\cdots+|\lambda^{r}|=n.$

Let $\mathcal{P}_{r,n}$ be the set of $r$-tuples of Young diagrams
${\bf\lambda}=(\lambda^{1},\ldots,\lambda^{r})$ with
$|\lambda^{1}|+\cdots+|\lambda^{r}|=n$. By filling each cell with a positive
integer in such a way that every $j\ (1\leq j\leq n)$ occurs once, we obtain
an $r$-tableau $T=(T^{1},\ldots,T^{r})$ of shape
${\bf{\bf\lambda}}=(\lambda^{1},\ldots,\lambda^{r})$. When the number $k$
occurs in the component $T^{i}$, we write $k\in T^{i}$. The set of
$r$-tableaux of shape ${\bf\lambda}$ is denoted by
${\mbox{Tab}}({\bf\lambda}).$ An $r$-tableau $T=(T^{1},\ldots,T^{r})$ is said
to be standard if the numbers are increasing on each column and each row of
$T^{\nu}\ (1\leq\nu\leq r)$. The set of $r$-standard tableaux of shape
${\bf\lambda}$ is denoted by ${\mbox{STab}}({\bf\lambda}).$

Let $S=(S^{1},\ldots,S^{r})\in{\mbox{STab}}({\bf\lambda})$. We associate a
word $w(S)$ in the following way. First we read each column of the component
$S^{1}$ from the bottom to the top starting from the left. We continue this
procedure for the tableau $S^{2}$ and so on. For word $w(S)$ we define index
$i(w(S))$ inductively as follows. The number 1 in the word $w(S)$ has the
index $i(1)=0.$ If the number $k$ has index $i(k)=p$ and the number has number
$k+1$ is sitting on the left (resp. right) of $k,$ then $k+1$ has index $p+1$
(resp. $p$). Finally, assigning the indices to the corresponding cells, we get
a shape ${\bf{\bf\lambda}}=(\lambda^{1},\ldots,\lambda^{r})$, each cell filled
with a nonnegative integer, which is denoted by
$i(S)=(i(S)^{1},\ldots,i(S)^{r}).$

Let $T=(T^{1},\ldots,T^{r})$ be an $r$-tableau of shape ${\bf\lambda}$. For
each component $T^{\nu}\ (1\leq\nu\leq r),$ the Young symmetrizer
${\mathbf{e}}_{T^{\nu}}$ of $T^{\nu}$ is defined by

${\mathbf{e}}_{T^{\nu}}=\frac{1}{\alpha_{T^{\nu}}}\sum_{\sigma\in R(T^{\nu})\
\tau\in C(T^{\nu})}\operatorname{sgn}(\tau)\tau\sigma,$

where $\alpha_{T^{\nu}}$ is the product of the hook lengths for the shape
$\lambda^{\nu}$, $R(T^{\nu})$ and $C(T^{\nu})$ are the row-stabilizer and
colomn-stabilizer of $T^{\nu}$ respectively.
We may regard a tableau $T$ on a Young diagram $\lambda$ as a map

$T:\\{\mbox{cells\ of}\ \lambda\\}\to{\mathbb{Z}}_{\geq 0},$

which assigns to a cell $\xi$ of $\lambda$ the number $T(\xi)$ written in the
cell $\xi$ in $T$.
For $S\in{\mbox{STab}}({\bf\lambda})$ and $T\in{\mbox{Tab}}({\bf\lambda})$,
Ariki, Terasoma and Yamada in [1] defined the higher Specht polynomial for
$G(r,n)$ by

$F_{T}^{S}=\prod_{\nu=1}^{r}\bigg{(}{\mathbf{e}}_{T^{\nu}}(x_{T^{\nu}}^{ri(S)^{\nu}})\prod_{k\in
T^{\nu}}x_{k}^{\nu}\bigg{)},$

where

$\displaystyle
x_{T^{\nu}}^{ri(S)^{\nu}}=\prod_{\xi\in\lambda^{\nu}}x_{T^{\nu}(\xi)}^{ri(S)^{\nu}(\xi)}.$

The following is the fundamental result in [1] on the higher Specht
polynomials for $G(r,n).$

######  Theorem 2.2.

  1. (1)

The space $\displaystyle
V_{S}(\lambda)=\sum_{T\in{\mbox{Tab}}(\lambda)}\mathbb{C}F_{T}^{S}$ affords an
irreducible representation of the reflection group $G(r,n)$.

  2. (2)

The set $\\{F_{T}^{S}\ |\ T\in{\mbox{STab}}({\bf\lambda})\\}$ gives a basis
over $\mathbb{C}$ for $V_{S}({\bf\lambda}).$

  3. (3)

For $S_{1}\in{\mbox{STab}}({\bf\lambda})$ and $S_{2}\in{\mbox{STab}}(\mu)$,
the representation $V_{S_{1}}({\bf\lambda})$ and $V_{S_{2}}(\mu)$ are
isomorphic if and only if $S_{1}$ and $S_{2}$ has the same shape, i.e.
$\lambda=\mu.$

  4. (4)

We have the irreducible decomposition

$\mathbb{C}[G(r,n)]=\bigoplus_{{\bf\lambda}\in\mathcal{P}_{r,n}}\bigoplus_{S\in{\mbox{STab}}({\bf\lambda})}V_{S}(\lambda)$

as representation of $G(r,n)$.

######  Theorem 2.3.

The higher Specht polynomials in ${{\cal
F}}=\\{F_{T}^{S};S,T\in{\mbox{STab}}(\lambda),\lambda\vdash n\\}$ form a basis
of the $\mathbb{C}[x_{1},...,x_{n}]^{G(r,n)}$-module
$\mathbb{C}[x_{1},...,x_{n}]$

##  3\. Decomposition Theorem

We are interested in studying the decomposition structure of $\pi_{+}(M)$,
where $M={\cal O}_{X},\ \pi:X=\operatorname{spec}({\cal O}_{X})\to
Y=\operatorname{spec}({\cal O}_{X}^{G(r,n)})$. Since ${\cal O}_{X}$ is a
holonomic $\mathcal{D}_{X}$-module [4, Chapter 10], $\pi_{+}({\cal O}_{X})$ is
a semisimple $\mathcal{D}_{Y}$-module by the Kashiwara decomposition theorem.
We construct the simple components of $\pi_{+}({\cal O}_{X})$ and provide
their multiplicities. Let us recall some useful facts from [9].

Let $\Delta:=Jac(\pi)$ be the Jacobian of $\pi$, $\Delta^{2}$ the discriminant
of $\pi$ we denote the complement of the branch locus and the discriminant by
$U$ and $V$, respectively. Assume now that $U,V$ are such that the respective
canonical modules are generated by volume forms $dx$, and $dy$, related by
$dx=\Delta dy$, where $\Delta$ is the Jacobian of $\pi$.

######  Proposition 3.1.

  * (i)

There is an isomorphism of $\mathcal{D}_{V}$-modules

$T:\pi_{+}(O_{U})\cong O_{U},\quad r(dy^{-1}\otimes dx)\mapsto r\Delta^{-1}.$

  * (ii)

$T(\pi_{+}(O_{X}))$ is isomorphic as a $\mathcal{D}_{Y}$-module to
$\pi_{+}({\cal O}_{X})$.

###### Proof.

See [9, Lemma 2.3]. ∎

It is more convenient to study $T(\pi_{+}(O_{X}))\cong\pi_{+}(O_{X})$, as a
submodule of $O_{U}$, than using the definition of $\pi_{+}(O_{X})$. Therefore
to reach our goal, we will first study the decomposition of $O_{U}$ into
irreducible components as a $\mathcal{D}_{V}$-module.
The following proposition enables us to reduce the study of the decomposition
factors of $\pi_{+}(O_{X})$ to the behavior of the direct image over the
complement to the branch locus, or even over the generic point. Let
$j:U\hookrightarrow X$ and $i:V\hookrightarrow Y$ be the inclusions.

######  Proposition 3.2.

Let $\pi:X\to Y$ be a finite map. Then

  * (i)

$\pi_{+}(O_{X})$ is semi-simple as a $\mathcal{D}_{Y}$-module.

  * (ii)

If $\pi_{+}(O_{X})=\oplus M_{k},\ k\in I$ is a decomposition into simple (non-
zero) $D_{Y}$-modules, then $\pi_{+}(O_{U})=\oplus i^{+}(M_{k}),\ k\in I$, is
a decomposition of $\pi_{+}(O_{U})$ into simple (non-zero)
$\mathcal{D}_{V}$-modules.

###### Proof.

See [9, Proposition 2.8]. ∎

###  3.1. Action description

As we want to study the polynomial representation of a ring of invariant
differential operators localized at $\Delta^{2},$ it is convenient to
precisely describe the action of that ring on the polynomials ring.

Let $\mathcal{D}_{X}=\mathbb{C}\langle
x_{1},\ldots,x_{n},\frac{\partial}{\partial
x_{1}},\ldots,\frac{\partial}{\partial x_{n}}\rangle$ be the ring of
differential operators associated with the polynomial ring ${\cal
O}_{X}=\mathbb{C}[x_{1},\ldots,x_{n}]$, and ${\cal
O}_{Y}=\mathbb{C}[x_{1},\ldots,x_{n}]^{G(r,n)}=\mathbb{C}[y_{1},\ldots,y_{n}]$
be the ring of invariant polynomials under the real reflection group $G(r,n).$

We denote by $\mathcal{D}_{Y}=\mathbb{C}\langle
y_{1},\ldots,y_{n},\frac{\partial}{\partial
y_{1}},\ldots,\frac{\partial}{\partial y_{n}}\rangle$ the ring of differential
operators associated with ${\cal O}_{Y}=\mathbb{C}[y_{1},\ldots,y_{n}]$. By
[8], $\mathcal{D}_{Y}$ is the ring of invariant differential operators under
the action of the reflection group $G(r,n)$.

Notations We adopt the following notations

$\displaystyle{\mathcal{O}}_{U}:=\mathbb{C}[x_{1},\ldots,x_{n},\Delta^{-1}],\
{\cal O}_{V}:=\mathbb{C}[y_{1},\ldots,y_{n},\Delta^{-2}],\
{\mathcal{D}}_{V}:=\mathbb{C}\langle
y_{1},\ldots,y_{n},\frac{\partial}{\partial
y_{1}},\ldots,\frac{\partial}{\partial y_{n}},\Delta^{-2}\rangle.$

######  Lemma 3.3.

${\mathcal{O}}_{U}$ is a ${\mathcal{D}}_{V}$-module.

###### Proof.

Let us make clear the action of ${\mathcal{D}}_{V}$ on ${\mathcal{O}}_{U}.$
Let $\displaystyle A=\displaystyle\bigg{(}\frac{\partial y_{j}}{\partial
x_{i}}\bigg{)}_{1\leq i,j\leq n}$ so that $\det(A)=\Delta$. We get the
following equation

$\left(\begin{array}[]{ccc}\frac{\partial}{\partial x_{n}}\\\ \vdots\\\
\frac{\partial}{\partial
x_{n}}\end{array}\right)=A\left(\begin{array}[]{ccc}\frac{\partial}{\partial
y_{1}}\\\ \vdots\\\ \frac{\partial}{\partial y_{n}}\end{array}\right).$

It follows that

$\left(\begin{array}[]{ccc}\frac{\partial}{\partial y_{1}}\\\ \vdots\\\
\frac{\partial}{\partial
y_{n}}\end{array}\right)=A^{-1}\left(\begin{array}[]{ccc}\frac{\partial}{\partial
x_{n}}\\\ \vdots\\\ \frac{\partial}{\partial x_{n}}\end{array}\right)$

and it is now clear that ${\mathcal{O}}_{U}$ is a ${\mathcal{D}}_{V}$-module.
∎

We know that ${\mathcal{O}}_{U}$ a ${\mathcal{D}}_{V}$\- semisimple module.
what are the simple components of ${\mathcal{O}}_{U}$ as
${\mathcal{D}}_{V}$-module and their multiplicities?

###  3.2. Simple components and their multiplicities

In this section, we state our main result. We use the representation theory of
the generalized symmetric group $G(r,n)$ to yield results on modules over the
ring of differential operators. It is well-known that

${\cal O}_{X}=\mathbb{C}[G(r,n)]\otimes{\cal O}_{Y}\ \mbox{as }\ {\cal
O}_{Y}\mbox{-modules}.$

Let us consider the multiplicative closed set
$S=\\{\Delta^{k}\\}_{k\in\mathbb{N}}\subset{\cal O}_{X}.$ It follows that:

$S^{-1}{\cal O}_{X}=\mathbb{C}[G(r,n)]\otimes S^{-1}{\cal O}_{Y}\ \mbox{as }\
S^{-1}{\cal O}_{Y}\mbox{-modules}.$

where $S^{-1}{\cal O}_{X}$ and $S^{-1}{\cal O}_{Y}$ are the localizations of
${\cal O}_{X}$ and ${\cal O}_{Y}$ at $S$ respectively. But $S^{-1}{\cal
O}_{X}={\mathcal{O}}_{U}$ and $S^{-1}{\cal O}_{Y}={\cal O}_{V}$ , whereby we
get

${\mathcal{O}}_{U}=\mathbb{C}[G(r,n)]\otimes{\cal O}_{V}\ \mbox{as}\
\mathbb{C}[G(r,n)]\mbox{-modules}.$

######  Lemma 3.4.

There exists an injective map

$\mathbb{C}[G(r,n)]\hookrightarrow\operatorname{Hom}_{\mathbb{C}}({\mathcal{O}}_{U},{\mathcal{O}}_{U}).$

###### Proof.

The $\mathbb{C}[G(r,n)]$-module $\mathbb{C}[G(r,n)]$ acts faithfully on itself
by multiplication, and this multiplication yields an injective map
$\mathbb{C}[G(r,n)]\hookrightarrow\operatorname{Hom}_{\mathbb{C}}\bigg{(}\mathbb{C}[G(r,n)],\mathbb{C}[G(r,n)]\bigg{)}.$
Since ${\cal O}_{V}$ is invariant under this action of $\mathbb{C}[G(r,n)]$,
we get the expected injective map. ∎

######  Proposition 3.5.

There exists an injective map

$\mathbb{C}[G(r,n)]\hookrightarrow\operatorname{Hom}_{\mathcal{D}_{V}}({\mathcal{O}}_{U},{\mathcal{O}}_{U}).$

###### Proof.

Since $\mathcal{D}_{V}=\mathbb{C}\langle y_{1},\ldots,y_{n},\partial
y_{1},\ldots,\partial y_{n},\Delta^{-2}\rangle$, we only need to show that
every element of $\mathbb{C}[G(r,n)]$ commutes with
$y_{1},\ldots,y_{n},\partial y_{1},\ldots,\partial y_{n}.$

  1. $\bullet$

It is clear that every element of $\mathbb{C}[G(r,n)]$ commutes with $y_{i},\
i=1,\ldots,n$.

  2. $\bullet$

Let us show that every element of $\mathbb{C}[G(r,n)]$ commutes with $\partial
y_{i},\ i=1,\ldots,n$. Let ${\bf D}$ be a derivation on the field
$K=\mathbb{C}(y_{1},\ldots,y_{n})$ of fractions of ${\cal O}_{V}$, then
$(K,{\bf D})$ is a differential field. Let $L=\mathbb{C}(x_{1},\ldots,x_{n})$
be the field of fractions of ${\mathcal{O}}_{U}$. We have that $K=L^{G(r,n)}$
is the fixed field and $L$ is a Galois extension of $K$, with Galois group
$G(r,n)$. Then by [3, Théorème 6.2.6] there exists a unique derivation on $L$
which extends ${\bf D},$ then $(L,{\bf D})$ is also a differential ring. In
this way, $\sigma^{-1}{\bf D}\sigma={\bf D}$ for every $\sigma\in G(r,n).$
Therefore $\sigma{\bf D}={\bf D}\sigma$ and $\sigma$ commute with ${\bf D}.$

∎

######  Corollary 3.6.

$\mathbb{C}[G(r,n)]\cong\operatorname{Hom}_{{\mathcal{D}}_{V}}({\mathcal{O}}_{U},{\mathcal{O}}_{U})$

###### Proof.

see [9, Corollary 26 ] ∎

Before we state our main result, let us recall some facts.
By Maschke’s Theorem [7, Chap XVIII], we know that $\mathbb{C}[G(r,n)]$ is a
semi-simple ring, and

$\mathbb{C}[G(r,n)]=\displaystyle\bigoplus_{\lambda\in{\cal
P}_{r,n}}R_{\lambda},$

where ${\cal P}_{r,n}$ be the set of $r$-tuples of Young diagrams and
$R_{\lambda}$ are simple rings. In fact $\displaystyle
R_{\bf\lambda}=\bigoplus_{S\in{\mbox{STab}}({\bf\lambda})}V_{S}({\bf\lambda})$
( see Theorem 2.2). We have the following corresponding decomposition of the
identity element of $\mathbb{C}[G(r,n)]$:

$1=\sum_{\lambda\in{\cal P}_{r,n}}r_{\lambda},$

where $r_{\lambda}$ is the identity element of $R_{\lambda}$, with
$r_{\lambda}^{2}=1$ and $r_{\lambda}r_{\mu}=0$ if $\lambda\neq\mu.$
$\\{r_{\lambda}\\}_{\lambda\in{\cal P}_{r,n}}$ is the set of primitive central
idempotents of $\mathbb{C}[G(r,n)]$. In fact
$R_{\lambda}=r_{\lambda}\mathbb{C}[G(r,n)]$, for $\lambda\in{\cal P}_{r,n}$.
Let $n\in\mathbb{N}^{*},{\bf\lambda}\in{\cal P}_{r,n}$, we set
$\displaystyle{\mbox{Tab}}(n)=\displaystyle\cup_{\lambda\in{\cal
P}_{r,n}}{\mbox{Tab}}(\lambda)$ and
$\displaystyle{\mbox{STab}}(n)=\cup_{\lambda\in{\cal
P}_{r,n}}{\mbox{STab}}(\lambda)$.

######  Theorem 3.7.

For every primitive idempotent $e\in\mathbb{C}[G(r,n)]$.

  1. (1)

$e{\mathcal{O}}_{U}$ is a nontrivial $\mathcal{D}_{V}$-submodule of
${\mathcal{O}}_{U},$

  2. (2)

The ${\mathcal{D}}_{V}$-module $e{\mathcal{O}}_{U}$ is simple,

  3. (3)

There exist $\lambda\in{\cal P}_{r,n}$ and a higher Specht polynomial
$F_{T}^{S}$ (with $S,T\in{\mbox{STab}}(\lambda))$ such that
$e{\mathcal{O}}_{U}={\mathcal{D}}_{V}F_{T}^{S}.$

###### Proof.

  1. (1)

Let $e\in\mathbb{C}[G(r,n)]$ be a primitive idempotent, we know that
$\mathbb{C}[G(r,n)]e$ is a $G(r,n)$\- irreducible representation. Theorem 2.3
states that there is $\lambda\in{\cal P}_{r,n}$ and
$S\in{\mbox{STab}}(\lambda)$ such that $\mathbb{C}[G(r,n)]e\cong
V_{S}(\lambda)$, with $V_{S}(\lambda)\subset{\mathcal{O}}_{U}.$ By [2, Chap
III, §4, Theorem 3.9 ], we have
$e\mathbb{C}[G(r,n)]e\cong\mathbb{C}e\neq\\{0\\}$. $\\{0\\}\neq
eV_{S}(\lambda)\subset e{\mathcal{O}}_{U}.$ Since $e$ commutes with every
element of ${\mathcal{D}}_{V}$ and ${\mathcal{O}}_{U}$ is a
${\mathcal{D}}_{V}$-module, it follows that $e{\mathcal{O}}_{U}$ is a
nontrivial ${\mathcal{D}}_{V}$-module.
In fact $V_{S}(\lambda)$ is a cyclic $\mathbb{C}[G(r,n)]$-module, i.e., there
exist $T,S\in{\mbox{STab}}(\lambda)$ and a higher Specht polynomial
$F_{T}^{S}$ such that $V_{S}(\lambda)=\mathbb{C}[G(r,n)]F_{T}^{S}$, so that
$\mathbb{C}[G(r,n)]e\cong\mathbb{C}[G(r,n)]F_{T}^{S}$. Then it follows that
there is a higher Specht polynomial $F_{T}^{S}$ such that $eF_{T}^{S}$ is a
scalar multiple of $F_{T}^{S}.$

  2. (2)

Assume that $1=\sum_{i=1}^{s}e_{i}$ where the $\\{e_{i}\\}_{1\leq i\leq s}$ is
the set of primitive idempotents of $\mathbb{C}[G(r,n)],$ then
${\mathcal{O}}_{U}=\sum_{i=1}^{s}e_{i}{\mathcal{O}}_{U}.$ Let $m\in
e_{i}{\mathcal{O}}_{U}\cap e_{j}{\mathcal{O}}_{U}$ with $i\neq j$ so that
$m=e_{i}m_{i}$ and $m=e_{j}m,$ but $e_{i}e_{j}=0$ then $e_{i}m=e_{i}e_{j}m=0$
hence $m=0.$ Therefore
${\mathcal{O}}_{U}=\oplus_{i=1}^{s}e_{i}{\mathcal{O}}_{U}$ and we get:

$\operatorname{Hom}_{{\mathcal{D}}_{V}}({\mathcal{O}}_{U},{\mathcal{O}}_{U})\cong\bigoplus_{i,j=1}^{s}\operatorname{Hom}_{{\mathcal{D}}_{V}}(e_{i}{\mathcal{O}}_{U},e_{j}{\mathcal{O}}_{U}),$

by Corollary 3.6 we know that
$\displaystyle\mathbb{C}[G(r,n)]\cong\bigoplus_{i,j=1}^{s}\operatorname{Hom}_{{\mathcal{D}}_{V}}(e_{i}{\mathcal{O}}_{U},e_{j}{\mathcal{O}}_{U}).$
For every $\lambda\in{\cal P}_{r,n}$, we pick a unique irreducible
representation $V_{S}(\lambda)$ for a certain $S\in{\mbox{STab}}(\lambda)$
which we denote by $V(\lambda):=V_{S}(\lambda)$. We also have, by [6,
Proposition 3.29], that
$\displaystyle\mathbb{C}[G(r,n)]\cong\bigoplus_{\lambda\in{\cal
P}_{r,n}}\mbox{End}_{\mathbb{C}}(V({\lambda})).$ But by the Wedderburn’s
decomposition Theorem [2, Chap II,§4, Theorem 4.2] we also know that

$\displaystyle\mathbb{C}[G(r,n)]=\bigoplus_{\lambda\in{\cal
P}_{r,n}}r_{\lambda}\mathbb{C}[G(r,n)]\ \mbox{and}\
r_{\lambda}\mathbb{C}[G(r,n)]\cong\mbox{Mat}_{f^{\lambda}}(\mathbb{C})\cong\mbox{End}_{\mathbb{C}}(\mathbb{C}^{f^{\lambda}})$

where $f^{\lambda}=\dim_{\mathbb{C}}(V({\bf\lambda})).$ Each $r$ standard
tableau $T$ corresponds a primitive idempotent $e_{T}$, so that
$r_{\lambda}=\displaystyle\sum_{T\in{\mbox{STab}}(\lambda)}e_{T}.$ In what
follows we denote $e_{i}$ the primitive idempotent associated with standard
tableau $T_{i},$ ( i.e. $e_{i}=e_{T_{i}}$.) Let us show that

$\displaystyle\mathbb{C}[G(r,n)]\cong\bigoplus_{\lambda\vdash
n}\bigg{(}\bigoplus_{T_{i},T_{j}\in{\mbox{STab}}(\lambda)}\operatorname{Hom}_{{\mathcal{D}}_{V}}(e_{i}{\mathcal{O}}_{U},e_{j}{\mathcal{O}}_{U})\bigg{)}\
\mbox{where}\ e_{i}=e_{T_{i}}.$

Let $x$ be an element of $\mathbb{C}[G(r,n)]$ and $r_{\lambda}$ the primitive
central idempotent associated with $\lambda\in{\cal P}_{r,n}.$ Then $x$
induces an ${\mathcal{D}}_{V}$-homomorphism
${\mathcal{O}}_{U}\to{\mathcal{O}}_{U},m\mapsto x\cdot m;$ the multiplication
by $x$. Since $r_{\lambda}$ is in the centre of $\mathbb{C}[G(r,n)]$,
$x\cdot(r_{\lambda}{\mathcal{O}}_{U})=(x\cdot
r_{\lambda}){\mathcal{O}}_{U}\subset r_{\lambda}{\mathcal{O}}_{U},$ which
means $x\in\displaystyle\bigoplus_{\lambda\in{\cal
P}_{r,n}}\operatorname{Hom}_{{\mathcal{D}}_{V}}(r_{\lambda}{\mathcal{O}}_{U},r_{\lambda}{\mathcal{O}}_{U}).$
It follows that

$\displaystyle\operatorname{Hom}_{{\mathcal{D}}_{V}}({\mathcal{O}}_{U},{\mathcal{O}}_{U})\cong\bigoplus_{\lambda\in{\cal
P}_{r,n}}\operatorname{Hom}_{{\mathcal{D}}_{V}}(r_{\lambda}{\mathcal{O}}_{U},r_{\lambda}{\mathcal{O}}_{U}).$

Then
$\operatorname{Hom}_{{\mathcal{D}}_{V}}(e_{i}{\mathcal{O}}_{U},e_{j}{\mathcal{O}}_{U})=\\{0\\}$
if $T_{i}\in{\mbox{STab}}(\lambda_{i}),T_{j}\in{\mbox{STab}}(\lambda_{j})$ and
$\lambda_{i}\neq\lambda_{j}.$ We get that

$\displaystyle\operatorname{Hom}_{{\mathcal{D}}_{V}}({\mathcal{O}}_{U},{\mathcal{O}}_{U})\cong\bigoplus_{\lambda\in{\cal
P}_{r,n}}\bigg{(}\bigoplus_{T_{i},T_{j}\in{\mbox{STab}}(\lambda)}\operatorname{Hom}_{{\mathcal{D}}_{V}}(e_{i}{\mathcal{O}}_{U},e_{j}{\mathcal{O}}_{U})\bigg{)}.$

The number of direct factors in the sum
$\displaystyle\bigoplus_{T_{i},T_{j}\in{\mbox{STab}}(\lambda)}\operatorname{Hom}_{{\mathcal{D}}_{V}}(e_{i}{\mathcal{O}}_{U},e_{j}{\mathcal{O}}_{U})$
is $(f^{\lambda})^{2}.$
Let us show that
$\operatorname{Hom}_{{\mathcal{D}}_{V}}(e_{i}{\mathcal{O}}_{U},e_{j}{\mathcal{O}}_{U})\cong\mathbb{C}$
if $T_{i},T_{j}\in{\mbox{Tab}}(\lambda).$ Consider the following commutative
diagram:

${\mathbb{C}[G(r,n)]}$${\displaystyle\operatorname{Hom}_{{\mathcal{D}}_{V}}({\mathcal{O}}_{U},{\mathcal{O}}_{U})}$${r_{\lambda}\mathbb{C}[G(r,n)]}$${\displaystyle\operatorname{Hom}_{{\mathcal{D}}_{V}}(r_{\lambda}{\mathcal{O}}_{U},r_{\lambda}{\mathcal{O}}_{U})}$$\alpha_{\lambda}$$\phi$$\psi$$\beta_{\lambda}$

where $\displaystyle\beta_{\lambda}:\bigoplus_{\mu\in{\cal
P}_{r,n}}\operatorname{Hom}_{{\mathcal{D}}_{V}}(r_{\mu}{\mathcal{O}}_{U},r_{\mu}{\mathcal{O}}_{U})\to\operatorname{Hom}_{{\mathcal{D}}_{V}}(r_{\lambda}{\mathcal{O}}_{U},r_{\lambda}{\mathcal{O}}_{U})$
and
$\displaystyle\alpha_{\lambda}:\bigoplus_{\mu\in{\cal
P}_{r,n}}r_{\mu}\mathbb{C}[G(r,n)]\to r_{\lambda}\mathbb{C}[G(r,n)]$ are
canonical projections et $\phi$ is the isomorphism in Corollary3.6. It follows
that $\psi$ is an isomorphism hence
$r_{\lambda}\mathbb{C}[G(r,n)]\cong\displaystyle\operatorname{Hom}_{{\mathcal{D}}_{V}}(r_{\lambda}{\mathcal{O}}_{U},r_{\lambda}{\mathcal{O}}_{U}).$
Now we identify $r_{\lambda}\mathbb{C}[G(r,n)]$ with either the set
$\mbox{Mat}_{f^{\lambda}}(\mathbb{C})$ of square matrices of order
$f^{\lambda}$ with coefficients in $\mathbb{C}$ either with
$\displaystyle\mbox{End}_{\mathbb{C}}(\mathbb{C}^{f^{\lambda}}).$ Let $E_{ij}$
be the square matrix of order $f^{\lambda}$ with 1 at the position $(i,j)$ and
0 elsewhere and $E_{i}=E_{i,i},$ then we identify the primitive idempotent
$e_{i}\in r_{\lambda}\mathbb{C}[G(r,n)]$ with $E_{i}$ in
$\mbox{Mat}_{f^{\lambda}}(\mathbb{C}).$ Let
$B=(a_{ij})\in\mbox{Mat}_{f^{\lambda}}(\mathbb{C})$ we get
$B=\sum_{i,j}a_{i,j}E_{i,j}=\sum_{i,j}E_{i}BE_{j}$, in fact $E_{i}BE_{j}$ is
the matrix with $a_{i,j}$ in the position $(i,j)$ and 0 elsewhere, if
$R=\mbox{Mat}_{f^{\lambda}}(\mathbb{C})$ we get that
$E_{i}RE_{j}\cong\mathbb{C}.$

This isomorphism $\psi$ implies that

$\displaystyle\bigoplus_{T_{i},T_{j}\in{\mbox{STab}}(\lambda)}E_{i}RE_{j}\cong\bigoplus_{T_{i},T_{j}\in{\mbox{STab}}(\lambda)}\operatorname{Hom}_{{\mathcal{D}}_{V}}(e_{i}{\mathcal{O}}_{U},e_{j}{\mathcal{O}}_{U});$

the restriction of $\psi$ to $E_{i}RE_{j}$ yields a map
$E_{i}RE_{j}\to\operatorname{Hom}_{{\mathcal{D}}_{V}}(e_{i}{\mathcal{O}}_{U},e_{j}{\mathcal{O}}_{U})$
and this map is surjective, moreover we have
$E_{i}RE_{j}\cong\operatorname{Hom}_{{\mathcal{D}}_{V}}(e_{i}{\mathcal{O}}_{U},e_{j}{\mathcal{O}}_{U})$.
Therefore
$\operatorname{Hom}_{{\mathcal{D}}_{V}}(e_{i}{\mathcal{O}}_{U},e_{i}{\mathcal{O}}_{U})\cong\mathbb{C}.$
Let us assume that $e_{i}{\mathcal{O}}_{U}$ is not simple
${\mathcal{D}}_{V}$-module, then $e_{i}{\mathcal{O}}_{U}$ may be written as
$e_{i}{\mathcal{O}}_{U}=\oplus_{j\in J}N_{j}$ where the $N_{j}$ are simple
${\mathcal{D}}_{V}$-modules and $|J|>1$. It follows that
$\dim_{\mathbb{C}}(\operatorname{Hom}_{{\mathcal{D}}_{V}}(e_{i}{\mathcal{O}}_{U},e_{i}{\mathcal{O}}_{U}))\geq|J|$
but
$\operatorname{Hom}_{{\mathcal{D}}_{V}}(e_{i}{\mathcal{O}}_{U},e_{i}{\mathcal{O}}_{U})\cong\mathbb{C}$
so we obtain that $J=1$, which necessary implies that $e_{i}{\mathcal{O}}_{U}$
is a simple ${\mathcal{D}}_{V}$-module.

  3. (3)

By the the proof (i) there exists a higher Specht polynomial $F_{T}^{S}$, with
$S,T\in{\mbox{STab}}({\bf\lambda})$ where $\lambda\vdash n$ such that
$e_{i}{\mathcal{O}}_{U}={\mathcal{D}}_{V}F_{T}^{S}.$

∎

######  Corollary 3.8.

With the above notations,
$e_{i}{\mathcal{O}}_{U}\cong_{{\mathcal{D}}_{V}}e_{j}{\mathcal{O}}_{U}$ if
only if $T_{i}$ and $T_{j}$ have the same shape i.e. if there is a partition
$\lambda\in{\cal P}_{r,n}$ such that $T_{i},T_{j}\in{\mbox{STab}}(\lambda).$

###### Proof.

The ${\mathcal{D}}_{V}$-modules $e_{i}{\mathcal{O}}_{U}$ are simple and
$\operatorname{Hom}_{{\mathcal{D}}_{V}}(e_{i}{\mathcal{O}}_{U},e_{j}{\mathcal{O}}_{U})\cong\mathbb{C}$
whenever there exists a partition $\lambda\in{\cal P}_{r,n}$ such that
$T_{i},T_{j}\in{\mbox{STab}}(\lambda).$ Since
$\operatorname{Hom}_{{\mathcal{D}}_{V}}(e_{i}{\mathcal{O}}_{U},e_{j}{\mathcal{O}}_{U})\neq\\{0\\},$
we conclude by using the Schur lemma.

∎

######  Proposition 3.9.

Let $\lambda\in{\cal P}_{r,n}$, $T\in{\mbox{STab}}(\lambda)$, and let $e$ be
the primitive idempotent associated with $T,$ denote by $F_{T}:=F_{T}^{S}$ the
corresponding higher Specht polynomial (for some
$S\in{\mbox{STab}}({\bf\lambda})$), in Theorem 3.7 (iii), such that
$e{\mathcal{O}}_{U}={\mathcal{D}}_{V}F_{T}^{S}$ then we have:

  1. (1)

$\displaystyle{\mathcal{O}}_{U}=\bigoplus_{{T\in{\mbox{STab}}(n)}}{\mathcal{D}}_{V}F_{T}=\bigoplus_{{\bf\lambda}\in{\cal
P}_{r,n}}\bigg{(}\bigoplus_{{T\in{\mbox{STab}}({\bf\lambda})}}{\mathcal{D}}_{V}F_{T}\bigg{)};$
(3.1)

  2. (2)

for each $\lambda\in{\cal P}_{r,n}$ fix an $r$-tableau
$T^{*}\in{\mbox{STab}}(\lambda)$, then

$\displaystyle{\mathcal{O}}_{U}=\bigoplus_{{\bf\lambda}\in{\cal
P}_{r,n}}f^{\bf\lambda}{\mathcal{D}}_{V}F_{T^{*}}$ (3.2)

where $f^{\bf\lambda}=\dim_{\mathbb{C}}(V(\lambda))$

###### Proof.

We have by the proof of Theorem 3.7 that

${\mathcal{O}}_{U}=\bigoplus_{T_{i}\in{\mbox{STab}}(n)}e_{i}{\mathcal{O}}_{U}$

and the $e_{i}{\mathcal{O}}_{U}$ are simple ${\mathcal{D}}_{V}$-modules. Since
to each primitive idempotent $e_{i}$ corresponds an $r$-diagram
$\lambda_{i}\in{\cal P}_{r,n}$ and a tableau
$T_{i}\in{\mbox{STab}}(\lambda_{i})$ such that
$e_{i}{\mathcal{O}}_{U}={\mathcal{D}}_{V}F_{T_{i}}$ then
${\mathcal{O}}_{U}=\bigoplus_{T\in{\mbox{STab}}(n)}{\mathcal{D}}_{V}F_{T}.$ By
Corollary 3.6, ${\mathcal{D}}_{V}F_{T_{j}}\cong{\mathcal{D}}_{V}F_{T_{j}}$ if
$T_{i},T_{j}\in{\mbox{STab}}(\lambda)$ and so we have $f^{\bf\lambda}$
isomorphic copies of ${\mathcal{D}}_{V}F_{T^{*}}$ in the direct sum (3.1). ∎

Using Proposition 3.1 and Proposition 3.2 we get the next theorem.

######  Theorem 3.10.

  * (i)

$N_{T}=\mathcal{D}_{Y}F_{T}$ is an irreducible $D_{Y}$-submodule of
$\pi_{+}({\cal O}_{X})$.

  * (ii)

There is a direct sum decomposition

$\pi_{+}({\cal O}_{X})=\bigoplus_{\lambda\in{\cal
P}_{r,n}}\bigoplus_{T\in{\mbox{STab}}(\lambda)}N_{T}$ (3.3)

  * (iii)

$\pi_{+}({\cal O}_{X})\cong\bigoplus_{\lambda\in{\cal
P}_{r,n}}f^{\lambda}N_{T^{*}}$ (3.4)

We get in Theorm 3.10 a decomposition of the $\pi_{+}({\cal O}_{X})$ into
irreducible $\mathcal{D}_{Y}$ modules generated by the higher Specht
polynomials.

##  4\. Using correspondence between $G$-representations and $D$-modules

In this section we us an equivalence of categories between group
representations and modules over differential ring to yield a better version
of decomposition the polynomial ring as ${\mathcal{D}}_{V}$-modules

######  Lemma 4.1.

Let $\lambda\in{\cal P}_{r,n},$ $S\in{\mbox{STab}}(\lambda)$, $V_{S}(\lambda)$
be the corresponding irreducible representation and $e_{T}$ the primitive
idempotent associated with an $r$-standard tableau
$T\in{\mbox{STab}}({\bf\lambda}).$ Then
$e_{T}(V_{S}({\lambda}))=\\{e_{T}(m)|m\in V_{S}({\lambda})\\}$ is a one
dimensional $\mathbb{C}$-vector space.

###### Proof.

In fact we have

$\displaystyle e_{T}V_{S}({\lambda})$ $\displaystyle\cong$ $\displaystyle
e_{T}\mathbb{C}[G(r,n)]e_{T}$ $\displaystyle\cong$ $\displaystyle
e_{T}\mathbb{C}[G(r,n)]e_{T}$ $\displaystyle\cong$
$\displaystyle\mathbb{C}e_{T}\ \mbox{by \
\cite[cite]{[\@@bibref{}{Boerner}{}{}, Chap III, \S 4, Theorem 3.9 ]}}$

∎

Recall that if $M$ is a semi-simple module over a ring $R$, and $N$ is simple
$R$-module, then the isotopic component of $M$ associated to $N$ is the sum
$\sum N^{\prime}\subset M$ of all $N^{\prime}\subset M$ such that
$N^{\prime}\cong N.$

######  Lemma 4.2.

Let $\lambda\in{\cal P}_{r,n},$ $S\in{\mbox{STab}}(\lambda)$ and
$V_{S}(\lambda)$ be the corresponidng irreducible module associated. Let
$M:={\mathcal{O}}_{U}$ and $M({\lambda})$be the isotopic component of $M$ (as
${\cal O}_{V}$-module) associated with $V_{S}({\lambda})$. Then $M({\lambda})$
is ${\mathcal{D}}_{V}$-module.

###### Proof.

We only have to proof that ${\mathcal{D}}_{V}\cdot M({\lambda})\subset
M({\lambda}).$ Let $D\in{\mathcal{D}}_{V}$ and $N$ be a
$\mathbb{C}[G(r,n)]$-module isomorphic to $V_{S}({\lambda})$, since $D$
commute with the elements of the group algebra $\mathbb{C}[G(r,n)]$, $D$ is an
$\mathbb{C}[G(r,n)]$-homomorphism from $N$ into $D(N)$. Then by virtue of the
Schur lemma $D(N)=0$ or $D(N)\cong N$ as a $\mathbb{C}[G(r,n)]$-module, and
$D(N)\subset M({\lambda})$. Hence ${\mathcal{D}}_{V}\cdot M({\lambda})\subset
M({\lambda}).$ ∎

######  Lemma 4.3.

Let $\lambda\in{\cal P}_{r,n},$ $S\in{\mbox{STab}}(\lambda)_{d}$, $M(\lambda)$
the isotopic component associated with $V_{S}(\lambda)$ and $e_{T}$ the
primitive idempotent associative with an $r$-standard tableau $T$ . Then
$e_{T}(M({\lambda}))$ si ${\mathcal{D}}_{V}$-module.

###### Proof.

Let $D\in{\mathcal{D}}_{V},$ we have
$D(e_{T}(M^{\lambda}))=e_{T}(D(M({\lambda}))\subset e_{T}(M({\lambda}))$, so
$e_{T}(M({\lambda}))$ si ${\mathcal{D}}_{V}$-module. ∎

Before we proceed, let us recall the correspondence between
$G$-representations and $D$-modules [9, Paragraph 2.4]. Let $L$ and $K$ be two
extensions fields a field $k$, denote by $T_{K/k}$ the $k$-linear derivations
of $K$. We say that a $T_{K/k}$-module $M$ is $L$-trivial  if
$L\otimes_{K}M\cong L^{n}$ as $T_{L/k}$-modules. Denote by
$\operatorname{Mod}^{L}(T_{K/k})$ the full subcategory of finitely generated
$T_{K/k}$-modules that are $L$-trivial. It is immediate that it is closed
under taking submodules and quotient modules. Using a lifting $\phi$ , $L$ may
be thought of as a $T_{K/k}$-module. If $G$ is a finite group let
$\operatorname{Mod}(k[G])$ be the category of finite-dimensional
representations of $k[G]$. Let now $k\to K\to L$ be a tower of fields such
that $K=L^{G}$. Note that the action of $T_{K/k}$ commutes with the action of
$G$. If $V$ is a $k[G]$-module, $L\otimes_{k}V$ is a $T_{K/k}$-module by
$D(l\otimes v)=D(l)\otimes v,\quad D\in T_{K/k}$, and $(L\otimes_{k}V)^{G}$ is
a $T_{K/k}$-submodule.

Let $L$ and $K$ be two fields, say that a $T_{K/k}$-module $M$ is $L$-trivial
if $L\otimes_{K}M\cong L^{n}$ as $T_{L/k}$-modules. Denote by
$\operatorname{Mod}^{L}(T_{K/k})$ the full subcategory of finitely generated
$T_{K/k}$-modules that are $L$-trivial. It is immediate that it is closed
under taking submodules and quotient modules. Using the lifting $\phi$ , $L$
may be thought of as a $T_{K/k}$-module. If $G$ is a finite group let
$\operatorname{Mod}(k[G])$ be the category of finite-dimensional
representations of $k[G]$. Let now $k\to K\to L$ be a tower of fields such
that $K=L^{G}$. Note that the action of $T_{K/k}$ commutes with the action of
$G$. If $V$ is a $k[G]$-module, $L\otimes_{k}V$ is a $T_{K/k}$-module
by$D(l\otimes v)=D(l)\otimes v,\quad D\in T_{K/k}$, and $(L\otimes_{k}V)^{G}$
is a $T_{K/k}$-submodule.

######  Proposition 4.4.

The functor

$\nabla:\operatorname{Mod}(k[G])\to\operatorname{Mod}(T_{K/k}),\quad
V\mapsto(L\otimes_{k}V)^{G}$

is fully faithful, and defines an equivalence of categories

$\operatorname{Mod}(k[G])\to\operatorname{Mod}^{L}(T_{K/k}).$

The quasi-inverse of $\nabla$ is the functor

$Loc:\operatorname{Mod}^{L}(T_{K/k})\to\operatorname{Mod}(k[G]),\quad
Loc(M)=(L\otimes_{K}M)^{\phi(T_{K/k})}.$

###### Proof.

see [9, Proposition 2.4] ∎

In the following proposition we take $G=G(r,n)$,
$K:=\mathbb{C}(y_{1},\ldots,y_{n})$ the field of fractions of ${\cal O}_{V}$
and $L=\mathbb{C}(x_{1},\ldots,x_{n})$ the field of fractions of
${\mathcal{O}}_{U}$ so that $K=L^{G(r,n)}.$ It is clear that $L$ is a Galois
extension of $K$ with Galois $G(r,n)$.

######  Proposition 4.5.

Let $\lambda\in{\cal P}_{r,n}$ $S\in{\mbox{STab}}(\lambda)$, $V_{S}(\lambda)$
be the corresponding irreducible representation and $e_{T}$ the primitive
idempotent associative with an $r$-standard tableau $T$. Set
$M_{T}:=e_{T}{\mathcal{O}}_{U}$.Then we have:

  1. (1)

$M_{T}=\nabla(V_{S}({\lambda}))$

  2. (2)

$M_{T}=e_{T}(M({\lambda}))$ is simple ${\mathcal{D}}_{V}$-module;

  3. (3)

$M({\lambda})=\displaystyle\bigoplus_{T\in{\mbox{STab}}(\lambda)}e_{T}(M({\lambda})).$

###### Proof.

  1. (1)

Let us consider the right $\mathbb{C}[G(r,n)]$-module
$V=e_{T}\mathbb{C}[G(r,n)]$ where $T\in{\mbox{STab}}(\lambda)$. This is the
image of $\mathbb{C}[G(r,n)]$ by right multiplication map
$e_{T}:\mathbb{C}[G(r,n)]\to\mathbb{C}[G(r,n)]$. By [9, Example 2.5], we may
turn this map to a left multiplication
$\mathbb{C}[G(r,n)]^{r}\to\mathbb{C}[G(r,n)]^{r}$ and get an image which is
isomorphic to $V_{S}({\lambda})$ for some $S\in{\mbox{STab}}(\lambda)$. So we
get an induced map

$\nabla(\mathbb{C}[G(r,n)]^{r})\to\nabla(V_{S}({\lambda}))\subset\nabla(\mathbb{C}[G(r,n)]^{r}),$

which is a multiplication by $e_{T}$ according to [9, Example 2.5]. Then
$\nabla(V_{S}({\lambda}))$ is egal to $e_{T}{\mathcal{O}}_{U}=M_{T}.$

  2. (2)

Since $V_{S}({\lambda})$ is a simple $\mathbb{C}[G(r,n)]$-module,
$\nabla(V_{S}({\lambda}))$ is also a simple ${\mathcal{D}}_{V}$-module.

  3. (3)

follows from the fact that $1=\sum_{T\in{\mbox{STab}}(n)}e_{T}$ and
$e_{T}(M({\lambda}))=0$ if $T\notin{\mbox{STab}}({\bf\lambda})$ .

∎

######  Theorem 4.6.

Let $T$ be an $r$-standard tableau of shape $\lambda$ where $\lambda\in{\cal
P}_{r,n}$ and $M_{T}$ be as in the above proposition. Then

  1. (1)

$M_{T}=\displaystyle\bigoplus_{S\in{\mbox{STab}}(\lambda)}{\cal
O}_{V}F_{T}^{S}$ as ${\mathcal{D}}_{V}$-module,

  2. (2)

${\mathcal{O}}_{U}=\displaystyle\bigoplus_{\lambda\in{\cal
P}_{r,n}}\bigg{(}\bigoplus_{S,T\in{\mbox{STab}}(\lambda)}{\cal
O}_{V}F_{T}^{S}\bigg{)}$ as a ${\mathcal{D}}_{V}$-module.

###### Proof.

  1. (1)

For $S\in{\mbox{STab}}({\bf\lambda})$ We know by Theorem 2.2 that the
polynomial $F_{T}^{S}$ generate a $\mathbb{C}[G(r,n)]$-module inside
${\mathcal{O}}_{U}$ which is isomorphic to $V({\lambda})$ . Then $F_{T}^{S}\in
M^{\lambda}$ and
$M^{\lambda}=\displaystyle\bigoplus_{S,T\in{\mbox{STab}}(\lambda)}\mathbb{C}[G(r,n)]F_{T}^{S}{\cal
O}_{V}$. Moreover $e_{T}(F_{T}^{S})=cF_{T}^{S},c\in\mathbb{C}$ and by Lemma
4.1 $e_{T}(\mathbb{C}[G(r,n)]F_{T}^{S})=\mathbb{C}F_{T}^{S}.$ Hence
$M_{T}=e_{T}(M^{\lambda})=\displaystyle\bigoplus_{S\in{\mbox{STab}}(\lambda)}{\cal
O}_{V}F_{T}^{S}.$

  2. (2)

follows from Proposition 3.9.

∎

######  Theorem 4.7.

Let $\lambda\in{\cal P}_{r,n}$ and ${\bf D}\in{\mathcal{D}}_{V}$ such that
${\bf D}(F_{T}^{S})\neq 0$ for a higher Specht polynomial $F_{T}^{S}$ with
$S,T\in{\mbox{STab}}(\lambda)$. Then the image of the
$\mathbb{C}[G(r,n)]$-module $V_{S}({\lambda})$ by ${\bf D}$ is an
$\mathbb{C}[G(r,n)]$-module isomorphic to $V_{S}({\lambda})$. In others words,
the actions of the differential operators of ${\mathcal{D}}_{V}$ on the higher
Specht polynomials generate isomorphic copies of the corresponding irreducible
$\mathbb{C}[G(r,n)]$-module.

###### Proof.

Let $\lambda\in{\cal P}_{r,n},{\bf D}\in{\mathcal{D}}_{V}$ such that ${\bf
D}(F_{T}^{S})\neq 0$ for $S,T\in{\mbox{STab}}(\lambda)$ and set $W^{S}_{\bf
D}({\lambda})={\bf D}(V_{S}({\lambda}))$ the image of the module
$V_{S}({\lambda})$ under the map ${\bf D}$. By Theorem 2.2, the
$\mathbb{C}$-vector space $V_{S}({\lambda})$ is equipped with a basis ${{\cal
F}}=\\{F_{T}^{S};T\in{\mbox{STab}}(\lambda)\\}$, then $W^{S}_{\bf
D}({\lambda})$ is the vector space spanned by the set $\\{{\bf D}(F_{T}^{S});\
T\in{\mbox{STab}}(\lambda)\\}$. The elements of
$\\{F_{T}^{S};T\in{\mbox{STab}}(\lambda)\\}$ are linearly independent over
${\mathcal{D}}_{V}$, otherwise the direct sum in Theorem 4.7 cannot hold. It
follows that the elements $\\{{\bf
D}(F_{T}^{S});T\in{\mbox{STab}}(\lambda)\\}$ are linear independent over
$\mathbb{C}$. Hence $\\{{\bf D}(F_{T}^{S});T\in{\mbox{STab}}(\lambda)\\}$ is a
basis of $W^{S}_{\bf D}({\lambda})$ over $\mathbb{C}$. Since ${\bf D}$ commute
with elements of $\mathbb{C}[G(r,n)],\ W^{S}_{\bf D}({\lambda})$ is an
$\mathbb{C}[G(r,n)]$-module isomorphic to $V_{S}({\lambda}).$ ∎

## Acknowledgments

The final version work has been done while the first author was visiting IMSP
at Benin, under the Staff Exchange program of the German Office of Academic
Exchange (DAAD). He warmly thanks the DAAD for the financial support.

## References

  * [1] Ariki, S. Terasoma, T. and Yamada, H., Higher Specht polynomials, Hiroshima Math, 27 (1997), no. 1, 177-188
  * [2] Boerner,H. Representations of Groups with special Consideration for the Needs of Modern Physics, North-Holland, New York, NY, (1970).
  * [3] Chambert-Loir, A, Algèbre corporelle, Les éditions de l’Ecole polytechnique (2005).
  * [4] Coutinho, S. C., A primer of algebraic $D$-modules, 33, Cambridge University Press, Cambridge,
  * [5] de Cataldo, M. A.A and Migliorini. L., The decomposition theorem, perverse sheaves and the topology of algebraic maps, Bull. Amer. Math. Soc. (N.S.) 46 (2009), no. 4, 535-633.
  * [6] Fulton, W. and Harris, J. Representation Theory. A First Course, Graduate Texts in Mathematics, 129. Readings in Mathematics. Springer-Verlag, New York, (1991).
  * [7] Lang, S, Algebra. Revisited third edition. Graduate Texts in Mathematics, 211. Springer-Verlag, New York, 2002.
  * [8] Levasseur, T., and Stafford,J.T. Invariant differential operators and a homomorphism of Harish-Chandra, J.Amer. Math. Soc. 8 (1995), 365-372. MR 95g:22029
  * [9] Nonkané, I., Specht polynomials and modules over the Weyl algebra, Afr. Mat, 30, Issue 11 pp 279-290 (2019).
  * [10] Nonkané, I., Differential operators and the symmetric groups, Journal of Physics: Conference Series 1730(1):012129, (2021).
  * [11] Nonkané, I., Representation Theory of Groups and $\mathcal{D}$-Modules, International Journal of Mathematics and Mathematical Sciences, vol. 2021, Article ID 6613869, 7 pages, (2021).
  * [12] Nonkané, I. and Latévi, M. L., Differential operators and reflection group of type $B_{n}$, arxiv: 2110.04643 [Math.RT] 9 OCtobre 2021 accepted to appear in Proceedings of 10th International Conference on Mathematical Modeling in Physical Science (2021), J of phys.: confer.ser.
  * [13] Sagan, Bruce. E., The symmetric group. Representations, combinatorial algorithms, and symmetric functions, Second edition. Graduate Texts in Mathematics, 203\. Springer-Verlag, New York, (2001).
  * [14] Weyl, Hermann, The theory of groups and quantum mechanics, Dover, New York, (1950).


# L2M: Practical posterior Laplace approximation
with optimization-driven second moment estimation

Christian S. Perone  Roberto Pereira Silveira  Thomas Paula

###### Abstract

Uncertainty quantification for deep neural networks has recently evolved
through many techniques. In this work, we revisit Laplace approximation, a
classical approach for posterior approximation that is computationally
attractive. However, instead of computing the curvature matrix, we show that,
under some regularity conditions, the Laplace approximation can be easily
constructed using the gradient second moment. This quantity is already
estimated by many exponential moving average variants of Adagrad such as Adam
and RMSprop, but is traditionally discarded after training. We show that our
method (L2M) does not require changes in models or optimization, can be
implemented in a few lines of code to yield reasonable results, and it does
not require any extra computational steps besides what is already being
computed by optimizers, without introducing any new hyperparameter. We hope
our method can open new research directions on using quantities already
computed by optimizers for uncertainty estimation in deep neural networks.

Uncertainty estimation, Laplace, posterior approximation

##  1 Introduction

Uncertainty quantification is an important component of learning systems. It
is often a required quantity for the deployment of deep learning models in
real-world applications, especially when it involves decision making.
Uncertainty is also widely used in many domains such as reinforcement learning
(Osband et al., 2018), active learning (Gal et al., 2017), out-of-distribution
detection (Lakshminarayanan et al., 2017), and planning (Tigas et al., 2019),
to name a few.

Although many methods of uncertainty estimation for modern neural networks
were recently developed, most of them rely on computationally expensive
training and management procedures, sometimes of multiple models, with
implementation complexity requiring specialized code for certain layer
implementations or post-training tuning/estimation of hyperparameters in some
cases.

Our contributions in this work are:

  * •

We show that under some regularity conditions, a diagonal Laplace
approximation can be constructed without computing anything besides what is
already being computed by widely used optimizers;

  * •

We qualitatively compare this approximation with methods such as deep
ensembles (Lakshminarayanan et al., 2017), MC Dropout (Gal & Ghahramani,
2016), Hamiltonian Monte Carlo (HMC) (Cobb & Jalaian, 2020), among others;

  * •

We also show that our approximation is orthogonal to methods such as
ensembling (Lakshminarayanan et al., 2017) and does not require changing
training procedures, estimating new quantities, or adding new hyperparameters.

To the best of our knowledge, it is the first time that the relationship
between the raw second moment estimation in optimizers and uncertainty
estimation is established.

###  1.1 Related Work

The use of the Laplace method for posterior approximation in neural networks
can be dated back to Wray L. Buntine & Andreas S. Weigend (1991) and MacKay
(1992). Recently, it was also employed by Ritter et al. (2018) in deep neural
networks where a Kronecker factorization of the curvature matrix was employed.
In Kristiadi et al. (2020), they also used Laplace approximation for the last
layer of networks to overcome the confidence of ReLUs. In Kirkpatrick et al.
(2017), a diagonal Laplace approximation was used to overcome catastrophic
forgetting in neural networks.

Another line of work to estimate uncertainty in neural networks is through the
use of ensembling methods as in Lakshminarayanan et al. (2017) where multiple
models are trained and then used to estimate the prediction variance. This
method, together with random priors (randomly initialized and non-trainable
networks) and bootstrapping was also employed in Osband et al. (2018).

Many variational approximations (Graves, 2011) were also employed for the
posterior approximation, including an efficient and appealing method called MC
Dropout by Gal & Ghahramani (2016), where dropout is left turned on during
inference time.

Another method that was recently developed is called SWA-Gaussian (SWAG)
(Maddox et al., 2019), which relies on SWA (Izmailov et al., 2018) running
averages for the first and second moment of parameters to build a Gaussian
approximation of the posterior. The main difference of SWAG to our method is
that it uses SGD and compute the moments of parameters, while our method re-
uses optimizer second moment estimation of the gradients. Computationally,
they have similar performance when using SWAG with SGD, but that is not the
case if SWAG is used with other optimizers such as Adam (Kingma & Ba, 2015).

##  2 Laplace approximation introduction

The main idea of the Laplace approximation (Azevedo-Filho & Shachter, 1994)
can be stated as follows: locate the posterior mode and then construct a
Gaussian distribution around the mode using as covariance matrix the inverse
of the curvature around the mode.

The Laplace approximation can be derived from the second-order Taylor
expansion around the mode (which can be found with optimization of a maximum a
posterior $\theta_{\text{MAP}}$ estimator):

$\log p(\theta|\mathcal{D})\approx\log
p(\theta_{\text{MAP}}|\mathcal{D})-\frac{1}{2}(\theta-\theta_{\text{MAP}})^{\intercal}\tilde{H}(\theta-\theta_{\text{MAP}})$
(1)

Where $\tilde{H}$ is $\mathop{\mathbb{E}}[H]$, the expected Hessian of the
negative log posterior. Given the property that we are expanding around
$\theta_{\text{MAP}}$ where the gradient is zero, the first order term of the
expansion becomes zero. By exponentiation we can then reach the functional
form of the Gaussian posterior approximation:

$\theta\sim\mathcal{N}(\theta_{\text{MAP}},\tilde{H}^{-1})$ (2)

During inference, we can approximate the mean or any other statistic by taking
$S$ Monte Carlo samples $\theta^{(s)}$ from this approximation:

$p(\mathcal{D^{*}}|\mathcal{D})=\int
p(\mathcal{D}^{*}|\theta)p(\theta|\mathcal{D})d\theta\approx\frac{1}{S}\sum^{S}_{S=1}p(\mathcal{D}^{*}|\theta^{(s)})$
(3)

Which gives us the posterior predictive distribution.

##  3 Laplace approximation in neural networks

The use of Laplace approximation of the posterior in neural networks can be
dated back to Wray L. Buntine & Andreas S. Weigend (1991) and MacKay (1992).
Only recently, Laplace approximation was used for larger modern deep neural
networks (Ritter et al., 2018), using a Kronecker factorization (Martens &
Grosse, 2015) of the curvature matrix.

Although the Kronecker factorization can yield a better approximation when
compared to a diagonal approximation of the curvature matrix, the Kronecker
factors still have to be computed on another step after training the network
and can require specialized computation depending on the layers (Grosse &
Martens, 2016) used in the network, making it difficult to use in practice.
However, it is important to note that our method can be used together with
KFAC (Martens & Grosse, 2015) instead of Adam, as the same approach described
in this work can be used with the KFAC approximation instead of the raw
diagonal second moment estimated by optimizers such as Adam (Kingma & Ba,
2015).

##  4 Diagonal Laplace approximation using optimizer second moment

###  4.1 Diagonal Empirical Fisher estimation

Many exponential moving average variants of Adagrad (Duchi et al., 2010) are
popular in the deep learning community. RMSprop (Tieleman & Hinton, 2012),
Adam (Kingma & Ba, 2015), and Adadelta (Zeiler, 2012) are some examples. In
this work, we will focus on Adam (Kingma & Ba, 2015), as it is one of the most
used optimizers for deep neural networks.

The uncentered second moment of the gradients can be seen as a diagonal
approximation of the Fisher Information Matrix (FIM) used in the Natural
Gradient Descent (Amari, 1998):

$\mathbf{F}_{\theta}=\mathop{\mathbb{E}}_{\begin{subarray}{c}y\sim
p_{\theta}(y|x)\\\ x\sim
p_{\text{data}}\end{subarray}}\left[\nabla_{\theta}\log
p_{\theta}(y|x)\,\nabla_{\theta}\log p_{\theta}(y|x)^{\intercal}\right]\,$ (4)

However, as noted by Kunstner et al. (2019), Adam optimizer does not use the
expectation with $y\sim p_{\theta}(y|x)$ (with $y$ from the model
distribution), but instead it uses the ground-truth labels. This matrix, which
is often called “empirical” Fisher (EF) 111We use the same commonly
nomenclature here as “empirical” Fisher, however, as pointed by others (Thomas
et al., 2019; Kunstner et al., 2019), this term can be inaccurate., is often
confused with the actual FIM. Also, in the preconditioning with the FIM, Adam
employs a square root of the Fisher, another departure from the natural
gradient.

###  4.2 Construction of the Laplace approximation

Under the condition where the model is realizable and there is enough data to
recover the true parameters, then at the minimum, the EF converges to the FIM,
as shown in (Kunstner et al., 2019), given the consistency of the maximum
likelihood estimator. Also, for optimizers using exponential moving averages
of the gradient’s second moment, the influence of gradients beyond a fixed
window size becomes negligibly small (Reddi et al., 2018).

Under the aforementioned regularity conditions regarding the convergence of
the empirical Fisher to the $\mathbf{F}_{\theta}$ matrix, one can easily
construct the posterior approximation using a diagonal Laplace approximation
with the minima $\theta_{\text{MAP}}$ and the empirical Fisher diagonal
$\operatorname{diag}(\mathbf{F}_{\theta})$, quantities already estimated by
many modern optimizers (and that are usually discarded after training):

$\theta\sim\mathcal{N}(\theta_{\text{MAP}},\operatorname{diag}(\mathbf{F}_{\theta})^{-1})$
(5)

The $\textbf{F}_{\theta}$ is connected to the Hessian through the Gauss-Newton
matrix (Botev et al., 2017), which is an approximation of the Hessian where
the second-order term is ignored, as it becomes small for models fitting data
well. For exponential family distributions, the Gauss-Newton coincides with
the Hessian, and as opposed to the Hessian – that can be degenerate for neural
networks – the $\textbf{F}_{\theta}$ and the Gauss-Newton are guaranteed to be
positive semi-definite (p.s.d.).

In practice, we also assume a Gaussian prior, that would be equivalent to the
L$2$ term in regularized loss where
$\theta\sim\mathcal{N}(0,\frac{1}{\lambda}\mathbb{I})$, $\lambda$ is the
penalty term, and $\mathbb{I}$ denotes the identity matrix. This leads to the
following approximation:

$\theta\sim\mathcal{N}(\theta_{\text{MAP}},\left[\operatorname{diag}(\mathbf{F}_{\theta})+\frac{1}{\lambda}+\epsilon\right]^{-1})$
(6)

Where $\epsilon$ is a small term to avoid division by zero. This approximation
is very attractive in computational terms and simplicity, as it takes leverage
of Adam’s gradient second moment buffer (that are nowadays discarded) and uses
the minima found by the optimizer as well. We call this approximation L2M
(from Laplace 2nd Moment).

It is also important to highlight that our method is orthogonal to methods
such as deep ensembles (Lakshminarayanan et al., 2017), since ensembles can be
used to expand the mode coverage of the posterior while still using our method
on each model of the ensemble, leveraging the local curvature information
around individual modes of the posterior.

###  4.3 Limitations

Although our approximation is scalable and easy to implement, it also comes
with limitations that are important to highlight. The Laplace approximation is
a Taylor expansion around the MAP estimate and, in our case, its curvature is
built with a diagonal approximation, which means that parameters are assumed
to be independent.

Also, the EF converges to the FIM only at the minimum (Kunstner et al., 2019),
and although it can have remarkable angle similarity (Thomas et al., 2019),
the ratio of traces from the EF and FIM can be quite different, which can lead
to potential variance underestimation. However, up to a multiplicative
constant, we can expect these two quantities to exhibit high similarity
(Thomas et al., 2019).

We note that priors other than Gaussian (which has L2 regularization
equivalence) would require changes in our method. Nevertheless, for many
applications, this approximation can be an excellent trade-off while providing
reasonable uncertainty estimation with a simple, easy to implement, and
scalable approach.

###  4.4 Note regarding the $\epsilon$ term

The $\epsilon$ term, also introduced in Adam (Kingma & Ba, 2015) to avoid
division by zero, can also be seen as a damping term improving the
conditioning of the Fisher or setting a trust region radius during
optimization, as noted by Choi et al. (2019). In our case, $\epsilon$ can have
an effect of reducing the posterior variance, as it is added to the Fisher’s
diagonal before the inversion.

##  5 Evaluation

Comparing different uncertainty estimation methods is a complex subject as
there is no ground truth for uncertainty. Therefore, many works compare
methods on the grounds of a qualitative evaluation on toy regression datasets
or on out-of-distribution detection tasks.

In Figure 1, we show how our method compares to other methods on a toy
regression dataset where $y\sim x^{3}+\mathcal{N}(0,3^{2})$ as in Ritter et
al. (2018). For this comparison, implemented in PyTorch (Paszke et al., 2019),
we employed a simple MLP with $2$ fully-connected layers, $40$ units and ReLU
activation. We used an Adam’s decoupled weight decay (Loshchilov & Hutter,
2019) of $0.1$ with a learning rate of $0.1$ and trained it for $5,000$
epochs. For the predictions, we sampled $500$ posterior samples and then
report the mean and $1$, $2$, and $3$ standard deviations above and below the
mean.

Figure 1: Qualitative comparison on a toy regression dataset. Red shades
denotes 1, 2, and 3 $\sigma^{2}$. The black line shows the mean prediction and
the crosses represents data points. First panel (from left to right):
Randomized prior functions (Osband et al., 2018). Second panel: MC Dropout
(Gal & Ghahramani, 2016). Third panel: Deep Ensembles (Lakshminarayanan et
al., 2017). Fourth panel: SWAG-Diagonal (Maddox et al., 2019). Fifth panel:
Hamiltonian Monte Carlo (Cobb & Jalaian, 2020). Sixth panel: L2M (our method).

As we can clearly see in Figure 1, our method (L2M) performs on par with other
methods, i.e., uncertainty is high in regions with no data points and low in
regions with several data points. The method here presented does not need lots
of hyperparameters to tune and does not need post-processing steps in order to
get to these results, while still being fast to get the posterior predictive
distribution.

We leave the evaluation of the integration of our method with deep ensembles
(Lakshminarayanan et al., 2017) for future work, as our method can be used on
each model of the ensemble to approximate the local curvature around the mode,
potentially leading to better uncertainty estimates. The application of our
method to classification problems is also left for future work.

##  6 Conclusion

In this paper, we presented L2M, a method for uncertainty quantification for
deep neural networks. We revisited Laplace approximation, a classical approach
for posterior approximation that is computationally attractive, and showed
that we can construct a posterior approximation with the gradient raw second
moment which is already computed by many exponential moving average variants
of Adagrad (Duchi et al., 2010). To the best of our knowledge, it is the first
time that this relationship between the raw second moment estimation in
optimizers and uncertainty estimation was established.

We qualitatively compared the approximations produced by our method with other
commonly used methods such as MC Dropout (Gal & Ghahramani, 2016), Deep
Ensembles (Lakshminarayanan et al., 2017), among others. Our results are
encouraging and open new research directions towards scalable approaches
leveraging raw second moment estimation in optimizers for uncertainty
estimation.

L2M has some limitations though. First, the curvature is built with a diagonal
approximation, which means that parameters are assumed to be independent.
Second, we rely on some regularity conditions such as the convergence of the
EF (“Empirical” Fisher) to the FIM (Fisher Information Matrix). Third, priors
other than Gaussian would require changes in our method, as the prior
equivalence with the Gaussian prior wouldn’t hold anymore. And last, we tested
our approach only in a toy regression dataset, which means it would need to be
further expanded to other larger datasets, network architectures and tasks.

For future work, we plan to conduct experiments on larger datasets. We also
plan to expand this method to handle general classification and language
modelling tasks with larger networks. Finally, we aim to explore the use of
this method for out-of-distribution problems and incorporate it into other
techniques such as deep ensembles (Lakshminarayanan et al., 2017).

## Software and Data

A PyTorch (Paszke et al., 2019) implementation of our method can be found at
the following repository below:

https://github.com/anonymous/L2M

## References

  * Abadi et al. (2015) Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., and Zheng, X.  TensorFlow: Large-scale machine learning on heterogeneous systems, 2015\.  URL https://www.tensorflow.org/.  Software available from tensorflow.org.
  * Amari (1998) Amari, S. I.  Natural Gradient Works Efficiently in Learning.  _Neural Computation_ , 10(2):251–276, feb 1998\.  ISSN 08997667.  doi: 10.1162/089976698300017746.
  * Azevedo-Filho & Shachter (1994) Azevedo-Filho, A. and Shachter, R. D.  Laplace’s Method Approximations for Probabilistic Inference in Belief Networks with Continuous Variables.  In _Uncertainty Proceedings 1994_ , pp. 28–36. Elsevier, feb 1994\.  doi: 10.1016/b978-1-55860-332-5.50009-2.
  * Botev et al. (2017) Botev, A., Ritter, H., and Barber, D.  Practical Gauss-Newton optimisation for deep learning.  In _34th International Conference on Machine Learning, ICML 2017_ , volume 2, pp. 860–880, 2017.  ISBN 9781510855144.
  * Choi et al. (2019) Choi, D., Shallue, C. J., Nado, Z., Lee, J., Maddison, C. J., and Dahl, G. E.  On Empirical Comparisons of Optimizers for Deep Learning.  2019\.
  * Cobb & Jalaian (2020) Cobb, A. D. and Jalaian, B.  Scaling hamiltonian monte carlo inference for bayesian neural networks with symmetric splitting.  _arXiv preprint arXiv:2010.06772_ , 2020.
  * Duchi et al. (2010) Duchi, J., Hazan, E., and Singer, Y.  Adaptive subgradient methods for online learning and stochastic optimization.  In _COLT 2010 - The 23rd Conference on Learning Theory_ , volume 12, pp. 257–269, 2010.  ISBN 9780982252925.
  * Gal & Ghahramani (2016) Gal, Y. and Ghahramani, Z.  Dropout as a bayesian approximation: Representing model uncertainty in deep learning.  In _Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48_ , ICML’16, pp. 1050–1059. JMLR.org, 2016.
  * Gal et al. (2017) Gal, Y., Islam, R., and Ghahramani, Z.  Deep Bayesian active learning with image data.  Technical report, 2017.
  * Graves (2011) Graves, A.  Practical variational inference for neural networks.  In _Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011_ , volume 24, 2011.  ISBN 9781618395993.
  * Grosse & Martens (2016) Grosse, R. and Martens, J.  A kronecker-factored approximate fisher matrix for convolution layers.  In _33rd International Conference on Machine Learning, ICML 2016_ , volume 2, pp. 851–874. International Machine Learning Society (IMLS), feb 2016.  ISBN 9781510829008.
  * Izmailov et al. (2018) Izmailov, P., Podoprikhin, D., Garipov, T., Vetrov, D., and Wilson, A. G.  Averaging weights leads to wider optima and better generalization.  In _34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018_ , volume 2, pp. 876–885. Association For Uncertainty in Artificial Intelligence (AUAI), mar 2018.  ISBN 9781510871601.
  * Kingma & Ba (2015) Kingma, D. P. and Ba, J. L.  Adam: a Method for Stochastic Optimization.  _International Conference on Learning Representations 2015_ , pp. 1–15, 2015.  ISSN 09252312.  doi: http://doi.acm.org.ezproxy.lib.ucf.edu/10.1145/1830483.1830503.
  * Kirkpatrick et al. (2017) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., Hassabis, D., Clopath, C., Kumaran, D., and Hadsell, R.  Overcoming catastrophic forgetting in neural networks.  _Proceedings of the National Academy of Sciences of the United States of America_ , 114(13):3521–3526, dec 2017.  ISSN 10916490.  doi: 10.1073/pnas.1611835114.
  * Kristiadi et al. (2020) Kristiadi, A., Hein, M., and Henni, P.  Being Bayesian, even Just a Bit, Fixes Overconfidence in ReLU Networks.  In _37th International Conference on Machine Learning, ICML 2020_ , volume PartF16814, pp. 5392–5402, 2020.  ISBN 9781713821120.
  * Kunstner et al. (2019) Kunstner, F., Balles, L., and Hennig, P.  Limitations of the empirical fisher approximation for natural gradient descent.  In _Advances in Neural Information Processing Systems_ , volume 32, 2019.
  * Lakshminarayanan et al. (2017) Lakshminarayanan, B., Pritzel, A., and Blundell, C.  Simple and scalable predictive uncertainty estimation using deep ensembles.  Technical report, 2017.
  * Loshchilov & Hutter (2019) Loshchilov, I. and Hutter, F.  Decoupled weight decay regularization.  In _7th International Conference on Learning Representations, ICLR 2019_. International Conference on Learning Representations, ICLR, nov 2019\.
  * MacKay (1992) MacKay, D. J. C.  A Practical Bayesian Framework for Backpropagation Networks.  _Neural Computation_ , 4(3):448–472, 1992.  ISSN 0899-7667.  doi: 10.1162/neco.1992.4.3.448.
  * Maddox et al. (2019) Maddox, W. J., Garipov, T., Izmailov, Vetrov, D., and Wilson, A. G.  A simple baseline for Bayesian uncertainty in deep learning.  Technical report, 2019.
  * Martens & Grosse (2015) Martens, J. and Grosse, R.  Optimizing neural networks with Kronecker-factored approximate curvature.  In _32nd International Conference on Machine Learning, ICML 2015_ , volume 3, pp. 2398–2407. International Machine Learning Society (IMLS), mar 2015.  ISBN 9781510810587.
  * Osband et al. (2018) Osband, I., Aslanides, J., and Cassirer, A.  Randomized prior functions for deep reinforcement learning.  Technical report, 2018.
  * Paszke et al. (2019) Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S.  PyTorch: An imperative style, high-performance deep learning library.  In _Advances in Neural Information Processing Systems_ , volume 32. Neural information processing systems foundation, dec 2019.
  * Reddi et al. (2018) Reddi, S. J., Kale, S., and Kumar, S.  On the convergence of Adam and beyond.  In _6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings_ , 2018.
  * Ritter et al. (2018) Ritter, H., Botev, A., and Barber, D.  A scalable laplace approximation for neural networks.  In _6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings_ , 2018.
  * Thomas et al. (2019) Thomas, V., Pedregosa, F., van Merriënboer, B., Mangazol, P.-A., Bengio, Y., and Roux, N. L.  On the interplay between noise and curvature and its effect on optimization and generalization.  2019\.
  * Tieleman & Hinton (2012) Tieleman, T. and Hinton, G.  Lecture 6.5—RmsProp: Divide the gradient by a running average of its recent magnitude.  COURSERA: Neural Networks for Machine Learning, 2012.
  * Tigas et al. (2019) Tigas, P., Filos, A., Mcallister, R., Rhinehart, N., Levine, S., and Gal, Y.  Robust Imitative Planning: Planning from Demonstrations Under Uncertainty.  Technical Report NeurIPS, 2019.
  * Wray L. Buntine & Andreas S. Weigend (1991) Wray L. Buntine and Andreas S. Weigend.  Bayesian Back-Propagation by Wray L. Buntine and Andreas S. Weigend.  _Complex Systems_ , 5(6):603–643, 1991.
  * Zeiler (2012) Zeiler, M. D.  ADADELTA: An Adaptive Learning Rate Method.  2012\.

## Appendix

### Appendix A - L2M approximation pseudo-code

This appendix shows a pseudo-code implementation of our method:

⬇

# Trained model and optimizer

model = (...)

optimizer = (...)

\par# Build L2M approximation

grad_second_moment = optimizer.second_moment()

prior_variance = 1.0/weight_decay

fisher_diag = grad_second_moment + \ prior_variance + eps

inv_fisher_diag = 1.0/fisher_diag

l2m_approximation = Normal(model.weights,

sqrt(inv_fisher_diag))

\par# Multiple posterior samples for inference

outputs = [] # predictive distribution

for s in S:

model.weights = l2m_approximation.sample()

output = model(input)

outputs.append(output)