# Supercharacters for parabolic contractions of finite groups of $A,B,C,D$ Lie
types

A.N. Panov

_Mathematical Department, Samara University, Samara, Russia<EMAIL_ADDRESS>
We construct supercharacter theories for the finite groups constructed by
parabolic constraction from simple groups of $A,B,C,D$ Lie types. In terms of
rook placements in the root systems we classify supecharacters and
superclasses.

Key Words: group representations, supercharacter theory, group contraction,
irreducible character, superclasses

2010 Mathematics Subject Classification: 20C15, 17B22

##  1 Introduction

The notion of a supercharacter theory was introduced by P. Diaconis and I.M.
Isaacs in the paper [1] in 2008. By definition, a supercharacter theory for a
given finite group $G$ is a pair $({\mathfrak{S}},{\cal K})$, where
${\mathfrak{S}}=\\{\chi_{1},\ldots,\chi_{M}\\}$ is a system of pairwise
disjoint characters (representations) of $G$ and ${\cal
K}=\\{K_{1},\ldots,K_{M}\\}$ is a partition of $G$ such that the characters
from ${\mathfrak{S}}$ are constant on subsets from ${\cal K}$ and
$\\{1\\}\in{\cal K}$. The characters from ${\mathfrak{S}}$ are called
_supercharacters_ , and subsets from ${\cal K}$ – _superclasses_ (see [1]).
Observe the number of supercharacters equals to the number of superclasses. An
example of supercharacter theory is the pair that consists of the system of
irreducible characters and the decomposition of group into conjugacy classes.
If a classification of irreducible representations is considered an extremely
difficult, ”wild” problem, we aim to construct a constructive supercharacter
theory. The classical example is the theory of basic characters for the
unitriangular group $\mathrm{UT}(n,{\mathbb{F}}_{q})$ developed by C. André in
[2, 3, 4]. This example was generalized to algebra groups in the same paper
[1] (by definition, an algebra group is a group of the form
${\mathfrak{G}}=1+J$, where $J$ is a finite dimensional associative nilpotent
algebra over a finite field).

In the present paper, we construct a supercharacter theory for parabolic
contractions of the groups ${\mathrm{GL}}(n)$, ${\mathrm{O}}(n)$ and
${\mathrm{Sp}}(n)$ defined over a finite field ${\mathbb{F}}_{q}$. For each
group we classify supercharacters and superclasses.

To construct a supercharacter theory for the parabolic contraction
${\mathrm{GL}}^{a}(n)$, we use the method of [10] (see Theorem 3.2). The case
of Borel contraction is studed in [11], where we apply the other method from
[12, 13, 14] that gives rise a supercharacter theory that is slightly
different from the one constructed in the present paper. Also, we classify
supercharacters and superclasses for the unipotent radical $U^{a}$ (see
Theorems 3.1 and 3.1). For the parabolic radicals of ${\mathrm{GL}}(n)$, the
similar classification is verified in [16].

A supercharacter theory for the Sylow subgroups in the orthogonal and
symplectic groups is constructed in [5, 6, 7, 17]. The author proposed a
supercharacter theory for parabolic subgroups in ${\mathrm{O}}(n)$ and
${\mathrm{Sp}}(n)$ in the paper [15]. This approach is developing in the
present paper for parabolic contractions of orthogonal and symplectic groups.
We obtain a description of supercharacters and superclasses in the exact form
(see Theorems 4.1 and 4.2).

##  2 Parabolic contractions

Let $G$ be a split reductive groups defined over a field $K$ and
${\mathfrak{g}}$ be its Lie algebra. Fix the Cartan subgroup $H$ contained in
the Borel subgroup $B$. The group $B$ has the unipotent radical $N_{+}$ with
the Lie algebra ${\mathfrak{n}}_{+}$. Applying the Killing form
$(\cdot,\cdot)$ of ${\mathfrak{g}}$, we identify ${\mathfrak{n}}^{*}$ with the
opposite subalgebra ${\mathfrak{n}}_{-}$.

Let $P$ be a parabolic subgroup containing $B$. The subgroup $P$ is a
semidirect product $P=LU_{+}$, where $L$ is the Levi subgroup containing $H$,
and $U_{+}$ is the unipotent radical contained in $N_{+}$. The Lie algebra
${\mathfrak{p}}$ of the group $P$ is a semidirect sum
${\mathfrak{p}}={\mathfrak{l}}\oplus{\mathfrak{u}}_{+}$ of the Lie subalgebra
${\mathfrak{l}}=\mathrm{Lie}(L)$ and the ideal
${\mathfrak{u}}_{+}=\mathrm{Lie}(U_{+})$.

As above we identify ${\mathfrak{u}}_{-}$ with ${\mathfrak{u}}_{+}^{*}$.
Consider the commutative Lie algebra ${\mathfrak{u}}_{-}^{a}$ that coincides
with ${\mathfrak{u}}_{-}$ as a linear space and the abelian group
$U_{-}^{a}=1+{\mathfrak{u}}_{-}^{a}$ with multiplication
$(1+{\lambda}_{1})(1+{\lambda}_{2})=1+{\lambda}_{1}+{\lambda}_{2}$.

_A parabolic contraction_ of the group $G$ is a semidirect product
$G^{a}=P\ltimes U_{-}^{a}$; the subgroup $P$ acts on $U^{a}_{-}$ by the
formula $p(1+{\lambda})p^{-1}=1+{\mathrm{Ad}}_{p}^{*}({\lambda})$, where $p\in
P$ and ${\lambda}\in{\mathfrak{u}}_{-}^{a}={\mathfrak{u}}^{*}$. Respectively,
its Lie algebra
${\mathfrak{g}}^{a}={\mathfrak{p}}\oplus{\mathfrak{u}}_{-}^{a}$ (_a parabolic
contraction_ of the Lie algebra ${\mathfrak{g}}$) is a semidirect sum of the
Lie subalgebra ${\mathfrak{p}}$ and the ideal ${\mathfrak{u}}_{-}^{a}$. These
definitions are matched with the general definition of Lie algebra contraction
from [8, 9].

##  3 Supercharacters and superclasses of the group ${\mathrm{GL}}^{a}(n)$

Let ${\mathrm{GL}}(n)$ be the general linear group over a finite field
${\mathbb{F}}_{q}$ of $q$ elements and of characteristic $\neq 2$. Fix a
parabolic subgroup $P$ of ${\mathrm{GL}}(n)$ and consider its parabolic
contraction ${\mathrm{GL}}^{a}(n)$. The parabolic subgroup $P$ is defined by a
decomposition of $[1,n]$ into the system of consecutive segments
$\mathcal{I}=I_{1}\sqcup\ldots\sqcup I_{\ell}$. Let $n_{i}=|I_{i}|$. Then
$L={\mathrm{GL}}(n_{1})\times\ldots\times{\mathrm{GL}}(n_{\ell})$.

Emphasize the Lie algebra $\mathfrak{gl}^{a}(n)$ of the group
${\mathrm{GL}}^{a}(n)$ is an associative algebra with respect to the
multiplication for which ${\mathfrak{p}}$ is a subalgebra with matrix
multiplication, ${\mathfrak{u}}_{-}^{a}$ is a subalgebra with zero
multiplication and

$t{\lambda}(x)={\lambda}(xt)\quad\mbox{and}\quad{\lambda}t(x)={\lambda}(tx),$
(1)

where $t\in{\mathfrak{p}}$, $x\in{\mathfrak{u}}_{+}$, and
${\lambda}\in{\mathfrak{u}}_{-}^{a}$. The group ${\mathrm{GL}}^{a}(n)$ is
realized as the subgroup of invertible elements of the associative algebra
$\mathfrak{gl}^{a}(n)$.

The group ${\mathrm{GL}}^{a}(n)$ is a semidirect product
${\mathrm{GL}}^{a}(n)=L\ltimes U^{a}$, where $U^{a}=U_{+}\ltimes U_{-}^{a}$.
The subgroup $U^{a}$ is an algebra subgroup $U^{a}=1+{\mathfrak{u}}^{a}$,
where ${\mathfrak{u}}^{a}={\mathfrak{u}}_{+}+{\mathfrak{u}}_{-}^{a}$ is the
nilradical of $\mathfrak{gl}^{a}(n)$.

Our goal is to construct a supercharacter theory for the group
${\mathrm{GL}}^{a}(n)$ following the general method from the paper [10] (where
it is refered to as the $GG$-supercharacter theory).

###  3.1 Supercharacter theory for $U^{a}$

Because $U^{a}$ is an algbera subgroup, it has the supercharacter theory of P.
Diaconis and I.M. Isaacs from [1]. The superclasses in algebra group
${\mathfrak{G}}=1+J$ is the subsets of the form
$1+{\mathfrak{G}}x{\mathfrak{G}}$, where $x\in J$. The supercharacters
$\chi_{\lambda}$ of ${\mathfrak{G}}$ are constructed by the following
procedure. Fix the nontrivial character $t\mapsto{\varepsilon}^{t}$ of the
additive group of field ${\mathbb{F}}_{q}$ with values in ${\mathbb{C}}^{*}$.
Consider the system of representatives $\\{{\lambda}\\}$ of the left-right
orbits of the group ${\mathfrak{G}}\times{\mathfrak{G}}$ in $J^{*}$. By each
${\lambda}$, we construct the linear character

$\xi_{\lambda}(1+x)={\varepsilon}^{{\lambda}(x)}$

of the stabilizer ${\mathfrak{G}}_{{\lambda},\mathrm{right}}$ of the linear
form ${\lambda}$ with respect to the right action of ${\mathfrak{G}}$ on
$J^{*}$ (see formula (1)). The supercharacter $\chi_{\lambda}$ of the algebra
group ${\mathfrak{G}}$ is induced from $\xi_{\lambda}$ to ${\mathfrak{G}}$.
There is a formula for supercharacter

$\chi_{\lambda}(1+x)=\frac{|{\mathfrak{G}}{\lambda}|}{|{\mathfrak{G}}{\lambda}{\mathfrak{G}}|}\sum_{\mu\in{\mathfrak{G}}{\lambda}{\mathfrak{G}}}{\varepsilon}^{\mu(x)}.$

So, the superclasses and supercharacters of an algebra group ${\mathfrak{G}}$
are uniquely determined by $({\mathfrak{G}}\times{\mathfrak{G}})$-orbits in
$J$ and $J^{*}$ respectively.

In our case, ${\mathfrak{G}}=U^{a}$ and a classification of $(U^{a}\times
U^{a})$-orbits in ${\mathfrak{u}}^{a}$ and $({\mathfrak{u}}^{a})^{*}$ is
considered an extremely difficult problem (see [10, 14, 16]). Therefore, we
consider the more ”coarse” supercharacter theory (see Theorem 3.1) in which
superclasses are attached to the
$({\mathrm{GL}}^{a}(n)\times{\mathrm{GL}}^{a}(n))$-orbits in
${\mathfrak{u}}^{a}$, and supercharacters a up to a constant multiple equal to
sums of functions ${\varepsilon}^{\mu(x)}$, where $\mu$ runs through a
$({\mathrm{GL}}^{a}(n)\times{\mathrm{GL}}^{a}(n))$-orbit in
$({\mathfrak{u}}^{a})^{*}$.

Let $\Delta$ denote the root system of the Lie algebra $\mathfrak{gl}(n)$. We
attach to each root ${\alpha}=\epsilon_{i}-\epsilon_{j}$, $1\leqslant
i,~{}j\leqslant n$, $i\neq j$, the pair $(i,j)$. To simplify notations, we
write ${\alpha}=(i,j)$. We refer to $i$ as a row number of the root ${\alpha}$
and to $j$ as a column number of root ${\alpha}$. Notations:
$i={\mathrm{row}}({\alpha})$ and $j={\mathrm{col}}({\alpha})$.

A positive root ${\alpha}\in\Delta_{+}$ is attached to a pair with $i<j$.
Respectively, for an negative root ${\alpha}\in\Delta_{-}$, we have $i>j$.

We introduce the definition of _sum of roots_ , which if different from usual
one: a root $\gamma$ is a sum $\gamma={\alpha}+\beta$ if ${\alpha}=(i,j)$,
$\beta=(j,k)$ and $\gamma=(i,k)$.

To each root ${\alpha}\in\Delta_{+}$, we attach the matrix unit
$E_{\alpha}\in{\mathfrak{n}}_{+}$ and to each root ${\alpha}\in\Delta_{-}$ –
the matrix unit $F_{\alpha}\in{\mathfrak{n}}_{-}^{a}$.

Consider the subset $\Delta_{+}({\mathfrak{u}}^{a})$ which consists of
${\alpha}\in\Delta_{+}$ such that $E_{\alpha}\in{\mathfrak{u}}_{+}$.
Respectively,
$\Delta_{-}({\mathfrak{u}}^{a})=\\{{\alpha}\in\Delta_{-}:~{}F_{\alpha}\in{\mathfrak{u}}_{-}^{a}\\}$
and
$\Delta({\mathfrak{u}}^{a})=\Delta_{+}({\mathfrak{u}}^{a})\cup\Delta_{-}({\mathfrak{u}}^{a})$.

We realize ${\mathfrak{u}}^{a}$ as the subspace of the linear space of
$(n\times n)$-matrices spanned by the union of two systems of matrix units

$\\{E_{\alpha}:~{}{\alpha}\in\Delta_{+}({\mathfrak{u}}^{a})\\}\cup\\{F_{\alpha}:~{}{\alpha}\in\Delta_{-}({\mathfrak{u}}^{a})\\}.$

The structure relations have the form

  * •

$E_{\alpha}E_{\beta}=E_{{\alpha}+\beta}$ if ${\alpha}+\beta$ is defined,
otherwise $E_{\alpha}E_{\beta}=0$.

  * •

$F_{\alpha}F_{\beta}=0$.

  * •

$E_{\alpha}F_{\beta}=F_{{\alpha}+\beta}$ if ${\alpha}+\beta$ is defined and
belongs to $\Delta_{-}({\mathfrak{u}}^{a})$, otherwise
$E_{\alpha}F_{\beta}=0$.

  * •

$F_{\alpha}E_{\beta}=F_{{\alpha}+\beta}$ if ${\alpha}+\beta$ is defined and
belongs to $\Delta_{-}({\mathfrak{u}}^{a})$, otherwise
$F_{\alpha}E_{\beta}=0$.

Observe that
${\mathfrak{n}}_{+}={\mathfrak{u}}_{+}\oplus{\mathfrak{n}}_{+}(L)$, where
${\mathfrak{n}}_{+}(L)={\mathfrak{n}}_{+}\cap{\mathfrak{l}}$. Respectively,
${\mathfrak{n}}_{-}^{a}={\mathfrak{u}}_{-}^{a}\oplus{\mathfrak{n}}^{a}_{-}(L)$,
where
${\mathfrak{n}}^{a}_{-}(L)=\\{{\lambda}\in{\mathfrak{n}}_{-}^{a}:~{}{\lambda}({\mathfrak{u}}_{+})=0\\}$.
Lemma 3.1. 1) ${\mathfrak{n}}^{a}_{-}(L)$ is a two sided ideal in the
associative algebra $\mathfrak{gl}^{a}(n)$.
2) $\mu_{L}x=x\nu_{L}=0$ for all $x\in{\mathfrak{u}}_{+}$ and
$\mu_{L},~{}\nu_{L}\in{\mathfrak{n}}^{a}_{-}(L)$.
Proof. The both statements follows from the fact that ${\mathfrak{u}}_{+}$ is
a two sided ideal in ${\mathfrak{n}}_{+}$. $\Box$

For an arbitrary element $u=1+X\in U^{a}$, where $X\in{\mathfrak{u}}^{a}$, we
consider the class

$K(u)=\\{1+AXB:~{}A,B\in{\mathrm{GL}}^{a}(n)\\}.$

The following Theorem 3.1 gives a classification of classes $K(u)$. We need
new definitions.

_A rook spacement_ in $\Delta$ is a subset $D\subset\Delta$ that has at most
one root in any row and colunm. The rook spacement $D$ is a union $D=D_{+}\cup
D_{-}$, where $D_{\pm}=D\cap\Delta_{\pm}$. In his papers, C. André used the
other term _basic subset_ for the root placements in $\Delta_{+}$.

To each rook placement $D\subset\Delta({\mathfrak{u}}^{a})$ and a map
$\phi:D\to{\mathbb{F}}_{q}^{*}$, we attach the element
$X_{D,\phi}\in{\mathfrak{u}}^{a}$ of the form

$X_{D,\phi}=\sum_{\gamma_{+}\in
D_{+}}\phi(\gamma_{+})E_{\gamma_{+}}+\sum_{\gamma_{-}\in
D_{-}}\phi(\gamma_{-})F_{\gamma_{-}}.$ (2)

If $\phi$ is identically equals to one on $D$, we take $X_{D,\phi}=X_{D}$.
Denote $u_{D}=1+X_{D}$.

For any rook placement $D\subset\Delta$ and pairs of segments $I_{k},I_{m}$,
where $k\neq m$, we define

$r_{ij}=r_{ij}(D)=\mathrm{rank}\,X_{D}(I_{k}\times I_{m})=|D\cap(I_{k}\times
I_{m})|,$ $n^{\prime}_{km}=\sum_{s<k}n_{s}+\sum_{t<m}r_{kt},$
$n^{\prime\prime}_{km}=\sum_{s<m}n_{s}+\sum_{t<k}r_{tm}.$

We say that a rook placement $D_{c}$ has a _canonical type_ if for each pair
$1\leqslant k,m\leqslant\ell$, $k\neq m$, the subset $D_{c}\cap(I_{k}\times
I_{m})$ consists of roots

$(n_{km}^{\prime}+1,n_{km}^{\prime\prime}+1),\ldots,(n_{km}^{\prime}+r_{km},n_{km}^{\prime\prime}+r_{km}).$

Let $W_{L}$ be a Weyl group in $L$. For each $w\in W_{L}$, the action of $w$
on the row numbers (respectively, the action of $w^{-1}$ on the column
numbers) of roots from $\Delta$ defines the left (respectively, the right)
action of $w$ on $\Delta$.
Lemma 3.2. For any rook placement $D$ there exist elements $w_{1},w_{2}\in
W_{L}$ such that $w_{1}Dw_{2}$ is a rook placement $D_{c}$ of canonical type.
The rook placement $D_{c}$ is uniquely determined by $D$.
Proof. For any rook placement $D$, there exists $w_{1}\in W_{L}$ such that the
system ${\mathrm{row}}(w_{1}D)$ has the required ordering. Multiplying by a
suitable $w_{2}\in W_{L}$ we obtain the rook placement $D_{c}=w_{1}Dw_{2}$ of
canonical type. Uniqueness follows from the fact that the numbers
$\\{r_{km}\\}$ don’t change if we replace $D$ by $w_{1}Dw_{2}$. $\Box$

In the case $P=B$, the next theorem is proved in [11].
Theorem 3.3. 1) Each class contains the element $u_{D}$ for some rook
placement $D$ in the rook system $\Delta({\mathfrak{u}}^{a})$.
2) Two elements $u_{D}$ and $u_{D^{\prime}}$ belong to a common class if and
only if $D$ and $D^{\prime}$ are conjugate with respect to the left-right
action of the double Weyl group $W_{L}\times W_{L}$ on the set of roots
$\Delta({\mathfrak{u}}^{a})$.
Proof. _Item 1._ Let $u=1+X$ and $X\in{\mathfrak{u}}^{a}$. The associative
algebra ${\mathfrak{u}}^{a}$ is a subalgebra in the factor algebra of
${\mathfrak{n}}^{a}={\mathfrak{n}}_{+}\ltimes{\mathfrak{n}}_{-}^{a}$ modulo
the two sided ideal ${\mathfrak{n}}^{a}_{-}(L)$. There exists
$\overline{X}\in{\mathfrak{n}}^{a}$ such that
$X=\overline{X}\bmod{\mathfrak{n}}^{a}_{-}(L)$. It implies from [11, Theorem
3.4] there exists $A,B\in{\mathrm{UT}}(n)\ltimes N_{-}^{a}$ such that

$A\overline{X}B=X_{\overline{D},\overline{\phi}}$

for some rook placement $\overline{D}\subset\Delta$ and a map
$\overline{\phi}:\overline{D}\to{\mathbb{F}}_{q}^{*}$. Decompose
$A=A_{1}(1+\mu_{L})$ and $B=(1+\nu_{L})B_{1}$, where
$A_{1},B_{1}\in{\mathrm{UT}}(n)\ltimes U_{-}^{a}$ and
$\mu_{L},\nu_{L}\in{\mathfrak{n}}^{a}_{-}(L)$. Since
$\overline{X}=x+{\lambda}$ with $x\in{\mathfrak{u}}_{+}$ and
${\lambda}\in{\mathfrak{u}}_{-}^{a}$, by Lemma 3.1, we get
$(1+\mu_{L})\overline{X}(1+\nu_{L})=\overline{X}$. Then

$A_{1}\overline{X}B_{1}=X_{\overline{D},\overline{\phi}}.$ (3)

We denote $D=\overline{D}\cap\Delta({\mathfrak{u}}^{a})$ and
$\phi=\overline{\phi}|_{D}$. Taking the equality (3) modulo the ideal
${\mathfrak{n}}^{a}_{-}(L)$, we obtain $A_{1}XB_{1}=X_{D,\phi}$. Acting by a
suitable $h\in H$, we get $hX_{D,\phi}=X_{D}\in K(u)$, this proves 1).
_Item 2._ Suppose that $AX_{D}B=X_{D^{\prime}}$, where
$A,B\in{\mathrm{GL}}^{a}(n)$. For any $X\in{\mathfrak{u}}^{a}$ and $1\leqslant
k,m\leqslant n$, $k\neq m$, we denote by $r_{km}(X)$ rank of the $I_{k}\times
I_{m}$ block of the matrix $X$. By Lemma 3.1, to prove statement 2), it is
sufficient to show that $r_{km}(X_{D})=r_{km}(X_{D^{\prime}})$ for all $k\neq
m$. We need the other realization of $\mathfrak{gl}^{a}(n)$ and also
${\mathfrak{u}}^{a}$.

Consider the subalgebra $\underline{\mathfrak{gl}}^{a}(n)$ of
$\mathrm{Mat}(2n)$ that consists of matrices $\left(\begin{array}[]{cc}A&B\\\
0&A\end{array}\right)$, where $A\in{\mathfrak{p}}$. The algebra
$\mathfrak{gl}^{a}(n)$ is isomorphic to the factor algebra of
$\underline{\mathfrak{gl}}^{a}(n)$ modulo the ideal
$\left\\{\left(\begin{array}[]{cc}0&B\\\ 0&0\end{array}\right)\right\\}$,
where $B\in{\mathfrak{p}}$. Consider the decomposition of segment $[1,2n]$
into consecutive segments

$I_{1},\ldots,I_{\ell},\tilde{I}_{1},\ldots,\tilde{I}_{\ell},$

where $I_{1},\ldots,I_{\ell}$ as above and for each $1\leqslant
m\leqslant\ell$ if $I_{m}=[a,b]$ then $\tilde{I}_{m}=[a+n,b+n]$. We realize
the algebra $\mathfrak{gl}^{a}(n)$ as the subspace in
$\underline{\mathfrak{gl}}^{a}(n)$ consisting of matrices with zero
$I_{k}\times\tilde{I}_{m}$, $k\leqslant m$, blocks.
Example. If $n=2$ and ${\mathfrak{p}}$ is a Borel subalgebra, then

$\mathfrak{gl}^{a}(n)=\left\\{\left(\begin{array}[]{cccc}a_{11}&a_{12}&0&0\\\
0&a_{22}&b_{22}&0\\\ 0&0&a_{11}&a_{12}\\\ 0&0&0&a_{22}\\\
\end{array}\right)\right\\},\quad{\mathfrak{u}}^{a}=\left\\{\left(\begin{array}[]{cccc}0&a_{12}&0&0\\\
0&0&b_{22}&0\\\ 0&0&0&a_{12}\\\ 0&0&0&0\\\ \end{array}\right)\right\\}$

Let $X\in{\mathfrak{u}}^{a}$. For any $1\leqslant k<m\leqslant\ell$, we denote
by $R_{km}(X)$ the rank of submatrix of $X$ with the system of rows and
columns $I_{k}\cup\ldots\cup I_{m}$. For any $1\leqslant m<k\leqslant\ell$, we
denote by $\tilde{R}_{km}(X)$ the rank of submatrix of $X$ with the system of
rows and columns $I_{k}\cup\ldots\cup
I_{\ell}\cup\tilde{I}_{1}\cup\ldots\cup\tilde{I}_{m}$. Observe that the ranks
of $R_{km}(X)$ and $\tilde{R}_{km}(X)$ preserve if we replace $X$ by $AXB$ for
any $A,~{}B\in{\mathrm{GL}}^{a}(n)$. We have
$r_{km}(X_{D})=r_{km}(X_{D^{\prime}})$ for any $k\neq m$. The canonical forms
$D$ and $D^{\prime}$ coincide (see Lemma 3.1). $\Box$

The proof of Theorem 3.1 implies the following consequence.
Corollary 3.4. The elements $u_{D}$ and $u_{D^{\prime}}$ belong to a common
class if and only if $r_{ij}(D)=r_{ij}(D^{\prime})$ for all $1\leqslant
i,~{}j\leqslant\ell$, $i\neq j$.

Turn to a classification of the
$({\mathrm{GL}}^{a}(n)\times{\mathrm{GL}}^{a}(n))$-orbits in
$({\mathfrak{u}}^{a})^{*}$ and construction of supercharacters for $U^{a}$. We
define the non-degenerate bilinear form on ${\mathfrak{u}}^{a}$ by the formula

$(X_{1},X_{2})={\lambda}_{2}(x_{1})+{\lambda}_{1}(x_{2}),$ (4)

where $X_{1}=x_{1}+{\lambda}_{1}$ and $X_{2}=x_{2}+{\lambda}_{2}$. One can
directly calculate

$(gX_{1},X_{2})=(X_{1},X_{2}g),$ (5)

for all $g\in G^{a}$. For any linear form
${\Lambda}\in({\mathfrak{u}}^{a})^{*}$, there exists
$X_{\Lambda}\in{\mathfrak{u}}^{a}$ such that ${\Lambda}(Y)=(X_{\Lambda},Y)$
for any $Y\in{\mathfrak{u}}^{a}$. Taking into account formula (5), one can
identify the left-right
$({\mathrm{GL}}^{a}(n)\times{\mathrm{GL}}^{a}(n))$-orbit of ${\Lambda}$ in
$({\mathfrak{u}}^{a})^{*}$ with the left-right orbit of $X_{\Lambda}$ in
${\mathfrak{u}}^{a}$.

For an arbitrary rook placement $D$ and a map $\phi:D\to{\mathbb{F}}_{q}^{*}$,
we define the linear form

${\Lambda}_{D,\phi}=\sum_{\gamma_{+}\in
D_{+}}\phi(\gamma_{+})E^{*}_{\gamma_{+}}+\sum_{\gamma_{-}\in
D_{-}}\phi(\gamma_{-})F^{*}_{\gamma_{-}}.$ (6)

If $\phi$ is identically equal to one, we denote
${\Lambda}_{D,\phi}={\Lambda}_{D}$.

Observe that ${\Lambda}_{D}(Y)=(X_{D^{t}},Y)$, where $D^{t}$ consists of all
roots opposite to the roots from $D$. It follows from Theorem 3.1 that any
$({\mathrm{GL}}^{a}(n)\times{\mathrm{GL}}^{a}(n))$-orbit in
$({\mathfrak{u}}^{a})^{*}$ contains a unique element of the form
${\Lambda}_{D}$.

Let us describe the stabilizer of ${\Lambda}_{D}$ with respect to the right
action of $U^{a}$ on $({\mathfrak{u}}^{a})^{*}$. For each
$\gamma\in\Delta({\mathfrak{u}}^{a})$, consider the subset
${\mathcal{S}}(\gamma)={\mathcal{S}}_{+}(\gamma)\cup{\mathcal{S}}_{-}(\gamma)$,
where
${\mathcal{S}}_{\pm}(\gamma)={\mathcal{S}}(\gamma)\cap\Delta_{\pm}(U^{a})$ is
defined as follows:

  1. 1.

If $\gamma\in I_{k}\times I_{m}$ for some $k<m$ (i.e. $\gamma>0$), then
${\mathcal{S}}_{-}(\gamma)=\varnothing$ and ${\mathcal{S}}_{+}(\gamma)$
consists of all roots ${\alpha}\in I_{k}\times I_{t}$ such that
${\mathrm{row}}({\alpha})={\mathrm{row}}(\gamma)$ and $k<t<m$.

  2. 2.

If $\gamma\in I_{k}\times I_{m}$ for some $k>m$ (i.e. $\gamma<0$), then
${\mathcal{S}}_{-}(\gamma)$ consists of all roots ${\alpha}\in I_{k}\times
I_{t}$ such that ${\mathrm{row}}({\alpha})={\mathrm{row}}(\gamma)$ and
$1\leqslant t<m$; ${\mathcal{S}}_{+}(\gamma)$ consists of all roots
${\alpha}\in I_{k}\times I_{t}$ such that
${\mathrm{row}}({\alpha})={\mathrm{row}}(\gamma)$ and $k<t\leqslant\ell$.

Let ${\mathfrak{u}}^{a}_{\gamma}$ be the subspace spanned by the matrix units
$E_{\beta}$, $\beta\notin{\mathcal{S}}_{+}(\gamma)$, and $F_{\beta}$,
$\beta\notin{\mathcal{S}}_{-}(\gamma)$. The subspace
${\mathfrak{u}}^{a}_{\gamma}$ is a subalgebra (more precisely, right ideal) in
${\mathfrak{u}}^{a}$. Respectively,
$U^{a}_{\gamma}=1+{\mathfrak{u}}^{a}_{\gamma}$ is an algebra subgroup in
$U^{a}$.

Define

${\mathcal{S}}(D)=\bigcup_{\gamma\in
D}{\mathcal{S}}(\gamma),\quad\quad{\mathfrak{u}}^{a}_{D}=\bigcap_{\gamma\in
D}{\mathfrak{u}}^{a}_{\gamma},\quad\quad
U^{a}_{D}=1+{\mathfrak{u}}^{a}_{D}=\bigcap_{\gamma\in D}U^{a}_{\gamma}.$

The next statement is proved by direct calculations.
Proposition 3.5. The subgroup $U^{a}_{D}$ is the stabilizer of ${\Lambda}_{D}$
with respect to the right action of $U^{a}$ on $({\mathfrak{u}}^{a})^{*}$.

The formula

$\xi_{D}=\sum_{p\in L}{\varepsilon}^{p{\Lambda}_{D}(X)},\quad u=1+X,\quad
X\in{\mathfrak{u}}^{a}_{D}$

defines a character of the group $U^{a}_{D}$. Consider the character
$\chi_{D}$ of ${\mathrm{GL}}^{a}(n)$ induced from the character $\xi_{D}$ of
the subgroup $U^{a}_{D}$. The following formula is true for $\chi_{D}$:

$\zeta_{D}(u)=\frac{|{\Lambda}_{D}U^{a}|}{|{\mathrm{GL}}^{a}(n){\Lambda}_{D}{\mathrm{GL}}^{a}(n)|}\sum_{\mu\in{\mathrm{GL}}^{a}(n){\Lambda}_{D}{\mathrm{GL}}^{a}(n)}{\varepsilon}^{\mu(X)}$
(7)

where $u=1+X$ (see [10, Corollary 3.1]). Denote
$K_{D}=1+{\mathrm{GL}}^{a}(n)X_{D}{\mathrm{GL}}^{a}(n)$.
Theorem 3.6. The system of characters $\\{\zeta_{D}\\}$ and the decomposition
of the group $U^{a}$ into classes $\\{K_{D}\\}$ give rise to a supercharacter
theory of the group $U^{a}$.
Proof. It follows from Theorem 3.1 and [10, theorem 3.3]. $\Box$

###  3.2  Supercharacter theory for ${\mathrm{GL}}^{a}(n)$

In this subsection, following the general method of paper [10], we construct a
supercharacter theory the group ${\mathrm{GL}}^{a}(n)$.

For each $\gamma\in\Delta({\mathfrak{u}}^{a})$, we consider the subgroup
$H_{\gamma}$ as follows. If $\gamma>0$ and $\gamma\in I_{k}\times I_{m}$,
$1\leqslant k<m\leqslant\ell$, then the subgroup $H_{\gamma}$ consists of all
$h=(h_{1},\ldots,h_{\ell})\in L$, $h_{i}\in{\mathrm{GL}}(n_{i})$ such that
$(h_{k},\ldots,h_{m})$ is a scalar submatrix.

If $\gamma<0$ and $\gamma\in I_{k}\times I_{m}$, $1\leqslant
m<k\leqslant\ell$, then the subgroup $H_{\gamma}$ consists of all
$h=(h_{1},\ldots,h_{\ell})\in L$ such that
$(h_{1},\ldots,h_{m},h_{k},\ldots,h_{\ell})$ is a scalar submatrix.
Example. If $n=4$, $P=B$, $\gamma_{1}=(1,3)$, and $\gamma_{2}=(3,1)$, then

$H_{\gamma_{1}}=\left\\{\left(\begin{array}[]{cccc}a&0&0&0\\\ 0&a&0&0\\\
0&0&a&0\\\ 0&0&0&b\\\ \end{array}\right)\right\\},\quad
H_{\gamma_{2}}=\left\\{\left(\begin{array}[]{cccc}a&0&0&0\\\ 0&b&0&0\\\
0&0&a&0\\\ 0&0&0&a\\\ \end{array}\right)\right\\}$

For a rook placement $D$, we define

$H_{D}=\bigcap_{\gamma\in D}H_{\gamma}.$

The next statement if proved by direct calculations.
Proposition 3.7. For any rook placement $D\subset\Delta({\mathfrak{u}}^{a})$,
the subgroup $H_{D}$ consists of all $h\in L$ such that
${\mathrm{Ad}}^{*}_{h}$ stabilizes all linear forms from
${\mathrm{GL}}^{a}(n){\Lambda}_{D}{\mathrm{GL}}^{a}(n)$.

Consider the subset ${\cal A}$ of all pairs $(D,\theta)$, where $D$ is a rook
placement contained in $\Delta({\mathfrak{u}}^{a})$, and $\theta$ is an
irreducible character of the subgroup $H_{D}$. Construct the subgroup
$G^{a}_{D}=H_{D}U^{a}_{D}$. The formula

$\xi_{\theta,{\Lambda}_{D}}(hu)=\theta(h){\varepsilon}^{{\Lambda}_{D}(X)},\quad
u=1+X,\quad X\in{\mathfrak{u}}^{a}_{D}$

defines the character of subgroup $G^{a}_{D}$. Consider the character
$\chi_{\alpha}$ of group ${\mathrm{GL}}^{a}(n)$ induced from the character

$\sum_{p\in L}\xi_{\theta,p{\Lambda}_{D}}$

of subgroup $G^{a}_{D}$. There is a formula for the character $\chi_{\alpha}$
(see. [10, Corollary 3.1]):

$\chi_{\alpha}(g)=\frac{|L|^{2}}{|H_{D}|}\cdot\dot{\theta}(h)\zeta_{D}(u),$

where $g=hu$, $\zeta_{D}(u)$ are from (7), $\dot{\theta}(h)=\theta(h)$ for
$h\in H_{D}$ and equals to zero for $h\notin H_{D}$.

Turn to superclasses of the group ${\mathrm{GL}}^{a}(n)$. Let $h\in L$.
Consider the smallest
${\mathrm{GL}}^{a}(n)\times{\mathrm{GL}}^{a}(n)$-invariant ideal
${\mathfrak{u}}^{a}_{h}$ in ${\mathfrak{u}}^{a}$ such that ${\mathrm{Ad}}_{h}$
is identical on ${\mathfrak{u}}^{a}/{\mathfrak{u}}^{a}_{h}$.
Example. Let $n=5$, $P=B$. We have

$h=\left(\begin{array}[]{cccc}a&0&0&0\\\ 0&a&0&0\\\ 0&0&b&0\\\ 0&0&0&a\\\
\end{array}\right),\mbox{where}~{}a\neq
b,\quad{\mathfrak{u}}^{a}_{h}=\left\\{\left(\begin{array}[]{cccc}0&0&*&*\\\
\,*&0&*&*\\\ \,*&*&0&*\\\ 0&0&*&0\\\ \end{array}\right)\right\\}$

Consider the subset ${\cal B}$ of all pairs $(D,h)$, where $D$ is a rook
placement contained in $\Delta({\mathfrak{u}}^{a})$, and $h\in H_{D}$. Let
$\omega_{D}$ be an orbit of the element $X_{D}$ with respect to
${\mathrm{GL}}^{a}(n)\times{\mathrm{GL}}^{a}(n)$. We denote

$K_{\beta}=CL_{L}(h)\cdot(1+\pi_{h}^{-1}\pi_{h}(\omega_{D})),$

where $CL_{L}(h)$ is the conjugacy class of $h$ in $L$.
Theorem 3.8. The system of characters $\\{\chi_{\alpha}:~{}{\alpha}\in{\cal
A}\\}$ and the decomposition into classes $\\{K_{\beta}:~{}\beta\in{\cal
B}\\}$ give rise to a supercharacter theory of the group
${\mathrm{GL}}^{a}(n)$.
Proof. It implies from [10, Theorem 3.3]. $\Box$

##  4 Contractions of the orthogonal and symplectic groups

As above ${\mathbb{F}}_{q}$ is a finite field of $q$ elements and of
characteristic $\neq 2$. In this section, we use the notation
${\mathbb{G}}={\mathrm{GL}}(M,{\mathbb{F}}_{q})$. Respectively, for the
subgroups in ${\mathbb{G}}$ we use new notations ${\mathbb{P}}$,
${\mathbb{L}}$, ${\mathbb{H}}$, ${\mathbb{U}}_{\pm}$ and ${\mathbb{N}}_{\pm}$.
The orthogonal (symplectic) subgroup $G$ is defined by the involutive
antiautomorphism $X\to X^{\dagger}$.

Let $M=2n+1$ for case $B_{n}$ and $M=2n$ for $C_{n}$ and $D_{n}$. Denote by
${\mathbb{I}}_{M}$ the $N\times N$ matrix in which all entries on the
antidiagonal are equal to $1$ and all other entries are zeros. Consider the
involutive antiautomorphism
$X^{\dagger}={\mathbb{I}}_{M}X^{t}{\mathbb{I}}_{M}$ of the algebra of $M\times
M$ matrices over the field ${\mathbb{F}}_{q}$. The orthogonal group $O(M)$
consists of all matrices $g\in{\mathbb{G}}$ such that $g^{\dagger}=g^{-1}$.
The symplectic group ${\mathrm{Sp}}(M)$ is defined similarly applying the
involutive automorphism $X^{\dagger}={\mathbb{J}}_{2n}X^{t}{\mathbb{J}}_{2n}$,
where $M=2n$ and
${\mathbb{J}}_{2n}=\left(\begin{array}[]{cc}0&{\mathbb{I}}_{n}\\\
-{\mathbb{I}}_{n}&0\end{array}\right).$ Respectively, the Lie algebra
${\mathfrak{g}}$ of group $G$ coincides with the Lie algebra
$\\{X\in\mathfrak{gl}(M):~{}X^{\dagger}=-X\\}$.

We enumerate the rows and columns of $M\times M$ matrices as follows:

$\begin{array}[]{c}n>\ldots>1>0>-1>\ldots>-n\quad\mbox{for}\quad M=2n+1,\\\
n>\ldots>1>-1>\ldots>-n\quad\mbox{for}\quad M=2n.\end{array}$

Consider the partition $\mathcal{I}$ of segment $I=[-n,n]$ in case $B_{n}$
(respectively, $I=[-n,n]\setminus\\{0\\}$ in cases $C_{n}$ and $D_{n}$) into
the system of consecutive segments. We enumerate the components of partition
$\mathcal{I}=\\{I_{\ell},\ldots,I_{0},\ldots,I_{-\ell}\\}$ comparably with the
enumeration of rows and columns. Suppose that the partition $\mathcal{I}$ is
symmetric relative to zero. By the partition $\mathcal{I}$, we construct the
parabolic subgroup ${\mathbb{P}}={\mathbb{L}}{\mathbb{U}}_{+}$ and the
complementary unipotent subgroup ${\mathbb{U}}_{-}$. As above
${\mathbb{U}}_{\pm}=1+{\mathcal{U}}_{\pm}$ are the algebra subgroups.

Intersection of these subgroups with $G$ defined the parabolic subgroup
$P=LU_{+}$ and the complementary unipotent subgroup $U_{-}$. The subgroup $L$
consists of all matrices
$\mathrm{diag}(A_{\ell},\ldots,A_{0},\ldots,A_{-\ell})$, where for each
$\ell\geqslant i\geqslant{-\ell}$ the transpose of matrix $A_{i}$ about its
anti-diagonal coincides with $A_{-i}^{-1}$. We preserve notations
${\mathfrak{u}}_{\pm}$ for Lie algebras of unipotent groups $U_{\pm}$. We
identify ${\mathfrak{u}}_{-}$ with ${\mathfrak{u}}^{*}$.

We extend the involutive antiautomorphism of subgroup ${\mathbb{P}}$ to the
one of ${\mathbb{G}}^{a}={\mathbb{P}}\ltimes{\mathbb{U}}_{-}^{a}$ setting
$(1+{\lambda})^{\dagger}=1-{\lambda}$, ${\lambda}\in{\mathcal{U}}_{-}^{a}$,
and $(gv)^{\dagger}=v^{\dagger}g^{\dagger}$, $g\in{\mathbb{P}}$,
$v=1+{\lambda}$. The parabolic contraction $G^{a}$ of orthogonal and
symplectic groups consists of all elements $gv\in{\mathbb{G}}^{a}$,
$g\in{\mathbb{P}}$ and $v\in{\mathbb{U}}_{-}^{a}$ obeying
$(gv)^{\dagger}=(gv)^{-1}$.

The group $G^{a}$ is a semidirect product of $L$ and the unipotent subgroup
$U^{a}=U\ltimes U_{-}^{a}$ with Lie algebra
${\mathfrak{u}}^{a}={\mathfrak{u}}_{+}\oplus{\mathfrak{u}}_{-}^{a}$.

The group ${\mathbb{G}}^{a}$ acts on ${\mathfrak{u}}^{a}$ by the formula

$g\centerdot X=gXg^{\dagger},$ (8)

where $g\in{\mathbb{G}}^{a}$ and $X\in{\mathfrak{u}}^{a}$. We define the
action of ${\mathbb{G}}^{a}$ on $({\mathfrak{u}}^{a})^{*}$ by

$g\centerdot{\lambda}(X)={\lambda}(g^{\dagger}Xg).$

Consider the Cayley map $f:{\mathbb{U}}^{a}\to{\mathcal{U}}^{a}$; its value
$f(u)$ at $u=1+X\in{\mathbb{U}}^{a}$ is calculated by the formula

$f(u)=\frac{2(u-1)}{u+1}=\frac{2X}{X+2}=\sum_{k=0}^{\infty}(-1)^{k}\frac{X^{k+1}}{2^{k}}.$

###  4.1 Supercharacter theory for orthogonal and symplectic $U^{a}$

In this section, we classify ${\mathbb{G}}^{a}$-orbits in ${\mathfrak{u}}^{a}$
and $({\mathfrak{u}}^{a})^{*}$ and construct supercharacter theory for
$U^{a}$. We need definition of rook placements for orthogonal and symplectic
groups.

Let $\Delta$ be the root system of the orthogonal or symplectic group $G$.
Ordering of the segment $I=[-n,n]$ defines the system $\Delta_{+}$ of positive
roots (respectively, the system of negative roots $\Delta_{-}$).

With each root ${\alpha}$, we associate some element of $I\times I$ according
to the following rule. For positive roots
${\alpha}=\epsilon_{i}-\epsilon_{j}$, $i>j$, and
${\alpha}=\epsilon_{i}+\epsilon_{j}$ we associate the pairs $(i,j)$ and
$(i,-j)$ respectively. Analogously, for ${\alpha}=\epsilon_{i}$, it is the
pair $(i,0)$, for ${\alpha}=2\epsilon_{i}$ – the pair $(i,-i)$. With the
negative root $-{\alpha}<0$ we associate the pair transpose to the pair of
${\alpha}$. For example, for ${\alpha}=-\epsilon_{i}-\epsilon_{j}$, it is
$(-j,i)$. For simplicity, if ${\alpha}=\epsilon_{i}-\epsilon_{j}$, $i>j$, we
write ${\alpha}=(i,j)$ (similarly in other cases).

For each root ${\alpha}=(i,j)$, we consider the pair
${\alpha}^{\prime}=(-j,-i)$ (observe that the last pair does not associate
with any root, except ${\alpha}={\alpha}^{\prime}=(i,-i)$).

To each root ${\alpha}\in\Delta_{+}$, we attach the root vector
$\mathcal{E}_{\alpha}=E_{\alpha}+\eta({\alpha})E_{{\alpha}^{\prime}}$, where
$\eta({\alpha})\in\\{1,0,-1\\}$. Respectively, to a root
${\alpha}\in\Delta_{-}$, we attach the root vector
$\mathcal{F}_{\alpha}=F_{\alpha}+\eta({\alpha})F_{{\alpha}^{\prime}}$. The
system of root vectors
$\\{\mathcal{E}_{\alpha}\\}\cup\\{\mathcal{F}_{\alpha}\\}$ is a basis in
${\mathfrak{u}}^{a}$. The relative elements of the dual basis we denote by
$\mathcal{E}^{*}_{\alpha}$ and $\mathcal{F}^{*}_{\alpha}$.

Similarly to the previous section, we define $\Delta_{+}({\mathfrak{u}}^{a})$
that consists of all ${\alpha}\in\Delta_{+}$ such that
$\mathcal{E}_{\alpha}\in{\mathfrak{u}}_{+}$. Respectively,
$\Delta_{-}({\mathfrak{u}}^{a})=\\{{\alpha}\in\Delta_{-}:~{}\mathcal{F}_{\alpha}\in{\mathfrak{u}}^{a}_{-}\\}$.
We define
$\Delta({\mathfrak{u}}^{a})=\Delta_{+}({\mathfrak{u}}^{a})\cup\Delta_{-}({\mathfrak{u}}^{a}).$

Definition 4.1. We say that a subset $D\subset\Delta$ is a rook placement if
there is at most one pair $(i,j)\in D\cup D^{\prime}$ in any row and column.
In other words, $D$ is a rook placement in orthogonal or symplectic case if
and only if $D\cup D^{\prime}$ is a rock placement in the root system of
${\mathrm{GL}}(M)$.

Example 4.2. Let $G=C_{2}$,
$\Delta_{+}=\\{{\alpha}_{1},{\alpha}_{2},{\alpha}_{1}+{\alpha}_{2},~{}2{\alpha}_{1}+{\alpha}_{2}\\}$,
$D=\\{{\alpha}_{1}+{\alpha}_{2},-{\alpha}_{2}\\}$. On the diagram the pairs
from $D\cup D^{\prime}$ are marked by $\otimes$. On the diagonal we put the
numbers of relative rows (columns).

$\left(\begin{array}[]{cccc}2&\cdot&\otimes&\cdot\\\ \cdot&1&\cdot&\otimes\\\
\cdot&\otimes&-1&\cdot\\\ \cdot&\cdot&\cdot&-2\end{array}\right)$

.

A rook placement $D\subset\Delta$ splits $D=D_{+}\cup D_{-}$, where
$D_{\pm}=D\cap\Delta_{\pm}$.

To the rook placement $D\subset\Delta$ and map
$\phi:D\to{\mathbb{F}}_{q}^{*}$, we attach the element
${\mathcal{X}}_{D,\phi}\in{\mathfrak{u}}^{a}$ and
${\Lambda}_{D,\phi}\in({\mathfrak{u}}^{a})^{*}$ defined by the formulas

${\mathcal{X}}_{D,\phi}=\sum_{\gamma\in
D_{+}}\phi(\gamma)\mathcal{E}_{\gamma}+\sum_{\gamma\in
D_{-}}\phi(\gamma)\mathcal{F}_{\gamma},$ ${\Lambda}_{D,\phi}=\sum_{\gamma\in
D_{+}}\phi(\gamma)\mathcal{E}^{*}_{\gamma}+\sum_{\gamma\in
D_{-}}\phi(\gamma)\mathcal{F}^{*}_{\gamma}.$

Fix the non-square element $\delta\in{\mathbb{F}}_{q}^{*}$. In the case
$C_{n}$, we say that a root $\gamma$ belongs to anti-diagonal if
$\gamma=(i,-i)$ for some $n\geqslant i\geqslant-n$.
In the case $B_{n}$ and $D_{n}$, we refer to pair $(D,\phi)$ as a  basic pair,
if $\phi(\gamma)=1$ for any $\gamma\in D$.
Definition 4.3. In the case $C_{n}$, we refer to pair $(D,\phi)$ as a basic
pair if
1) $\phi(\gamma)\in\\{1,\delta\\}$.
2) $\phi(\gamma)=\delta$ implies $\gamma$ belongs to anti-diagonal.
3) For any $\ell\geqslant k\geqslant-\ell$, the subset $D\cap(I_{k}\times
I_{-k})$ is contained in the anti-diagonal, and there is at most one root
$\gamma$ in this subset such that $\phi(\gamma)=\delta$.

By definition, we let $d_{k}(\phi)$ equal to $-1$ if there is a root from 3),
otherwise we take $d_{k}(\phi)=1$. Below we introduce notations that are
similar to the relative notations for series $A$ (the difference in
substitution $D\cup D^{\prime}$ for $D$ and in changing the row and column
enumeration). For any rock placement $D\subset\Delta$ and pair of segments
$I_{k},I_{m}$, where $k\neq m$ and $k+m\geqslant 0$, we define

$r_{ij}=r_{ij}(D)=|(D\cup D^{\prime})\cap(I_{k}\times I_{m})|,$
$n^{\prime}_{km}=n-\sum_{s>k}n_{s}-\sum_{t>m}r_{kt},$
$n^{\prime\prime}_{km}=n-\sum_{s>m}n_{s}-\sum_{t>k}r_{tm}.$

Definition 4.4. We say that a basic pair $(D_{c},\phi_{c})$ has canonical type
if for each pair $k\neq m$, $k+m\geqslant 0$, the subset
$D_{c}\cap(I_{k}\times I_{m})$ coincides with the subset

$\\{(n_{km}^{\prime}-1,n_{km}^{\prime\prime}-1),\ldots,(n_{km}^{\prime}-r_{km},n_{km}^{\prime\prime}-r_{km})\\},$
(9)

in the cases $B_{n}$, $D_{n}$ and $C_{n}$ (for $k+m>0$). In the case $C_{n}$
and $m=-k$, the subset $D_{c}\cap(I_{k}\times I_{-k})$ coincides with

$\\{(n_{k,-k}^{\prime}-1,-n_{k,-k}^{\prime}+1)),\ldots,(n_{k,-k}^{\prime}-r_{k,-k},-n_{k,-k}^{\prime}+r_{k,-k})\\},$
(10)

furthermore the values of $\phi_{c}$ at all roots from (10) apart from
possibly the first root equal to one.

Let $W_{\mathbb{L}}$ be the Weyl group of the subgroup ${\mathbb{L}}$ in
${\mathbb{G}}$. The action of $W_{\mathbb{L}}$ on ${\mathfrak{u}}^{a}$ is
obtained by restriction of the ${\mathbb{G}}^{a}$-action on
${\mathfrak{u}}^{a}$ (see formula (8)). Under this action the elements
${\mathcal{X}}_{D,\phi}$ transform to elements of the same type; this fact
defines an action of $W_{\mathbb{L}}$ on the set of basic pairs
$\\{(D,\phi)\\}$.

Lemma 4.5. For any basic pair $(D,\phi)$, there exists an element $w\in
W_{\mathbb{L}}$ such that $w\centerdot(D,\phi)$ has the canonical type
$(D_{c},\phi_{c})$. The canonical pair $(D_{c},\phi_{c})$ is uniquely
determined by $(D,\phi)$.
Proof. We proceed the proof for $D_{n}$. The other series are treated
similarly. We prove by the induction method on $n$.

Let ${\cal D}=D\cup D^{\prime}$. Decompose the square $I\times I$ into four
parts $I\times I=I_{11}\cup I_{12}\cup I_{21}\cup I_{22}$, where
$I_{11}=[0,n]\times[1,n]$, $I_{12}=[0,n]\times[-n,0]$,
$I_{21}=[-n,-1]\times[1,n]$, $I_{22}=[-n,-1]\times[-n,0]$.

We take ${\cal D}_{ij}={\cal D}\cap I_{ij}$. Observe that ${\cal
D}_{11}^{\prime}={\cal D}_{22}$, ${\cal D}_{12}^{\prime}={\cal D}_{12}$ and
${\cal D}_{21}^{\prime}={\cal D}_{21}$.

Each element $w\in{\mathbb{L}}$ has the form
$w=\mathrm{diag}(A_{\ell},\ldots,A_{0},\ldots,A_{-\ell})$, where
$A_{i}\in{\mathrm{GL}}(n_{i})$. Decompose $w=w_{1}w_{2}$, where
$w_{1}=\mathrm{diag}(A_{\ell},\ldots,A_{0},E,\ldots,E)$ and
$w_{2}=\mathrm{diag}(E,\ldots,E,A_{-1},\ldots,A_{-\ell})$.

Then $w\centerdot{\cal D}_{11}=w_{1}{\cal D}_{11}w_{2}^{\dagger}$,
$w\centerdot{\cal D}_{12}=w_{1}{\cal D}_{12}w_{1}^{\dagger}$,
$w\centerdot{\cal D}_{21}=w_{2}{\cal D}_{21}w_{2}^{\dagger}$. Replacing ${\cal
D}$ by suitable $w\centerdot{\cal D}$, we consider ${\cal D}_{11}$ to be of
canonical type in $I_{11}$ (see Lemma 3.1).

Suppose that $\tilde{I}_{12}$ is a square obtained by deleting in $I_{12}$ the
rows from ${\mathrm{row}}({\cal D}_{11})$ and columns from
${\mathrm{col}}({\cal D}_{22})$ (here the number of rows equals to the number
of columns as ${\cal D}_{22}={\cal D}_{11}^{\prime}$). Since there are no
roots from ${\cal D}_{12}$ in rows from ${\mathrm{row}}({\cal D}_{11})$ and
columns from ${\mathrm{col}}({\cal D}_{22})$, we have ${\cal
D}_{12}\subset\tilde{I}_{12}$. Applying the induction assumpsion, we can
replace ${\cal D}_{12}$ by some $w_{1}{\cal D}_{12}w_{1}^{\dagger}$ of
canonical type, where $w_{1}$ stabilizes the rows from ${\mathrm{row}}({\cal
D}_{11})$. Analogously for $w_{2}$ and ${\cal D}_{21}$.

Uniqueness follows from the fact that the numbers $\\{r_{km}\\}$ does not
change under the action of ${\mathbb{L}}$. $\Box$

For parabolic subgroups, the next theorem is proved in [15, Proposition 3.3].
Theorem 4.6. 1) Each ${\mathbb{G}}$-orbit in ${\mathfrak{u}}^{a}$ contains an
element of the form ${\mathcal{X}}_{D,\phi}$ for some basic pair
$\\{D,\phi\\}$.
2) Two elements ${\mathcal{X}}_{D,\phi}$ and
${\mathcal{X}}_{D^{\prime},\phi^{\prime}}$ for basic pairs $(D,\phi)$ and
$(D^{\prime},\phi^{\prime})$ belong to a common ${\mathbb{G}}^{a}$-orbit if
and only if $(D,\phi)$ and $(D^{\prime},\phi^{\prime})$ belong to a common
$W_{\mathbb{L}}$-orbit.
Proof. _Item 1_. Our goal is to prove statement 1) for the Borel contraction.
In this case $P=B=HN$ is a Borel subgroup in the orthogonal or symplectic
group, and ${\mathbb{P}}={\mathbb{B}}={\mathbb{H}}{\mathbb{N}}$ is a Borel
subgroup in ${\mathbb{G}}$. The unitriangular group ${\mathbb{N}}$ is an
algebra subgroup ${\mathbb{N}}=1+{\mathcal{N}}$. The group ${\mathbb{G}}^{a}$
acts on ${\mathcal{N}}^{a}={\mathcal{N}}_{+}\ltimes{\mathcal{N}}_{-}^{a}$. To
prove statement 1), we use the method analogous to [11, Proposition 3.3].
1) Let $X=x+{\lambda}\in{\mathfrak{n}}^{a}$, where $x\in{\mathfrak{n}}_{+}$
and ${\lambda}\in{\mathfrak{n}}_{-}^{a}$. We consider $X,~{}x,~{}{\lambda}$ as
the elements from ${\mathcal{N}}^{a}$ obeying $Y^{\dagger}=-Y$. It follows
from [5, 6] that there exists $a\in{\mathbb{N}}_{+}$ such that
$axa^{\dagger}={\mathcal{X}}_{D_{+},\phi_{+}}$, where $D_{+}$ is a rock
placement in $\Delta_{+}$ and $\phi:D_{+}\to{\mathbb{F}}_{q}^{*}$. Take
$x_{+}={\mathcal{X}}_{D_{+},\phi_{+}}$. We obtain
$aXa^{\dagger}=x_{+}+a{\lambda}a^{\dagger}$.
2) In what follows $X=x_{+}+{\lambda}$. In this subitem, we consider
decomposition (11) for ${\mathcal{N}}_{-}^{a}={\mathcal{N}}_{+}^{*}$ and dual
decomposition ${\mathfrak{n}}_{-}^{a}$ (see (12) and (13)); we show that
applying ${\mathbb{N}}^{a}$-action we get a similar element with ${\lambda}$
in the subspace (13). As above ${\cal D}=D\cup D^{\prime}$.

Consider decomposition
${\mathcal{N}}_{+}={\mathcal{L}}_{+}\oplus{\mathcal{L}}_{0}$ into direct sum
of two left ideals

${\mathcal{L}}_{+}=\mathrm{span}\\{E_{ij}:~{}j\notin{\mathrm{row}}({\cal
D}_{+})\\}=\\{y\in{\mathcal{N}}_{+}:~{}yx_{+}=0\\},$
${\mathcal{L}}_{0}=\mathrm{span}\\{E_{ij}:~{}j\in{\mathrm{row}}({\cal
D}_{+})\\}.$

Then
${\mathcal{N}}_{-}^{a}={\mathcal{L}}_{+}^{\perp}\oplus{\mathcal{L}}_{0}^{\perp}$,
where

${\mathcal{L}}_{+}^{\perp}=\mathrm{span}\\{F_{km}:~{}k\in{\mathrm{row}}({\cal
D}_{+})\\}=x_{+}{\mathcal{N}}_{-}^{a},\quad{\mathcal{L}}_{0}^{\perp}=\mathrm{span}\\{F_{km}:~{}k\notin{\mathrm{row}}({\cal
D}_{+})\\}$

We have
${\mathcal{N}}_{-}^{a}=x_{+}{\mathcal{N}}_{-}^{a}\oplus{\mathcal{L}}_{0}^{\perp}$.

Similarly, consider decomposition
${\mathcal{N}}_{+}={\mathcal{R}}_{+}\oplus{\mathcal{R}}_{0}$ into direct sum
of two right ideals, where ${\mathcal{R}}_{+}={\mathcal{L}}_{+}^{\dagger}$ and
${\mathcal{R}}_{0}={\mathcal{L}}_{0}^{\dagger}$.

Since the subspaces ${\mathcal{L}}_{+}$, ${\mathcal{L}}_{0}$,
${\mathcal{R}}_{+}$, and ${\mathcal{R}}_{0}$ are spanned by vectors from
standard basis $\\{E_{ij}\\}$ in ${\mathcal{N}}_{+}$, we obtain

${\mathcal{N}}_{-}^{a}=\left(x_{+}{\mathcal{N}}_{-}^{a}+{\mathcal{N}}_{-}^{a}x_{+}\right)\oplus\left({\mathcal{L}}_{0}^{\perp}\cap{\mathcal{R}}_{0}^{\perp}\right)$
(11)

Both subspaces in this sum are invariant under action of involutive
antiautomorphism. Therefore, ${\mathfrak{n}}_{-}^{a}$ is a direct sum of
intersections of these subspaces with ${\mathfrak{n}}_{-}^{a}$:

$\left(x_{+}{\mathcal{N}}_{-}^{a}+{\mathcal{N}}_{-}^{a}x_{+}\right)\cap{\mathfrak{n}}_{-}^{a}=\left\\{x_{+}Q^{\dagger}+Qx_{+}:~{}Q\in{\mathcal{N}}_{-}^{a}\right\\},$
(12)
$\left({\mathcal{L}}_{0}^{\perp}\cap{\mathcal{R}}_{0}^{\perp}\right)\cap{\mathfrak{n}}_{-}^{a}=\left\\{V\in{\mathfrak{n}}_{-}^{a}:~{}V({\mathcal{L}}_{0})=0\right\\}=\left\\{V\in{\mathfrak{n}}_{-}^{a}:~{}V({\mathcal{R}}_{0})=0\right\\}.$
(13)

Return to the element $X=x_{+}+{\lambda}\in{\mathfrak{n}}$. The linear form
${\lambda}\in{\mathfrak{n}}_{-}^{a}$ splits
${\lambda}=x_{+}Q^{\dagger}+Qx_{+}+{\lambda}_{0}$, where ${\lambda}_{0}$
belongs to subspace (13) in decomposition ${\mathfrak{n}}_{-}^{a}$. We get

$(1-Q)X(1-Q^{\dagger})=x_{+}-x_{+}Q^{\dagger}-Qx_{+}+{\lambda}=x_{+}+{\lambda}_{0}.$

3) So, we can assume that $X=x_{+}+{\lambda}_{0}$, where
${\lambda}_{0}({\mathcal{L}}_{0})={\lambda}_{0}({\mathcal{R}}_{0})=0$ (recall
that ${\mathcal{R}}_{0}={\mathcal{L}}_{0}^{\dagger}$). Let us show that in
${\mathbb{G}}^{a}$-orbit of the element $X$ there is an element of the form
$x_{+}+{\lambda}^{\prime}$, where

${\lambda}^{\prime}={\mathcal{X}}_{D_{-}^{\prime},\phi_{-}^{\prime}}$

for some rook placement $D_{-}^{\prime}\subset\Delta_{-}$ and
$\phi_{-}^{\prime}:D_{-}^{\prime}\to{\mathbb{F}}_{q}^{*}$.

For ${\lambda}_{0}$ there exists $a\in{\mathbb{N}}_{+}$ such that
$a{\lambda}_{0}a^{\dagger}={\lambda}^{\prime}$, where ${\lambda}^{\prime}$ has
the mentioned above form. Let us show that we can chose $a\in{\mathbb{N}}_{+}$
such that $ax_{+}a^{\dagger}=x_{+}$.

Consider the algebra subgroups $L_{+}=1+{\mathcal{L}}_{+}$ and
$L_{0}=1+{\mathcal{L}}_{0}$ in ${\mathbb{N}}_{+}$. Since ${\mathcal{N}}_{+}$
is a direct sum of left ideal
${\mathcal{N}}_{+}={\mathcal{L}}_{+}+{\mathcal{L}}_{0}$, each
$a\in{\mathbb{N}}_{+}$ can be uniquely presented in the form $a=a_{+}a_{0}$,
where $a_{+}\in L_{+}$ and $a_{0}\in L_{0}$.

By definition of ${\mathcal{L}}_{+}$, we have $a_{+}x_{+}=x_{+}$. From
$x_{+}^{\dagger}=-x_{+}$, we get $x_{+}a_{+}^{\dagger}=x_{+}$.

Let us show that $a_{0}{\lambda}_{0}={\lambda}_{0}$ for any $a_{0}\in L_{0}$.
Indeed, $a_{0}=1+y_{0}$, where $y_{0}\in{\mathcal{L}}_{0}$. Then
$a_{0}{\lambda}(x)={\lambda}_{0}(xa)={\lambda}_{0}(x)+{\lambda}_{0}(xy_{0})$.
Since ${\mathcal{L}}_{0}$ is a left ideal in ${\mathcal{N}}_{+}$, it follows
${\lambda}_{0}(xy_{0})=0$. This proves $a_{0}{\lambda}_{0}={\lambda}_{0}$.
From ${\lambda}_{0}^{\dagger}=-{\lambda}_{0}$, we obtain
${\lambda}_{0}a_{0}^{\dagger}={\lambda}_{0}$. We have

$a_{+}Xa_{+}^{\dagger}=a_{+}x_{+}a_{+}^{\dagger}+a_{+}{\lambda}_{0}a_{+}^{\dagger}=x_{+}+a_{+}a_{0}{\lambda}a_{0}^{\dagger}a_{+}^{\dagger}=x_{+}+a{\lambda}_{0}a^{\dagger}=x_{+}+{\lambda}^{\prime}.$

4) So, each ${\mathbb{N}}^{a}$-orbit of the element $X\in{\mathfrak{n}}^{a}$
has an element $X^{\prime}=x_{+}+{\lambda}^{\prime}$, where
$x_{+}={\mathcal{X}}_{D_{+},\phi_{+}}$ and
${\lambda}^{\prime}={\mathcal{X}}_{D_{-}^{\prime},\phi_{-}^{\prime}}$.
Finally, as in subitem 3), replacing $X^{\prime}$ by a suitable element of the
form $(1+Q)X^{\prime}(1+Q^{\dagger})$, we can delete rocks of ${\cal
D}_{-}^{\prime}$ belonging to rows and columns from ${\cal D}_{+}$. We obtain
the element ${\mathcal{X}}_{D,\phi}$, where $D=(D_{+},D_{-})$ is a rook
placement in $\Delta$ and $\phi$ is a map $D\to{\mathbb{F}}_{q}^{*}$
coinciding with $\phi_{+}$ on $D_{+}$ and with $\phi_{-}^{\prime}$ on $D_{-}$.
Finally, replacing $X\to hXh^{\dagger}$ by a suitable $h\in{\mathbb{H}}$, we
make $(D,\phi)$ to be a basic pair. The statement if Item 1 is proved.
Item 2. The proof of statement 1) for parabolic contraction is proceed
similarly to Item 1 from Theorem 3.1. One can prove statement 2) similarly to
Item 2 of Theorem 3.1 applying Lemma 4.1. $\Box$
Corollary 4.7. Two elements ${\mathcal{X}}_{D,\phi}$ and
${\mathcal{X}}_{D^{\prime},\phi^{\prime}}$ for basic pairs $(D,\phi)$ and
$(D^{\prime},\phi^{\prime})$ belong to a common ${\mathbb{G}}^{a}$-orbit if
and only if $r_{km}(D)=r_{km}(D)$ for all $\ell\geqslant k,m\geqslant-\ell$,
$k\neq m$, and $d_{k}(\phi)=d_{k}(\phi^{\prime})$ for all $\ell\geqslant
k\geqslant-\ell$, $k\neq 0$.
Proof. Analogously to Theorem 3.1 and [15, Proposition 3.3]. $\Box$

Using identification of ${\mathfrak{u}}^{a}$ and $({\mathfrak{u}}^{a})^{*}$,
we obtain the following statement.
Proposition 4.8. 1) Each ${\mathbb{G}}$-orbit in $({\mathfrak{u}}^{a})^{*}$
contains an element of the form ${\Lambda}_{D,\phi}$ for some basic pair
$\\{D,\phi\\}$.
2) Two elements ${\Lambda}_{D,\phi}$ and
${\Lambda}_{D^{\prime},\phi^{\prime}}$ for basic pairs $(D,\phi)$ and
$(D^{\prime},\phi^{\prime})$ belong to a common ${\mathbb{G}}^{a}$-orbit if
and only if $(D,\phi)$ and $(D^{\prime},\phi^{\prime})$ belong to a common
$W_{\mathbb{L}}$-orbit.
Definition 4.9. We say that element $u$ and $u^{\prime}$ from $U^{a}$ belong
to a common class if the elements $f(X)$ and $f(X^{\prime})$ belong to a
connon ${\mathbb{G}}^{a}$-orbit in ${\mathcal{U}}^{a}$ ( here $f$ is the
Cayley map).

Denote $u_{D,\phi}=f^{-1}({\mathcal{X}}_{D,\phi})$. Theorem 4.1 yields a
classification of classes in the group $U^{a}$.

For each basic pair $(D,\phi)$ we consider a function on $U^{a}$ of the form

$\sigma_{D,\phi}(u)=\sum_{\mu\in{\mathbb{G}}^{a}\centerdot{\Lambda}_{D,\phi}}{\varepsilon}^{\mu(f(u))}$
(14)

Denote by $K_{D,\phi}$ the preimage of ${\mathbb{G}}^{a}$-orbit of
${\mathcal{X}}_{D,\phi}$ under the Cayley map.
Theorem 4.10. The system of functions $\\{\sigma_{D,\phi}\\}$ and the
decomposition of group $U^{a}$ into classes $\\{K_{D,\phi}\\}$ give rise to a
supercharacter theory of the group $U^{a}$.
Proof. It follows from [7] that the system of functions

$\sigma_{{\lambda}}(u)=\sum_{\mu\in{\mathbb{U}}^{a}\centerdot{\lambda}}{\varepsilon}^{\mu(f(u))}$

and subsets $K_{x}=f^{-1}({\mathbb{U}}^{a}\centerdot x)$, where ${\lambda}$
and $x$ run through the systems of representatives of
${\mathbb{U}}^{a}$-orbits in $({\mathfrak{u}}^{a})^{*}$ and ${\mathfrak{u}}$
respectively, give rise to a supercharacter theory of the group $U^{a}$. In
particular, the functions $\sigma_{{\lambda}}$ are characters of some
representations and pairwise orthogonal (disjoint). Therefore,
$\sigma_{D,\phi}$ are also characters and pairwise orthogonal. Theorem 4.1
implies that the decomposition into subsets $K_{D,\phi}$ form a partition of
the group $U^{a}$ more coarse than the decomposition $\\{K_{x}\\}$. Finally,
if $u$ and $u^{\prime}$ belong to a common class, then there exists
$g\in{\mathbb{G}}^{a}$, such that $f(u)=g\centerdot f(u^{\prime})$.
Substituting for (14), we verify that the values of $\sigma_{D,\phi}$ at $u$
and $u^{\prime}$ coincide. $\Box$

###  4.2 Supercharacter theory for orthogonal and symplectic $G^{a}$

In subsection 3.2, by a pair $\gamma=(i,j)$, $i\neq j$, we construct the
subgroup $H_{\gamma}$ in $L\subset{\mathrm{GL}}(n)$. In notations of this
section, it is the subgroup ${\mathbb{H}}_{\gamma}$ in the block-diagonal
group ${\mathbb{L}}$.

Let $D$ be a rook placement in $\Delta({\mathfrak{u}}^{a})$. For each root
$\gamma$, we define the subgroup $H_{\gamma}$ in $L$ as intersection of
${\mathbb{H}}_{\gamma}\cap{\mathbb{H}}_{\gamma^{\prime}}$ with orthogonal or
symplectic group $G$.

Define

$H_{D}=\bigcap_{\gamma\in D}H_{\gamma}.$

Denote by ${\mathfrak{u}}_{D}^{a}$ the intersection of all
$\mathrm{Ker}\,g\centerdot{\Lambda}_{D,\phi}$, where $g\in{\mathbb{G}}^{a}$.
The subset ${\mathfrak{u}}_{D}^{a}$ is a ${\mathbb{G}}_{a}$-invariant ideal in
${\mathfrak{u}}^{a}$.
Lemma 4.11. For each $h\in H_{D}$ the operator ${\mathrm{Ad}}_{h}$ is
identical on ${\mathfrak{u}}^{a}/{\mathfrak{u}}^{a}_{D}$.
Proof. Consider the subspace $W_{D}^{*}\subset({\mathfrak{u}}^{a})^{*}$
spanned by $g\centerdot{\Lambda}_{D,\phi}$, where $g\in{\mathbb{G}}^{a}$. The
subspace $W_{D}^{*}$ is invariant with respect to action of ${\mathbb{L}}$.
Therefore, $W_{D}^{*}$ decomposes into ${\mathbb{L}}$-irreducible components
$W_{km}^{*}$ related to segments of decomposition of $I=[-n,n]$. If $\gamma\in
I_{k}\times I_{m}$, $k>m$, then $W_{D}^{*}$ contains all components
$W_{ij}^{*}$, $k\geqslant i>j\geqslant m$. If $\gamma\in I_{k}\times I_{m}$,
$k<m$, then $W_{D}^{*}$ contains all components $W_{ij}^{*}$, $i\leqslant
k<m\leqslant j$. For each $h\in H_{\gamma}$, the operator $Ad^{*}_{h}$ is
identical on each mentioned above compotent $W_{ij}^{*}$. The subspace
$W_{D}^{*}$ is a direct sum of all enumerated components $W_{ij}^{*}$
constructed for roots $\gamma\in D$. Therefore, for each $h\in H_{D}$ the
operator $Ad^{*}_{h}$ is identical on $W_{D}^{*}$. It implies that for $h\in
H_{D}$ the operator $Ad_{h}$ is identical on
${\mathfrak{u}}^{a}/{\mathfrak{u}}^{a}_{D}$. $\Box$
Example4.12. For root system $D=\\{{\alpha}_{1}+{\alpha}_{2},-{\alpha}_{2}\\}$
of $G=C_{2}$ (see example 4.1), we have
$W^{*}_{D}=\mathrm{span}\\{\mathcal{F}^{*}_{-{\alpha}_{2}},~{}\mathcal{F}^{*}_{-{\alpha}_{1}-{\alpha}_{2}},~{}\mathcal{F}^{*}_{-2{\alpha}_{1}-{\alpha}_{2}},~{}\mathcal{E}^{*}_{{\alpha}_{1}},~{}\mathcal{E}^{*}_{{\alpha}_{1}+{\alpha}_{2}},\mathcal{E}^{*}_{{\alpha}_{2}}\\},$

${\mathfrak{u}}_{D}^{a}=\mathrm{span}\\{\mathcal{F}_{-{\alpha}_{1}},~{}\mathcal{E}_{2{\alpha}_{1}+{\alpha}_{2}}\\},\quad\quad
H_{D}=\\{\pm E\\}.$

On this diagram, the roots of $D\cup D^{\prime}$ is marked as $\otimes$. The
squares relative to basic elements of ${\mathfrak{u}}_{D}^{a}$ are marked by
$\bullet$. On diagonal we put the numbers of relative rows (columns).

$\left(\begin{array}[]{cccc}2&\cdot&\otimes&\bullet\\\
\bullet&1&\cdot&\otimes\\\ \cdot&\otimes&-1&\cdot\\\
\cdot&\cdot&\bullet&-2\end{array}\right)$

.

Consider the set ${\mathfrak{A}}$ of all triples
${\mathfrak{a}}=(D,\phi,\theta)$, where $(D,\phi)$ is a basic pair from
$\Delta({\mathfrak{u}}^{a})$, and $\theta$ in an irreducible representation of
the subgroup $H_{D}$. Below we attach to each ${\mathfrak{a}}$ the character
$\sigma_{\mathfrak{a}}$ of subgroup $G^{a}$.

Denote by $U^{a}_{D}$ the normal subgroup $f^{-1}({\mathfrak{u}}^{a}_{D})$ in
$U^{a}$. Lemma 4.2 implies that the subgroup $G_{D}^{a}=H_{D}\ltimes
U_{D}^{a}$ is abelian modulo $U^{a}_{D}$. The characters $\sigma_{D,\phi}$
defined by formula (14) can be taken modulo $U^{a}_{D}$ (see formula (14)).
The formula

$\xi_{\mathfrak{a}}(g)=\theta(h)\sigma_{D,\phi}(u),$

$g=hu$, $h\in H_{D}$, $u\in U^{a}$, defines the character of subgroup
$G_{D}^{a}$. Consider the character

$\sigma_{\mathfrak{a}}={\mathrm{Ind}}(\xi_{\mathfrak{a}},G_{D}^{a},G^{a}).$

We turn to construction of superclasses. For each $h\in L$, we consider the
smallest ${\mathbb{G}}^{a}$-invariant ideal ${\mathfrak{u}}^{a}_{h}$ in
${\mathfrak{u}}^{a}$ such that ${\mathrm{Ad}}_{h}$ is identical on
${\mathfrak{u}}^{a}/{\mathfrak{u}}^{a}_{h}$. Let $U_{h}^{a}$ be the
corresponding normal subgroup in $U^{a}$, and $\Pi_{h}$ be the natural
projection $U^{a}$ onto $U^{a}/U^{a}_{h}$.

Consider the set ${\mathfrak{B}}$ of all triples $\mathfrak{b}=(D,\phi,h)$,
where $(D,\phi)$ is a basic pair in $\Delta({\mathfrak{u}}^{a})$ and $h\in
H_{D}$. By each triple $\mathfrak{b}$, we define the class

$K_{\mathfrak{b}}=\mathrm{CL}_{L}(h)\cdot\Pi_{h}^{-1}\Pi_{h}\left(K_{D,\phi}\right),$

where $\mathrm{CL}_{L}(h)$ is the conjugacy class of $h$ in $L$, and
$K_{D,\phi}$ from Theorem 4.1.
Theorem 4.13. The system of characters
$\\{\sigma_{\mathfrak{a}}:~{}{\mathfrak{a}}\in{\mathfrak{A}}\\}$ and
decomposition of $G^{a}$ into classes
$\\{K_{\mathfrak{b}}:~{}\mathfrak{b}\in{\mathfrak{B}}\\}$ give rise to a
supercharacter theory of the orthogonal or symplectic $G^{a}$.
Proof. The characters $\\{\sigma_{\mathfrak{a}}\\}$ are orthogonal because

$\sigma_{\mathfrak{a}}(g)=c\,\dot{\theta}(h)\sigma_{D,\phi}(u),$

where $\sigma_{D,\phi}$ defined by formula (14), $g=hu$, $h\in L$, $u\in
U^{a}$, and $\dot{\theta}(h)=\theta(h)$ for $h\in H_{D}$ and is zero for
$h\notin H_{D}$.

Let us show that $\sigma_{\mathfrak{a}}$ is constant on $K_{\mathfrak{b}}$,
where $\mathfrak{b}=(D^{\prime},\phi^{\prime},h)$, $h\in H_{D^{\prime}}$. If
$h\notin H_{D}$, then $\sigma(K_{\mathfrak{b}})=0$.

Suppose $h\in H_{D}$. Then
${\mathfrak{u}}^{a}_{h}\subset{\mathfrak{u}}_{D}^{a}$ and $U_{h}^{a}\subset
U_{D}^{a}$. The $\sigma_{\mathfrak{a}}$ and class $K_{\mathfrak{b}}$ can be
taken modulo $U^{a}_{h}$. The character $\sigma_{\mathfrak{a}}$ is constant on
class $K_{\mathfrak{b}}$ because $\sigma_{D,\phi}$ is constant on
$K_{D^{\prime},\phi^{\prime}}$ for $U^{a}/U^{a}_{h}$. $\Box$

##  5 Funding

The paper is supported by the RFBR grant 20-01-00091a.

## References

  * [1] Diaconis P., Isaacs I.M., Supercharacters and superclasses for algebra groups.,Trans.Amer.Math.Soc., 360 (2008), 2359-2392.
  * [2] André C.A.M., Basic characters of the unitriangular group, J. Algebra, 175 (1995), 287-319.
  * [3] André C.A.M., Basic sums of coadjoint orbits of the unitriangular group, J. Algebra, 176(1995), 959-1000.
  * [4] André C.A.M., The basic character table of the unitriangular group, J. Algebra, 241 (2001), 437-471.
  * [5] André C.A.M., Neto A.M., Supercharacters of the finite the Sylow subgroup of the finite symplectic and orthogonal groups, Pacific Math. Journal, 239 (2009), 201-230.
  * [6] André C.A.M., Neto A.M., A supercharacter theory for the Sylow p-subgroups of the finite symplectic and orthogonal groups, J. Algebra, 322 (2009), 1273-1294.
  * [7] André C.A.M., Freitas J.P., Neto A.M., A supercharacter theory for involutive algebra groups, J. Algebra, 430 (2015), 159-190.
  * [8] Inönü E., Wigner E.P., On the contraction of groups and their representations, Proc. Nat-Acad. Sci., 39 (1953), 510-524.
  * [9] Gorbatsevich V.V., Onishchik A.L., Vinberg E.B. (1994). Lie groups and Lie algebras III. Encyclopaedia Math. Sci., vol. 41, Berlin: Springer.
  * [10] Panov A.N. Supercharacter theories for algebra group extensions. J. Group theory, 24 (2021), 263-283.
  * [11] Panov A.N., Supercharacter theory for the Borel group contraction of ${\mathrm{GL}}(n,{\mathbb{F}}_{q})$, Vestnik St. Peterburg Univ. Math., 53 (2020), no 2, 162-173.
  * [12] Panov A.N. (2016). Supercharacter theory for groups of invertible elements of reduced algebras, St. Petersburg Math. Journal, 27 (2016), 1035-1047.
  * [13] Panov A.N., Supercharacters for finite groups of triangular type, Comm.Algebra, 46 (2018), no.3, 1032-1046.
  * [14] Panov A.N., Towards a supercharacter theory of parabolic subgroups in ${\mathrm{GL}}(n,{\mathbb{F}}_{q})$, Algebr. Represent. theory, 21 2018, no. 5, 1133–1149.
  * [15] Panov A.N., Two supercharacter theories for the parabolic subgroups in orthogonal and symplectic groups, J. Algebra, 539 (2019), 37-53.
  * [16] Thiem N., Supercharacter theories of type $A$ unipotent radicals and unipotent polytopes, Algebraic Combinatorics, 1 ( 2018), no. 1, 23-45.
  * [17] Andrews S., Supercharacters of unipotent groups defined by involutions, J. Algebra, 425 (2015), 1-30.


††thanks: These two authors contributed equally to this work.††thanks: These
two authors contributed equally to this work.††thanks: Corresponding author:
<EMAIL_ADDRESS>Corresponding author<EMAIL_ADDRESS>Corresponding author<EMAIL_ADDRESS>
# Detecting Extragalactic Axion-like Dark Matter with Polarization
Measurements of Fast Radio Bursts

Bao Wang  0000-0003-3635-5375 Purple Mountain Observatory, Chinese Academy of
Sciences, Nanjing 210023, China  School of Astronomy and Space Sciences,
University of Science and Technology of China, Hefei 230026, China  Xuan Yang
Purple Mountain Observatory, Chinese Academy of Sciences, Nanjing 210023,
China  School of Astronomy and Space Sciences, University of Science and
Technology of China, Hefei 230026, China  Jun-Jie Wei  0000-0003-0162-2488
Purple Mountain Observatory, Chinese Academy of Sciences, Nanjing 210023,
China  School of Astronomy and Space Sciences, University of Science and
Technology of China, Hefei 230026, China  Song-Bo Zhang  Purple Mountain
Observatory, Chinese Academy of Sciences, Nanjing 210023, China  Xue-Feng Wu
Purple Mountain Observatory, Chinese Academy of Sciences, Nanjing 210023,
China  School of Astronomy and Space Sciences, University of Science and
Technology of China, Hefei 230026, China

###### Abstract

Axions or axion-like particles (ALPs) are one of the most promising candidates
of dark matter (DM). A prevalent method to detect axion-like DM is to seek the
periodic oscillation feature in the polarization angles of linearly polarized
light emitted from astrophysical sources. In this work, we use the time-
resolved polarization measurements of the hyperactive repeating fast radio
burst, FRB 20220912A, detected by the Five-hundred-meter Aperture Spherical
radio Telescope (FAST) to search for extragalactic axion-like DM for the first
time. Given a DM density profile of FRB 20220912A’s host, we obtain upper
limits on the ALP-photon coupling constant of $g_{a\gamma}<(2.9\times
10^{-11}-1.1\times 10^{-9})\,\mathrm{GeV}^{-1}$ for the ALP masses
$m_{a}\sim(1.4\times 10^{-21}-5.2\times 10^{-20})$ eV. Persistent polarimetric
observations with FAST would further improve the constraints. We prove that
FRBs offer an alternative way to detect axion-like DM on extragalactic
distance scales, complementary to other galactic DM probes.

##  I Introduction

Nowadays, one of the most important tasks that remain unresolved in modern
physics is the detection of dark matter (DM) particles, and numerous
candidates of DM have been proposed. Axion-like particles (ALPs), a type of
ultralight bosons, have emerged as the most prevailing candidates in the
search for DM Preskill  _et al._ (1983); Marsh (2016); Choi  _et al._ (2021).
Thanks to the special properties of ultralight scalar particles ($m_{a}\sim
10^{-22}\;{\rm eV}$), they can provide a natural solution to the challenges
encountered in small-scale structures of the Universe Hu  _et al._ (2000); Hui
_et al._ (2017). A lot of strategies for finding axion-like DM have been
explored, including photons from axion conversion Horns  _et al._ (2013);
Payez  _et al._ (2015); Anastassopoulos  _et al._ (2017); Du  _et al._ (2018),
nuclear magnetic resonance Graham and Rajendran (2013); Budker  _et al._
(2014), periodic oscillations of linearly polarized light Ivanov  _et al._
(2019); Chen  _et al._ (2020); Liu  _et al._ (2020); Yuan.  _et al._ (2021),
and other many terrestrial or astronomical experiments Dessert  _et al._
(2020); Meyer  _et al._ (2020).

Among the various axion-like DM detection methods, detecting periodic
oscillations of polarized light has been regarded as a promising approach in
astrophysics Fujita  _et al._ (2019); Caputo  _et al._ (2019); Ivanov  _et
al._ (2019); Chen  _et al._ (2020); Liu  _et al._ (2020); Yuan.  _et al._
(2021); Liu  _et al._ (2023a). When light propagates through the ALP field,
photons would interact with ALPs, and the interaction term is
$\mathcal{L}_{a\gamma}=\frac{1}{4}g_{a\gamma}aF_{\mu\nu}\tilde{F}^{\mu\nu}$,
where $g_{a\gamma}$ represents the coupling strength between the axion field
($a$) and the electromagnetic field ($F_{\mu\nu}$). The interaction leads to
modifications in the dispersion relations Carroll  _et al._ (1990); Harari and
Sikivie (1992). The left- and right-handed circular polarization modes of
light experience opposite corrections due to their different dispersion
relations. This effect is known as cosmic birefringence, resulting in changes
in the polarization angles (PAs) of the light. Therefore, in the presence of
an ALP field, if the light is linearly polarized, its PAs will oscillate with
the ALP field, with an amplitude proportional to $g_{a\gamma}$. So far, ALP-
photon couplings have been provided increasingly stringent constraints from a
multitude of astrophysical observations Fujita  _et al._ (2019); Caputo  _et
al._ (2019); Ivanov  _et al._ (2019); Chen  _et al._ (2020); Liu  _et al._
(2020); Yuan.  _et al._ (2021); Liu  _et al._ (2023a).

Fast radio bursts (FRBs) are brief and intense radio transients that originate
from cosmological distances Xiao  _et al._ (2021); Zhang  _et al._ (2023a);
Zhang (2023). They have been powerful astrophysical laboratories for studying
cosmology and fundamental physics Wei  _et al._ (2015); Wu  _et al._ (2016);
Yang and Zhang (2016); Macquart  _et al._ (2020); Wei and Wu (2021); Yang  _et
al._ (2022); James  _et al._ (2022); Liu  _et al._ (2023b); Wang and Wei
(2023). FRBs also have the potential to play an important role in the
detection of DM, such as constraining millicharged DM from timing of FRBs
Caputo  _et al._ (2019), bounding the dark photon DM parameter space with the
time delays of photons from FRBs Landim (2020), probing compact DM with FRB
microstructure Sammons  _et al._ (2020) or with strongly lensed FRBs Muñoz
_et al._ (2016); Wang and Wang (2018); Liao  _et al._ (2020); Ho  _et al._
(2023), and constraining primordial black hole DM with lensed FRBs Krochek and
Kovetz (2022). More recently, Gao  _et al._ (2023) proposed a future-feasible
method to hunt axion DM with gravitationally lensed FRBs, indicating the
potential of FRBs for searching for axion DM.

To date, no studies have utilized real polarization observations of repeating
FRBs to directly constrain axion-like DM. The repetition pattern of FRBs
enables us to monitor their polarization properties long-term to detect axion-
like DM on extragalactic distance scales, complementary to other galactic DM
probes. One intriguing sample for such a study is FRB 20220912A, an active
repeating source with highly linear polarization Ravi  _et al._ (2023); Zhang
_et al._ (2023b) and its local environment is very clean Feng  _et al._
(2023). Its long-term polarization observations from the Five-hundred-meter
Aperture Spherical radio Telescope (FAST) Zhang  _et al._ (2023b) provide a
remarkable opportunity to detect extragalactic axion-like DM through searching
for a periodic oscillation in the PAs.

In this work, we constrain the ALP-photon coupling constant $g_{a\gamma}$
using polarization data of FRBs for the first time. We analyze the
polarization angle variations of linearly polarized emission from FRB
20220912A observed by FAST. All currently available observations from October
28th, 2022 to December 5th, 2022 are adopted for our study. The observational
time coverage of $\sim 38$ days with a cadence of $\sim 1$ day is sensitive to
ALPs with mass ranging from $1.4\times 10^{-21}$ eV to $5.2\times 10^{-20}$
eV. By estimating the changes in linear PAs, we can place upper limits on
$g_{a\gamma}$ directly. Finally, we also predict further constraints from
continued polarization observations of FRB 20220912A in the future.

##  II Models

###  II.1 ALP-photon Coupling

ALPs can be described as a scalar field $a(x,t)$ with mass $m_{a}$, where $x$
is the spatial coordinates and $t$ is the time. The ALP field can interact
with the electromagnetic field, and its dynamics can be captured by the
Lagrangian terms Wilczek (1987):

$\mathcal{L}=-\frac{1}{4}F_{\mu\nu}F^{\mu\nu}+\frac{1}{2}\partial^{\mu}a\partial_{\mu}a-\frac{1}{2}m_{a}^{2}a^{2}+\frac{1}{4}g_{a\gamma}aF_{\mu\nu}\tilde{F}^{\mu\nu},$
(1)

where $F^{\mu\nu}$ denotes the electromagnetic field tensor,
$\tilde{F}^{\mu\nu}=\frac{1}{2}\epsilon^{\mu\nu\lambda\sigma}F_{\lambda\sigma}$
is the dual of $F^{\mu\nu}$, and $g_{a\gamma}$ represents the ALP-photon
coupling constant which characterizes the strength of interaction. This
coupling leads to a modification in the dispersion relation:

$\omega_{\pm}\simeq k\pm\frac{1}{2}g_{a\gamma}n^{\mu}\partial_{\mu}a,$ (2)

where $n^{\mu}$ is null tangent vector of light, $k$ is the wave vector111The
natural unit system $\hbar$=c=1., and the frequency $\omega_{\pm}$ corresponds
to two circular polarization states. When two vertically polarized
electromagnetic waves of these two states propagate, a phase shift occurs
between them due to the disparity in their phase velocities. This phase shift
leads to the rotation of the polarization plane, known as cosmic
birefringence. Specifically, the frequency difference between the two
polarization components is
$\Delta\omega=\omega_{+}-\omega_{-}=g_{a\gamma}n^{\mu}\partial_{\mu}a$. If
waves emitted from the source at position $x_{1}$ at time $t_{1}$ are received
by an observer at position $x_{2}$ and time $t_{2}$, the ALP-induced PA shift
is then expressed as

$\phi=\frac{1}{2}\int_{C}\Delta\omega{\rm
d}s=\frac{1}{2}g_{a\gamma}\left[a(x_{2},t_{2})-a(x_{1},t_{1})\right],$ (3)

where $C$ is the propagation path of waves. The equation of motion for the ALP
field is given by the Klein-Gordon equation. When we neglect the backaction
term, the solution simplifies and exhibits as an oscillating form:

$a(x,t)=a_{0}(x)\sin\left(m_{a}t+\delta\right),$ (4)

where $\delta$ is the position-dependent phase. $a_{0}(x)$ is the amplitude
that relates to the energy density of the ALP field $\rho$ (or equivalently
the energy density of DM, if the dominant DM is assumed to be made up of
ALPs), i.e., $\rho=\frac{1}{2}m_{a}a_{0}^{2}$. The oscillation period of the
ALP field is given by $T=2\pi/m_{a}$, which depends on the ALP mass. If the
energy density of the ALP field at the observer is much lower than the one at
the source (i.e., $a(x_{2},t_{2})\ll a(x_{1},t_{1})$), Eq. (3) can be
converted to an oscillatory expression,

$\phi(t)=-\frac{\sqrt{2}}{2}g_{a\gamma}\,m_{a}^{-1}\rho^{\frac{1}{2}}\sin\left(2\pi\frac{t}{T^{\prime}}+\delta\right),$
(5)

where $T^{\prime}=T(1+z)$ is the observed period on Earth, taking into account
of the cosmic expansion. Eq. (5) describes that the PAs have the periodic
oscillation characteristic when linearly polarized waves are coupled with
ALPs.

###  II.2 DM Density Profile

Outside the solitonic cores of galaxies, the DM density distribution $\rho(r)$
can be approximately described by the generalized Navarro-Frenk-White (NFW)
profile Zhao (1996):

$\rho(r)=\frac{\rho_{0}}{\left(r/r_{\mathrm{s}}\right)^{\beta}\left(1+r/r_{\mathrm{s}}\right)^{3-\beta}},$
(6)

where $r$ is the distance from the galaxy center, $\rho_{0}$ is the
characteristic density, $r_{s}$ is the scale radius, and $\beta$ is the power-
law index. Also, $\rho(r)\propto r^{-\beta}$ when $r\ll r_{s}$ and
$\rho(r)\propto r^{\beta-3}$ when $r\gg r_{s}$. For the case of $\beta=1$, Eq.
(6) is reduced to the original NFW profile Navarro  _et al._ (1996). In
principle, these coefficients ($\rho_{0}$, $r_{s}$, and $\beta$) can be
determined by fitting the rotation curves of galaxies. The physical origins of
FRBs are still unknown, but some of them have been localized in extragalactic
host galaxies. Once we have enough observational information about the FRB
host galaxy, we can estimate the DM density $\rho$ in the vicinity of the FRB
source.

Figure 1: Linear polarization measurements of the bursts from FRB 20220912A
detected by FAST. The upper and lower panels denote the observed PAs
$\phi_{\rm obs}$ and the ALP-induced PA shifts $\phi(t)$, respectively. Each
burst is depicted by a yellow point, the green points represent the daily
medians, and the light green shaded area encompasses the $1\sigma$ confidence
range. The ALP-induced PA shifts are estimated through $\phi(t)=\phi_{\rm
obs}-\left\langle\phi_{\rm bkg}\right\rangle$, where $\left\langle\phi_{\rm
bkg}\right\rangle=-24.95^{\circ}$ (see section IV).

##  III Observations of FRB 20220912A

###  III.1 The Host Galaxy

The Deep Synoptic Array localised the repeater FRB 20220912A to a host galaxy,
PSO J347.2702+48.7066, at redshift $z=0.0771$ Ravi  _et al._ (2023). The host
galaxy has a stellar mass of approximately $10^{10}\,M_{\odot}$, a star-
formation rate of $\gtrsim 0.1\,M_{\odot}\,{\rm yr^{-1}}$, and an effective
radius of $2.2\,{\rm kpc}$. Gordon  _et al._ (2023) compared the optical host
luminosities of repeating and nonrepeating FRBs across redshift, and defined a
demarcation at luminosity $10^{9}\,L_{\odot}$ below which a host can be
classified as a dwarf galaxy. FRB 20220912A sits just above the borderline at
$\approx 1.1\times 10^{9}\,L_{\odot}$ (see Fig. 9 of Gordon  _et al._ (2023)),
suggesting that its host may be a dwarf galaxy.

Since they have higher fractions of DM compared to more massive systems, dwarf
galaxies are deemed as ideal systems to probe the DM density profile Cooke
_et al._ (2022). However, we currently lack rotation curve observations of the
host of FRB 20220912A to investigate its DM density distribution. Here we use
the DM density profile of a dwarf galaxy, NGC 4451 (with a similar stellar
mass of $\sim 10^{10}\,M_{\odot}$ and a similar radius of $\sim 2.2\,{\rm
kpc}$ Cooke  _et al._ (2022)), as a reference. Note that the differences in DM
density profiles between NGC 4451 and FRB 20220912A’s host have negligible
effects within the precision range of our study. Based on the stellar rotation
curve, Cooke  _et al._ (2022) determined the coefficients of the generalized
NFW profile (Eq. (6)) for NGC 4451, i.e., $\rho_{0}=0.41\,M_{\odot}\,{\rm
pc^{-3}}$, $r_{s}=2.2\,{\rm kpc}$, and $\beta=0$. Furthermore, a recent
milliarcsecond localization of FRB 20220912A shows that its transverse offset
from the host galaxy center is $r\approx 0.8$ kpc Hewitt  _et al._ (2023).
With this information, an estimate on the DM density at the location of FRB
20220912A from Eq. (6) is $\rho\sim 0.16\,M_{\odot}\,{\rm pc^{-3}}$. This
value is much larger than the DM density near our Earth, which is $\sim
0.01\,M_{\odot}\,{\rm pc^{-3}}$ estimated by the Galactic NFW profile Nesti
and Salucci (2013).

###  III.2 The Polarization Angles

After the initial discovery of FRB 20220912A, subsequent observations from
multiple telescopes have consistently detected a large number of bursts from
this specific source Bhusare  _et al._ (2022); Fedorova and Rodin (2022);
Herrmann (2022); Kirsten  _et al._ (2022); Ould-Boukattine  _et al._ (2022);
Pelliciari  _et al._ (2022); Perera  _et al._ (2022); Rajwade  _et al._
(2022); Sheikh  _et al._ (2022); Yu  _et al._ (2022); Feng  _et al._ (2023);
Zhang  _et al._ (2023b). It is noteworthy that FRB 20220912A was monitored by
FAST for a period of several dozen days, during which a total of 1076 bursts
were recorded Zhang  _et al._ (2023b). Most of these bursts exhibit nearly
100% linear polarization. The rotation measure (RM) of FRB 20220912A is very
close to 0 and did not show any variation during the FAST observation period,
indicating that FRB 20220912A is located in a relatively clean local
environment Feng  _et al._ (2023); Zhang  _et al._ (2023b). The non-variable
RM also means that the PAs of FRB 20220912A are relatively stable.

As described in section II, when a linearly polarized light propagates in an
external ALP field, the corresponding PAs would have a time-dependent change
due to the ALP-photon coupling effect. By analyzing the PA variations, we thus
can constrain the ALP-photon coupling strength, thereby facilitating the
potential detection of axion-like DM. Here, the PAs of FRB 20220912A are
processed from the raw data of FAST. The central frequency, bandwidth, number
of frequency channels, and sampling time for the raw data were 1.25 GHz, 0.5
GHz, 4096, and 49.152 $\rm\mu s$, respectively. We use the GPU accelerated
transient detection software _HEIMDALL_
222https://sourceforge.net/projects/heimdall-astro and process the data on
FAST’s high-performance-computer facilities. The dispersion measure range that
we search is between 200 and 250 $\rm pc\,cm^{-3}$. The signal-to-noise ratio
threshold and maximum boxcar are set as 6.5 and 512, respectively. The RM is
searched from $-2000$ to $2000$ $\rm rad\,m^{-2}$ in steps of 1 $\rm
rad\,m^{-2}$ using the _rmfit_ program van Straten  _et al._ (2012). After
correcting the data with the best-fitted RMs, we derive the position angles of
the linearly polarized component. During a total of 9.2 hours of observations
between October 28th, 2022 and December 5th, 2022 (corresponding to MJD 59880
and MJD 59918), we obtain 700 bursts with RM measurements. Fig. 1 displays PA
measurements of FRB 20220912A as a function of time. One can see from this
plot that PAs vary greatly within a day, but they are relatively stable on the
timescale of months. Our focus is primarily on the PA variations in the order
of days, so we calculate the median value of the PA in each day. In this case,
the minimum time interval is 1 day, while the total observational time is 38
days. According to the theory presented in section II, the lower and upper
limits of the ALP mass $m_{a}$ depend on the total observational time and the
minimum time interval, respectively. That is, the sensitive ALP mass $m_{a}$
falls within the range of $1.4\times 10^{-21}$ eV to $5.2\times 10^{-20}$ eV.

##  IV Constraints on the ALP-Photon Coupling

Variations in PAs of FRBs are complex and puzzling Cho  _et al._ (2020); Luo
_et al._ (2020); Xu  _et al._ (2022). The prevailing understanding is that
these variations are mainly attributed to the significant fluctuations in the
magnetic fields surrounding FRB sources. However, if the axion-like DM exits
in the host galaxy and envelops the FRB sources, the observed PA ($\phi_{\rm
obs}$) is expected to be composed of two components: one from the PA
contribution of the astrophysical background (e.g., the magnetic field),
$\phi_{\rm bkg}$, and the other one is the ALP-induced PA shift, $\phi(t)$,
i.e.,

$\begin{split}\phi_{\rm obs}&=\phi_{\rm bkg}+\phi(t)\\\
&\simeq\left\langle\phi_{\rm bkg}\right\rangle+\Delta\phi_{\rm
bkg}+\phi(t),\end{split}$ (7)

where $\left\langle\phi_{\rm bkg}\right\rangle$ represents the mean value of
the PA caused by the background magnetic field and $\Delta\phi_{\rm bkg}$
corresponds to the PA fluctuation arising from the changes of the magnetic
field. Since $\Delta\phi_{\rm bkg}$ is unpredictable, we assume that the
observed PA fluctuations are attributed to the ALP-photon coupling effect,
i.e., $\phi_{\rm obs}=\left\langle\phi_{\rm bkg}\right\rangle+\phi(t)$, which
will conduct conservative upper limits on the ALP-photon coupling constant
$g_{a\gamma}$ for different ALP mass.

In this work, we employ two analysis methods to search for the ALP-photon
coupling effect. The first way is to estimate the standard deviation
($\Delta\phi$) of the ALP-induced PA shift $\phi(t)$. Given the randomness of
the value of the phase $\delta$ (see Eq. (5)), we use the standard deviation
of $\phi(t)$ to characterize its oscillation amplitude Liu  _et al._ (2020).
This yields

$\begin{split}\Delta\phi&\equiv\sqrt{\left\langle\phi^{2}(t)\right\rangle}\\\
&\simeq 1.96^{\circ}\left(\frac{\rho}{0.16\,M_{\odot}\,{\rm
pc^{-3}}}\right)^{\frac{1}{2}}\\\
&\times\left(\frac{m_{a}}{10^{-21}\,\mathrm{eV}}\right)^{-1}\left(\frac{g_{a\gamma}}{10^{-11}\,\mathrm{GeV}^{-1}}\right).\end{split}$
(8)

The mean value of the observed daily median PAs is $\langle\phi_{\rm
Med}\rangle=-24.95^{\circ}\pm 4.16^{\circ}$. Here the mean $-24.95^{\circ}$ is
regarded as the mean background $\left\langle\phi_{\rm bkg}\right\rangle$, and
the $1\sigma$ standard deviation $4.16^{\circ}$ is regarded as $\Delta\phi$.
From Eq. (8), we can see that $\Delta\phi$ is proportional to $g_{a\gamma}$
for a given ALP mass $m_{a}$. With the ALP mass ranging from $1.4\times
10^{-21}$ eV to $5.2\times 10^{-20}$ eV, the corresponding upper limits on
$g_{a\gamma}$ can be obtained.

Additionally, the Markov Chain Monte Carlo (MCMC) fitting method is also
applied for our analyses. Firstly, we construct a grid in the parameter space
of ($m_{a}$, $\delta$). For a set of fixed ALP mass $m_{a}$ from $1.4\times
10^{-21}$ eV to $5.2\times 10^{-20}$ eV and fixed phase $\delta$ in the range
of [0, 2$\pi$], we perform a fitting procedure to simultaneously optimize the
ALP-photon coupling constant $g_{a\gamma}$ and the mean background
$\left\langle\phi_{\mathrm{bkg}}\right\rangle$. This process involves
minimizing the $\chi^{2}$ value by adjusting the two parameters $g_{a\gamma}$
and $\left\langle\phi_{\mathrm{bkg}}\right\rangle$ with the MCMC technology.
The standard $\chi^{2}$ statistic is given by

$\chi^{2}=\sum_{i=1}^{N}\left[\frac{\phi_{\mathrm{obs},i}-\phi(t_{i})-\left\langle\phi_{\mathrm{bkg}}\right\rangle}{\sigma_{i}}\right]^{2},$
(9)

where $\phi_{\mathrm{obs},i}$ and $\sigma_{i}$ are the observed PA and its
error. Then we average over $g_{a\gamma}$ for the same $m_{a}$, which is
equivalent to marginalizing over the nuisance parameter $\delta$.

The resulting constraints on $g_{a\gamma}$ from both the two methods are shown
in Fig. 4, along with other constraints from different astrophysical sources.
For the first method, we obtain an upper limit of $g_{a\gamma}<2.1\times
10^{-11}\,\mathrm{GeV}^{-1}$ for an ALP mass $m_{a}\sim 10^{-21}$ eV. For the
second one, the obtained upper limit is $g_{a\gamma}<2.7\times
10^{-10}\,\mathrm{GeV}^{-1}$ for the same ALP mass. The worse result obtained
in the second method is probably due to the large daily fluctuations of PAs
after using the complete data in MCMC, which can be regarded as a pessimistic
estimation. It should be emphasized that these results are the first
constraints on the ALP-photon coupling derived from FRB polarization
measurements. Finally, we also forecast a future constraint. If the
polarization observations of FRB 20220912A last for up to one year, the limit
would be further improved by one order of magnitude, yielding
$g_{a\gamma}<3.0\times 10^{-12}\,\mathrm{GeV}^{-1}$.

Recently, Gao  _et al._ (2023) proposed gravitationally lensed FRBs as probes
for hunting Galactic axion DM. They predicted that, with a single lensed FRB,
the ALP-photon coupling could be constrained to be $g_{a\gamma}<7.3\times
10^{-11}\,\mathrm{GeV}^{-1}$ for an axion mass $m_{a}\sim 10^{-21}$ eV. This
forecast limit is similar to our result of $g_{a\gamma}<2.1\times
10^{-11}\,\mathrm{GeV}^{-1}$ from real FRB polarization observations.
Furthermore, as shown in Fig. 4, our result is slightly better than the
constraint from the CERN Axion Solar Telescope (CAST) ($g_{a\gamma}<6.6\times
10^{-11}\,\mathrm{GeV}^{-1}$) Anastassopoulos  _et al._ (2017) and comparable
with the constraint from the supernovae observed by the Fermi Large Area
Telescope (LAT) ($g_{a\gamma}<2.6\times 10^{-11}\,\mathrm{GeV}^{-1}$) Meyer
_et al._ (2020). In contrast to the galactic probes, such as pulsars Liu  _et
al._ (2020), black hole Sgr A∗ Yuan.  _et al._ (2021), and protoplanetary disk
Fujita  _et al._ (2019), our method can detect ALPs on kiloparsec scales,
which highlights the potential of FRBs for detecting extragalactic axion-like
DM.

Figure 2: The resulting constraints on the ALP-photon coupling constant
$g_{a\gamma}$ for different ALP mass, obtained using the polarization
measurements of FRB 2022012A (green shaded area). The green straight and
irregular lines correspond to the upper limits of $g_{a\gamma}$ derived from
the standard deviation method and the MCMC method, respectively. The green
dashed line represents the future constraints from continued polarization
observations of FRB 2022012A up to one year. Other constraints from different
astrophysical sources are also displayed for comparisons, including the CAST
experiment (yellow shaded area) Anastassopoulos  _et al._ (2017), the X-ray
observation of super star clusters (blue solid line) Dessert  _et al._ (2020),
the Fermi-LAT observation of supernovae (red dot-dashed line) Meyer  _et al._
(2020), the supernova SN1987A (gray dot-dashed line) Payez  _et al._ (2015),
the VLBA polarization observations of jets from active galaxies (orange shaded
area) Ivanov  _et al._ (2019), the polarized pulsar light (gray shaded area)
Liu  _et al._ (2020), and the polarization measurements of the black hole Sgr
A∗ (red solid line) Yuan.  _et al._ (2021). Please refer to this
link444https://cajohare.github.io/AxionLimits/docs/ap.html for a more complete
summary.

##  V Summary

Linearly polarized light emitted from FRBs can interact with the ALPs,
producing a periodic oscillation of the PAs. Searching for this periodic
oscillation feature in FRB polarization measurements can thus help us to
detect the axion-like DM. In this work, we attempted to detect extragalactic
ALPs in the host galaxies of the hyperactive repeating FRBs for the first
time. With the state-of-the-art observations the PAs of the bursts from the
repeating source FRB 20220912A taken by FAST, we obtained upper limits on the
ALP-photon coupling constant of $g_{a\gamma}<(2.9\times 10^{-11}-1.1\times
10^{-9})\,\mathrm{GeV}^{-1}$ for the ALP masses $m_{a}\sim(1.4\times
10^{-21}-5.2\times 10^{-20})$ eV. If the polarization observations of FRB
20220912A are expected to continue for one year, the $g_{a\gamma}$ limit would
be improved to be $g_{a\gamma}<3.0\times 10^{-12}\,\mathrm{GeV}^{-1}$ for an
ALP mass $m_{a}\sim 1.4\times 10^{-22}$ eV. Extragalactic FRBs provide an
alternative way to detect axion-like DM on kiloparsec scales, complementary to
other galactic DM probes.

###### Acknowledgements.

Bao Wang thanks Lei Lei for helpful discussions on dark matter. This work is
partially supported by the National SKA Program of China (2022SKA0130100), the
National Natural Science Foundation of China (grant Nos. 12373053, 12321003,
and 12041306), the Key Research Program of Frontier Sciences (grant No. ZDBS-
LY-7014) of Chinese Academy of Sciences, International Partnership Program of
Chinese Academy of Sciences for Grand Challenges (114332KYSB20210018), the CAS
Project for Young Scientists in Basic Research (grant No. YSBR-063), the CAS
Organizational Scientific Research Platform for National Major Scientific and
Technological Infrastructure: Cosmic Transients with FAST, and the Natural
Science Foundation of Jiangsu Province (grant No. BK20221562).

## References

  * Preskill  _et al._ (1983) J. Preskill, M. B. Wise, and F. Wilczek, Physics Letters B 120, 127 (1983).
  * Marsh (2016) D. J. E. Marsh, Physics Reports 643, 1 (2016), arXiv:1510.07633 [astro-ph.CO] .
  * Choi  _et al._ (2021) K. Choi, S. H. Im, and C. S. Shin, Annual Review of Nuclear and Particle Science 71, 225 (2021), arXiv:2012.05029 [hep-ph] .
  * Hu  _et al._ (2000) W. Hu, R. Barkana, and A. Gruzinov, Phys. Rev. Lett. 85, 1158 (2000), arXiv:astro-ph/0003365 [astro-ph] .
  * Hui  _et al._ (2017) L. Hui, J. P. Ostriker, S. Tremaine, and E. Witten, Phys. Rev. D 95, 043541 (2017), arXiv:1610.08297 [astro-ph.CO] .
  * Horns  _et al._ (2013) D. Horns, _et al._ , J. Cosmol. Astropart. Phys. 2013, 016 (2013), arXiv:1212.2970 [hep-ph] .
  * Payez  _et al._ (2015) A. Payez, _et al._ , J. Cosmol. Astropart. Phys. 2015, 006 (2015), arXiv:1410.3747 [astro-ph.HE] .
  * Anastassopoulos  _et al._ (2017) V. Anastassopoulos, _et al._ , Nature Physics 13, 584 (2017), arXiv:1705.02290 [hep-ex] .
  * Du  _et al._ (2018) N. Du, _et al._ , Phys. Rev. Lett. 120, 151301 (2018), arXiv:1804.05750 [hep-ex] .
  * Graham and Rajendran (2013) P. W. Graham and S. Rajendran, Phys. Rev. D 88, 035023 (2013), arXiv:1306.6088 [hep-ph] .
  * Budker  _et al._ (2014) D. Budker, P. W. Graham, M. Ledbetter, S. Rajendran, and A. O. Sushkov, Physical Review X 4, 021030 (2014), arXiv:1306.6089 [hep-ph] .
  * Ivanov  _et al._ (2019) M. M. Ivanov, _et al._ , J. Cosmol. Astropart. Phys. 2019, 059 (2019), arXiv:1811.10997 [astro-ph.CO] .
  * Chen  _et al._ (2020) Y. Chen, J. Shu, X. Xue, Q. Yuan, and Y. Zhao, Phys. Rev. Lett. 124, 061102 (2020), arXiv:1905.02213 [hep-ph] .
  * Liu  _et al._ (2020) T. Liu, G. Smoot, and Y. Zhao, Phys. Rev. D 101, 063012 (2020), arXiv:1901.10981 [astro-ph.CO] .
  * Yuan.  _et al._ (2021) G.-W. Yuan., _et al._ , J. Cosmol. Astropart. Phys. 2021, 018 (2021), arXiv:2008.13662 [astro-ph.HE] .
  * Dessert  _et al._ (2020) C. Dessert, J. W. Foster, and B. R. Safdi, Phys. Rev. Lett. 125, 261102 (2020), arXiv:2008.03305 [hep-ph] .
  * Meyer  _et al._ (2020) M. Meyer, T. Petrushevska, and Fermi-LAT Collaboration, Phys. Rev. Lett. 125, 119901 (2020).
  * Fujita  _et al._ (2019) T. Fujita, R. Tazaki, and K. Toma, Phys. Rev. Lett. 122, 191101 (2019), arXiv:1811.03525 [astro-ph.CO] .
  * Caputo  _et al._ (2019) A. Caputo, _et al._ , Phys. Rev. D 100, 063515 (2019), arXiv:1902.02695 [astro-ph.CO] .
  * Liu  _et al._ (2023a) T. Liu, X. Lou, and J. Ren, Phys. Rev. Lett. 130, 121401 (2023a), arXiv:2111.10615 [astro-ph.HE] .
  * Carroll  _et al._ (1990) S. M. Carroll, G. B. Field, and R. Jackiw, Phys. Rev. D 41, 1231 (1990).
  * Harari and Sikivie (1992) D. Harari and P. Sikivie, Physics Letters B 289, 67 (1992).
  * Xiao  _et al._ (2021) D. Xiao, F. Wang, and Z. Dai, Science China Physics, Mechanics, and Astronomy 64, 249501 (2021), arXiv:2101.04907 [astro-ph.HE] .
  * Zhang  _et al._ (2023a) J.-G. Zhang, _et al._ , Science China Physics, Mechanics, and Astronomy 66, 120412 (2023a), arXiv:2307.01605 [astro-ph.CO] .
  * Zhang (2023) B. Zhang, Reviews of Modern Physics 95, 035005 (2023), arXiv:2212.03972 [astro-ph.HE] .
  * Wei  _et al._ (2015) J.-J. Wei, H. Gao, X.-F. Wu, and P. Mészáros, Phys. Rev. Lett. 115, 261101 (2015), arXiv:1512.07670 [astro-ph.HE] .
  * Wu  _et al._ (2016) X.-F. Wu, _et al._ , Astrophys. J. Lett. 822, L15 (2016), arXiv:1602.07835 [astro-ph.HE] .
  * Yang and Zhang (2016) Y.-P. Yang and B. Zhang, Astrophys. J. Lett. 830, L31 (2016), arXiv:1608.08154 [astro-ph.HE] .
  * Macquart  _et al._ (2020) J. P. Macquart, _et al._ , Nature (London) 581, 391 (2020), arXiv:2005.13161 [astro-ph.CO] .
  * Wei and Wu (2021) J.-J. Wei and X.-F. Wu, Frontiers of Physics 16, 44300 (2021), arXiv:2102.03724 [astro-ph.HE] .
  * Yang  _et al._ (2022) K. B. Yang, Q. Wu, and F. Y. Wang, Astrophys. J. Lett. 940, L29 (2022), arXiv:2211.04058 [astro-ph.HE] .
  * James  _et al._ (2022) C. W. James, _et al._ , Mon. Not. R. Astron. Soc. 516, 4862 (2022), arXiv:2208.00819 [astro-ph.CO] .
  * Liu  _et al._ (2023b) Y. Liu, H. Yu, and P. Wu, Astrophys. J. Lett. 946, L49 (2023b), arXiv:2210.05202 [astro-ph.CO] .
  * Wang and Wei (2023) B. Wang and J.-J. Wei, Astrophys. J. 944, 50 (2023), arXiv:2211.02209 [astro-ph.CO] .
  * Landim (2020) R. G. Landim, European Physical Journal C 80, 913 (2020), arXiv:2005.08621 [astro-ph.CO] .
  * Sammons  _et al._ (2020) M. W. Sammons, _et al._ , Astrophys. J. 900, 122 (2020), arXiv:2002.12533 [astro-ph.CO] .
  * Muñoz  _et al._ (2016) J. B. Muñoz, E. D. Kovetz, L. Dai, and M. Kamionkowski, Phys. Rev. Lett. 117, 091301 (2016), arXiv:1605.00008 [astro-ph.CO] .
  * Wang and Wang (2018) Y. K. Wang and F. Y. Wang,  Astron. Astrophys. 614, A50 (2018), arXiv:1801.07360 [astro-ph.CO] .
  * Liao  _et al._ (2020) K. Liao, S. B. Zhang, Z. Li, and H. Gao, Astrophys. J. Lett. 896, L11 (2020), arXiv:2003.13349 [astro-ph.CO] .
  * Ho  _et al._ (2023) S. C. C. Ho, _et al._ , Astrophys. J. 950, 53 (2023), arXiv:2304.04990 [astro-ph.HE] .
  * Krochek and Kovetz (2022) K. Krochek and E. D. Kovetz, Phys. Rev. D 105, 103528 (2022), arXiv:2112.03721 [astro-ph.CO] .
  * Gao  _et al._ (2023) R. Gao, _et al._ , arXiv e-prints , arXiv:2312.12997 (2023), arXiv:2312.12997 [astro-ph.CO] .
  * Ravi  _et al._ (2023) V. Ravi, _et al._ , Astrophys. J. Lett. 949, L3 (2023), arXiv:2211.09049 [astro-ph.HE] .
  * Zhang  _et al._ (2023b) Y.-K. Zhang, _et al._ , Astrophys. J. 955, 142 (2023b), arXiv:2304.14665 [astro-ph.HE] .
  * Feng  _et al._ (2023) Y. Feng, _et al._ , arXiv e-prints , arXiv:2304.14671 (2023), arXiv:2304.14671 [astro-ph.HE] .
  * Wilczek (1987) F. Wilczek, Phys. Rev. Lett. 58, 1799 (1987).
  * Zhao (1996) H. Zhao, Mon. Not. R. Astron. Soc. 278, 488 (1996), arXiv:astro-ph/9509122 [astro-ph] .
  * Navarro  _et al._ (1996) J. F. Navarro, C. S. Frenk, and S. D. M. White, Astrophys. J. 462, 563 (1996), arXiv:astro-ph/9508025 [astro-ph] .
  * Gordon  _et al._ (2023) A. C. Gordon, _et al._ , Astrophys. J. 954, 80 (2023), arXiv:2302.05465 [astro-ph.GA] .
  * Cooke  _et al._ (2022) L. H. Cooke, _et al._ , Mon. Not. R. Astron. Soc. 512, 1012 (2022), arXiv:2203.00694 [astro-ph.GA] .
  * Hewitt  _et al._ (2023) D. M. Hewitt, _et al._ , arXiv e-prints , arXiv:2312.14490 (2023), arXiv:2312.14490 [astro-ph.HE] .
  * Nesti and Salucci (2013) F. Nesti and P. Salucci, J. Cosmol. Astropart. Phys. 2013, 016 (2013), arXiv:1304.5127 [astro-ph.GA] .
  * Bhusare  _et al._ (2022) Y. Bhusare, _et al._ , The Astronomer’s Telegram 15806, 1 (2022).
  * Fedorova and Rodin (2022) V. A. Fedorova and A. E. Rodin, The Astronomer’s Telegram 15713, 1 (2022).
  * Herrmann (2022) W. Herrmann, The Astronomer’s Telegram 15691, 1 (2022).
  * Kirsten  _et al._ (2022) F. Kirsten, _et al._ , The Astronomer’s Telegram 15727, 1 (2022).
  * Ould-Boukattine  _et al._ (2022) O. S. Ould-Boukattine, _et al._ , The Astronomer’s Telegram 15817, 1 (2022).
  * Pelliciari  _et al._ (2022) D. Pelliciari, _et al._ , The Astronomer’s Telegram 15695, 1 (2022).
  * Perera  _et al._ (2022) B. Perera, _et al._ , The Astronomer’s Telegram 15734, 1 (2022).
  * Rajwade  _et al._ (2022) K. Rajwade, _et al._ , The Astronomer’s Telegram 15791, 1 (2022).
  * Sheikh  _et al._ (2022) S. Sheikh, _et al._ , The Astronomer’s Telegram 15735, 1 (2022).
  * Yu  _et al._ (2022) Z. Yu, _et al._ , The Astronomer’s Telegram 15758, 1 (2022).
  * van Straten  _et al._ (2012) W. van Straten, P. Demorest, and S. Oslowski, Astronomical Research and Technology 9, 237 (2012), arXiv:1205.6276 [astro-ph.IM] .
  * Cho  _et al._ (2020) H. Cho, _et al._ , Astrophys. J. Lett. 891, L38 (2020), arXiv:2002.12539 [astro-ph.HE] .
  * Luo  _et al._ (2020) R. Luo, _et al._ , Nature (London) 586, 693 (2020), arXiv:2011.00171 [astro-ph.HE] .
  * Xu  _et al._ (2022) H. Xu, _et al._ , Nature (London) 609, 685 (2022), arXiv:2111.11764 [astro-ph.HE] .