# Norm tori of étale algebras and unramified Brauer groups

Eva Bayer-Fluckiger and Ting-Yu Lee

(Date: August 27, 2024)

###### Abstract.

Let $k$ be a field, and let $L$ be an étale $k$-algebra of finite rank. If
$a\in k^{\times}$, let $X_{a}$ be the affine variety defined by
$N_{L/k}(x)=a$. Assuming that $L$ has at least one factor that is a cyclic
field extension of $k$, we give a combinatorial description of the unramified
Brauer group of $X_{a}$.

##  0\. Introduction

Let $k$ be a field, let $L$ be an étale $k$-algebra of finite rank, and let
$N_{L/k}:L\to k$ be the norm map. Let $a\in k^{\times}$, and let $X_{a}$ be
the affine $k$-variety determined by

${\rm N}_{L/k}(t)=a.$

Let $X_{a}^{c}$ be a smooth compactification of $X_{a}$. The aim of this paper
is to describe the group ${\rm Br}(X_{a}^{c})/{\rm Im}({\rm Br}(k))$ under the
hypothesis that $L$ has at least one cyclic factor. We first give a
combinatorial description of a group associated to the étale algebra $L$ (see
§3), and then give an explicit isomorphism between this group and ${\rm
Br}(X_{a}^{c})/{\rm Im}({\rm Br}(k))$ (see §4, in particular Theorem 4.3).

Let us illustrate our results by a special case. Let $p$ be a prime number and
let $n\geqslant 1$ be an integer; assume that ${\rm char}(k)\not=p$ and let
$F$ be a Galois extension of $k$ with Galois group
$\mathbf{Z}/p^{n}\mathbf{Z}\times\mathbf{Z}/p^{n}\mathbf{Z}$. Suppose that $L$
is a product of $r$ linearly disjoint cyclic subfields of $F$ of degree
$p^{n}$. Then we have (see Theorem 4.9) :

Theorem.

$\mathrm{Br}(X_{a}^{c})/\mathrm{Im}(\mathrm{Br}(k))\simeq(\mathbf{Z}/p^{n}\mathbf{Z})^{r-2}.$

We also give explicit generators of this group, as follows. With the above
notation, let $K$ be one of the cyclic subfields of degree $p^{n}$ of $F$, and
let $\chi$ be an injective morphism from ${\rm Gal}(K/k)$ to
$\mathbf{Q}/\mathbf{Z}$. Let us write $L=K\times K\textquoteright$, with
$K\textquoteright=\underset{i\in I}{\prod}K_{i}$, where $K_{i}$ is a cyclic
subfield of $F$ of degree $p^{n}$ of $F$ for all $i\in I$, and assume that $K$
and the fields $K_{i}$ are linearly disjoint in $F$. For all $i\in I$, set
$N_{i}=N_{K_{i}/k}(y_{i})$, considered as elements of $k(X_{a})^{\times}$.
Assume that the cardinal of $I$ is $r-1$, so that $L$ is a product of the $r$
linearly independent cyclic subfields $K$ and $K_{i}$ of $F$ of degree
$p^{n}$. Let $I^{\prime}$ be a subset of cardinal $r-2$ of $I$.

Let $(N_{i},\chi)$ denote the class of the cyclic algebra over $k(X_{a})$
associated to $\chi$ and the element $N_{i}\in k(X_{a})^{\times}$.

Theorem. The group $\mathrm{Br}(X_{a}^{c})/\mathrm{Im}(\mathrm{Br}(k))$ is
generated by the elements $(N_{i},\chi)$ for $i\in I^{\prime}$.

This is also proved in Theorem 4.9. Note that the above results are
generalizations of [BP 20], Theorems 11.1 and 11.2.

The paper is organized as follows. Throughout the paper, $K$ is a finite
cyclic extension of $k$, and $L=K\times K^{\prime}$, where $K^{\prime}$ is an
étale $k$-algebra of finite rank. Sections 1 and 2 are preliminary : in
particular, it is shown in §2 that we may assume $K$ be cyclic of prime power
degree. Sections 3 and 4 contain the description of the unramified Brauer
group. When $k$ is a global field, we obtain additional results concerning the
“locally trivial" Brauer group (cf. §5). Finally, in §6 we apply Theorem 4.3
to give an alternative proof of [BLP 19] Theorem 7.1 for $k$ a global field
with $char(k)\neq p$; we show that the Brauer-Manin map of [BLP 19] is the
Brauer-Manin pairing, and hence deduce the Hasse principle from results of [Sa
81] and [DH 17].

##  1\. Definitions and notation

Generalities

Let $k$ be a field, let $k_{s}$ be a separable closure of $k$ and let
$\mathcal{G}_{k}={\rm Gal}(k_{s}/k)$ be the absolute Galois group of $k$. We
fix once and for all this separable closure $k_{s}$, and all separable
extensions of $k$ that will appear in the paper will be contained in $k_{s}$.
We use standard notation in Galois cohomology; in particular, if $M$ is a
discrete $\mathcal{G}_{k}$-module and $i$ is an integer $\geq 0$, we set
$H^{i}(k,M)=H^{i}(\mathcal{G}_{k},M)$. A $\mathcal{G}_{k}$-lattice will be a
torsion free $\bf Z$-module of finite rank on which $\mathcal{G}_{k}$ acts
continuously. For a $k$-torus $T$, we denote by $\hat{T}={\rm
Hom}(T,\mathbb{G}_{m})$ its character group; it is a
$\mathcal{G}_{k}$-lattice.

Let $G$ be a finite group. A $G$-lattice is by definition a ${\bf Z}$-torsion
free ${\bf Z}[G]$-module of finite rank. If $g\in G$, we denote by $\langle
g\rangle$ the cyclic subgroup of $G$ generated by $g$. Let $M$ be a
$G$-lattice. Set

${\mbox{{Sh}}}^{2}_{\rm cycl}(G,M)={\rm Ker}[H^{2}(G,M)\to\prod_{g\in
G}H^{2}(\langle g\rangle,M)].$

We recall a result of Colliot-Thélène and Sansuc (cf. [CTS 87] Prop. 9.5)

######  Theorem 1.1.

Let $G$ be a finite group, let $T$ be a $k$-torus, and assume that the
character group of $T$ is a $G$-lattice via a surjection $\mathcal{G}_{k}\to
G$. Let $T^{c}$ be a smooth compactification of $T$. We have ${\rm
Br}(T^{c})/{\rm Br}(k)\simeq{\mbox{{Sh}}}^{2}_{\rm cycl}(G,\hat{T})$.

Proof. See [BP 20], Theorem 1.3.

Norm equations

Let $L$ be an étale $k$-algebra of finite rank; in other words, a product of a
finite number of separable extensions of $k$. Let
$T_{L/{k}}=R^{(1)}_{L/k}({\bf G}_{m})$ be the $k$-torus defined by

$1\to T_{L/{k}}\to R_{L/k}({\bf G}_{m}){\buildrel{\rm
N}_{L/{k}}\over{\longrightarrow}}{\bf G}_{m}\to 1.$

Let $a\in k^{\times}$. Let $X_{a}$ be the affine $k$-variety associated to the
norm equation

${\rm N}_{L/k}(t)=a.$

The variety $X_{a}$ is a torsor under $T_{L/k}$; let $X_{a}^{c}$ be a smooth
compactification of $X_{a}$. We have a natural map ${\rm Br}(k)\to{\rm
Br}(X_{a}^{c})$; if $a=1$ then $X_{1}=T_{L/k}$, and the map ${\rm
Br}(k)\to{\rm Br}(T_{L/k}^{c})$ is injective, and moreover we have an
injection

${\rm Br}(X^{c})/{\rm Im}({\rm Br}(k))\to{\rm Br}(T^{c})/{\rm Br}(k)$

(see for instance [BP 20], §5). Recall a result from [BP 20], Theorem 6.1 :

######  Theorem 1.2.

Assume that $L=K\times K\textquoteright$, where $K/k$ is a cyclic extension
and $K\textquoteright$ an étale $k$-algebra. Then the map ${\rm
Br}(X^{c})/{\rm Im}({\rm Br}(k))\to{\rm Br}(T^{c})/{\rm Br}(k)$ is an
isomorphism.

Global fields

If $k$ is a global field, we denote by $V_{k}$ be the set of all places of
$k$; if $v\in V_{k}$, we denote by $k_{v}$ the completion of $k$ at $v$.

For any $k$-torus $T$, set

${\mbox{{Sh}}}^{i}(k,T)={\rm Ker}(H^{i}(k,T)\rightarrow\underset{v\in
V_{k}}{\prod}H^{i}(k_{v},T)).$

If $M$ is a $\mathcal{G}_{k}$-module, set

${\mbox{{Sh}}}^{i}(k,M)={\rm Ker}(H^{i}(k,M)\rightarrow\underset{v\in
V_{k}}{\prod}H^{i}(k_{v},M)),$

and let ${\mbox{{Sh}}}_{\omega}^{i}(k,M)$ be the set of $x\in H^{i}(k,M)$ that
map to 0 in $H^{i}(k_{v},M)$ for almost all $v\in V_{k}$.

##  2\. Norm equations and étale algebras

In the sequel, we consider norm equations of étale algebras having at least
one cyclic factor. The aim of this section is to introduce some notation and
prove some results that will be used throughout the paper.

Let $K$ be a cyclic extension of $k$, and let $K\textquoteright$ be an étale
$k$-algebra of finite rank; set $L=K\times K\textquoteright$. We first show
that it is enough to consider the case when $K/k$ is cyclic of prime power
degree.

Reduction to the prime power degree case

Let $\mathcal{P}$ be the set of prime numbers dividing $[K:k]$. For all
$p\in\mathcal{P}$, let $K[p]$ be the largest subfield of $K$ such that
$[K[p]:k]$ is a power of $p$, and set $L[p]=K[p]\times K\textquoteright$.
Recall from [BLP 19] the following result

######  Proposition 2.1.

Assume that $k$ is a global field. We have

${\mbox{{Sh}}}^{2}(k,\hat{T}_{L/k})\simeq\underset{p\in\mathcal{P}}{\oplus}{\mbox{{Sh}}}^{2}(k,\hat{T}_{L[p]/k}).$

Proof. This follows from [BLP 19], Lemma 3.1 and Proposition 5.16.

Let $k\textquoteright/k$ be a Galois extension of minimal degree splitting
$T_{L/k}$, and let $G={\rm Gal}(k\textquoteright/k)$.

######  Proposition 2.2.

We have ${\mbox{{Sh}}}_{\rm
cycl}^{2}(G,\hat{T}_{L/k})\simeq\underset{p\in\mathcal{P}}{\oplus}{\mbox{{Sh}}}^{2}_{\rm
cycl}(G,\hat{T}_{L[p]/k}).$

Proof. Let us write $K\textquoteright=\underset{i\in I}{\prod}K_{i}$, where
the $K_{i}$ are finite field extensions of $k$. Let $H$ be the subgroup of $G$
such that $K=(k\textquoteright)^{H}$, and for all $i\in I$, let $H_{i}$ be the
subgroup of $G$ such that $K_{i}=(k\textquoteright)^{H_{i}}$. Set
$M=\hat{T}_{L/k}$. We have the exact sequence of $G$-modules

$0\to{\bf Z}\to{\bf Z}[G/H]\oplus\underset{i\in I}{\oplus}{\bf Z}[G/H_{i}]\to
M\to 0.$

For all $p\in\mathcal{P}$, let $H[p]$ be the subgroup of $G$ such that
$K[p]=(k\textquoteright)^{H[p]}$. Set $M[p]=\hat{T}_{L[p]/k}$. We have the
exact sequence of $G$-modules

$0\to{\bf Z}\to{\bf Z}[G/H[p]]\oplus\underset{i\in I}{\oplus}{\bf
Z}[G/H_{i}]\to M[p]\to 0.$

Let $\ell\textquoteright/\ell$ be an unramified extension of number fields
with Galois group $G$ (cf. [F 62]). Set $L_{0}=(\ell\textquoteright)^{H}$,
$L_{0}[p]=(\ell\textquoteright)^{H[p]}$, and
$L_{i}=(\ell\textquoteright)^{H_{i}}$. Let $E=L_{0}\times\underset{i\in
I}{\prod}L_{i}$ and $E[p]=L_{0}[p]\times\underset{i\in I}{\prod}L_{i}$. We
have $\hat{T}_{E/\ell}\simeq M$ and $\hat{T}_{E[p]/\ell}\simeq M[p]$. By
Proposition 2.1 we have
${\mbox{{Sh}}}^{2}(\ell,M)\simeq\underset{p\in\mathcal{P}}{\oplus}{\mbox{{Sh}}}^{2}(\ell,M[p]).$
Since $\ell\textquoteright/\ell$ is unramified, we have ${\mbox{{Sh}}}_{\rm
cycl}^{2}(G,M)\simeq{\mbox{{Sh}}}^{2}(\ell,M)$ and ${\mbox{{Sh}}}_{\rm
cycl}^{2}(G,M[p])\simeq{\mbox{{Sh}}}^{2}(\ell,M[p])$ (see [BP 20], Proposition
3.1), hence ${\mbox{{Sh}}}_{\rm
cycl}^{2}(G,M)\simeq\underset{p\in\mathcal{P}}{\oplus}{\mbox{{Sh}}}^{2}_{\rm
cycl}(G,M[p]).$

######  Proposition 2.3.

Assume that $k$ is a global field We have

${\mbox{{Sh}}}_{\omega}^{2}(k,\hat{T}_{L/k})\simeq\underset{p\in\mathcal{P}}{\oplus}{\mbox{{Sh}}}_{\omega}^{2}(k,\hat{T}_{L[p]/k}).$

Proof. This follows from Proposition 2.2 and [BP 20], Corollary 3.4.

The prime power degree case

Let $p$ be a prime number, and assume that $K/k$ is cyclic of degree a power
of $p$. Let us write $K\textquoteright=\underset{i\in I}{\prod}K_{i}$, where
the $K_{i}$ are finite field extensions of $k$, and let $[K:k]=p^{n}$.

######  Notation 2.4.

For all integers $1\leq m\leq n$, let $K(m)$ be the unique subfield of $K$ of
degree $p^{m}$ over $k$. The $K_{i}$-algebra $K(m)\otimes_{k}K_{i}$ is a
product of cyclic extensions of $K_{i}$; let $p^{e_{i}(m)}$ be the degree of
these extensions, and set $\mathcal{E}(m)=\\{e_{i}(m)\ |\ i\in I\\}$. For all
$i\in I$, let us chose one of the cyclic factors $E_{i}/K_{i}$ of
$K\otimes_{k}K_{i}$. For all $m$ let $E_{i}(m)$ be the subfield of $E_{i}$
which corresponds to a cyclic factor of $K(m)\otimes_{k}K_{i}$.

Let $\mathcal{K}$ be a Galois extension of $k$ containing $K$ and all the
fields $K_{i}$, and let $G={\rm Gal}(\mathcal{K}/k)$. If $F$ is a subfield of
$\mathcal{K}$, we denote by $G_{F}$ the subgroup of $G$ such that
$F=\mathcal{K}^{G_{F}}$.

For all integers $0\leq m\leq n$, let $\Gamma_{i}^{m}$ be the set of conjugacy
classes of elements $g\in G$ such that $\langle
g\rangle\cap(G_{E_{i}(n-m)}\setminus G_{E_{i}(n-m+1)})\neq\emptyset$.

######  Notation 2.5.

Assume moreover that $k$ is a global field. Let $V_{i}^{m}$ be the set of
places $v$ of $k$ such that there exists a place $w$ of $K_{i}$ above $v$
having the property that $K\otimes_{k}(K_{i})_{w}$ is a product of fields
extensions of degree _at least_ $p^{m}$ of $(K_{i})_{w}$.

######  Proposition 2.6.

Assume that $k$ is a global field. Let $V_{rm}$ be the set of places of $k$
which are ramified in $\mathcal{K}$. For all integers $0\leq m\leq n$, sending
a place $v\in V_{i}^{m}\setminus V_{rm}$ to the conjugacy class of its
Frobenius element $\mathfrak{f}_{v}\in G$ gives rise to a surjection from
$V_{i}^{m}\setminus V_{rm}$ onto $\Gamma_{i}^{m}$.

In order to prove the Proposition, we need the following lemma.

######  Lemma 2.7.

Let $F$ be a field, and let $E$ be a cyclic extension of $F$ of prime degree.
Let $M$ be an extension of $E$, and assume that $M$ is a Galois extension of
$F$. Set $G_{F}={\mathrm{Gal}}(M/F)$ and $G_{E}={\mathrm{Gal}}(M/E)$. Let
$v:M^{\times}\to{\bf Z}$ be a discrete valuation of $M$; assume that the
restriction $v_{F}$ of $v$ to $F^{\times}$ is surjective, and that the residue
field of $v$ is perfect.

Let $D_{M/F}$ be the decomposition group of $v$. Then $v_{F}$ is inert in $E$
if and only if $D_{M/F}\cap(G_{F}\setminus G_{E})\not=\varnothing.$

Proof. Let $G_{E/F}$ be the Galois group of the extension $E/F$, and let
$D_{E/F}$ be the decomposition group of $v_{F}$; note that $v_{F}$ is inert in
$E$ if and only if $D_{E/F}=G_{E/F}$. Since $E/F$ is cyclic of prime degree,
this amounts to saying that $D_{E/F}$ is not trivial.

We have the exact sequence

$1\to G_{E}\to G_{F}\to G_{E/F}\to 1.$

The image of $D_{M/F}$ by the homomorphism $G_{F}\to G_{E/F}$ is equal to
$D_{E/F}$ (see for instance [Se 79], Chap. I, Proposition 22). Hence $D_{E/F}$
is non trivial if and only if $D_{M/F}\cap(G_{F}\setminus
G_{E})\not=\varnothing.$

######  Proposition 2.8.

Let $F$, $E$ and $M$ be as in Lemma 2.7. Assume moreover that $k$ is a
subfield of $F$, and that $M$ is a Galois extension of $k$. Let
$G=\mathrm{Gal}(M/k)$. Let $v_{k}:k^{\times}\to{\bf Z}$ be a discrete
valuation such that the extensions of $v_{k}$ to $M$ are unramified; let
$\mathcal{D}$ be the set of corresponding decomposition groups. The following
are equivalent

(a) There exists an extension of $v_{k}$ to $F$ that is inert in $E$.

(b) There exists $D\in\mathcal{D}$ such that $D\cap(G_{F}\setminus
G_{E})\not=\varnothing.$

Proof. Let us prove that (a) implies (b). Let $v_{F}$ be an extension of
$v_{k}$ to $F$ that is inert in $E$, let $v$ be an extension of $v_{F}$ to
$M$, and let $D$ be the decomposition group of $v$. By Lemma 2.7, we have (b).
Conversely, assume that (b) holds. Let $D\in\mathcal{D}$ be as in (b), and let
$v$ be the corresponding valuation. Let $v_{F}$ be the restriction of $v$ to
$F$. By Lemma 2.7, we see that $v_{F}$ is inert in $E$, hence (a) holds.

Proof of Proposition 2.6 If $v\in V_{i}^{m}\setminus V_{rm}$, then by
definition there exists a place of $E_{i}(n-m)$ that is inert in the extension
$E_{i}(n)/E_{i}(n-m)$, and hence also in $E_{i}(n-m+1)/E_{i}(n-m)$; therefore
by Proposition 2.8 the conjugacy class of its Frobenius element
$\mathfrak{f}_{v}$ belongs to $\Gamma_{i}^{m}$.

Conversely, if the conjugacy class of $g\in G$ belongs to $\Gamma_{i}^{m}$,
then by Chebotarev’s density theorem there exists an umramified place $v$ such
that its Frobenius element $\mathfrak{f}_{v}$ is the conjugacy class of $g$.
Since the conjugacy class of $g$ belongs to $\Gamma_{i}^{m}$, there is a place
$w$ of $E_{i}(n-m)$ above $v$ such that $w$ is inert in
$E_{i}(n-m+1)/E_{i}(n-m)$. Therefore $w$ is also inert in
$E_{i}(n)/E_{i}(n-m)$ as $E_{i}(n)/E_{i}(n-m)$ is cyclic of $p$-power degree
and is unramified at $w$. This implies that $v\in V_{i}^{m}\setminus V_{rm}$.

We get immediately the following corollary.

######  Corollary 2.9.

For $i$, $j\in I$, the map defined in Proposition 2.6 induces a surjection
from $V_{i}^{m}\cap V_{j}^{m}\setminus V_{rm}$ to
$\Gamma^{m}_{i}\cap\Gamma^{m}_{j}$.

######  Remark 2.10.

Keep the notation in Proposition 2.6. For each conjugacy class in
$\Gamma_{i}^{m}$, by Chebotarev’s density theorem there are infinite many
unramified places $v\in V_{i}^{m}$ mapped to it.

##  3\. Norm equations - unramified Brauer group

We keep the notation of the previous section. In particular, $k$ is a field,
$K$ is a cyclic extension of $k$, and $L=K\times K\textquoteright$ where
$K\textquoteright$ is an étale $k$-algebra of finite rank. Let
$T_{L/{k}}=R^{(1)}_{L/k}({\bf G}_{m})$ be the $k$-torus defined by

$1\to T_{L/{k}}\to R_{L/k}({\bf G}_{m}){\buildrel{\rm
N}_{L/{k}}\over{\longrightarrow}}{\bf G}_{m}\to 1.$

Let $a\in k^{\times}$, and let $X_{a}$ be the affine $k$-variety associated to
the norm equation ${\rm N}_{L/k}(t)=a.$ The variety $X_{a}$ is a torsor under
$T_{L/k}$. Let $T_{L/k}^{c}$ be a smooth compactification of $T_{L/k}$, and
let $X_{a}^{c}$ be a smooth compactification of $X_{a}$.

The aim of this section is to describe the group ${\rm Br}(X_{a}^{c})/{\rm
Im}({\rm Br}(k))$. Using the results of §2, we can assume that $K/k$ is of
degree $p^{n}$, where $p$ is a prime number.

We use the notation of §2 (see Notation 2.4). In addition, we need the
following

######  Notation 3.1.

For all integers $n\geq 1$, we denote by $C(I,{\bf Z}/p^{n}{\bf Z})$ the set
of maps $I\to{\bf Z}/p^{n}{\bf Z}$.

If $1\leq m\leq n$, let $\pi_{n,m}$ be the projection $C(I,{\bf Z}/p^{n}{\bf
Z})\to C(I,{\bf Z}/p^{m}{\bf Z})$.

For $x\in{\bf Z}/p^{m}\bf{Z}$ and $y\in{\mathbf{Z}}/p^{r}{\bf Z}$, we denote
by $\delta(x,y)$ the maximum integer $d\leq\min\\{m,r\\}$ such that $x=y$ (mod
$p^{d}\bf{Z}$).

We start with some special cases, in which the results are especially simple.

$K/k$ cyclic of degree $p$

Assume first that $[K:k]=p$, and that $K$ is not contained in any of the
fields $K_{i}$. Then for all $i\in I$, $E_{i}$ is a cyclic field extension of
degree $p$ of $K_{i}$. Let $\Gamma_{i}=\Gamma_{i}^{1}$ be the set of conjugacy
classes of elements $g\in G$ such that $\langle
g\rangle\cap(G_{K_{i}}-G_{E_{i}})\neq\emptyset$ (cf. Notation 2.4).

Let $C(L)$ be the group

$\\{c\in C(I,{\bf Z}/p{\bf Z})\ |\ c(i)=c(j)\ {\rm if}\
\Gamma_{i}\cap\Gamma_{j}\not=\varnothing\\},$

and $D$ be the subgroup of constant maps $I\to{\bf Z}/p{\bf Z}$.

As a consequence of Theorem 3.8, we’ll show the following

######  Proposition 3.2.

Assume that $K/k$ is cyclic of degree $p$, and that $K$ is not contained in
any of the fields $K_{i}$. Then we have

${\mbox{{Sh}}}^{2}_{\rm cycl}(G,\hat{T}_{L/k})\simeq C(L)/D.$

By Theorem 1.1, this implies the following

######  Corollary 3.3.

Assume that $K/k$ is cyclic of degree $p$, and that $K$ is not contained in
any of the fields $K_{i}$. Then we have

${\rm Br}(T^{c})/{\rm Br}(k)\simeq C(L)/D.$

$K/k$ of degree $p^{n}$ and $K$ linearly disjoint of all the $K_{i}$

For all integers $m$ with $1\leq m\leq n$ set

$C^{m}=\\{c\in C(I,{\bf Z}/p^{m}{\bf Z})\ |c(i)=c(j)\ {\rm if}\
\Gamma_{i}^{m}\cap\Gamma_{j}^{m}\not=\varnothing\\}.$

Set

$C(L)=\\{c\in C^{n}\ |\ \pi_{n,m}(c)\in C^{m}\ {\rm for}\ {\rm all}\ m\leq
n\\},$

and denote by $D$ the subgroup of constant maps $I\to{\bf Z}/p^{n}{\bf Z}$.

######  Proposition 3.4.

Assume that $K/k$ is cyclic of degree $p^{n}$, and that $K$ is linearly
disjoint of all the fields $K_{i}$. Then we have

${\mbox{{Sh}}}^{2}_{\rm cycl}(G,\hat{T}_{L/k})\simeq C(L)/D.$

As in the case where $K/k$ is of degree $p$, this follows from Theorem 3.8,
and has the immediate corollary

######  Corollary 3.5.

Assume that $K/k$ is cyclic of degree $p^{n}$, and that $K$ is linearly
disjoint of all the fields $K_{i}$. Then we have

${\rm Br}(T^{c})/{\rm Br}(k)\simeq C(L)/D.$

The general case

Recall that $K/k$ is cyclic of degree $p^{n}$, and that we use Notation 2.4.
Recall that $\mathcal{E}=\mathcal{E}(n)$.

######  Notation 3.6.

For all $e\in\mathcal{E}$, set $I_{e}=\\{i\in I\ |\ e_{i}(n)=e\\}$. Denote by
$\hat{e}$ the maximum element in $\mathcal{E}$. Note that the index $i$
belongs to $I_{e}$ if and only if $K\cap K_{i}$ is an extension of degree
$p^{n-e}$ of $k$. As $K$ is a cyclic extension, this means that given $0\leq
m\leq n$, the $e_{i}(m)$ are the same for all $i\in I_{e}$ and we denote it by
$e(m)$.

For all integers $m$ with $1\leq m\leq n$ set

$C^{m}=\\{c\in\underset{e\in\mathcal{E}}{\oplus}\ C(I_{e},{\bf
Z}/p^{e-e(n-m)}{\bf Z})\ |c(i)=c(j)\ {\rm if}\
\Gamma_{i}^{m}\cap\Gamma_{j}^{m}\not=\varnothing\\}.$

We still denote by $\pi_{n,m}$ the map from
$\underset{e\in\mathcal{E}}{\oplus}\ C(I_{e},{\bf Z}/p^{e}{\bf Z})$ to
$\underset{e\in\mathcal{E}}{\oplus}\ C(I_{e},{\bf Z}/p^{e-e(n-m)}{\bf Z})$
induced by the natural projection.

Set

$C(L)=\\{c\in C^{n}\ |\ \pi_{n,m}(c)\in C^{m}\ {\rm for}\ {\rm all}\ m\leq
n\\},$

and denote by $D$ the image of constant maps $I\to{\bf Z}/p^{n}{\bf Z}$ in
$C^{n}$ under the natural projection .

######  Remark 3.7.

If $E_{i}(n-m+1)\supsetneq E_{i}(n-m)$, then $K(n-m)\supseteq K\cap K_{i}$. In
this case $e_{i}(n)\geq m$ and $e_{i}(n)-e_{i}(n-m)=m$.

The main results of this section are

######  Theorem 3.8.

Assume that $K/k$ is cyclic of degree $p^{n}$. Then we have

${\mbox{{Sh}}}^{2}_{\rm cycl}(G,\hat{T}_{L/k})\simeq C(L)/D.$

By Theorem 1.1, this implies the following

######  Corollary 3.9.

Assume that $K/k$ is cyclic of degree $p^{n}$. then we have

${\rm Br}(T^{c})/{\rm Br}(k)\simeq C(L)/D.$

The proof of Theorem 3.8 will be given below, using some arithmetic results of
[BLP 19]. We start by recalling and developing some results concerning global
fields.

Global fields

Assume that $k$ is a global field. Recall that $K/k$ is cyclic of degree
$p^{n}$, and that we use notations 2.4 as well as 3.6. In addition, for global
fields, we also use notation 2.5.

For all integers $m$ with $1\leq m\leq n$ set

$C_{\rm arith}^{m}=\\{c\in\underset{e\in\mathcal{E}}{\oplus}\ C(I_{e},{\bf
Z}/p^{e-e(n-m)}{\bf Z})\ |c(i)=c(j)\ {\rm if}\ V_{i}^{m}\cap
V_{j}^{m}\not=\varnothing\\}$

and

$C_{\omega}^{m}=\\{c\in\underset{e\in\mathcal{E}}{\oplus}\ C(I_{e},{\bf
Z}/p^{e-e(n-m)}{\bf Z})\ |c(i)=c(j)\ {\rm if}\ V_{i}^{m}\cap V_{j}^{m}{\rm\
is\ infinite}\\}$

Set

$C_{\rm arith}(L)=\\{c\in C_{\rm arith}^{n}\ |\ \pi_{n,m}(c)\in C_{\rm
arith}^{m}\ {\rm for}\ {\rm all}\ m\leq n\\}$

and

$C_{\omega}(L)=\\{c\in C_{\omega}^{n}\ |\ \pi_{n,m}(c)\in C_{\omega}^{m}\ {\rm
for}\ {\rm all}\ m\leq n\\}.$

######  Theorem 3.10.

Assume that $K/k$ is cyclic of degree $p^{n}$. Then we have

(1) ${\mbox{{Sh}}}^{2}(k,\hat{T}_{L/k})\simeq C_{\rm arith}(L)/D.$

(2) ${\mbox{{Sh}}}_{\omega}^{2}(k,\hat{T}_{L/k})\simeq C_{\omega}(L)/D.$

Proof. Recall some notation of [BLP 19].

For $a=(a_{i})\in\underset{e\in\mathcal{E}}{\oplus}\underset{i\in
I_{e}}{\oplus}({\bf Z}/p^{e}{\bf Z})$ and
$r\in\mathbf{Z}/p^{\hat{e}}\mathbf{Z}$, set

$I_{r}(a)=\\{i\in I\ |\ a_{i}=r\ ({\rm mod}\ p^{e_{i}(n)}\mathbf{Z})\\}.$

Set

$G(K,K^{\prime})=\\{a=(a_{i})\in\underset{e\in\mathcal{E}}{\oplus}\underset{i\in
I_{e}}{\oplus}({\bf Z}/p^{e}{\bf Z})\ |\ \underset{r\in{\bf Z}/p^{\hat{e}}{\bf
Z}}{\cap}\ \underset{i\notin
I_{r}(a)}{\cup}V_{i}^{\delta(r,a_{i})+1}=\varnothing\\};$
$G_{\omega}(K,K^{\prime})=\\{a=(a_{i})\in\underset{e\in\mathcal{E}}{\oplus}\underset{i\in
I_{e}}{\oplus}({\bf Z}/p^{e}{\bf Z})\ |\ \underset{r\in{\bf Z}/p^{\hat{e}}{\bf
Z}}{\cap}\ \underset{i\notin I_{r}(a)}{\cup}V_{i}^{\delta(r,a_{i})+1}\ \mbox{
is finite}\\}.$

With the notation of [BLP 19], we have
${\mbox{{Sh}}}^{2}(k,\hat{T}_{L/k})\simeq{\mbox{{Sh}}}(K,K\textquoteright)=G(K,K^{\prime})/D$,
where $D$ is the subgroup generated by $(1,1,...,1)$ (see [BLP 19], Theorem
5.3 and Lemma 3.1). Similarly, it is shown in [Lee 21], Theorem 2.5 that
${\mbox{{Sh}}}_{\omega}^{2}(k,\hat{T}_{L/k})\simeq
G_{\omega}(K,K^{\prime})/D$. Hence it suffices to show that
$G(K,K^{\prime})\simeq C_{\rm arith}(L)$ and that
$G_{\omega}(K,K^{\prime})\simeq C_{\omega}(L)$.

We show that $G(K,K^{\prime})\simeq C_{\rm arith}(L)$; the proof of
$G_{\omega}(K,K^{\prime})\simeq C_{\omega}(L)$ is the same.

Let

$f:\underset{e\in\mathcal{E}}{\oplus}\underset{i\in I_{e}}{\oplus}({\bf
Z}/p^{e}{\bf Z})\to\underset{e\in\mathcal{E}}{\oplus}\ C(I_{e},{\bf
Z}/p^{e}{\bf Z})$

be the map sending $(a_{i})\in\underset{i\in
I_{e}}{\oplus}({\mathbf{Z}}/p^{e}\mathbf{Z})$ to $c:I_{e}\to{\bf Z}/p^{e}{\bf
Z}$ such that $c(i)=a_{i}$. We claim that the isomorphism $f$ gives rise to an
isomorphism

$G(K,K^{\prime})\to C_{\rm arith}(L).$

For $c\in\underset{e\in\mathcal{E}}{\oplus}\ C(I_{e},{\bf Z}/p^{e}{\bf Z})$,
we denote $\pi_{n,m}(c)$ by $c_{m}$.

Let $a=(a_{i})\in G(K,K^{\prime})$ and $c=f(a)$. We show that $c_{m}\in C^{m}$
for $1\leq m\leq n$.

Suppose that $V^{m}_{i}\cap V^{m}_{j}\neq\varnothing$. By Remark 3.7, we have
$e_{i}-e_{i}(n-m)=e_{j}-e_{j}(n-m)=m$. Let $v\in V^{m}_{i}\cap V^{m}_{j}$. As
$a\in G(K,K^{\prime})$, there is $r\in\mathbf{Z}/p^{\hat{e}}\mathbf{Z}$ such
that $v\notin\underset{l\notin I_{r}(a)}{\cup}V_{l}^{\delta(r,a_{l})+1}$. If
$i\notin I_{r}(a)$, then $\delta(r,a_{i})+1>m$ since $v\notin
V_{i}^{\delta(r,a_{i})+1}$. Hence $c(i)=r$ (mod $p^{m}\bf Z$). If $i\in
I_{r}(a)$, then $c(i)=r$ (mod $p^{e_{i}(n)}\bf Z$). In both cases we have
$c_{m}(i)=r$ (mod $p^{m}\bf Z$). The same argument works for $j$. Therefore
$c_{m}(i)=c_{m}(j)$ and $c_{m}\in C^{m}$.

Let $c\in C^{n}$ such that $c_{m}\in C^{m}$ for $1\leq m\leq n$. Let
$a=f^{-1}(c)$. If $c\in D$, then clearly $a\in D$. Suppose that $c\notin D$.
We claim that $\underset{r\in{\bf Z}/p^{\hat{e}}{\bf Z}}{\cap}\
\underset{i\notin I_{r}(a)}{\cup}V_{i}^{\delta(r,a_{i})+1}=\varnothing.$

Suppose not. Let $v\in\underset{r\in{\mathbf{Z}}/p^{\hat{e}}{\bf Z}}{\cap}\
\underset{i\notin I_{r}(a)}{\cup}V_{i}^{\delta(r,a_{i})+1}$. Choose
$r_{0}\in\mathbf{Z}/p^{\hat{e}}\mathbf{Z}$. Since $c\notin D$, there is
$r_{1}\in\mathbf{Z}/p^{\hat{e}}\mathbf{Z}$ and $i\in I_{r_{1}}(a)\setminus
I_{r_{0}}(a)$ such that $v\in V_{i}^{\delta(r_{0},a_{i})+1}$. For the same
reason, there is $r_{2}\in\mathbf{Z}/p^{\hat{e}}\mathbf{Z}$ and $j\in
I_{r_{2}}(a)\setminus I_{r_{1}}(a)$ such that $v\in
V_{j}^{\delta(r_{1},a_{j})+1}$.

By the choice of $i$ and $j$, we have
$\delta(r_{0},a_{i})=\delta(r_{0},r_{1})$ and
$\delta(r_{1},a_{j})=\delta(r_{1},r_{2})$. Suppose that
$\delta(r_{0},r_{1})\geq\delta(r_{1},r_{2})$. Then $v\in V_{i}^{m}\cap
V_{j}^{m}$, where $m=\delta(r_{1},r_{2})+1$. Hence $c_{m}(i)=c_{m}(j)$ and
$\delta(a_{i},a_{j})\geq m=\delta(r_{1},r_{2})+1$, which contradicts that
$\delta(r_{1},r_{2})\geq\delta(a_{i},a_{j})$. Therefore
$\delta(r_{0},r_{1})<\delta(r_{1},r_{2})$.

We can continue the above process to get an infinite sequence of
$r_{l}\in\mathbf{Z}/p^{\hat{e}}\mathbf{Z}$ such that
$\delta(r_{l},r_{l+1})<\delta(r_{l+1},r_{l+2})$. It is a contradiction as
$\delta(r_{l},r_{l+1})$ ranges from $0$ to $\hat{e}$. Hence
$\underset{r\in{\bf Z}/p^{\hat{e}}{\bf Z}}{\cap}\ \underset{i\notin
I_{r}(a)}{\cup}V_{i}^{\delta(r,a_{i})+1}=\varnothing$ and $a\in
G(K,K^{\prime})$. As a consequence $f$ induces an isomorphism
$G(K,K^{\prime})\to C_{arith}(L)$.

######  Corollary 3.11.

Let $k$ be a global field. Then
${\mbox{{Sh}}}_{\omega}^{2}(k,\hat{T}_{L/k})\simeq C(L)/D$.

Proof. By Corollary 2.9 and Remark 2.10, the two sets $C^{m}$ and
$C^{m}_{\omega}$ are the same. Our claim then follows from Theorem 3.10.

Proof of Theorem 3.8. Recall that $\mathcal{K}$ is a Galois extension of $k$
containing $K$ and all the fields $K_{i}$, and that $G={\rm
Gal}(\mathcal{K}/k)$; if $F$ is a subfield of $\mathcal{K}$, we denote by
$G_{F}$ the subgroup of $G$ such that $F=\mathcal{K}^{G_{F}}$.

Note that $k$ is not necessarily a global field here. However there is always
an unramified extension $\ell^{\prime}/\ell$ with Galois group
$\mathrm{Gal}(\ell^{\prime}/\ell)\simeq G$ ([F 62]). Hence we can regard
$\hat{T}_{L/K}$ as a $\mathrm{Gal}(\ell^{\prime}/\ell)$-module.

To be precise, set $F=(\ell^{\prime})^{G_{K}}$,
$L_{i}=(\ell^{\prime})^{G_{K_{i}}}$ and $E=F\times\underset{i\in
I}{\prod}L_{i}$. By construction, the $G$-lattices $\hat{T}_{E/\ell}$ and
$\hat{T}_{L/k}$ are isomorphic.

Since the extension $\ell^{\prime}/\ell$ is unramified, we have
${\mbox{{Sh}}}^{2}(\ell,\hat{T}_{E/\ell})\simeq{\mbox{{Sh}}}_{\omega}^{2}(\ell,\hat{T}_{E/\ell})\simeq{\mbox{{Sh}}}^{2}_{\rm
cycl}(G,\hat{T}_{E/\ell})$. By Corollary 3.11 the group
${\mbox{{Sh}}}^{2}_{\rm cycl}(G,\hat{T}_{E/\ell})$ is isomorphic to $C(E)/D$.
However $C(L)$ only depends on the group $G$. Hence $C(L)\simeq C(E)$.
Therefore ${\mbox{{Sh}}}^{2}_{\rm
cycl}(G,\hat{T}_{L/k})\simeq{\mbox{{Sh}}}^{2}_{\rm
cycl}(G,\hat{T}_{E/\ell})\simeq C(L)/D$.

##  4\. Unramified Brauer groups and generators

We keep the notation of the previous sections. Recall that $p$ is a prime
number, $K/k$ a cyclic field extension of degree $p^{n}$, and $L=K\times
K\textquoteright$, where $K\textquoteright$ is an étale $k$-algebra of finite
rank. In the previous section, we introduced a group $C(L)$ and proved that
${\rm Br}(X_{a}^{c})/{\rm Im}({\rm Br}(k))\simeq C(L)/D$.

The aim of this section is to give more precise information about the
isomorphism $C(L)/D\to{\rm Br}(X_{a}^{c})/{\rm Im}({\rm Br}(k))$.

Let ${\rm Br_{ur}}(k(X_{a}))$ be the subgroup of ${\rm Br}(k(X_{a}))$
consisting of all elements which are unramified at all discrete valuations of
$k(X_{a})$ with residue fields containing $k$ and with fields of fraction
$k(X_{a})$; recall that ${\rm Br_{ur}}(k(X_{a}))$ is isomorphic to ${\rm
Br}(X_{a}^{c})$ (see Cesnavius [C 19], Theorem 1.2).

As in the previous sections, let us write $K\textquoteright=\underset{i\in
I}{\prod}K_{i}$, where the $K_{i}$ are finite separable field extensions of
$k$.

######  Notation 4.1.

We denote by $\mathcal{G}_{k}$ the absolute Galois group of $k$,
$\mathcal{G}_{k(X_{a})}$ the absolute Galois group of $k(X_{a})$. Let $R$ be a
discrete valuation ring of $k(X_{a})$ with residue field $\kappa_{R}$
containing $k$ and with field of fractions $k(X_{a})$. We denote by
$\mathcal{G}_{R}$ the absolute Galois group of $\kappa_{R}$.

######  Notation 4.2.

For all $i\in I$, let $\\{\beta_{ij}\\}$ be a basis of $K_{i}$ over k. Let
$y_{i}=\underset{j}{\sum}\beta_{ij}x_{ij}$, where $x_{ij}$ are variables. Set
$N_{i}=N_{K_{i}\otimes k(X_{a})/k(X_{a})}(y_{i})$ considered as an element of
$k(X_{a})^{\times}$. We define $N=N_{K\otimes k(X_{a})/k(X_{a})}(y)$ in a
similar way. Fix an isomorphism $\chi:{\rm
Gal}(K/k)\to\mathbf{Z}/p^{n}\mathbf{Z}$. Then $\chi$ gives rise to a morphism
$\tilde{\chi}:\mathcal{G}_{k(X_{a})}\to\mathbf{Z}/p^{n}\mathbf{Z}$ and a
morphism $\chi_{R}:\mathcal{G}_{R}\to\mathbf{Z}/p^{n}\mathbf{Z}$. Let
$(N_{i},\tilde{\chi})$ denote the class of the cyclic algebra over $k(X_{a})$
associated to $\tilde{\chi}$ and the element $N_{i}\in k(X_{a})^{\times}$.
([GS 06] Prop. 4.7.3)

The main result of this section is

######  Theorem 4.3.

Assume $char(k)\neq p$. Then the map

$u:C(L)\to{\rm Br}(k(X_{a}))$

given by

$u(c)=\underset{i\in I}{\sum}c(i)(N_{i},\tilde{\chi})$

induces an isomorphism

$C(L)/D\to{\rm Br}(X_{a}^{c})/{\rm Im}({\rm Br}(k)).$

######  Remark 4.4.

Note that $(N_{i},\tilde{\chi})\in\mathrm{Br}(k(X_{a}))$ has order at most
$p^{e_{i}(n)}$, so $c(i)(N_{i},\tilde{\chi})$ is well-defined for
$c(i)\in\mathbf{Z}/p^{e_{i}(n)}\mathbf{Z}$ (ref. [BLP 19] Lemma 6.1).

We start with following lemmas.

######  Lemma 4.5.

The group $u(D)$ is contained in the image of $\mathrm{Br}(k)$ in
$\mathrm{Br}(k(X^{c}))$.

Proof. Since $N\cdot\underset{i\in I}{\prod}N_{i}=c$, we have $\underset{i\in
I}{\sum}(N_{i},\tilde{\chi})=(c/N,\tilde{\chi})=(c,\tilde{\chi})$, which is
the image of $(c,\chi)$ in $\mathrm{Br}(k(X_{a}))$. Hence
$u(D)\subseteq\mathrm{Im}(\mathrm{Br}(k))$.

The following lemma can be found in [Lee 21] §3. Here we use the notation
$C(L)$ to simplify the proof.

######  Lemma 4.6.

  1. (1)

Let $c\in C(L)\setminus D$. Pick $i_{0}\in I_{\hat{e}}$. Let $\hat{c}\in D$ be
the image of the constant map from $I$ to $c(i_{0})$. Set $m$ to be the
maximal integer such that $\pi_{n,m}(c)=\pi_{n,m}(\hat{c})$. Choose
$r\in\mathbf{Z}/p^{\hat{e}}\mathbf{Z}$ such that $\delta(r,c(i_{0}))=m$.
Consider the element
$c^{\prime}\in\underset{e\in\mathcal{E}}{\oplus}C(I_{e},\mathbf{Z}/p^{e}\mathbf{Z})$
defined as follows:

(4.1)
$c^{\prime}(i)=\left\\{\begin{array}[]{ll}\pi_{\hat{e},e_{i}(n)}(r),&\hbox{if
$e_{i}(n)>m$ and $m=\delta(c(i),c(i_{0}))$};\\\
\pi_{\hat{e},e_{i}(n)}(c(i_{0})),&\hbox{otherwise.}\end{array}\right.$

Then $c^{\prime}\in C(L)\setminus D$.

  2. (2)

Suppose that $k$ is a global field and $c\in C_{arith}(L)\setminus D$. Then
the $c^{\prime}$ defined above is in $C_{arith}(L)\setminus D$.

Proof. As $c$ is not in $D$, by the choice of $m$ there is some $i\in I$ such
that $e_{i}(n)>m$ and $\delta(c(i),c(i_{0}))=m$. Hence $c^{\prime}(i)\neq
c^{\prime}(i_{0})\ ({\rm mod}\ p^{e_{i}(n)}\mathbf{Z})$ by our construction
and $c^{\prime}\notin D$.

Now we show that $\pi_{n,l}(c^{\prime})\in C^{l}(L)$ for $0\leq l\leq n$. If
$l\leq m$, then by the choice of $r$ we have
$\pi_{n,l}(c(i_{0}))=\pi_{n,l}(r)$. Clearly $\pi_{n,l}(c^{\prime})\in
C^{l}(L)$.

Suppose $l>m.$ If $\Gamma_{i}^{l}\cap\Gamma_{j}^{l}\neq\varnothing,$ then by
Remark 3.7 $e_{i}(n)$ and $e_{j}(n)$ are at least $l$ and $c(i)=c(j)\ ({\rm
mod}\ p^{l}\mathbf{Z})$. Hence $\delta(c(i_{0}),c(i))=m$ if and only if
$\delta(c(i_{0}),c(j))=m$. By construction $c^{\prime}(i)=c^{\prime}(j)\ ({\rm
mod}\ p^{l}\mathbf{Z})$ and $\pi_{n,l}(c^{\prime})\in C^{l}(L)$.

The proof of statement (2) is similar.

######  Lemma 4.7.

Assume that $char(k)\neq p$. Let $R$ be a discrete valuation ring as in
Notation 4.1. Denote by $\partial_{R}$ the residue map from
$\mathrm{Br}(k(X_{a}))$ to $H^{1}(\kappa_{R},\mathbf{Q}/\mathbf{Z})$. Suppose
that the order of $\partial_{R}(N_{i},\tilde{\chi})$ and the order of
$\partial_{R}(N_{j},\tilde{\chi})$ are both at least $p^{m}$. Then
$\Gamma_{i}^{m}\cap\Gamma_{j}^{m}$ is nonempty.

Proof. Let $\nu_{R}$ be the discrete valuation associated to $R$. Denote the
completion of $k(X_{a})$ with respect to $\nu_{R}$ by $k(X_{a})_{R}$. Choose
an extension $\omega_{R}$ of $\nu_{R}$ to a separable closure of
$k(X_{a})_{R}$. By the construction of $\tilde{\chi}$,
$\partial_{R}(N_{i},\tilde{\chi})=\nu_{R}(N_{i})\chi_{R}$. (See [GS 06] 6.8.4
and 6.8.5.) Write $\nu_{R}(N_{i})$ as $p^{m_{i}}q_{i}$ where $p\nmid q_{i}$.
Let $p^{n_{R}}$ be the order of $\chi_{R}$. As the order of
$\partial_{R}(N_{i},\tilde{\chi})\geq p^{m}$, we have $n_{R}-m_{i}\geq m$.

Since $\nu_{R}(N_{i})=p^{m_{i}}q_{i}$, there is some factor $M_{i}$ of
$K_{i}\otimes_{k}k(X_{a})_{R}$ such that $p^{m_{i}+1}$ does not divide
$\nu_{R}(N_{M_{i}/k(X_{a})_{R}}(y_{M_{i}}))$, where $y_{M_{i}}$ is the
projection of $y_{i}$ in $M_{i}$. Let $\omega_{i,R}$ be the restriction of
$\omega_{R}$ to $M_{i}$. Write the inert degree of $w_{i,R}$ over $\nu_{R}$ as
$p^{f_{i}}q^{\prime}_{i}$ where $p\nmid q^{\prime}_{i}$. As $p^{m_{i}+1}$ does
not divide $\nu_{R}(N_{M_{i}/k(X_{a})_{R}}(y_{M_{i}}))$, we have $f_{i}\leq
m_{i}$.

Choose a factor $M$ of $K\otimes_{k}k(X_{a})_{R}$ and let $\overline{M}$ be
its residue field. Let $\overline{M}_{i}$ be the residue field of $w_{i,R}$.
Both fields are considered as subfields of a separable closure
$\kappa_{R}^{s}$ of $\kappa_{R}$.

As $f_{i}\leq m_{i}$ and $n_{R}-m_{i}\geq m$, the cyclic extension
$\overline{M}\overline{M}_{i}/\overline{M_{i}}$ is of degree at least $p^{m}$.
Choose $g_{R}\in\mathcal{G}_{R}$ such that $\chi_{R}(g_{R})$ generates the
image of $\chi_{R}$ in $\mathbf{Q}/\mathbf{Z}$. Let $\mathcal{H}_{i}$ be the
subgroup of $\mathcal{G}_{R}$ which fixes $\overline{M}_{i}$. We claim that
there are some $h_{i}\in\mathcal{G}_{R}$ and some $\sigma_{R}\in\langle
h_{i}^{-1}g_{R}h_{i}\rangle\cap\mathcal{H}_{i}$ such that
$\chi_{R}(\sigma_{R})$ is of order at least $p^{m}$.

Consider the group action of $\langle g_{R}\rangle$ on the set of left cosets
of $\mathcal{H}_{i}$ in $\mathcal{G}_{R}$. As
$\lvert\mathcal{G}_{R}/\mathcal{H}_{i}\lvert=p^{f_{i}}q^{\prime}_{i}$ with
$p\nmid q_{i}^{\prime}$ , there is some $h_{i}\in\mathcal{G}_{R}$ such that
$p^{f_{i}+1}$ does not divide the order of the orbit of
$h_{i}\mathcal{H}_{i}$. Hence the stabilizer of $h_{i}\mathcal{H}_{i}$ is
$\langle g_{R}^{p^{f_{i}^{\prime}}r_{i}}\rangle$ for some $f_{i}^{\prime}\leq
f_{i}$ and some $r_{i}$ coprime to $p$. Let
$\sigma_{R}=h_{i}^{-1}g_{R}^{p^{f_{i}^{\prime}}r_{i}}h_{i}$. Then
$\chi_{R}(\sigma_{R})=\chi_{R}(g_{R}^{p^{f_{i}^{\prime}}r_{i}})$, which is of
order $p^{n_{R}-f^{\prime}_{i}}$. Since $f^{\prime}_{i}\leq f_{i}\leq m_{i}$
and $n_{R}-m_{i}\geq m$, the order of $\chi_{R}(\sigma_{R})$ is at least
$p^{m}$.

Let $g$ and $\sigma$ be the image of $g_{R}$ and $\sigma_{R}$ in $G$. Then
$\sigma$ fixes $K_{i}$ and $\sigma$ is an element of order at least $p^{m}$ in
$\mathrm{Gal}(K/k)$. Hence the conjuacy class of $g$ belongs to
$\Gamma_{i}^{m}$. The same argument proves that the conjuacy class of $g$
belongs to $\Gamma_{j}^{m}$. Hence $\Gamma^{m}_{i}\cap\Gamma^{m}_{j}$ is
nonempty.

Next we prove that for all $c\in C(L)$, the element $\underset{i\in
I}{\sum}c(i)(N_{i},\tilde{\chi})$ is unramified.

######  Proposition 4.8.

Suppose that $char(k)\neq p$. The image of $u$ is an unramified subgroup of
$\mathrm{Br}(k(X_{a}))$.

Proof. By Lemma 4.5 we can assume that $c(i)=0$ for some $i\in I_{\hat{e}}$.

Let $R$ be a discrete valuation ring of $k$ with residue field $\kappa_{R}$
containing $k$ and with field of fractions $k(X_{a})$. Let $\nu_{R}$ be the
discrete valuation associated to $R$. Denote by $\partial_{R}$ the residue map
from $\mathrm{Br}(k(X_{a}))$ to $H^{1}(\kappa,\mathbf{Q}/\mathbf{Z})$. We
claim that $u(c)=\underset{i\in I}{\sum}c(i)(N_{i},\tilde{\chi})$ is
unramified at $R$.

Let $J(c)=\\{i\in I\mid c(i)\neq 0\ {\rm in}\
\mathbf{Z}/p^{e_{i}(n)}\mathbf{Z})\\}$. Let $m$ be the maximum integer such
that $\pi_{n,m}(c)=0$ and set $J_{m}(c)=\\{i\in J(c)\mid m=\delta(0,c(i))\\}$.
We prove by induction on $\lvert J(c)\rvert$. If $|J(c)|=0$, then $c=0$ and
our claim is trivial. Suppose that our claim is true for $\lvert
J(c)\rvert\leq h$.

Let $|J(c)|=h+1$. Then $c\notin D$ and $J_{m}(c)$ is nonempty. Pick $j\in
J_{m}(c)$ and choose $r\in\mathbf{Z}/p^{\hat{e}}\mathbf{Z}$ such that $c(j)=r$
$({\rm mod}\ p^{e_{j}(n)}\mathbf{Z})$. Let $c^{\prime}$ be defined as in Lemma
4.6. We first prove the $u(c^{\prime})$ is unramified at $R$, i.e.
$\partial_{R}(u(c^{\prime}))=0.$

Since $c(i)=0$, by the definition of $c^{\prime}$ we have
$u(c^{\prime})=\underset{s\in J_{m}(c)}{\sum}r(N_{s},\tilde{\chi})$. Hence
$\partial_{R}(u(c^{\prime}))=\underset{s\in
J_{m}(c)}{\sum}r\cdot\nu_{R}(N_{s})\chi_{R}$. Suppose that
$\partial_{R}(u(c^{\prime}))$ is not zero. Then there is some $s\in J_{m}(c)$
such that $r\cdot\nu_{R}(N_{s})\chi_{R}\neq 0$. As $\delta(0,r)=m$, the order
of $\nu_{R}(N_{s})\chi_{R}$ is at least $p^{m+1}$.

By Lemma 4.5, there is some $t\in I\setminus J_{m}(c)$ such that
$r\cdot\nu_{R}(N_{t})\chi_{R}(g)\neq 0$ and the order of
$\nu_{R}(N_{t})\chi_{R}$ is at least $p^{m+1}$. By Lemma 4.7 the set
$\Gamma_{s}^{m+1}\cap\Gamma_{t}^{m+1}$ is nonempty. As $\delta(0,r)=m$, we
have $c^{\prime}(s)\neq c^{\prime}(t)$ $({\rm mod}\ p^{m+1}\mathbf{Z})$. This
contradicts that $c^{\prime}\in C(L)$. Therefore
$\partial_{R}(u(c^{\prime}))=0$ and $u(c^{\prime})$ is unramified.

Consider the element $c-c^{\prime}\in C(L)$. By our construction of
$c^{\prime}$, the cardinality of $J(c-c^{\prime})$ decreases by at least one.
By induction hypothesis $u(c-c^{\prime})$ is unramified. Hence $u(c)$ is
unramified.

Proof of Theorem 4.3. By Lemma 4.5 and Proposition 4.8, the map

$u:C(L)/D\to\mathrm{Br}(X^{c}_{a})/\mathrm{Im}(\mathrm{Br}(k))$

is well-defined.

A similar argument as in [BP 20] Thm. 12.2 proves the injectivity of $u$.
Consider the projection from $X_{a}$ to the affine space $\mathbb{A}^{d}$,
where $d=\underset{i\in I}{\sum}[K_{i};k]$ and the coordinates are given by
$x_{ij}$ defined in Notation 4.2.

Let $M$ be the function field of $\mathbb{A}^{d}$. Denote by $\chi_{M}$ the
image of $\chi$ in $H^{1}(M,\mathbf{Q}/\mathbf{Z})$. Suppose that
$u(c)=\alpha\in\mathrm{Im}(\mathrm{Br}(k))$. Then $u(c)-\alpha$ is in the
kernel of $\mathrm{Br}(M)\to\mathrm{Br}(k(X_{a}))$,which is generated by
$(a\underset{i\in I}{\prod}N_{i}^{-1},\chi)$ (see [BP 20] Lemma 12.3).
Therefore $u(c)-\alpha=r(a\underset{i\in I}{\prod}N_{i}^{-1},\chi).$

Consider the discrete valuation $v_{N_{i}}$ on $M$ and let $\kappa_{N_{i}}$ be
its residue field. Denote by $\chi_{N_{i}}$ the image of $\chi$ in
$H^{1}(\kappa_{N_{i}},\mathbf{Q}/\mathbf{Z})$. We claim that $\chi_{N_{i}}$ is
of order $p^{e_{i}(n)}$. Let $M_{i}$ be the function field of the subvariety
of $\mathbb{A}^{[K_{i}:k]}$ defined by $N_{i}$. Then
$\kappa_{N_{i}}=M_{i}(x_{jl})$ where $x_{jl}$ are defined as in Notation 4.2
with $j\neq i$. Hence $\kappa_{N_{i}}$ is purely transcendental over $M_{i}$.
Let $\chi_{M_{i}}$ be the image of $\chi$ in
$H^{1}(M_{i},\mathbf{Q}/\mathbf{Z})$. It suffices to prove the order of
$\chi_{M_{i}}$ is $p^{e_{i}(n)}$. Note that
$K\otimes_{k}M_{i}\otimes_{k}K_{i}\simeq K\otimes_{k}(\prod
K_{i}(x))\simeq\prod E_{i}(n)(x)$, where $x$ is a variable (see Notation 2.4
for $E_{i}(n)$). On the other hand
$K\otimes_{k}M_{i}\otimes_{k}K_{i}\simeq(\prod\tilde{M}_{i})\otimes_{k}K_{i}$,
where $\tilde{M}_{i}$ is a factor of $K\otimes_{k}M_{i}$. If
$[\tilde{M}_{i}:M_{i}]<p^{e_{i}(n)}$, then
$\tilde{M}_{i}\otimes_{k}K_{i}=\tilde{M}_{i}\otimes_{M_{i}}M_{i}\otimes_{k}K_{i}$
is a product of extensions of $K_{i}(x)$ of degree less then $p^{e_{i}(n)}$,
which is a contradiction. Hence $[\tilde{M}_{i}:M_{i}]=p^{e_{i}(n)}$ and
$\chi_{M_{i}}$ is of order $p^{e_{i}(n)}$.

As $\chi_{N_{i}}$ is of order $p^{e_{i}(n)}$, after taking residue of
$u(c)-\alpha$ at $v_{N_{i}}$, we see that $c(i)=-r$ (mod
$p^{e_{i}(n)}\mathbf{Z}$). Hence $c\in D$ and $u$ is injective.

Since $u$ is injective, $\lvert
C(L)/D\rvert\leq\lvert\mathrm{Br}(X_{a}^{c})/\mathrm{Im}(\mathrm{Br}(k))\rvert$.
By Theorem 3.8 and Theorem 1.1, the order of $C(L)/D$ is equal to the order of
$\mathrm{Br}(T^{c}_{L/k})/\mathrm{Br}(k)$. By Theorem 1.2, the map $u$ is
surjective and hence is an isomorphism.

We now prove the results announced in the introduction :

######  Proposition 4.9.

Let $k$ be a field of $char(k)\neq p$. Let $F$ be a bicyclic extension of $k$
with Galois group
$\mathbf{Z}/p^{n}\mathbf{Z}\times\mathbf{Z}/p^{n}\mathbf{Z}$. Let $K$ and
$K_{i}$ be linearly disjoint cyclic subfields of $F$ with degree $p^{n}$ for
$i=1,...,m$. Then

$\mathrm{Br}(X_{a}^{c})/\mathrm{Im}(\mathrm{Br}(k))\simeq(\mathbf{Z}/p^{n}\mathbf{Z})^{m-1},$

and is generated by $(N_{i},\chi)$ for $i=1...m-1$.

Proof. There is a number field $\ell$ and an unramified Galois extension
$\ell^{\prime}/\ell$ such that
$\mathrm{Gal}(\ell^{\prime}/\ell)\simeq\mathrm{Gal}(F/k)$ ([F 62]). Set
$F=(\ell^{\prime})^{G_{K}}$, $L_{i}=(\ell^{\prime})^{G_{K_{i}}}$ and
$E=F\times\underset{i\in I}{\prod}L_{i}$. By construction, the $G$-lattices
$\hat{T}_{E/\ell}$ and $\hat{T}_{L/k}$ are isomorphic.

By [Lee 21] Proposition 7.3, the group
${\mbox{{Sh}}}^{2}_{\omega}(\ell,\hat{T}_{E/\ell})\simeq(\mathbf{Z}/p^{n}\mathbf{Z})^{m-1}$.
Since
${\mbox{{Sh}}}^{2}_{\omega}(\ell,\hat{T}_{E/\ell})\simeq{\mbox{{Sh}}}^{2}_{\rm
cycl}(G,\hat{T}_{E/\ell})\simeq{\mbox{{Sh}}}^{2}_{\rm cycl}(G,\hat{T}_{L/k})$,
we have $C(L)/D\simeq(\mathbf{Z}/p^{n}\mathbf{Z})^{m-1}$ by Theorem 3.8. The
assertion then follows from Theorem 4.3.

##  5\. Global fields

We keep the notation of the previous section, and in addition we assume that
$k$ is a global field. Denote by ${\mbox{{B}}}(X_{a}^{c})$ the subgroup of
$\mathrm{Br}(X_{a}^{c})/\mathrm{Im}(\mathrm{Br}(k))$ consisting of locally
trivial elements; and by ${\mbox{{B}}}_{\omega}(X_{a}^{c})$ the subgroup
consisting of elements which are trivial at almost all places of $k$.

######  Theorem 5.1.

Suppose that $k$ is a global field with $char(k)\neq p$. Then

  1. (1)

$u$ induces an isomorphism between $C(L)/D$ and
${\mbox{{B}}}_{\omega}(X_{a}^{c})$.

  2. (2)

$u$ induces an isomorphism between $C_{arith}(L)/D$ and
${\mbox{{B}}}(X_{a}^{c})$.

Proof. First note that [Sa 81] 6.1.4 remains true over global fields. Hence
${\mbox{{B}}}_{\omega}(X_{a}^{c})\simeq{\mbox{{B}}}_{\omega}(X_{a})$ and
${\mbox{{B}}}(X_{a}^{c})\simeq{\mbox{{B}}}(X_{a})$.

By [Sa 81] 6.8 and 6.9 (ii), we have
${\mbox{{B}}}(X_{a})\simeq{\mbox{{Sh}}}^{2}(k,\hat{T}_{L/k})$
(resp.${\mbox{{B}}}_{\omega}(X_{a})\simeq{\mbox{{Sh}}}_{\omega}^{2}(k,\hat{T}_{L/k})$).
By Theorem 3.10 and Corollary 3.11, we conclude that
$C(L)/D\simeq{\mbox{{B}}}_{\omega}(X_{a}^{c})$ and
$C_{arith}(L)/D\simeq{\mbox{{B}}}(X_{a}^{c})$.

As ${\mbox{{B}}}_{\omega}(X_{a}^{c})$ is a subgroup of
$\mathrm{Br}(X_{a}^{c})/\mathrm{Im}(\mathrm{Br}(k))$ with the same
cardinality, the first statement follows from Theorem 4.3.

To see that $u$ gives rise to the desired isomorphism in (2), it is sufficient
to show that $u(c)_{v}\in\mathrm{Br}(X_{a}^{c}\times_{k}k_{v})$ is in the
image of $\mathrm{Br}(k_{v})$ for $c\in C_{arith}(L)$ and for $v\in V_{k}$.

By Lemma 4.5 we can assume that $c(i_{0})=0$ for some $i_{0}\in I_{\hat{e}}$.
Let $J(c)=\\{i\in I\mid c(i)\neq 0\ {\rm in}\
\mathbf{Z}/p^{e_{i}(n)}\mathbf{Z})\\}$. Set $m$ be the maximum integer such
that $\pi_{n,m}(c)=0$ and set $J_{m}(c)=\\{i\in J(c)\mid m=\delta(0,c(i))\\}$.
We prove by induction on $\lvert J(c)\rvert$. If $|J(c)|=0$, then $c=0$ and
our claim is trivial. Suppose that our claim is true for $\lvert
J(c)\rvert\leq h$.

Let $|J(c)|=h+1$. Then $c\notin D$ and $J_{m}(c)$ is nonempty. Pick $j\in
J_{m}(c)$ and choose $r\in\mathbf{Z}/p^{\hat{e}}\mathbf{Z}$ such that $c(j)=r$
$({\rm mod}\ p^{e_{j}(n)}\mathbf{Z})$. Let $c^{\prime}$ be defined as in Lemma
4.6.

For $v\in V_{k}$, let $\chi_{v}$ be the image of $\chi$ in
$H^{1}(k_{v},\mathbf{Q}/\mathbf{Z})$, and $\chi_{i,w}$ be the image of $\chi$
in $H^{1}((K_{i})_{w},\mathbf{Q}/\mathbf{Z})$ where $w$ is a place of $K_{i}$.

By the definition of $C_{arith}(L)$, the set $V^{m+1}_{i}\cap V^{m+1}_{j}$ is
empty for any $i\in J_{m}(c)$ and for any $j\notin J_{m}(c)$.

Let $v\in V_{k}$. Suppose that $v\notin\underset{i\in
J_{m}(c)}{\cup}V^{m+1}_{i}$. Then for all $i\in J_{m}(c)$, $\chi_{i,w}$ is of
order at most $m$ for all $w\mid v$. By the projection formula,
$(N_{i},\chi)_{v}$ has order at most $p^{m}$. Hence
$c^{\prime}(i)(N_{i},\chi)_{v}=0$ and $u(c^{\prime})_{v}=0$ in this case.

Suppose that $v\in V^{m+1}_{i}$ for some $i\in J_{m}(c)$. Let $d\in D$ be the
image of the constant map from $I$ to $r$. Set $\overline{c}=d-c^{\prime}$. As
the set $V^{m+1}_{i}\cap V^{m+1}_{j}$ is empty for any $j\notin J_{m}(c)$,
$v\notin\underset{j\notin J_{m}(c)}{\cup}V^{m+1}_{j}$. The same argument shows
that $u(\overline{c})_{v}=0$. Hence $u(c^{\prime})_{v}$ is in the image of
$\mathrm{Br}(k_{v})$.

Since the cardinality of $J(c-c^{\prime})$ decreases by at least one, by
induction hypothesis $u(c-c^{\prime})\in{\mbox{{B}}}(X_{a}^{c})$. In
combination with $u(c^{\prime})\in{\mbox{{B}}}(X_{a}^{c})$, we see that $u(c)$
is in ${\mbox{{B}}}(X_{a}^{c})$.

######  Example 5.2.

Let $k=\mathbf{Q}(i)$. Let $K=k(\sqrt[4]{17})$, $K_{1}=k(\sqrt[4]{17\times
13})$ and $K_{2}=k(\sqrt[4]{13})$. By [Lee 21] Example 7.4,
${\mbox{{Sh}}}^{2}_{\omega}(k,\hat{T}_{L/k})\simeq\mathbf{Z}/4\mathbf{Z}$ and
${\mbox{{Sh}}}^{2}(k,\hat{T}_{L/k})\simeq\mathbf{Z}/2\mathbf{Z}$. By Theorem
4.3 and Theorem 5.1, the element $(N_{2},\chi)$ generates the group
$\mathrm{Br}(X_{a}^{c})/\mathrm{Im}(\mathrm{Br}(k))$ and $2(N_{2},\chi)$
generates ${\mbox{{B}}}(X_{a}^{c})$.

More generally we have the following.

######  Proposition 5.3.

Let $k$ be a global field of $char(k)\neq p$. Let $F$ be a bicyclic extension
of $k$ with Galois group
$\mathbf{Z}/p^{n}\mathbf{Z}\times\mathbf{Z}/p^{n}\mathbf{Z}$. Let $K$ and
$K_{i}$ be linearly disjoint cyclic subfields of $F$ with degree $p^{n}$ for
$i=1,...,m$. Assume moreover that $F\otimes_{k}k_{v}$ is a product of cyclic
extensions for all $v\in V_{k}$. Then $(N_{i},\chi)$ generates
${\mbox{{B}}}(X_{a}^{c})$.

Proof. By [Lee 21] Prop. 7.3, we have
${\mbox{{Sh}}}^{2}(k,\hat{T}_{L/k})\simeq(\mathbf{Z}/p^{n}\mathbf{Z})^{m-1}$.
We apply Theorem 3.10 (1) and Theorem 5.1 (2) to conclude.

##  6\. An application to Hasse principles

In this section we apply Theorem 4.3 to give an alternative proof of [BLP 19]
Theorem 7.1 for $k$ a global field with $char(k)\neq p$. Moreover we can
assume that $K/k$ is a cyclic extension of $p$-power degree. (See §2 and [BLP
19] 6.3.)

We use the notation of the previous sections. In particular, $X_{a}$ is the
affine variety defined in the introduction, $K/k$ is a cyclic extension of
$p$-power degree, and $K_{i}/k$ is a finite separable extension for all $i\in
I$. Recall that $\chi$ an injective morphism from ${\rm Gal}(K/k)$ to
$\mathbf{Q}/\mathbf{Z}$.

Let $\chi_{v}$ be the image of $\chi$ in $H^{1}(k_{v},\mathbf{Q}/\mathbf{Z})$.
Let $\mathrm{inv}$ be the Hasse invariant map ${\rm
inv}:\mathrm{Br}(k_{v})\to\mathbf{Q}/\mathbf{Z}$.

Denote by $K_{i}^{v}$ the algebra $K_{i}\otimes_{k}k_{v}$. Suppose that there
is a local point $(x_{i}^{v})\in\underset{v\in V_{k}}{\prod}X_{a}(k_{v})$,
where $x_{i}^{v}\in K_{i}^{v}$ for $i\in I$ and $x_{0}^{v}\in
K\otimes_{k}k_{v}$. Define $\alpha_{a}:C_{arith}(L)/D\to\mathbf{Q}/\mathbf{Z}$
as

$\alpha_{a}(c)=\underset{v\in V_{k}}{\sum}\underset{i\in I}{\sum}c(i){\rm
inv}(N_{K_{i}^{v}/k_{v}}(x_{i}^{v}),\chi_{v})$

######  Theorem 6.1.

Suppose that there is a local point $(x_{i}^{v})\in\underset{v\in
V_{k}}{\prod}X_{a}(k_{v})$. Then the map $\alpha_{a}$ is the Brauer-Manin
pairing and $X_{a}$ has a $k$-point if and only if $\alpha_{a}=0$.

Proof. First we consider the case where $k$ is a number field. By Theorem 5.1
and [Sa 81] Lemma 6.2, the map $\alpha_{a}$ is the Brauer-Manin pairing of
$X_{a}^{c}$. Our claim then follows from Sansuc’s result [Sa 81] Cor. 8.7.

For $k$ a global function field, we apply Theorem 5.1 and [DH 17] Theorem 2.5
to conclude.

## References

  * [BLP 19] E. Bayer-Fluckiger, T-Y. Lee, R. Parimala,  Hasse principles for multinorm equations, Adv. Math. 356 (2019).
  * [BP 20] E. Bayer-Fluckiger, R. Parimala,  On unramified Brauer groups of torsors over tori, Doc. Math. 25 (2020), 1263-1284.
  * [C 19] K. Cesnavicius, Purity for the Brauer group, Duke Math. J. 168 (2019), 1461-1486.
  * [CTHSk 03] J-L. Colliot-Thélène, D. Harari, A. Skorobogatov, Valeurs d’un polynôme à une variable représentées par une norme, Number theory and algebraic geometry, London Math. Soc. Lecture Note Ser. 303, Cambridge Univ. Press, Cambridge (2003), 69-89.
  * [CTHSk 05] J-L. Colliot-Thélène, D. Harari, A. Skorobogatov, Compactification équivariante d’un tore (d’après Brylinski et Künnemann), Expo. Math 23 (2005), 161-170.
  * [CTSk 19] J-L. Colliot-Thélène, A. Skorobogatov, The Brauer-Grothendieck group, Springer International Publishing (2021).
  * [CTS 77] J-L. Colliot-Thélène, J-J. Sansuc, La R-équivalence sur les tores, Ann. Sc ENS 10 (1977), 175-229.
  * [CTS 87] J-L. Colliot-Thélène, J-J. Sansuc, Principal homogeneous spaces under flasque tori : applications, J. Algebra 106 (1987), 148-205.
  * [DH 17] C. Demarche and D. Harari, Local-global principles for homogeneous spaces of reductive groups over global fields, preprint, https://arxiv.org/abs/2107.08906.
  * [F 62] A. Fröhlich, On non-ramified extensions with prescribed Galois group, Mathematika 9 (1962), 133-134.
  * [GS 06] P. Gille, T. Szamuely, Central simple algebras and Galois cohomology, Cambridge University Press (2006).
  * [Lee 21] T-Y. Lee, The Tate-Shafarevich groups of multinorm-one tori, to appear in J. Pure Appl. Algebra.
  * [Sa 81] J.-J. Sansuc, Groupe de Brauer et arithmétique des groupes algébriques linéaires sur un corps de nombres, J. Reine Angew. Math. 327 (1981) 12–80.
  * [Se 79] J-P. Serre, Local fields, Graduate Texts in Mathematics 67 Springer-Verlag, New York-Berlin (1979).

Eva Bayer–Fluckiger

EPFL-FSB-MATH

Station 8

1015 Lausanne, Switzerland

<EMAIL_ADDRESS>

Ting-Yu Lee

NTU-MATH

Astronomy Mathematics Building 5F,

No. 1, Sec. 4, Roosevelt Rd.,

Taipei 10617, Taiwan (R.O.C.)

<EMAIL_ADDRESS>


# How to Learn and Represent Abstractions:
An Investigation using Symbolic Alchemy

Badr AlKhamissi
Sony AI
&Akshay Srinivasan
Sony AI
&Zeb-Kurth Nelson
DeepMind
&Sam Ritter
DeepMind

###### Abstract

Alchemy is a new meta-learning environment rich enough to contain interesting
abstractions, yet simple enough to make fine-grained analysis tractable.
Further, Alchemy provides an optional symbolic interface that enables meta-RL
research without a large compute budget. In this work, we take the first steps
toward using Symbolic Alchemy to identify design choices that enable deep-RL
agents to learn various types of abstraction. Then, using a variety of
behavioral and introspective analyses we investigate how our trained agents
use and represent abstract task variables, and find intriguing connections to
the neuroscience of abstraction. We conclude by discussing the next steps for
using meta-RL and Alchemy to better understand the representation of abstract
variables in the brain.

##  1 Introduction

Humans display the remarkable ability to use abstractions to guide their
behavior. They are able to abstract over the sensorimotor details of a
situation to derive the general principles involved, and to use those
principles to behave effectively. A classic example is the use of abstract
knowledge to guide behavior in a restaurant; for example, people know that
they must pay at the end, even though they have never seen that exact
restaurant or paid for that specific meal (Wallis et al.,, 2001).
Neuroscientific studies have made progress in understanding how the brain
might represent abstract rules (Mansouri et al.,, 2020), establishing that
prefrontal cortex plays a significant role (Milner,, 1963), with more recent
work implicating the hippocampus as well (Samborska et al.,, 2021). One
limitation of these studies is that they use extremely simple tasks, where
optimal behavior can be derived as a nearly trivial function of the inputs. As
a result, there is a large gap between the abstractions studied in the
experiments and the richness at play in human life.

A recent development presents an opportunity to begin to bridge this gap. The
Alchemy benchmark was proposed by Wang et al., (2021) to make fine-grained
analysis and interpretation of meta-RL agents possible while maintaining the
complexity needed for more interesting conceptual abstractions to be learnt.
In the present work, we use a biologically inspired model on the symbolic
version of the Alchemy benchmark (Wang et al.,, 2021) to investigate the
learning and representation of more complex abstractions than those studied
previously in neuroscience. Our model builds on the meta-RL framework of Wang
et al., (2018) who showed that by considering the prefrontal cortex (PFC) as
its own meta-RL system, that is driven by dopamine-based synaptic learning,
one can account for a wide range of behavioral and neurophysiological
findings. Our model builds on that by extending their core recurrent network
with an episodic memory via a modified transformer block to represent the
hippocampus, similarly to (Ritter et al.,, 2018).

The main contributions of the paper can be summarized as follows: (1) we show
a way in which researchers can achieve high performance in Symbolic Alchemy –
albeit with some tricks – without having access to vast computational
resources as is usually required for deep-RL research. To be specific, all
experiments done for this paper were run on a single-GPU (Tesla T4) machine.
(2) We propose a hypothesis based on empirical results into why previous
agents failed to solve Symbolic Alchemy despite being much more powerful than
the one used in this work. (3) We release a tool for visualizing the chemistry
and latent space of any given episode in Symbolic Alchemy to better help
researchers debug the behavior of their agents. (4) We present a new kind of
behavioral analysis that can be done on Alchemy to test whether the agent
succeeded or failed in acquiring specific pieces of abstract knowledge. (5)
Finally, we demonstrate that, just as in animals (Wallis et al.,, 2001; Wallis
and Miller,, 2003; Muhammad et al.,, 2006), single-units of the LSTM and
transformer encode abstract task variables. Moreover, single-unit analyses
revealed evidence for distinct functional roles for LSTM and transformers
units. We draw a connection between this observation and the differential
roles of PFC and hippocampus observed in recent neuroscience experiments
(Samborska et al.,, 2021).

##  2 Methods

###  2.1 Symbolic Alchemy

The Alchemy benchmark was proposed by Wang et al., (2021) to make fine-grained
analysis and interpretation of meta-RL agents possible while maintaining the
complexity needed for interesting conceptual abstractions to be learnt. Unlike
other meta-RL task distributions, Alchemy’s accessibility allows us to compare
our model’s performance against a Bayesian learner referred to as an ‘Ideal
Observer’. Wang et al., (2021) also develop a ‘Random Heuristic’ that we use
as a reference for evaluating our agent’s understanding of specific abstract
principles. The task itself is divided into episodes, each of which consists
of $10$ trials. In our experiments, the number of timesteps per trial was set
to $15$ to speedup training.

#### Stones and Potions

The goal of the agent within each trial is to transform three stones into a
more valuable form – the value of which is tied to the stone’s perceptual
features – by applying a sequence of different potions on each of them. The
agent can then collect the reward associated with a stone by dropping it into
a central cauldron. The stone’s appearance can change along only one of three
dimensions at a time: size, color and shape. Each potion on the other hand is
characterized by one of $6$ hues that dictates the transformative effect it
has on a specific stone according to some ‘chemistry’ that is sampled from a
structured generative process at the beginning of each episode. There are $12$
potions in each trial, each of which is consumed (i.e. can not be re-used)
once applied on a stone.

#### Chemistry

The chemistry dictates the causal structure that governs each episode as well
as the possible set of stone perceptual features that can occur. It can be
visualized as a cube, where each vertex correspond to a specific stone value
in the latent space, which can be one of (-3, -1, +1, +15), or a specific
appearance in the perceptual space. The potion effects run along the edges of
the cube as shown in Figure 1. Edges can be missing, creating a bottleneck
that the agent must pass through to reach a high rewarding state. This may
require passing through intermediate lower value states first.

#### Abstract Principles

There are certain rules and constraints that span across episodes which makes
Alchemy a meta-learning benchmark. The agent is expected to learn to identify
and exploit those regularities in order to achieve high performance in the
task. This includes:

  * •

Consistency: Potions of the same color will always have the same effect on
stones with the same visual features within an episode.

  * •

Parallelism: Each potion color has the same direction of effect regardless of
the other features (e.g. a red potion will always change the color of a stone
from blue to purple irrespective of the size and shape of that stone).

  * •

Missing Edges: Overlaid on that parallelism, some edges can be disabled for an
episode. Therefore once discovered the agent shouldn’t attempt to traverse
that missing edge.

  * •

Potion Pairs: Potions come in pairs with opposite effects (red/green,
yellow/orange, pink/turquoise). The agent should know those pairs since they
are consistent across all episodes. For example, if the effect of the orange
potion is to increase the size of a stone, then the effect of the yellow
potion will be to decrease the size of the stone.

#### State and Action Spaces

In the input representation, each stone is represented by its three perceptual
features, its reward in the current latent state and whether it has been
deposited into the cauldron or not. For the potions, we use a modified
representation from the one proposed by Wang et al., (2021); instead of
representing the state of each of the $12$ potions as elements of a single
vector, we represent the remaining number of potions per hue. Therefore the
state space is comprised of a $21$-dimensional vector. The action space is
also modified in a similar manner, where the agent chooses which color to
apply to which stone, or whether to deposit a specific stone to the cauldron.
When a specific stone-color combination is chosen, a wrapper then randomly
selects one of the available potions with that color and apply it to the
selected stone. Alternatively a no-op action can be chosen, which makes the
number of possible actions $22$.

#### Rewards and Penalties

In addition to the rewards the agent receives whenever depositing a stone into
the cauldron, we found that penalizing the agent in three specific scenarios
sped up the rate of convergence considerably. Specifically, we gave a reward
of $-0.2$ if the action taken results in a null transition (i.e. does not have
any effect on the stone) but given that it is not a no-op action. In the
second case, the agent was penalized with a reward of $-1$ whenever it chose
to use a potion hue that is not available (i.e. it should learn to never use a
hue when its corresponding entry is zero in the input representation) or a
stone that has already been cached in the cauldron. Finally, an additional
penalty of $-1$ was given when the agent chose the same potion color
consecutively on the same stone.

###  2.2 Visualizing the Latent Space

In order to simplify the process of qualitatively debugging the agent’s
behavior, we developed and are
releasing111https://github.com/bkhmsi/symbolic_alchemy_visualizer a tool for
Symbolic Alchemy that visualizes the topology of the underlying causal graph
for a given episode. Overlaid on it are the positions of the stones in the
latent space along with arrows indicating the direction of effect of the
available potions. The Ideal Observer’s belief about a particular potion color
or edge is also indicated using the opacity of the corresponding object. That
way we can evaluate the agent’s actions with respect to the belief state of an
agent that has perfect understanding of the structure of the task.

Figure 1: Visualizations of chemistries created using the Symbolic Alchemy
Visualization Tool, which we are releasing for public use. The coordinates on
the cube’s vertices indicate the latent state and thus reward of the stone in
that position. Translucent arrows indicate that the agent has not yet
discovered the effect of that color. The stones are represented by a red, blue
and green spheres. The available potions are shown floating above the cube
with their corresponding color, and when a potion is consumed it will no
longer be visible. In some chemistries, edges of the graph may be missing as
can be seen in the 2nd and 3rd snapshots.

###  2.3 Agent Architecture

Figure 2: In the center of the architecture is an LSTM that takes as input the
action and reward of the previous timestep, an encoded version of the current
state, and a distilled representation of the relevant memory entries. The
encoder is just a two-layered MLP. The episodic memory stores the features of
the stone being transformed $\widetilde{s}_{t}$, a one-hot encoding of the
applied potion color $p_{t}$, the reward $r_{t}$, and the resultant features
of the stone after transformation $\widetilde{s}_{t+1}$. The transformer block
architecture is fully described in Section 2.3. The output of the LSTM is
finally given to the policy and value networks, each of which is a linear
layer, to generate an action and the value estimate respectively.

We use a biologically inspired architecture that maps to functions and neural
structures in the brain. Specifically, we build on the work of Wang et al.,
(2018) in which the prefrontal cortex (PFC) is conceptualized as forming a
gated recurrent neural network (characterized as an LSTM (Hochreiter and
Schmidhuber,, 1997)) and augment it with an episodic memory which is connected
to the LSTM via a single modified transformer block from Ritter et al., (2020)
(see Figure 2).

The LSTM, which has been shown to have analogies with prefrontal gating
theories (Chatham and Badre,, 2015), takes as input on each time-step an
encoded version of the current state, the reward and action it had taken at
the preceding time-step and a compressed representation of the previous
memories experienced in the current episode. Since the LSTM learns to distill
that information over the course of an episode in its hidden state, it can be
considered as a form of working-memory (Lara and Wallis,, 2015).

On the other hand, the transformer architecture takes as input the current
state $s_{t}$ and the entries of the episodic memory $\\{m_{i}\\}_{i=0}^{t}$
on each time-step. The state $s_{t}$ is then concatenated to each $m_{i}$.
This matrix is then transformed feature-wise by a shared linear layer, before
being passed to the planner module of Ritter et al., (2020) described using
the following equations: $s^{*}_{t}=\operatorname{ReLU}(x+\phi(x))$ where
$x=\\{[m_{i},s_{t}]\\}_{i=0}^{t}$ and
$\phi(x)=\operatorname{MHA}(\operatorname{LayerNorm}(x))$ where
$\operatorname{MHA}$ is the multi-head dot-product self-attention mechanism
described by Vaswani et al., (2017) with layer-normalization (Ba et al.,,
2016) applied to the input. The output is an attended view of the agent’s past
relevant experience in the episode given the current state. This is then
passed to a two-layer shared MLP, the output of which is pooled using a
feature-wise max operation. Since the transformer we use is referred to as an
Episodic Planner Network (EPN) in (Ritter et al.,, 2020) we refer to our agent
as A2C EPN.

###  2.4 Experimental Setup

The agent is trained using the synchronous version of the Advantage Actor-
Critic (A2C) RL algorithm (Mnih et al.,, 2016) with a batch-size of $8$. The
value coefficient was set to $\beta_{V}=0.5$ and the initial entropy
coefficient to $\beta_{E}=0.1$ which was decayed in a linear fashion
throughout training. The encoder is a stack of $2$ affine layers with $32$
units each, and an $\operatorname{ELU}$ non-linearity in-between (Clevert et
al.,, 2016). The LSTM has $256$ hidden units, while the transformer block
contains $4$ attention heads with a dimensionality of $64$. The MLP is
similarly a stack of two affine layers with $64$ units each, with an
$\operatorname{ELU}$ non-linearity in-between. The episodic memory operated
with a maximum size of $150$ entries, and was reset after each episode. We
used the Adam optimizer (Kingma and Ba,, 2014) with an initial learning rate
of $7.5e-4$. The learning rate was decayed linearly from the start of
training. The gradient was clipped at a maximum norm of $100$. We found that
starting with a small discount factor of $\gamma=0.7$ worked best. The
resultant model was then finetuned using a higher value ($\gamma=0.95$) after
convergence. The number of unroll steps is $20$. The code is made open-
source222https://github.com/bkhmsi/alchemist.

##  3 Results

The results and analyses presented in this section have all been done on the
$1000$ held-out evaluation episodes provided by the benchmark using the agent
described in Section 2.3.

###  3.1 Performance

The A2C EPN agent achieves an average episode score of $272.85$ with the
modified representations, which is significantly higher than the much more
powerful VMPO agent with a gated transformer XL network (Song et al.,, 2020;
Parisotto et al.,, 2019) used by Wang et al., (2021). In an attempt to
identify which aspects contributed to this leap in performance, we re-trained
our agent but using the canonical representation in each of the input, output
and memory separately as shown in Table 1 (more details about each in the
Appendix A.2). The results show a significant drop to almost the same
performance reported by Wang et al., (2021) when using the canonical action
space. Interestingly, evaluating the same agent but without committing
anything into the episodic memory led to the same performance (see Score w/o
Memory column in Table 1), indicating that the model was not making use of its
long-term history to appraise the value of future actions.

Figure 3: Top: Behavioral results showing the agent’s lack of understanding of several abstract principles that span across episodes. In each plot, we report the number of times the agent took a null transition that it should have avoided if it understood the corresponding concept. The Ideal Observer is zero in all cases demonstrating perfect understanding of each principle. Our agent performs significantly better than the Random Heuristic but still takes a lot of actions where it could know better in the Parallelism and Potion Pairs analyses. Bottom-left: The average episode score as a function of the number of missing edges for each agent. Bottom-right: The percentage of episodes where the Idea Observer scores more than the A2C EPN agent at a given trial. Table 1: Evaluation episode scores comparing the effect of the canonical representation (indicated by an ‘o’) proposed by the Alchemy benchmark to the modified one (indicated by an ‘x’) described in Section 2.1. The VMPO result taken from (Wang et al.,, 2021). The no bottleneck column indicate the average score of the evaluation episodes with no missing edges. Agent | Input | Output | Mem | Score $\pm$ SEM |  | Score
---
(w/o Mem)
| No
---
Bottleneck
A2C EPN | x | x | x | 272.85 $\pm$ 1.78 | 168.88 | 309.90
o | x | x | 243.83 $\pm$ 2.21 | 171.59 | 300.79
o | o | x | 156.34 $\pm$ 1.57 | 156.70 | 181.41
o | o | o | 158.91 $\pm$ 1.60 | 153.40 | 182.10
VMPO | o | o | - | 155.40 $\pm$ 1.60 | - | -
Random Heuristic | - | - | - | 146.07 $\pm$ 1.55 | - | 172.11
Ideal Observer | - | - | - | 284.42 $\pm$ 1.59 | - | 313.31

This inability to exploit the memory can be largely attributed to the notion
of positional output, where the agent is required to choose an instance of the
potion and not the abstract color that causes the transformative effect. Since
each potion slot can have more than one color across different trials, the
same action will have different effects within the same episode. For instance,
action $a_{2}$ in trial $k$ can be a pink potion while in another trail can
correspond to an orange potion. In order to have an appropriate mapping
between each instance and its abstract color one will need a more suitable
architecture that would be able to recapitulate the information we gave to the
system via the custom encoding. The series of experiments we presented show
where such an architecture search could begin. Specifically, an architecture
design that gets the right information into the memory and provides
sufficiently flexible neural networks to process the memories in order to
reproduce the information contained in our modified representation. In this
work we used the custom encoding since it enabled us to pinpoint where the
architectural problem lies and perform interesting analyses that we present in
the following sections with a small compute budget.

To measure the effect of reward shaping on the final performance, we evaluated
the A2C EPN agent without giving it any additional rewards or penalties. This
achieved a score of $236.06\pm 2.18$, which suggests that exploration is a
bottleneck. In other words, the problem does not really have to do much with
learning from examples to represent abstract variables, but instead is just
about having a data distribution that’s sufficiently broad.

We found it useful as well to compare the results with respect to the number
of missing edges in the graph (see Figure 3), since the A2C EPN agent’s
performance approached that of the Ideal Observer in the case where there are
no bottlenecks. In an effort to bridge this gap, we found that this difference
is more pronounced in earlier trials. It is especially visible in the first
trial in the case where there are no missing edges as shown in Figure 3, where
the A2C EPN agent performs similar to or better than the Ideal Observer in far
more number of episodes after the first trial. This can be mitigated in future
work by incorporating more advanced exploration methods.

#### Action Types

Similar to the analysis done in Wang et al., (2021), Figure 4 shows the number
of times, throughout the first and last trial, the agent applied a potion that
worsened, improved or had no effect on the value of the stone and the number
of times the agent deposited a stone into the central cauldron as a function
of its reward. It can be seen that the A2C EPN agent almost never deposits a
negative reward stone and more importantly, it adapts its strategy throughout
the course of the episode. Concretely, in the first trial it performs a lot of
exploratory actions by trying out potions that do not have an effect on the
stone (the orange area) while in the last trial it shifts towards a more
exploitative strategy by performing more actions that improved the value of
the stone as it acquired knowledge about the episode’s chemistry. This ability
to adapt is indicative of good meta-learning performance and is similar in
behavior to what we see from the Ideal Observer as shown in the Appendix.

Figure 4: Comparing different action types throughout the first and last trial
in a similar manner to Wang et al., (2021). The A2C EPN agent shows analogous
behavior to that of the Ideal Observer (see Figure 6 in Appendix), indicating
an ability to adapt strategies (exploration vs exploitation) throughout the
course of the episode.

###  3.2 Behavioral Tests

In the following behavioral tests we report the number of times the agent took
a null transition (i.e. applying a potion that has no effect on a stone) that
wasn’t the result of a missing edge or choosing a potion color or stone that
was not available, but due to a lack of understanding of a specific abstract
principle. We compare our agent’s behavior to that of the Ideal Observer and
the Random Heuristic.

To put it formally, let $t:\langle x,y,z\rangle\xrightarrow{p}\langle
w,v,u\rangle$ be a transition that the agent has observed earlier in the
episode, where $x$, $y$ and $z$ are the color, size and shape of the stone in
question respectively, $p$ is the color of the potion being applied to the
stone, and $w$, $v$ and $u$ are the stone perceptual features after applying
the potion on it. Note that $\langle x,y,z\rangle$ can be equal to $\langle
w,v,u\rangle$, and we call that a ‘null transition’. Since there are no
rotated graphs in the evaluation episodes, each feature dimension can only
take one of two values. This implies that each stone can only be in one of
$2^{3}=8$ possible perceptual states. Figure 3 shows the results of each
tested abstract principle.

#### Consistency

Here we count the number of times the agent applied a potion with color $p$ on
a stone with features $\langle x,y,z\rangle$ after observing that $t$ is a
null transition. In other words, we have evidence that the agent does not
understand consistency if it applied a potion on a stone after observing that
this particular potion color has no effect on a stone with the same visual
features earlier in the episode.

#### Parallelism

In this test, we calculate the frequency in which the agent applied a potion
with color $p$ on a stone with features $\langle a,b,\bar{z}\rangle$ for the
first time after it observed that $t$ is not a null transition, where
$\bar{z}$ is the other possible shape, assuming that $p$ is responsible for
transforming the shape of the stone. For example, suppose the agent observed a
red potion transforming a large blue round stone to pointy. Subsequently in
the episode, it should never apply a red potion to any pointy stone, including
small or purple stones.

#### Missing Edges

The agent will show a lack of understanding of missing edges if it re-applied
potion $p$ on a stone with the same latent state as the stone with $\langle
x,y,z\rangle$ after observing that $t$ is a null transition as a result of a
missing edge. In other words, we compute the frequency in which the agent
attempted to transform a stone after ‘discovering’ that this color has no
effect on the current latent state due to a missing edge in the graph. Note
that the agent does not have access to this information, but must identify it
by experimentation.

#### Potion Pairs

To test whether the agent demonstrate an understanding that potions come in
pairs with opposite effects, we count the number of steps in which the agent
made a null transition as a result of applying a potion color $\bar{p}$ that
it has not seen the effect of before, but has observed the effect of its
opposite color previously in the episode (i.e. $t$ was not a null transition).
For example, an agent should know that yellow potions decreases the size of
stones after only observing that an orange potion transformed a small stone to
a large one (without having to see the effect of a yellow potion before).
Therefore, it should never apply a yellow potion to a small stone as to avoid
a null transition.

###  3.3 Single-Unit Activations

(a)

(b)

(c)

(d)

Figure 5: Top: LSTM units. (a) The activation of unit $192$ as a function of
the latent state. This unit is only responsive when stone $3$ is in latent
state $8$ (the state with the highest reward). (b) The same unit as in (a) but
as a function of the stone’s perceptual state. It implies that the unit is
responsive regardless of the stone’s perceptual features. (c) The activations
of unit $84$ has a magnitude proportional to the reward of its corresponding
state regardless of which stone is used. Note that states $\\{4,6,7\\}$ have a
reward of $+1$ while the rewards of states $\\{2,3,5\\}$ is $-1$. (d) The
activations of unit $26$ are more positive in the states with negative reward.
Bottom: Transformer units. (a) Unit $34$ has a high activation when the agent
choose the turquoise potion while a negative activation when it chooses its
opposite color. (b) Unit $56$ is the opposite of (a). (c) Unit $60$ is
similarly responsive when the agent chooses potion with a specific color (here
yellow) and has a negative response for its opposite color. (d) Unit 9 is
selective when the agent chooses an action that uses stone 1 regardless of the
potion color.

Inspired by single-cell recordings in neuroscience, where single neurons are
usually shown to be selective for a specific abstract concept (Wallis et al.,,
2001), we probed our model to see if it will give rise to similar selectivity
by analysing the activations of single units in the LSTM and that of the
transformer (specifically, the output of the feature-wise max). Figure 5 shows
the activations of a few units averaged across all steps in the $1000$
evaluation episodes. The latent and perceptual state of the stone that the
agent chose is recorded along with the activation of each unit in each
timestep. Note that there are $8$ latent states that correspond to the $8$
vertices of the cube, and $8$ possible perceptual states as previously
mentioned in all of the held-out episodes. In the plots, we denote each state
using a single number by binarizing its features. The top row shows the
activations of some LSTM units while the bottom row shows activations recorded
from the output the transformer.

Interestingly, we found distinct functional specialization between the
transformer and the LSTM units. Specifically, $15.6\%$ of transformer units
showed some understanding of the potion pairs abstract concept. Specifically,
each of those units had a positive activation for one potion color and a
negative activation for its opposite color (some samples shown in Figure 5),
whereas no LSTM units did. The LSTM units, on the other hand, were mostly
selective to stone-reward combinations. For instance, several units were only
responsive when a specific set of stones had either a positive or negative
reward and an opposite activation otherwise irrespective of the actual value
or the stones’ visual features. This also shows some notion of abstraction
since the agent understands which stones it needs to deposit to the cauldron
regardless of its exact representation.

However, there were no single units that were responsive to a particular
perceptual state or any single visual feature, nor were there units responsive
to specific latent states except to the reward associated with that state.
This led us to conjecture that the model reduces the cubic structure of the
latent space to a two-dimensional form (i.e. a square), where each vertex
correspond to one of the four possible rewards. This is inline with the
observation that the agent is unable to handle bottlenecks in the underlying
causal graph.

##  4 Discussion and Future Work

In this paper, we present an agent that is capable of meta-learning a set of
abstract principles that underpins the complexity of the Alchemy benchmark.
Further, we present a battery of analytical tools that can be used to test for
specific pieces of abstract knowledge. Our design choices were motivated by
the fact that strong deep-RL agents were unable to solve this task (Wang et
al.,, 2021), thus we gradually simplified the problem in order to identify
exactly where the bottleneck lies. To that end, we reached some conclusions
that can guide researchers in searching for better suited architectures.
Specifically, our experiments show that more efficient outer-loop exploration
and an architecture for handling positional output are required. We show as
well that researchers with a limited compute budget can use Symbolic Alchemy
in order to analyze their deep-RL agents in a principled manner.

Finally, we found evidence for single-unit representations of abstract
variables such as potion pairs, and functional dissociation between ‘cortical’
and ‘hippocampal’ units which were modeled as an LSTM and transformer
respectively. This connects with what’s been seen in recordings from rodent
and primate cortex (Wallis et al.,, 2001; Samborska et al.,, 2021; Mansouri et
al.,, 2020). This opens the way for future work to use meta-RL to carry out
more detailed simulations of classic neuroscience experiments to better
understand the mechanisms underlying the observed results.

## Acknowledgments

Special thanks to Muhammad ElNokrashy, as well as Michael Spranger of Sony AI,
for many helpful discussions, suggestions and comments on the paper.

## References

  * Ba et al., (2016) Ba, J., Kiros, J., and Hinton, G. E. (2016).  Layer normalization.  ArXiv, abs/1607.06450.
  * Chatham and Badre, (2015) Chatham, C. and Badre, D. (2015).  Multiple gates on working memory.  Current Opinion in Behavioral Sciences, 1:23–31.
  * Clevert et al., (2016) Clevert, D.-A., Unterthiner, T., and Hochreiter, S. (2016).  Fast and accurate deep network learning by exponential linear units (elus).  arXiv: Learning.
  * Hochreiter and Schmidhuber, (1997) Hochreiter, S. and Schmidhuber, J. (1997).  Long short-term memory.  Neural Comput., 9(8):1735–1780.
  * Kingma and Ba, (2014) Kingma, D. P. and Ba, J. (2014).  Adam: A method for stochastic optimization.  cite arxiv:1412.6980Comment: Published as a conference paper at the 3rd International Conference for Learning Representations, San Diego, 2015.
  * Lara and Wallis, (2015) Lara, A. H. and Wallis, J. D. (2015).  The role of prefrontal cortex in working memory: A mini review.  Frontiers in Systems Neuroscience, 9:173.
  * Mansouri et al., (2020) Mansouri, F., Freedman, D., and Buckley, M. (2020).  Emergence of abstract rules in the primate brain.  Nature reviews. Neuroscience, 21.
  * Milner, (1963) Milner, B. (1963).  Effects of different brain lesions on card sorting: The role of the frontal lobes.  Archives of neurology, 9(1):90–100.
  * Mnih et al., (2016) Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K. (2016).  Asynchronous methods for deep reinforcement learning.  In Balcan, M. F. and Weinberger, K. Q., editors, Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 1928–1937, New York, New York, USA. PMLR.
  * Muhammad et al., (2006) Muhammad, R., Wallis, J., and Mller, E. (2006).  A comparison of abstract rules in the prefrontal cortex, premotor cortex, inferior temporal cortex, and striatum.  Journal of cognitive neuroscience, 18:974–89.
  * Parisotto et al., (2019) Parisotto, E., Song, H. F., Rae, J. W., Pascanu, R., Gülçehre, Ç., Jayakumar, S. M., Jaderberg, M., Kaufman, R. L., Clark, A., Noury, S., Botvinick, M. M., Heess, N., and Hadsell, R. (2019).  Stabilizing transformers for reinforcement learning.  CoRR, abs/1910.06764.
  * Ritter et al., (2020) Ritter, S., Faulkner, R., Sartran, L., Santoro, A., Botvinick, M., and Raposo, D. (2020).  Rapid Task-Solving in Novel Environments.  page 16.
  * Ritter et al., (2018) Ritter, S., Wang, J., Kurth-Nelson, Z., and Botvinick, M. (2018).  Episodic control as meta-reinforcement learning.  bioRxiv.
  * Samborska et al., (2021) Samborska, V., Butler, J. L., Walton, M. E., Behrens, T. E., and Akam, T. (2021).  Complementary Task Representations in Hippocampus and Prefrontal Cortex for Generalising the Structure of Problems.  preprint, Neuroscience.
  * Song et al., (2020) Song, H. F., Abdolmaleki, A., Springenberg, J. T., Clark, A., Soyer, H., Rae, J. W., Noury, S., Ahuja, A., Liu, S., Tirumala, D., Heess, N., Belov, D., Riedmiller, M. A., and Botvinick, M. (2020).  V-mpo: On-policy maximum a posteriori policy optimization for discrete and continuous control.  ArXiv, abs/1909.12238.
  * Vaswani et al., (2017) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. (2017).  Attention is all you need.  In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
  * Wallis et al., (2001) Wallis, J., Anderson, K., and Mller, E. (2001).  Single neurons in prefrontal cortex encode abstract rules.  Nature, 411:953–6.
  * Wallis and Miller, (2003) Wallis, J. D. and Miller, E. K. (2003).  From rule to response: Neuronal processes in the premotor and prefrontal cortex.  Journal of Neurophysiology, 90(3):1790–1806.  PMID: 12736235.
  * Wang et al., (2018) Wang, J., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J., Hassabis, D., and Botvinick, M. (2018).  Prefrontal cortex as a meta-reinforcement learning system.  Nature Neuroscience, 21:14.
  * Wang et al., (2021) Wang, J. X., King, M., Porcel, N., Kurth-Nelson, Z., Zhu, T., Deck, C., Choy, P., Cassin, M., Reynolds, M., Song, F., Buttimore, G., Reichert, D. P., Rabinowitz, N., Matthey, L., Hassabis, D., Lerchner, A., and Botvinick, M. (2021).  Alchemy: A structured task distribution for meta-reinforcement learning.  page 16.

##  Appendix A Appendix

###  A.1 Action Type for Ideal Observer and Random Heuristic

Figure 6: The type of action that the Ideal Observer take throughout the first
and last trial. It can be seen that similar to the A2C EPN agent, it adapts
its strategy across the episode.

Figure 7: The type of action that the Random Heuristic take throughout the
first and last trial. The agent’s strategy is the same in both trials
demonstrating an inability to adapt strategies.

###  A.2 Input, Output and Memory Representations

The canonical input and output representation are described in detail in Wang
et al., (2021). The canonical memory representation on the other hand stores
each entry as $[s_{t},a_{t},s_{t+1}]$ where $s_{t}$ is the current input
state, $a_{t}$ is a one-hot encoding of the executed action at timestep $t$
and $s_{t+1}$ is the state at the following timestep that contains the
transformed stone.