# On the Automorphism Group of a Binary Form Associated with Algebraic
Trigonometric Quantities

Anton Mosunov

###### Abstract

Let $F(x,y)$ be a binary form of degree at least three and non-zero
discriminant. In this article we compute the automorphism group
$\operatorname{\operatorname{Aut}}F$ for four families of binary forms. The
first two families that we are interested in are homogenizations of minimal
polynomials of $2\cos\left(\frac{2\pi}{n}\right)$ and
$2\sin\left(\frac{2\pi}{n}\right)$, which we denote by $\Psi_{n}(x,y)$ and
$\Pi_{n}(x,y)$, respectively. The remaining two forms that we consider are
homogenizations of Chebyshev polynomials of first and second kinds, denoted
$T_{n}(x,y)$ and $U_{n}(x,y)$, respectively.

##  1 Introduction

Let $F(x,y)$ denote a binary form with complex coefficients of degree $d\geq
1$. Then for a matrix $M=\left(\begin{smallmatrix}s&u\\\
t&v\end{smallmatrix}\right)$, with complex entries we define a new binary form
$F_{M}(x,y)$ as follows:

$F_{M}(x,y)=F(sx+uy,tx+vy).$

If $K$ is a subfield of $\mathbb{C}$, we say that $M$ is a $K$-_automorphism_
of $F$ if $F_{M}=F$ and $M$ has entries in $K$. The set of all
$K$-automorphisms of $F$ forms a group and is denoted by
$\operatorname{\operatorname{Aut}}_{K}F$. We put
$\operatorname{\operatorname{Aut}}F=\operatorname{\operatorname{Aut}}_{\mathbb{Q}}F$.

In what follows, we restrict our attention to binary forms $F$ with integer
coefficients, degree $d\geq 3$ and non-zero discriminant $D_{F}$. In this case
$\operatorname{\operatorname{Aut}}F$ is a finite subgroup of
$\operatorname{\operatorname{GL}}_{2}(\mathbb{Q})$ [18]. It is a fact that
every finite subgroup of $\operatorname{\operatorname{GL}}_{2}(\mathbb{Q})$ is
$\operatorname{\operatorname{GL}}_{2}(\mathbb{Q})$-conjugate to one of the
groups listed in Table 1 [12].

The automorphism group $\operatorname{\operatorname{Aut}}F$ arises in the
analysis of _Thue equations_. A Thue equation is a Diophantine equation of the
form

$F(x,y)=h,$ (1)

where $h$ is a fixed integer. In 1909, Thue [19] proved that this equation has
only finitely many solutions in integers $x$ and $y$. A solution $(x,y)$ to
such equation is called _primitive_ if $x$ and $y$ are coprime. By fixing a
finite subgroup $G$ of $\operatorname{GL}_{2}(\mathbb{Z})$, Stewart [17,
Section 6] constructed binary forms $F$ such that
$\operatorname{\operatorname{Aut}}F=G$. If we now let
$M=\left(\begin{smallmatrix}s&u\\\ t&v\end{smallmatrix}\right)$ to be an
element of $\operatorname{\operatorname{Aut}}F$ and assume that $(x,y)$ is a
primitive solution to (1), then $(sx+uy,tx+vy)$ is also a primitive solution.
Therefore, for infinitely many integers $h$, the equation (1) has at least
$|\operatorname{\operatorname{Aut}}F|$ primitive solutions.

Group | Generators | Group | Generators
---|---|---|---
$\bm{C}_{1}$ | $\begin{pmatrix}1&0\\\ 0&1\end{pmatrix}$ | $\bm{D}_{1}$ | $\begin{pmatrix}0&1\\\ 1&0\end{pmatrix}$
$\bm{C}_{2}$ | $\begin{pmatrix}-1&0\\\ 0&-1\end{pmatrix}$ | $\bm{D}_{2}$ | $\begin{pmatrix}0&1\\\ 1&0\end{pmatrix},\begin{pmatrix}-1&0\\\ 0&-1\end{pmatrix}$
$\bm{C}_{3}$ | $\begin{pmatrix}0&1\\\ -1&-1\end{pmatrix}$ | $\bm{D}_{3}$ | $\begin{pmatrix}0&1\\\ 1&0\end{pmatrix},\begin{pmatrix}0&1\\\ -1&-1\end{pmatrix}$
$\bm{C}_{4}$ | $\begin{pmatrix}0&1\\\ -1&0\end{pmatrix}$ | $\bm{D}_{4}$ | $\begin{pmatrix}0&1\\\ 1&0\end{pmatrix},\begin{pmatrix}0&1\\\ -1&0\end{pmatrix}$
$\bm{C}_{6}$ | $\begin{pmatrix}0&-1\\\ 1&1\end{pmatrix}$ | $\bm{D}_{6}$ | $\begin{pmatrix}0&1\\\ 1&0\end{pmatrix},\begin{pmatrix}0&1\\\ -1&1\end{pmatrix}$
Table 1: Representatives of equivalence classes of finite subgroups of
$\operatorname{\operatorname{GL}}_{2}(\mathbb{Q})$ under conjugation.

In 2019, Stewart and Xiao [18] proved that the number of integers $R_{F}(Z)$
of absolute value at most $Z$ which are represented by $F$ is asymptotic to
$C_{F}Z^{2/d}$ for some positive number $C_{F}$. The number $C_{F}$ can be
computed as follows. Let

$\\{(x,y)\in\mathbb{R}^{2}\colon|F(x,y)|\leq 1\\}$

be the _fundamental region_ of $F$, and let $A_{F}$ denote its area. Then
$C_{F}=W_{F}A_{F}$, where $W_{F}$ is an explicit function of
$\operatorname{\operatorname{Aut}}F$ described in [18, Theorem 1.2]. In [18,
Corollary 1.3], Stewart and Xiao determined
$\operatorname{\operatorname{Aut}}F$, $W_{F}$, $A_{F}$ and $C_{F}$ in the case
when $F(x,y)=ax^{d}+by^{d}$ is a binomial form. Hooley [5, 6] determined the
value of $A_{F}$ in the case when $F$ is a cubic form. In turn, Bean [1]
determined the value of $A_{F}$ in the case when $F$ is a quartic form. These
results enabled Xiao [22, Theorems 3.1 and 4.1] to compute
$\operatorname{\operatorname{Aut}}F$, $W_{F}$ and $C_{F}$ in the case when $F$
is a cubic form or a quartic form.

In this article we compute $\operatorname{\operatorname{Aut}}F$ and $W_{F}$,
and estimate $C_{F}$ for four families of binary forms. Let $\Psi_{n}(x)$ and
$\Pi_{n}(x)$ denote the minimal polynomials of
$2\cos\left(\frac{2\pi}{n}\right)$ and $2\sin\left(\frac{2\pi}{n}\right)$,
respectively. The first two families that we are interested in are
$\Psi_{n}(x,y)$ and $\Pi_{n}(x,y)$, which are homogenizations of $\Psi_{n}(x)$
and $\Pi_{n}(x)$, respectively. By [21, Lemma],

$\Psi_{n}(x,y)=\prod\limits_{\begin{subarray}{c}1\leq k<\frac{n}{2}\\\
\gcd(k,n)=1\end{subarray}}\left(x-2\cos\left(\frac{2\pi k}{n}\right)y\right).$
(2)

Further, since
$\sin\left(\frac{2\pi}{n}\right)=\cos\left(\frac{2\pi(n-4)}{4n}\right)$, we
see that $\sin\left(\frac{2\pi}{n}\right)$ is an algebraic conjugate of
$\cos\left(\frac{2\pi}{c(n)}\right)$, where $c(n)$ is the denominator of
$\frac{n-4}{4n}$ (in lowest terms). Consequently,

$\Pi_{n}(x,y)=\Psi_{c(n)}(x,y).$ (3)

The formula for $c(n)$ can be found in [11, Corollary 1.2].

Next, let $T_{n}(x)$ and $U_{n}(x)$ denote Chebyshev polynomials of first and
second kinds, respectively. The other two families that we are interested in
are $T_{n}(x,y)$ and $U_{n}(x,y)$, which are homogenizations of $T_{n}(x)$ and
$U_{n}(x)$, respectively. It is known [10] that

$T_{n}(x,y)=2^{n-1}\prod\limits_{k=0}^{n-1}\left(x-\cos\left(\frac{(2k+1)\pi}{2n}\right)y\right)$

and

$U_{n}(x,y)=2^{n}\prod\limits_{k=1}^{n}\left(x-\cos\left(\frac{k\pi}{n+1}\right)y\right).$

Let $\varphi(n)$ denote the Euler’s totient function. Define
$\operatorname{\operatorname{Aut}}|F|$ as the group of all $2\times 2$
matrices $M$, with rational entries, such that $F_{M}=F$ or $F_{M}=-F$. Note
that $\operatorname{\operatorname{Aut}}|F|$ contains finitely many elements,
and that $\operatorname{\operatorname{Aut}}F$ is a normal subgroup of
$\operatorname{\operatorname{Aut}}|F|$ of index at most $2$. Our first result
is stated in Theorem 1.1.

| $\operatorname{\operatorname{Aut}}\Psi_{n}$ | $\operatorname{\operatorname{Aut}}|\Psi_{n}|$
---|---|---
$n$ | $\operatorname{Rep}\Psi_{n}$ | Generators | $\operatorname{Rep}|\Psi_{n}|$ | Generators
$7,18$ | $\bm{C}_{3}$ | $\begin{pmatrix}-1&-1\\\ 1&0\end{pmatrix}$ | $\bm{D}_{3}$ | $\begin{pmatrix}-1&-1\\\ 1&0\end{pmatrix},\begin{pmatrix}-1&0\\\ 0&-1\end{pmatrix}$
$9,14$ | $\bm{C}_{3}$ | $\begin{pmatrix}-1&1\\\ -1&0\end{pmatrix}$ | $\bm{D}_{3}$ | $\begin{pmatrix}-1&1\\\ -1&0\end{pmatrix},\begin{pmatrix}-1&0\\\ 0&-1\end{pmatrix}$
$15$ | $\bm{C}_{4}$ | $\begin{pmatrix}-1&2\\\ -1&1\end{pmatrix}$ | $\bm{C}_{4}$ | $\begin{pmatrix}-1&2\\\ -1&1\end{pmatrix}$
$24$ | $\bm{D}_{4}$ | $\begin{pmatrix}0&1\\\ 1&0\end{pmatrix},\begin{pmatrix}0&1\\\ -1&0\end{pmatrix}$ | $\bm{D}_{4}$ | $\begin{pmatrix}0&1\\\ 1&0\end{pmatrix},\begin{pmatrix}0&1\\\ -1&0\end{pmatrix}$
$30$ | $\bm{C}_{4}$ | $\begin{pmatrix}1&2\\\ -1&-1\end{pmatrix}$ | $\bm{C}_{4}$ | $\begin{pmatrix}1&2\\\ -1&-1\end{pmatrix}$
Table 2: $\operatorname{\operatorname{Aut}}\Psi_{n}$ and
$\operatorname{\operatorname{Aut}}|\Psi_{n}|$ for
$n\in\\{7,9,14,15,18,24,30\\}$. Here $\operatorname{Rep}\Psi_{n}$ and
$\operatorname{Rep}|\Psi_{n}|$ denote representatives of the equivalence
classes of $\operatorname{\operatorname{Aut}}\Psi_{n}$ and
$\operatorname{\operatorname{Aut}}|\Psi_{n}|$, respectively, under
$\operatorname{\operatorname{GL}}_{2}(\mathbb{Q})$ conjugation.

######  Theorem 1.1.

Let $n$ be a positive integer such that $n\notin\\{1,2,3,4,5,6,8,10,12\\}$ and
let $d=\varphi(n)/2$, so that $\deg\Psi_{n}=d$ and $d\geq 3$.

  1. 1.

If $d\geq 5$ is odd, then
$\operatorname{\operatorname{Aut}}\Psi_{n}=\\{I\\}\cong\bm{C}_{1}$ and
$\operatorname{\operatorname{Aut}}|\Psi_{n}|=\\{\pm I\\}\cong\bm{C}_{2}$,
where $I$ denotes the $2\times 2$ identity matrix.

  2. 2.

If $d\geq 6$ is even and $n\not\equiv 0\pmod{4}$, then
$\operatorname{\operatorname{Aut}}\Psi_{n}=\operatorname{\operatorname{Aut}}|\Psi_{n}|=\\{\pm
I\\}\cong\bm{C}_{2}$.

  3. 3.

If $n\equiv 0\pmod{4}$ and $n\neq 24$, then

$\operatorname{\operatorname{Aut}}\Psi_{n}=\operatorname{\operatorname{Aut}}|\Psi_{n}|=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}1&0\\\
0&-1\end{pmatrix}\right\rangle\cong\bm{D}_{2}.$

  4. 4.

If $n\in\\{7,9,14,15,18,24,30\\}$, then
$\operatorname{\operatorname{Aut}}\Psi_{n}$ and
$\operatorname{\operatorname{Aut}}|\Psi_{n}|$ are as in Table 2.

Since $\deg\Psi_{n}\in\\{3,4\\}$ if and only if
$n\in\\{7,9,14,15,16,18,20,24,30\\}$, we see that Theorem 1.1 covers all
possible cases. The proof relies on the careful analysis of roots of
$\Psi_{n}(x)$ given in (2). For example, in Lemma 3.5 we use the fact that,
for every odd positive integer $n$, if $\alpha_{i}=2\cos\left(\frac{2\pi
i}{n}\right)$ is a root of $\Psi_{n}(x)$, then so is
$\alpha_{2i}=\alpha_{i}^{2}-2$, while in Lemma 3.6 we use the fact that no
root of $\Psi_{n}(x)$ exceeds $2$ in absolute value.

From Theorem 1.1 we deduce the following.

| $\operatorname{\operatorname{Aut}}\Pi_{n}$ | $\operatorname{\operatorname{Aut}}|\Pi_{n}|$
---|---|---
$n$ | $\operatorname{Rep}\Pi_{n}$ | Generators | $\operatorname{Rep}|\Pi_{n}|$ | Generators
$28,36$ | $\bm{C}_{3}$ | $\begin{pmatrix}-1&1\\\ -1&0\end{pmatrix}$ | $\bm{D}_{3}$ | $\begin{pmatrix}-1&1\\\ -1&0\end{pmatrix},\begin{pmatrix}-1&0\\\ 0&-1\end{pmatrix}$
$60$ | $\bm{C}_{4}$ | $\begin{pmatrix}1&2\\\ -1&-1\end{pmatrix}$ | $\bm{C}_{4}$ | $\begin{pmatrix}1&2\\\ -1&-1\end{pmatrix}$
$24$ | $\bm{D}_{4}$ | $\begin{pmatrix}0&1\\\ 1&0\end{pmatrix},\begin{pmatrix}0&1\\\ -1&0\end{pmatrix}$ | $\bm{D}_{4}$ | $\begin{pmatrix}0&1\\\ 1&0\end{pmatrix},\begin{pmatrix}0&1\\\ -1&0\end{pmatrix}$
Table 3: $\operatorname{\operatorname{Aut}}\Pi_{n}$ and
$\operatorname{\operatorname{Aut}}|\Pi_{n}|$ for $n\in\\{24,28,36,60\\}$. Here
$\operatorname{Rep}\Pi_{n}$ and $\operatorname{Rep}|\Pi_{n}|$ denote
representatives of the equivalence classes of
$\operatorname{\operatorname{Aut}}\Pi_{n}$ and
$\operatorname{\operatorname{Aut}}|\Pi_{n}|$, respectively, under
$\operatorname{\operatorname{GL}}_{2}(\mathbb{Q})$ conjugation.

######  Corollary 1.2.

Let $n$ be a positive integer such that $n\notin\\{1,2,3,4,6,8,12,20\\}$. Let

$d=\begin{cases}\varphi(n)&\text{if $\gcd(n,8)<4$,}\\\ \varphi(n)/4&\text{if
$\gcd(n,8)=4$ and $n\neq 4$,}\\\ \varphi(n)/2&\text{if
$\gcd(n,8)>4$,}\end{cases}$ (4)

so that $\deg\Pi_{n}=d$ and $d\geq 3$ [14, III.4].

  1. 1.

If $n\equiv 4\pmod{8}$ and $d\geq 5$ is odd, then
$\operatorname{\operatorname{Aut}}\Pi_{n}=\\{I\\}\cong\bm{C}_{1}$ and
$\operatorname{\operatorname{Aut}}|\Pi_{n}|=\\{\pm I\\}\cong\bm{C}_{2}$.

  2. 2.

If $n\equiv 4\pmod{8}$ and $d\geq 6$ is even, then
$\operatorname{\operatorname{Aut}}\Pi_{n}=\operatorname{\operatorname{Aut}}|\Pi_{n}|=\\{\pm
I\\}\cong\bm{C}_{2}$.

  3. 3.

If $n\not\equiv 4\pmod{8}$ and $n\neq 24$, then

$\operatorname{\operatorname{Aut}}\Pi_{n}=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}1&0\\\
0&-1\end{pmatrix}\right\rangle\cong\bm{D}_{2}.$

  4. 4.

If $n\in\\{24,28,36,60\\}$, then $\operatorname{\operatorname{Aut}}\Pi_{n}$
and $\operatorname{\operatorname{Aut}}|\Pi_{n}|$ are as in Table 3.

###### Proof.

This is a direct consequence of (3) and Theorem 1.1. ∎

Since $\deg\Pi_{n}\in\\{3,4\\}$ if and only if
$n\in\\{5,10,16,24,28,36,60\\}$, we see that Corollary 1.2 covers all possible
cases.

Our second result is stated in Theorem 1.3.

######  Theorem 1.3.

For an integer $n\geq 3$, let $T_{n}(x,y)$ and $U_{n}(x,y)$ denote the
homogenizations of the $n$-th Chebyshev polynomials of first and second kinds,
respectively.

  1. 1.

If $n$ is odd, then

$\operatorname{\operatorname{Aut}}T_{n}=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix}\right\rangle\cong\bm{C}_{2},\quad\operatorname{\operatorname{Aut}}|T_{n}|=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}1&0\\\
0&-1\end{pmatrix}\right\rangle\cong\bm{D}_{2},$
$\operatorname{\operatorname{Aut}}U_{n}=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix}\right\rangle\cong\bm{C}_{2},\quad\operatorname{\operatorname{Aut}}|U_{n}|=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}1&0\\\
0&-1\end{pmatrix}\right\rangle\cong\bm{D}_{2}.$

  2. 2.

If $n$ is even, then

$\operatorname{\operatorname{Aut}}T_{n}=\operatorname{\operatorname{Aut}}|T_{n}|=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}1&0\\\
0&-1\end{pmatrix}\right\rangle\cong\bm{D}_{2},$
$\operatorname{\operatorname{Aut}}U_{n}=\operatorname{\operatorname{Aut}}|U_{n}|=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}1&0\\\
0&-1\end{pmatrix}\right\rangle\cong\bm{D}_{2}.$

The article is organized as follows. In Section 2 we use Theorem 1.1,
Corollary 1.2 and Theorem 1.3 to derive estimates for the quantities
$C_{\Psi_{n}}$, $C_{\Pi_{n}}$, $C_{T_{n}}$ and $C_{U_{n}}$. In Section 3 we
prove seven preliminary lemmas. Readers may skip the proofs in Section 3 and
refer only to the results when reading proofs of Theorems 1.1 and 1.3, which
are outlined in Sections 4 and 5, respectively.

##  2 Computation of $C_{F}$

$n$ | $W_{\Psi_{n}}$ | $A_{\Psi_{n}}$ | $C_{\Psi_{n}}$ | $W_{\Pi_{n}}$ | $A_{\Pi_{n}}$ | $C_{\Pi_{n}}$
---|---|---|---|---|---|---
5 | — | $\infty$ | — | 1/4 | 5.78302 | 1.44575
7 | 1/3 | 8.31171 | 2.77057 | 1/4 | 5.38644 | 1.34661
9 | 1/3 | 7.64379 | 2.54793 | 1/4 | 5.63543 | 1.40886
10 | — | $\infty$ | — | 1/4 | 5.78302 | 1.44575
11 | 1 | 6.12984 | 6.12984 | 1/4 | 5.27188 | 1.31797
13 | 1/2 | 5.8883 | 2.94415 | 1/4 | 5.26356 | 1.31589
14 | 1/3 | 8.31171 | 2.77057 | 1/4 | 5.38644 | 1.34661
15 | 1/4 | 6.31617 | 1.57904 | 1/4 | 5.84408 | 1.46102
16 | 1/4 | 6.08123 | 1.52031 | 1/4 | 6.08123 | 1.52031
17 | 1/2 | 5.66529 | 2.83265 | 1/4 | 5.26355 | 1.31589
Table 4: Invariants associated with $\Psi_{n}$ and $\Pi_{n}$ for
$n\in\\{5,7,9,10,11,\ldots,17\\}$.

In this section we estimate the quantity $C_{F}=W_{F}A_{F}$ for binary forms
$\Psi_{n}$, $\Pi_{n}$, $T_{n}$ and $U_{n}$. For small values of $n$ the
constants $C_{\Psi_{n}}$, $C_{\Pi_{n}}$, $C_{T_{n}}$ and $C_{U_{n}}$, along
with other invariants, can be found in Tables 4 and 5. The quantities
$A_{\Psi_{n}}$, $A_{\Pi_{n}}$, $A_{T_{n}}$ and $A_{U_{n}}$ were estimated (but
not computed) by the author in [11]. In particular, using lower- and upper-
bounds on $A_{\Psi_{n}}$, $A_{\Pi_{n}}$, $A_{T_{n}}$ and $A_{U_{n}}$
established in [11], one can prove that

$\lim\limits_{n\rightarrow\infty}A_{\Psi_{n}}=\lim\limits_{n\rightarrow\infty}A_{\Pi_{n}}=\frac{16}{3}$
(5)

and

$\lim\limits_{n\rightarrow\infty}A_{T_{n}}=\lim\limits_{n\rightarrow\infty}A_{U_{n}}=\frac{8}{3}.$
(6)

It remains to compute the rational numbers $W_{\Psi_{n}}$, $W_{\Pi_{n}}$,
$W_{T_{n}}$ and $W_{U_{n}}$. To do so, we use the formula provided in [18,
Theorem 1.2]. Let $\Lambda$ be the sublattice of $\mathbb{Z}^{2}$ consisting
of $(u,v)$ in $\mathbb{Z}^{2}$ for which $A\left(\begin{smallmatrix}u\\\
v\end{smallmatrix}\right)$ is in $\mathbb{Z}^{2}$ for all $A$ in
$\operatorname{\operatorname{Aut}}F$. Put $m=d(\Lambda)$, where $d(\Lambda)$
is the determinant of $\Lambda$. By [18, Theorem 1.2], the value of $W_{F}$
depends on $m$ when $\operatorname{\operatorname{Aut}}F$ is isomorphic to $\bf
C_{3}$, $\bf C_{4}$, $\bf C_{6}$, $\bf D_{1}$ or $\bf D_{2}$. When
$\operatorname{\operatorname{Aut}}F$ is isomorphic to $\bf D_{3}$, $\bf D_{4}$
or $\bf D_{6}$, in addition to being dependent on $m$, the value $W_{F}$
depends on the quantities $m_{i}=d(\Lambda_{i})$. The lattices $\Lambda_{i}$
are generated in a similar fashion as $\Lambda$ by certain _subgroups_ $G_{i}$
of $\operatorname{\operatorname{Aut}}F$, whose order is either $2$ or $3$.
When $\operatorname{\operatorname{Aut}}F$ is isomorphic to $\bf C_{1}$ or $\bf
C_{2}$, the value $W_{F}$ is equal to $1$ or $1/2$, respectively. In the
special case when $\operatorname{Aut}F$ is a subgroup of
$\operatorname{\operatorname{GL}}_{2}(\mathbb{Z})$, we have $m=1$ and
$m_{i}=1$, and so the formula for $W_{F}$ becomes especially simple:

$W_{F}=\frac{1}{|\operatorname{Aut}F|}.$ (7)

Notice that all automorphism groups in Theorem 1.1, Corollary 1.2 and Theorem
1.3 are subgroups of $\operatorname{GL}_{2}(\mathbb{Z})$, so the above formula
applies. For this reason we omit the calculations and directly state our
results. Notice that in the following propositions we do not provide explicit
formulas for $C_{\Psi_{n}}$, $C_{\Pi_{n}}$, $C_{T_{n}}$ and $C_{U_{n}}$,
because, as it was mentioned above, the quantities $A_{\Psi_{n}}$,
$A_{\Pi_{n}}$, $A_{T_{n}}$ and $A_{U_{n}}$ were _estimated_ , but not
_computed_.

######  Proposition 2.1.

Let $n$ be a positive integer such that $n\notin\\{1,2,3,4,5,6,8,10,12\\}$ and
let $d=\varphi(n)/2$, so that $\deg\Psi_{n}=d$ and $d\geq 3$. Then

$W_{\Psi_{n}}=\begin{cases}1&\text{if $d\geq 5$ is odd,}\\\ 1/2&\text{if
$d\geq 6$ is even and $n\not\equiv 0\\!\\!\\!\pmod{4}$,}\\\ 1/3&\text{if
$n\in\\{7,9,14,18\\}$,}\\\ 1/4&\text{if $n\in\\{15,30\\}$,}\\\ 1/4&\text{if
$n\equiv 0\\!\\!\\!\pmod{4}$ and $n\neq 24$,}\\\ 1/8&\text{if
$n=24$.}\end{cases}$

Consequently, if we let $R_{\Psi_{n}}(Z)$ denote the number of integers of
absolute value at most $Z$ which are represented by $\Psi_{n}$, then

$R_{\Psi_{n}}(Z)\sim W_{\Psi_{n}}A_{\Psi_{n}}Z^{2/d},$

with lower- and upper-bound on $A_{\Psi_{n}}$ given in [11, Theorem 1.1].

###### Proof.

The formula for $W_{\Psi_{n}}$ is a direct consequence of Theorem 1.1 and (7).
The asymptotic formula for $R_{\Psi_{n}}(Z)$ follows from [18, Theorem 1.1]. ∎

Combining Proposition 2.1 with $C_{F}=W_{F}A_{F}$ and (5), we find that

$\lim\limits_{k\rightarrow\infty}C_{\Psi_{4k}}=\frac{4}{3}.$

Further, note that $\deg\Psi_{n}\geq 3$ is odd if and only if $n=p^{j}$ or
$n=2p^{j}$, where $p\equiv 3$ (mod $4$) is prime and $j$ is a positive
integer. Let $\mathcal{S}=\\{3,6,7,9,11,14,18,\ldots\\}$ denote the set of all
such integers. Then

$\lim\limits_{\begin{subarray}{c}n\rightarrow\infty\\\
n\in\mathcal{S}\end{subarray}}C_{\Psi_{n}}=\frac{16}{3}\quad\text{and}\quad\lim\limits_{\begin{subarray}{c}n\rightarrow\infty\\\
n\notin\mathcal{S},\ 4\nmid n\end{subarray}}C_{\Psi_{n}}=\frac{8}{3}.$

$n$ | $W_{T_{n}}$ | $A_{T_{n}}$ | $C_{T_{n}}$ | $W_{U_{n}}$ | $A_{U_{n}}$ | $C_{U_{n}}$
---|---|---|---|---|---|---
$3$ | 1/2 | 5.78286 | 2.89143 | 1/2 | 4.46217 | 2.23086
$4$ | 1/4 | 4.30008 | 1.07502 | 1/4 | 3.50332 | 0.87583
$5$ | 1/2 | 3.78568 | 1.89284 | 1/2 | 3.19719 | 1.59859
$6$ | 1/4 | 3.52082 | 0.880205 | 1/4 | 3.04985 | 0.762463
$7$ | 1/2 | 3.35841 | 1.6792 | 1/2 | 2.96434 | 1.48217
$8$ | 1/4 | 3.24832 | 0.812081 | 1/4 | 2.90894 | 0.727235
$9$ | 1/2 | 3.16867 | 1.58434 | 1/2 | 2.87035 | 1.43517
$10$ | 1/4 | 3.10831 | 0.777077 | 1/4 | 2.84203 | 0.710508
$11$ | 1/2 | 3.06096 | 1.53048 | 1/2 | 2.82042 | 1.41021
$12$ | 1/4 | 3.02282 | 0.755705 | 1/4 | 2.80343 | 0.700857
Table 5: Invariants associated with $T_{n}$ and $U_{n}$ for
$n\in\\{3,4,\ldots,12\\}$.

######  Proposition 2.2.

Let $n$ be a positive integer such that $n\notin\\{1,2,3,4,6,8,12,20\\}$. Let
$d$ be as in (4), so that $\deg\Pi_{n}=d$. Then

$W_{\Pi_{n}}=\begin{cases}1&\text{if $n\equiv 4\\!\\!\\!\pmod{8}$ and $d\geq
5$ is odd,}\\\ 1/2&\text{if $n\equiv 4\\!\\!\\!\pmod{8}$ and $d\geq 6$ is
even,}\\\ 1/3&\text{if $n\in\\{28,36\\}$,}\\\ 1/4&\text{if $n=60$,}\\\
1/4&\text{if $n\not\equiv 4\\!\\!\\!\pmod{8}$ and $n\neq 24$,}\\\ 1/8&\text{if
$n=24$.}\end{cases}$

Consequently, if we let $R_{\Pi_{n}}(Z)$ denote the number of integers of
absolute value at most $Z$ which are represented by $\Pi_{n}$, then

$R_{\Pi_{n}}(Z)\sim W_{\Psi_{n}}A_{\Psi_{n}}Z^{2/d},$

with lower- and upper-bound on $A_{\Pi_{n}}=A_{\Psi_{c(n)}}$ given in [11,
Theorem 1.1].

###### Proof.

The formula for $W_{\Pi_{n}}$ is a direct consequence of Corollary 1.2 and
(7). The asymptotic formula for $R_{\Pi_{n}}(Z)$ follows from [18, Theorem
1.1]. ∎

Combining Proposition 2.2 with $C_{F}=W_{F}A_{F}$ and (5), we find that

$\lim\limits_{\begin{subarray}{c}n\rightarrow\infty\\\ n\not\equiv
4\\!\\!\\!\\!\\!\pmod{8}\end{subarray}}C_{\Pi_{n}}=\frac{4}{3}.$

Further, note that $\deg\Pi_{n}\geq 3$ is odd if and only if $n=4p^{j}$, where
$p\equiv 3$ (mod $4$) is prime and $j$ is a positive integer. Let
$\mathcal{T}=\\{12,28,36,44,76,92,108,\ldots\\}$ denote the set of all such
integers. Then

$\lim\limits_{\begin{subarray}{c}n\rightarrow\infty\\\
n\in\mathcal{T}\end{subarray}}C_{\Pi_{n}}=\frac{16}{3}\quad\text{and}\quad\lim\limits_{\begin{subarray}{c}n\rightarrow\infty\\\
\begin{subarray}{c}n\notin\mathcal{T},\
8\mid(n-4)\end{subarray}\end{subarray}}C_{\Pi_{n}}=\frac{8}{3}.$

######  Proposition 2.3.

Let $n$ be an integer such that $n\geq 3$. Then

$W_{T_{n}}=W_{U_{n}}=\begin{cases}1/2&\text{if $n$ is odd,}\\\ 1/4&\text{if
$n$ is even.}\end{cases}$

Consequently, if we let $R_{T_{n}}(Z)$ and $R_{U_{n}}(Z)$ denote the number of
integers of absolute value at most $Z$ which are represented by $T_{n}$ and
$U_{n}$, respectively, then

$R_{T_{n}}(Z)\sim W_{T_{n}}A_{T_{n}}Z^{2/n},$ $R_{U_{n}}(Z)\sim
W_{U_{n}}A_{U_{n}}Z^{2/n},$

with lower- and upper-bounds on $A_{T_{n}}$ and $A_{U_{n}}$ given in [11,
Theorem 1.3] and [11, Theorem 1.4], respectively.

###### Proof.

The formulas for $W_{T_{n}}$ and $W_{U_{n}}$ can be established with Theorem
1.3 and (7). The asymptotic formulas for $R_{T_{n}}(Z)$ and $R_{U_{n}}(Z)$
follow from [18, Theorem 1.1]. ∎

Combining Proposition 2.3 with $C_{F}=W_{F}A_{F}$ and (6), we find that

$\lim\limits_{k\rightarrow\infty}C_{T_{2k+1}}=\lim\limits_{k\rightarrow\infty}C_{U_{2k+1}}=\frac{4}{3}$

and

$\lim\limits_{k\rightarrow\infty}C_{T_{2k}}=\lim\limits_{k\rightarrow\infty}C_{U_{2k}}=\frac{2}{3}.$

##  3 Preliminary Lemmas

In this section we summarize some facts that will become useful to us when
proving Theorems 1.1 and 1.3.

######  Lemma 3.1.

Let $n$ be a positive integer. The Galois group of the field
$\mathbb{Q}\left(2\cos\left(\frac{2\pi}{n}\right)\right)$ is Abelian and it
consists of field automorphisms $\sigma_{k}$ defined by
$\sigma_{k}\left(2\cos\left(\frac{2\pi}{n}\right)\right)=2\cos\left(\frac{2\pi
k}{n}\right)$, where $k$ is an integer coprime to $n$.

###### Proof.

Let $\zeta_{n}=e^{\frac{2\pi i}{n}}$. By [3, Theorem 14.5.26], the Galois
group of the cyclotomic field $\mathbb{Q}\left(\zeta_{n}\right)$ is Abelian,
because it is isomorphic to $(\mathbb{Z}/n\mathbb{Z})^{\times}$, the
multiplicative group of invertible elements in $\mathbb{Z}/n\mathbb{Z}$. Since
$2\cos\left(\frac{2\pi}{n}\right)=\zeta_{n}+\zeta_{n}^{-1}$, we see that
$\mathbb{Q}(\zeta_{n}+\zeta_{n}^{-1})$ is a subfield of
$\mathbb{Q}(\zeta_{n})$, so the Galois group of
$\mathbb{Q}(\zeta_{n}+\zeta_{n}^{-1})$ is also Abelian. Furthermore, the
Galois group of $\mathbb{Q}(\zeta_{n})$ consists of field automorphisms
$\tau_{k}$ defined by $\tau_{k}(\zeta_{n})=\zeta_{n}^{k}$, where $k$ is an
integer coprime to $n$. Restricting the field automorphism $\tau_{k}$ to the
field $\mathbb{Q}(\zeta_{n}+\zeta_{n}^{-1})$ gives us the field automorphism
$\sigma_{k}$ defined by
$\sigma_{k}(\zeta_{n}+\zeta_{n}^{-1})=\zeta_{n}^{k}+\zeta_{n}^{-k}$. ∎

######  Lemma 3.2.

Let $n\geq 3$ be an integer and let $d=\varphi(n)/2$, so that
$\deg\Psi_{n}=d$.

  1. 1.

If $n\equiv 0\pmod{4}$, then $\Psi_{n}(x)=g(x^{2})$, where $g(x)$ is the
minimal polynomial of $2+2\cos\left(\frac{4\pi}{n}\right)$.

  2. 2.

If $n$ is odd, then $-2\cos\left(\frac{2\pi}{n}\right)$ is a conjugate of
$2\cos\left(\frac{\pi}{n}\right)$. Consequently,

$\Psi_{n}(x)=(-1)^{d}\Psi_{2n}(-x).$

###### Proof.

  1. 1.

Suppose that $n\equiv 0\pmod{4}$. Recall that $2\cos^{2}(x)=1+\cos(2x)$ for
any $x\in\mathbb{R}$. Therefore,

$4\cos^{2}\left(\frac{2\pi}{n}\right)=2\left(1+\cos\left(\frac{4\pi}{n}\right)\right)=2+2\cos\left(\frac{2\pi}{n/2}\right).$

Let $g(x)$ denote the minimal polynomial of
$2+2\cos\left(\frac{2\pi}{(n/2)}\right)$. Note that $\deg g(x)=\varphi(n/2)/2$
and

$g\left(4\cos^{2}\left(\frac{2\pi}{n}\right)\right)=0.$

Since for any positive integer $n$ divisible by $4$ it is the case that
$\varphi(n)/2=\varphi(n/2)$, we have

$\deg\Psi_{n}(x)=\frac{\varphi(n)}{2}=2\cdot\frac{\varphi(n/2)}{2}=2\deg
g(x)=\deg g(x^{2}).$

Since the polynomials $g(x^{2})$ and $\Psi_{n}(x)$ have equal degrees, both
vanish at $2\cos\left(\frac{2\pi}{n}\right)$, and the leading coefficient of
$g(x^{2})$ is positive, we conclude that $\Psi_{n}(x)=g(x^{2})$.

  2. 2.

Suppose that $n$ is odd. Note that

$-2\cos\left(\frac{2\pi}{n}\right)=2\cos\left(\pi+\frac{2\pi}{n}\right)=2\cos\left(\frac{2\pi(n+2)}{2n}\right).$

Since $\gcd(2n,n+2)=1$, we see that $-2\cos\left(\frac{2\pi}{n}\right)$ is a
conjugate of $2\cos\left(\frac{\pi}{n}\right)$. Thus
$\Psi_{2n}\left(-2\cos\left(\frac{2\pi}{n}\right)\right)=0$. But then
$2\cos\left(\frac{2\pi}{n}\right)$ is a root of $(-1)^{d}\Psi_{2n}(-x)$, and
since the leading coefficient of this polynomial is positive, it must be equal
to the minimal polynomial of $2\cos\left(\frac{2\pi}{n}\right)$.

∎

######  Lemma 3.3.

Let $F(x,y)\in\mathbb{Z}[x,y]$ be a binary form. Let $r$ be a non-zero
rational number and let $S\in\operatorname{GL}_{2}(\mathbb{Q})$. Then

$\operatorname{\operatorname{Aut}}rF_{S}=S^{-1}(\operatorname{\operatorname{Aut}}F)S\quad\text{and}\quad\operatorname{\operatorname{Aut}}|rF_{S}|=S^{-1}\left(\operatorname{\operatorname{Aut}}|F|\right)S.$

###### Proof.

We see that $F_{M}=F$ if and only if

$F_{S}=(F_{M})_{S}=F_{MS}=\left((F_{S})_{S^{-1}}\right)_{MS}=(F_{S})_{S^{-1}MS}.$

We conclude that $M\in\operatorname{\operatorname{Aut}}F$ if and only if
$S^{-1}MS\in\operatorname{\operatorname{Aut}}F_{S}$. This means that
$\operatorname{\operatorname{Aut}}F_{S}=S^{-1}\left(\operatorname{\operatorname{Aut}}F\right)S$.
Since
$\operatorname{\operatorname{Aut}}rF_{S}=\operatorname{\operatorname{Aut}}F_{S}$,
the result follows. The equality
$\operatorname{\operatorname{Aut}}|rF_{S}|=S^{-1}\left(\operatorname{\operatorname{Aut}}|F|\right)S$
can be proved analogously. ∎

######  Lemma 3.4.

Let

$D_{2}=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}1&0\\\ 0&-1\end{pmatrix}\right\rangle.$

Every finite subgroup of $\operatorname{\operatorname{GL}}_{2}(\mathbb{Q})$
that properly contains $D_{2}$ is either of the form

$\left\langle\begin{pmatrix}-1&0\\\ 0&1\end{pmatrix},\begin{pmatrix}0&t\\\
-1/t&0\end{pmatrix}\right\rangle$

or of the form

$\left\langle\begin{pmatrix}-1&0\\\ 0&1\end{pmatrix},\begin{pmatrix}1/2&t/2\\\
-3/(2t)&1/2\end{pmatrix}\right\rangle$

for some non-zero $t\in\mathbb{Q}$.

###### Proof.

Let $G=\operatorname{\operatorname{GL}}_{2}(\mathbb{Q})$ and let $H$ be a
finite subgroup of $G$ that properly contains $D_{2}$. According to the
classification of finite subgroups of $G$ given in Table 1, every finite
subgroup of $G$ that contains a group isomorphic to $\bm{D}_{2}$ and has more
than $4$ elements is $G$-conjugate to either $\bm{D}_{4}$ or $\bm{D}_{6}$. We
consider these two cases separately.

  1. 1.

Suppose that $H$ is $G$-conjugate to $\bm{D}_{4}$. That is, there exists some
matrix $A\in G$ such that $H=A\bm{D}_{4}A^{-1}$. Since $D_{2}\subsetneq H$, we
also have

$D_{2}=ANA^{-1}$

for some subgroup $N$ of $\bm{D}_{4}$ that is isomorphic to $\bm{D}_{2}$. Note
that $\bm{D}_{4}$ contains exactly two subgroups isomorphic to $\bm{D}_{2}$,
namely $\bm{D}_{2}$ itself and $D_{2}$. Thus we consider two separate cases,
i.e., $N=\bm{D}_{2}$ and $N=D_{2}$.

    1. (a)

Suppose that $D_{2}=A\bm{D}_{2}A^{-1}$. A straightforward calculation shows
that every matrix $A\in G$ such that $D_{2}=A\bm{D}_{2}A^{-1}$ must be of the
form

$\begin{pmatrix}a&-a\\\
b&b\end{pmatrix}\,\,\,\textnormal{or}\,\,\,\begin{pmatrix}a&a\\\
b&-b\end{pmatrix}$

for some non-zero $a,b\in\mathbb{Q}$. Independently of the form of $A$, we
have

$\displaystyle H$ $\displaystyle=\left\langle A\begin{pmatrix}0&1\\\
1&0\end{pmatrix}A^{-1},A\begin{pmatrix}0&1\\\
-1&0\end{pmatrix}A^{-1}\right\rangle$
$\displaystyle=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}0&a/b\\\ -b/a&0\end{pmatrix}\right\rangle$

Upon setting $t=a/b$, the result follows.

    2. (b)

Suppose that $D_{2}=AD_{2}A^{-1}$. A straightforward calculation shows that
every matrix $A\in G$ such that $D_{2}=AD_{2}A^{-1}$ must be of the form

$\begin{pmatrix}a&0\\\
0&b\end{pmatrix}\,\,\,\textrm{or}\,\,\,\begin{pmatrix}0&a\\\ b&0\end{pmatrix}$

for some non-zero $a,b\in\mathbb{Q}$. Consequently,

$\displaystyle H$ $\displaystyle=\left\langle\begin{pmatrix}0&a/b\\\
b/a&0\end{pmatrix},\begin{pmatrix}0&a/b\\\ -b/a&0\end{pmatrix}\right\rangle$
$\displaystyle=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}0&a/b\\\ -b/a&0\end{pmatrix}\right\rangle$

Upon setting $t=a/b$, the result follows.

  2. 2.

Suppose that $H$ is $G$-conjugate to $\bm{D}_{6}$. That is, there exists some
matrix $A\in G$ such that $H=A\bm{D}_{6}A^{-1}$. Since $D_{2}\subsetneq
D_{6}$, we also have

$D_{2}=ANA^{-1}$

for some subgroup $N$ of $\bm{D}_{6}$ that is isomorphic to $\bm{D}_{2}$. Note
that $\bm{D}_{6}$ contains exactly three subgroups isomorphic to $\bm{D}_{2}$,
namely $\bm{D}_{2}$ itself,

$D_{2}^{(1)}=\left\langle\begin{pmatrix}-1&0\\\
0&-1\end{pmatrix},\begin{pmatrix}1&0\\\ 1&-1\end{pmatrix}\right\rangle,$

and

$D_{2}^{(2)}=\left\langle\begin{pmatrix}-1&0\\\
0&-1\end{pmatrix},\begin{pmatrix}1&-1\\\ 0&-1\end{pmatrix}\right\rangle.$

Thus we consider three separate cases, i.e., $N=\bm{D}_{2}$, $N=D_{2}^{(1)}$,
and $N=D_{2}^{(2)}$.

    1. (a)

Suppose that $D_{2}=A\bm{D}_{2}A^{-1}$ for some $A\in G$. As it was explained
previously, every matrix $A$ which satisfies $D_{2}=A\bm{D}_{2}A^{-1}$ must be
of the form

$\begin{pmatrix}a&-a\\\
b&b\end{pmatrix}\,\,\,\textrm{or}\,\,\,\begin{pmatrix}a&a\\\
b&-b\end{pmatrix}$

for some non-zero $a,b\in\mathbb{Q}$. Therefore,

$\displaystyle H$ $\displaystyle=\left\langle A\begin{pmatrix}0&1\\\
1&0\end{pmatrix}A^{-1},A\begin{pmatrix}0&1\\\
-1&1\end{pmatrix}A^{-1}\right\rangle$
$\displaystyle=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}1/2&a/(2b)\\\
-3b/(2a)&1/2\end{pmatrix}\right\rangle$

Upon setting $t=a/b$, the result follows.

    2. (b)

Suppose that $D_{2}=AD_{2}^{(1)}A^{-1}$ for some $A\in G$. A straightforward
calculation shows that $A$ must be of the form

$\begin{pmatrix}a&-2a\\\
b&0\end{pmatrix}\,\,\,\textrm{or}\,\,\,\begin{pmatrix}a&0\\\
b&-2b\end{pmatrix}$

Therefore,

$H=\left\langle\begin{pmatrix}-1/2&-3a/(2b)\\\
-b/(2a)&1/2\end{pmatrix},\begin{pmatrix}1/2&3a/(2b)\\\
-b/(2a)&1/2\end{pmatrix}\right\rangle$

Upon setting $t=3a/b$, the result follows.

    3. (c)

Suppose that $D_{2}=A\bm{D}_{2}^{(2)}A^{-1}$ for some $A\in G$. A
straightforward calculation shows that $A$ must be of the form

$\begin{pmatrix}-2a&a\\\
0&b\end{pmatrix}\,\,\,\textrm{or}\,\,\,\begin{pmatrix}0&a\\\
-2b&b\end{pmatrix}$

for some non-zero $a,b\in\mathbb{Q}$. Therefore,

$H=\left\langle\begin{pmatrix}-1/2&-3a/(2b)\\\
-b/(2a)&1/2\end{pmatrix},\begin{pmatrix}1/2&-3a/(2b)\\\
b/(2a)&1/2\end{pmatrix}\right\rangle$

Upon setting $t=3a/b$, the result follows.

∎

######  Lemma 3.5.

Let $n$ be an odd positive integer such that $\varphi(n)\geq 10$. Let $j$ be
an integer coprime to $n$. If

$2\cos\left(\frac{2\pi
j}{n}\right)=\frac{2\cos\left(\frac{2\pi}{n}\right)v-u}{-2\cos\left(\frac{2\pi}{n}\right)t+s}$
(8)

for some rationals $s$, $t$, $u$ and $v$, then $s\neq 0$, $s=v$ and $t=u=0$.

###### Proof.

For an integer $i$, let $\alpha_{i}=2\cos\left(\frac{2\pi i}{n}\right)$. Put
$\alpha=\alpha_{1}$. Since $n$ is odd, it follows from Lemma 3.1 that there
exists a field automorphism $\sigma_{2}$ in the Galois group of
$\mathbb{Q}(\alpha)$ such that $\sigma_{2}(\alpha_{\ell})=\alpha_{2\ell}$ for
each $\ell$ coprime to $n$. Therefore,

$\alpha_{2j}=\sigma_{2}(\alpha_{j})=\sigma_{2}\left(\frac{v\alpha-u}{-t\alpha+s}\right)=\frac{v\sigma_{2}(\alpha)-u}{-t\sigma_{2}(\alpha)+s}=\frac{v\alpha_{2}-u}{-t\alpha_{2}+s}.$

Since for any $x\in\mathbb{R}$ it is the case that
$2\cos(2x)=(2\cos(x))^{2}-2$, we conclude that $\alpha_{2i}=\alpha_{i}^{2}-2$
for all $i$. Therefore,

$\left(\frac{v\alpha-u}{-t\alpha+s}\right)^{2}-2=\alpha_{j}^{2}-2=\alpha_{2j}=\frac{v\alpha_{2}-u}{-t\alpha_{2}+s}=\frac{v(\alpha^{2}-2)-u}{-t(\alpha^{2}-2)+s}.$

From the above equality we obtain

$\left(-t(\alpha^{2}-2)+s\right)\left((v\alpha-u)^{2}-2(-t\alpha+s)^{2}\right)=(-t\alpha+s)^{2}\left(v(\alpha^{2}-2)-u\right).$

We conclude that the polynomial

$\displaystyle(2t^{3}-t^{2}v-tv^{2})x^{4}$
$\displaystyle+(-4st^{2}+2stv+2tuv)x^{3}$
$\displaystyle+(2s^{2}t-s^{2}v-2st^{2}+sv^{2}-4t^{3}+t^{2}u+2t^{2}v-tu^{2}+2tv^{2})x^{2}$
$\displaystyle+(4s^{2}t+8st^{2}-2stu-4stv-2suv-4tuv)x$
$\displaystyle+(-2s^{3}-4s^{2}t+s^{2}u+2s^{2}v+su^{2}+2tu^{2})$

vanishes at $\alpha$. Since the degree of $\alpha$ is
$\frac{\varphi(n)}{2}\geq 5$ and the above polynomial has degree at most $4$,
it must be the case that this polynomial is identically equal to zero. That
is,

$\begin{array}[]{r l}t(t-v)(2t+v)&=0,\\\ t(-2st+sv+uv)&=0,\\\
2s^{2}t-s^{2}v-2st^{2}+sv^{2}-4t^{3}+t^{2}u+2t^{2}v-tu^{2}+2tv^{2}&=0,\\\
2s^{2}t+4st^{2}-stu-2stv-suv-2tuv&=0,\\\
-2s^{3}-4s^{2}t+s^{2}u+2s^{2}v+su^{2}+2tu^{2}&=0.\end{array}$ (9)

Depending on the value of $t$, we consider the following three cases.

  1. 1.

Suppose that $t=0$. Then the first two equations in (9) vanish, while the
third and the fourth equations simplify to $sv(v-s)=0$ and $suv=0$,
respectively. Note that $s\neq 0$, for otherwise the denominator of (8)
vanishes. Thus the last two equations further reduce to $v(v-s)=0$ and $uv=0$.
If $v=0$, then the number $\alpha_{j}=-u/s$ is rational, in contradiction to
the fact that $\deg\alpha_{j}\geq 5$. Thus it must be the case that $v\neq 0$,
$s=v$ and $u=0$.

  2. 2.

Suppose that $t=v$ and $t\neq 0$. Then the second equation in (9) simplifies
to $v(u-s)=0$. But then $s=u$, and

$\alpha_{j}=\frac{v\alpha-u}{-t\alpha+s}=\frac{v\alpha-u}{-v\alpha+u}=-1,$

in contradiction to the fact that $\deg\alpha_{j}\geq 5$.

  3. 3.

Suppose that $v=-2t$ and $t\neq 0$. Then the second equation simplifies to
$v(2s+u)=0$. But then $u=-2s$,

$\alpha_{j}=\frac{v\alpha-u}{-t\alpha+s}=\frac{-2t\alpha+2s}{-t\alpha+s}=2,$

in contradiction to the fact that $\deg\alpha_{j}\geq 5$.

∎

######  Lemma 3.6.

For a positive integer $n$, the minimal polynomial $\Psi_{n}(x)$ of
$2\cos\left(\frac{2\pi}{n}\right)$ is reciprocal if and only if $n=3$ or
$n=24$.

###### Proof.

With the Mathematica command MinimalPolynomial[2*cos(2*Pi/n)] we can compute
$\Psi_{n}$ for every $1\leq n\leq 24$ and verify that reciprocal polynomials
appear only for $n=3$ and $n=24$. They are $x+1$ and $x^{4}-4x^{2}+1$.

Next, we implement the formulas

$\operatorname{tr}(n)=\sum\limits_{\begin{subarray}{c}1\leq k<\frac{n}{2}\\\
\gcd(k,n)=1\end{subarray}}2\cos\left(\frac{2\pi
k}{n}\right),\quad\operatorname{norm}(n)=\prod\limits_{\begin{subarray}{c}1\leq
k<\frac{n}{2}\\\ \gcd(k,n)=1\end{subarray}}2\cos\left(\frac{2\pi
k}{n}\right),$
$\operatorname{rtr}(n)=\operatorname{norm}(n)\sum\limits_{\begin{subarray}{c}1\leq
k<\frac{n}{2}\\\ \gcd(k,n)=1\end{subarray}}\left(2\cos\left(\frac{2\pi
k}{n}\right)\right)^{-1}$

in the computer algebra system PARI/GP. Notice that if a polynomial $\Psi_{n}$
is reciprocal, then $|\operatorname{tr}(n)|=|\operatorname{rtr}(n)|$. Running
our PARI/GP code, we can verify that for $25\leq n\leq 745$ the equality
$|\operatorname{tr}(n)|=|\operatorname{rtr}(n)|$ occurs only when $4\mid n$.

From Part 1 of Lemma 3.2 we know that if $4\mid n$, then
$\Psi_{n}(x)=g(x^{2})$, where $g(x)$ is the minimal polynomial of
$2+2\cos\left(\frac{4\pi}{n}\right)$. Thus $\Psi_{n}(x)$ is reciprocal if and
only if $g(x)$ is reciprocal. We then implement the formulas

$\operatorname{tr}^{\prime}(n)=\sum\limits_{\begin{subarray}{c}1\leq
k<\frac{n}{4}\\\ \gcd(k,n/2)=1\end{subarray}}\left(2+2\cos\left(\frac{2\pi
k}{n}\right)\right),\quad\operatorname{norm}^{\prime}(n)=\prod\limits_{\begin{subarray}{c}1\leq
k<\frac{n}{4}\\\ \gcd(k,n/2)=1\end{subarray}}\left(2+2\cos\left(\frac{2\pi
k}{n}\right)\right),$
$\operatorname{rtr}^{\prime}(n)=\operatorname{norm}^{\prime}(n)\sum\limits_{\begin{subarray}{c}1\leq
k<\frac{n}{4}\\\ \gcd(k,n/2)=1\end{subarray}}\left(2+2\cos\left(\frac{2\pi
k}{n}\right)\right)^{-1}$

in PARI/GP. Notice that if a polynomial $\Psi_{n}$ with $4\mid n$ is
reciprocal, then
$|\operatorname{tr}^{\prime}(n)|=|\operatorname{rtr}^{\prime}(n)|$. Running
our PARI/GP code, we can verify that
$|\operatorname{tr}^{\prime}(n)|\neq|\operatorname{rtr}^{\prime}(n)|$ for all
$4\mid n$ such that $25\leq n\leq 745$. We conclude that every polynomial
$\Psi_{n}(x)$ with $25\leq n\leq 745$ is not reciprocal.

It remains to prove that there are no reciprocal polynomials with $n\geq 746$.
For a positive integer $n$, let $g(n)$ denote the Jacobsthal’s function; that
is, $g(n)$ is equal to the smallest positive integer $m$ such that every
sequence of $m$ consecutive integers contains an integer coprime to $n$. It
was proven by Kanold [8] that

$g(n)\leq 2^{\omega(n)},$

where $\omega(n)$ denotes the number of distinct prime factors of $n$.111The
author is grateful to Prof. Jeffrey Shallit for pointing out that better
bounds exist, e.g., [7, 20]. However, Kanold’s bound is sufficient for our
purposes. Combining the above upper bound with the inequality [15]

$\omega(n)\leq 1.3841\frac{\log n}{\log\log n},$

which holds for all $n\geq 3$, we get

$g(n)<n^{\frac{0.96}{\log\log n}}.$

Now, consider the interval
$\left[\frac{1}{2\pi}\arccos\left(\frac{1}{4}\right),\frac{1}{4}\right)$. We
claim that this interval contains a rational number $j/n$ with $j$ coprime to
$n$. In other words, we would like to locate an integer $j$ coprime to $n$
such that

$\frac{1}{2\pi}\arccos\left(\frac{1}{4}\right)n\leq j<\frac{1}{4}n.$

We see that such an integer $j$ has to belong to the interval
$\left[\frac{1}{2\pi}\arccos\left(\frac{1}{4}\right)n,\frac{1}{4}n\right)$,
whose length exceeds $n/25$. Since our interval is half-closed, it contains at
least $\lfloor n/25\rfloor$ consecutive integers. However, for all $n\geq 746$
we have

$n^{\frac{0.96}{\log\log n}}<\frac{n}{25}-1,$

and this inequality implies that

$g(n)<n^{\frac{0.96}{\log\log
n}}<\frac{n}{25}-1<\left\lfloor\frac{n}{25}\right\rfloor.$

This means that the interval
$\left[\frac{1}{2\pi}\arccos\left(\frac{1}{4}\right)n,\frac{1}{4}n\right)$
contains an integer $j$ that is coprime to $n$. But then

$\arccos\left(\frac{1}{4}\right)\leq\frac{2\pi j}{n}<\frac{\pi}{2},$

and consequently

$0<\alpha_{j}\leq\frac{1}{2}.$

If we now assume that $\Psi_{n}(x)$ is reciprocal, then the number
$\alpha_{j}^{-1}$ is a conjugate of $\alpha_{j}$, so there exists some $\ell$
such that $\alpha_{\ell}=\alpha_{j}^{-1}$. Thus $\alpha_{\ell}\geq 2$. On the
other hand, $\alpha_{\ell}\leq 2$, which means that $\ell=0$. Since
$\gcd(\ell,n)=1$, we conclude that $n=1$, and this contradicts our assumption
that $n\geq 746$. ∎

######  Lemma 3.7.

Let $k$ and $\ell$ be positive integers such that $k<\ell$ and
$k,\ell\notin\\{1,2,3,4,6\\}$. Then
$\mathbb{Q}\left(2\cos\left(\frac{2\pi}{k}\right)\right)=\mathbb{Q}\left(2\cos\left(\frac{2\pi}{\ell}\right)\right)$
if and only if $k$ is odd and $\ell=2k$.

###### Proof.

Suppose that $k$ is odd and $\ell=2k$. Then the degrees of
$\mathbb{Q}\left(2\cos\left(\frac{2\pi}{k}\right)\right)$ and
$\mathbb{Q}\left(2\cos\left(\frac{2\pi}{\ell}\right)\right)$ are both equal to
$\frac{\varphi(k)}{2}$. Further,

$2\cos\left(\frac{2\pi}{k}\right)=\left(2\cos\left(\frac{\pi}{k}\right)\right)^{2}-2=\left(2\cos\left(\frac{2\pi}{\ell}\right)\right)^{2}-2,$

which means that
$\mathbb{Q}\left(2\cos\left(\frac{2\pi}{k}\right)\right)\subseteq\mathbb{Q}\left(2\cos\left(\frac{2\pi}{\ell}\right)\right)$.
Since $\mathbb{Q}\left(2\cos\left(\frac{2\pi}{k}\right)\right)$ is a subfield
of $\mathbb{Q}\left(2\cos\left(\frac{2\pi}{\ell}\right)\right)$ of the same
degree, it must be the case that the two fields are identical.

Conversely, suppose that
$\mathbb{Q}\left(2\cos\left(\frac{2\pi}{k}\right)\right)=\mathbb{Q}\left(2\cos\left(\frac{2\pi}{\ell}\right)\right)$.
Then degrees of these number fields are equal, i.e.,
$\frac{\varphi(k)}{2}=\frac{\varphi(\ell)}{2}$. We claim that there exists a
prime that divides $k$ but not $\ell$, or vice versa. For suppose that this is
not the case and

$k=\prod_{i=1}^{t}p_{i}^{e_{i}},\,\,\,\ell=\prod_{i=1}^{t}p_{i}^{f_{i}}$

for some positive integers $t,e_{1},\ldots,e_{t},f_{1},\ldots,f_{t}$ and
distinct primes $p_{1},\ldots,p_{t}$. Then

$\prod_{i=1}^{t}p_{i}^{e_{i}-1}(p_{i}-1)=\varphi(k)=\varphi(\ell)=\prod_{i=1}^{t}p_{i}^{f_{i}-1}(p_{i}-1).$

After dividing both sides by $\prod_{i=1}^{t}(p_{i}-1)$, we obtain

$\prod_{i=1}^{t}p_{i}^{e_{i}-1}=\prod_{i=1}^{t}p_{i}^{f_{i}-1},$

which means that $e_{i}=f_{i}$ for all $i=1,2,\ldots,t$. But then $k=\ell$, in
contradiction to our assumption that $k$ and $\ell$ are distinct. This
completes the proof of our claim.

Now, it follows from the result of Lehmer [9, Theorem 3.8] that the
discriminant $D_{k}$ of the field
$\mathbb{Q}\left(2\cos\left(\frac{2\pi}{k}\right)\right)$ can be computed as
follows:

$D_{k}=\begin{cases}2^{(j-1)2^{j-2}-1}&\textrm{if $k=2^{j}$, $j>2$,}\\\
p^{(jp^{j}-(j+1)p^{j-1}-1)/2}&\textrm{if $k=p^{j}$ or $2p^{j}$, $p>2$
prime,}\\\
\left(\prod_{i=1}^{\omega(k)}p_{i}^{e_{i}-1/(p_{i}-1)}\right)^{\frac{\varphi(k)}{2}}&\textrm{if
$\omega(k)>1,k\neq 2p^{j}$.}\end{cases}$ (10)

An analogous formula applies to the discriminant $D_{\ell}$ of
$\mathbb{Q}\left(2\cos\left(\frac{2\pi}{\ell}\right)\right)$, and of course we
must have $D_{\ell}=D_{k}$. Suppose that $k=2^{j}$, $j>2$. Then $D_{k}$ is a
power of $2$. Since there is a prime that divides $k$ but not $\ell$ or vice
versa, it must be the case that an odd prime $q$ divides $\ell$. But then it
follows from (10) that $q\mid D_{\ell}$, so $D_{k}\neq D_{\ell}$. Thus, this
case is impossible, and so at least one odd prime divides $k$, i.e.,

$k=2^{r}\prod\limits_{i=1}^{t}p_{i}^{e_{i}}$

for some non-negative integer $r$, positive integers $t,e_{1},\ldots,e_{t}$,
and distinct odd primes $p_{1},\ldots,p_{t}$.

Notice how in (10), for every odd prime $p$, $p\mid k$ if and only if $p\mid
D_{k}$. Similarly, for every odd prime $q$, $q\mid\ell$ if and only if $q\mid
D_{\ell}$. Since $D_{k}=D_{\ell}$, we conclude that, for every odd prime $p$,
$p\mid k$ if and only if $p\mid\ell$. Thus,

$\ell=2^{s}\prod\limits_{i=1}^{t}p_{i}^{f_{i}}$

for some non-negative integer $s$ and positive integers $f_{1},\ldots,f_{t}$.
Further, since there exists a prime that divides $k$ but not $\ell$ or vice
versa, it must be the case that either $k$ or $\ell$ is odd. At this point, we
consider four cases.

  1. 1.

If $k=p^{j}$ for some odd prime $p$, then it follows from (10) that $D_{k}$ is
odd. Further, $\ell=2^{s}p^{m}$ for some positive integers $s$ and $m$ (recall
that there must be a prime that divides $\ell$, but not $k$). Further, it must
be the case that $s=1$, for otherwise it follows from (10) that $D_{\ell}$ is
even. Thus, $\ell=2p^{m}$, and so

$p^{(jp^{j}-(j+1)p^{j-1}-1)/2}=D_{k}=D_{\ell}=p^{(mp^{m}-(m+1)p^{m-1}-1)/2}.$

Since the function $f_{p}(x)=xp^{x}-(x+1)p^{x-1}-1$ is monotonously increasing
on the interval $[1,+\infty)$, we conclude that $m=j$, and so
$\ell=2p^{j}=2k$.

  2. 2.

If $k=2p^{j}$ for some odd prime $p$, then it follows from (10) that $D_{k}$
is odd. Further, $\ell=2^{s}p^{m}$ for some non-negative integer $s$ and
positive integer $m$. Since there must exist a prime that divides $k$ but not
$\ell$, we conclude that $\ell=2^{s}\geq 8$ or $\ell=p^{m}$. The former is
impossible, since $D_{\ell}$ has to be odd. Thus, $\ell=p^{m}$, and so

$p^{(jp^{j}-(j+1)p^{j-1}-1)/2}=D_{k}=D_{\ell}=p^{(mp^{m}-(m+1)p^{m-1}-1)/2}.$

Since the function $f_{p}(x)=xp^{x}-(x+1)p^{x-1}-1$ is monotonously increasing
on the interval $[1,+\infty)$, we conclude that $m=j$. But then
$\ell=p^{j}<2p^{j}=k$, which is impossible, since we assumed that $k<\ell$.

  3. 3.

If $k$ is odd and it is not an odd prime power, then $t\geq 2$ and

$D_{k}=\left(\prod_{i=1}^{t}p_{i}^{e_{i}-1/(p_{i}-1)}\right)^{\frac{\varphi(k)}{2}}.$

Since $D_{k}$ is odd, $D_{\ell}$ is odd, which in turn implies that
$\ell=2^{s}\prod_{i=1}^{t}p_{i}^{f_{i}}$ for $s\in\\{0,1\\}$. Thus,

$D_{\ell}=\left(\prod_{i=1}^{t}p_{i}^{f_{i}-1/(p_{i}-1)}\right)^{\frac{\varphi(\ell)}{2}}.$

Since $D_{k}=D_{\ell}$, the unique factorization tells us that
$\frac{\varphi(k)}{2}\left(e_{i}-\frac{1}{p_{i}-1}\right)=\frac{\varphi(\ell)}{2}\left(f_{i}-\frac{1}{p_{i}-1}\right)$
for every $i=1,\ldots,t$. Since
$\frac{\varphi(k)}{2}=\frac{\varphi(\ell)}{2}$, we conclude that $e_{i}=f_{i}$
for every $i=1,\ldots,t$. Since $k\neq\ell$, we conclude that $s=1$, and so
$\ell=2k$.

  4. 4.

If $k=2^{r}\prod_{i=1}^{t}p_{i}^{e_{i}}$ is even and it is not twice an odd
prime power, then $\ell$ must be odd. Consequently, $D_{\ell}$ is odd. But
then $D_{k}=D_{\ell}$ is odd, which is only possible when $r=1$. Thus,

$\left(\prod_{i=1}^{t}p_{i}^{e_{i}-1/(p_{i}-1)}\right)^{\frac{\varphi(k)}{2}}=D_{k}=D_{\ell}=\left(\prod_{i=1}^{t}p_{i}^{f_{i}-1/(p_{i}-1)}\right)^{\frac{\varphi(\ell)}{2}}.$

Once again, we find that $e_{i}=f_{i}$ for all $i=1,\ldots,t$, meaning that
$k=2\prod_{i=1}^{t}p_{i}^{e_{i}}$ and
$\ell=\prod_{i=1}^{t}p_{i}^{e_{i}}=\frac{k}{2}<k$, which contradicts $k<\ell$.

∎

##  4 Automorphisms of $\Psi_{n}(x,y)$

In this section we prove Theorem 1.1. Let $n$ be a positive integer such that
$n\notin\\{1,2,3,4,5,6,8,10,12\\}$ and let $d=\varphi(n)/2$, so that
$\deg\Psi_{n}=d$ and $d\geq 3$. In Sections 4.1, 4.2 and 4.3 we consider three
cases separately:

  * •

$d\geq 4$ and $n\equiv 0\pmod{4}$;

  * •

$d\geq 5$ and $n\not\equiv 0\pmod{4}$; and

  * •

$d=3,4$ and $n\not\equiv 0\pmod{4}$.

###  4.1 Case $d\geq 4$ and $n\equiv 0\pmod{4}$

Let $n\geq 16$ be an integer such that $n\equiv 0\pmod{4}$. Then it follows
from Part 1 of Lemma 3.2 that $\Psi_{n}(x)=g(x^{2})$ for some
$g(x)\in\mathbb{Z}[x]$. Consequently, there exists a binary form
$G(x,y)\in\mathbb{Z}[x,y]$ such that $\Psi_{n}(x,y)=G(x^{2},y^{2})$.
Therefore,

$D_{2}=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}1&0\\\ 0&-1\end{pmatrix}\right\rangle$

is a subgroup of $\operatorname{\operatorname{Aut}}\Psi_{n}$.

We claim that $D_{2}$ is a _proper_ subgroup of
$\operatorname{\operatorname{Aut}}|\Psi_{n}|$ if and only if $n=24$. Since
$D_{2}\subseteq\operatorname{\operatorname{Aut}}\Psi_{n}\subseteq\operatorname{\operatorname{Aut}}|\Psi_{n}|$,
this result would imply that
$\operatorname{\operatorname{Aut}}\Psi_{n}=\operatorname{\operatorname{Aut}}|\Psi_{n}|=D_{2}$
for any positive integer $n\geq 16$ such that $n\equiv 0$ (mod $4$) and $n\neq
24$.

By Lemma 3.4, if $D_{2}$ is a proper subgroup of
$\operatorname{\operatorname{Aut}}|\Psi_{n}|$, then there exists a non-zero
$t\in\mathbb{Q}$ such that

$\operatorname{\operatorname{Aut}}|\Psi_{n}|\cong\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}0&t\\\ -1/t&0\end{pmatrix}\right\rangle$

or

$\operatorname{\operatorname{Aut}}|\Psi_{n}|\cong\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}1/2&t/2\\\
-3/(2t)&1/2\end{pmatrix}\right\rangle.$

We will consider these two options separately. In each case, we will make use
of the formula

$|\Psi_{m}(0)|=\begin{cases}0&\textrm{if $m=4$,}\\\ 2&\textrm{if $m=2^{k}$ for
$k\geq 3$,}\\\ p&\textrm{if $m=4p^{k}$ for $k\geq 1$, where $p$ is an odd
prime,}\\\ 1&\textrm{otherwise.}\end{cases}$ (11)

The proof of (11) can be found in [2].

  1. 1.

Suppose that there exist integers $a\neq 0$ and $b\geq 1$ such that
$\gcd(a,b)=1$ and $M\in\operatorname{\operatorname{Aut}}|\Psi_{n}|$, where

$M=\left(\begin{matrix}0&a/b\\\ -b/a&0\end{matrix}\right).$

Then

$\displaystyle\Psi_{n}(x,y)$
$\displaystyle=\pm\Psi_{n}\left(\frac{a}{b}y,-\frac{b}{a}x\right)$
$\displaystyle=\pm(ab)^{-d}\Psi_{n}\left(a^{2}y,-b^{2}x\right).$

Thus

$(ab)^{d}\Psi_{n}(x,y)=\pm\Psi_{n}(a^{2}y,-b^{2}x).$

By plugging $x=1$ and $y=0$ into the above equation, we see that
$c_{0}=\pm(a/b)^{d}$, where $c_{0}$ denotes the constant coefficient of
$\Psi_{n}(x)$. Since $c_{0}$ is an integer, it must be the case that $t=a/b$
is an integer such that $t^{d}=\pm c_{0}$. By (11), the value of $|c_{0}|$ is
squarefree, and since $d\geq 2$ is even and $t^{d}=|c_{0}|$, we conclude that
$c_{0}=\pm 1$. Therefore, $t=a/b=\pm 1$, which means that
$M=\left(\begin{smallmatrix}0&1\\\ -1&0\end{smallmatrix}\right)$ or
$M=\left(\begin{smallmatrix}0&-1\\\ 1&0\end{smallmatrix}\right)$. But then

$\operatorname{\operatorname{Aut}}|\Psi_{n}|=\left\langle\begin{pmatrix}0&1\\\
1&0\end{pmatrix},\begin{pmatrix}0&1\\\ -1&0\end{pmatrix}\right\rangle.$

Now, suppose that there exists $A$ in
$\operatorname{\operatorname{Aut}}|\Psi_{n}|$ such that
$A\notin\operatorname{\operatorname{Aut}}\Psi_{n}$. Plugging $x=0$ and $y=1$
into $\Psi_{n}(x,y)=-\Psi_{n}(y,-x)$, we find that $c_{0}=-1$. It follows from
(11) that $n\neq 4p^{k}$ for any odd prime $p$ and any positive integer $k$.
From Part 1 of Lemma 3.2 we know that $\Psi_{n}(\alpha)=0$ if and only if
$\Psi_{n}(-\alpha)=0$, so

$c_{0}=\prod\limits_{\begin{subarray}{c}1\leq k<\frac{n}{2}\\\
\gcd(k,n)=1\end{subarray}}\left(2\cos\left(\frac{2\pi
k}{n}\right)\right)=\prod\limits_{\begin{subarray}{c}1\leq k<\frac{n}{4}\\\
\gcd(k,n)=1\end{subarray}}\left(-4\cos\left(\frac{2\pi
k}{n}\right)^{2}\right).$

Since $c_{0}$ is negative, the number $N$ of integers in the interval
$[1,n/4)$ that are coprime to $n$ must be odd. If we write $n=2^{s}t$, where
$s\geq 2$ is an integer and $t$ is odd, then through elementary number
theoretic observations we find that
$N=\frac{d}{2}=2^{s-2}\frac{\varphi(t)}{2}$. Since $N$ is odd, we find that
$s=2$ and $t=p^{k}$ for some prime $p\equiv 3$ (mod $4$), in contradiction to
the fact that $n\neq 4p^{k}$ for any odd prime $p$ and any positive integer
$k$. Hence
$\operatorname{\operatorname{Aut}}|\Psi_{n}|=\operatorname{\operatorname{Aut}}\Psi_{n}$,
and so $\Psi_{n}(x,y)=\Psi_{n}(y,x)$, which makes $\Psi_{n}(x)$ a reciprocal
polynomial. It follows from Lemma 3.6 that $n=24$ and
$\Psi_{n}(x,y)=x^{4}-4x^{2}y^{2}+y^{4}$.

  2. 2.

Suppose that there exist integers $a\neq 0$ and $b\geq 1$ such that
$\gcd(a,b)=1$ and $M\in\operatorname{\operatorname{Aut}}|\Psi_{n}|$, where

$M=\begin{pmatrix}1/2&a/(2b)\\\ -3b/(2a)&1/2\end{pmatrix}.$

We will show that this is impossible.

Since $M\in\operatorname{\operatorname{Aut}}|\Psi_{n}|$,

$\displaystyle\Psi_{n}(x,y)$
$\displaystyle=\pm\Psi_{n}\left(\frac{1}{2}x+\frac{a}{2b}y,-\frac{3b}{2a}x+\frac{1}{2}y\right)$
$\displaystyle=\pm(2ab)^{-d}\Psi_{n}\left(abx+a^{2}y,-3b^{2}x+aby\right).$

Thus

$(2ab)^{d}\Psi_{n}(x,y)=\pm\Psi_{n}\left(abx+a^{2}y,-3b^{2}x+aby\right).$ (12)

By plugging $x=0$ and $y=1$ into the above equation, we obtain
$c_{0}2^{d}b^{d}=\pm\Psi_{n}(a,b)$. Thus $\Psi_{n}(a,b)$ is divisible by $b$.
Since the leading coefficient of $\Psi_{n}(x,y)$ is equal to one, we see that

$a^{d}\equiv\Psi(a,0)\equiv\Psi_{n}(a,b)\equiv 0\pmod{b}.$

Then $b\mid a^{d}$, and since $a$ and $b\geq 1$ are coprime, we conclude that
$b=1$ and $c_{0}2^{d}=\pm\Psi_{n}(a)$. By plugging $x=1$ and $y=0$ into (12),
we obtain $(2a)^{d}=\pm\Psi_{n}(a,-3)$. Since $\Psi_{n}(x)=g(x^{2})$, we see
that

$\pm c_{0}(-3)^{d}\equiv\Psi_{n}(a,-3)\equiv 0\pmod{a^{2}},$

which means that $a^{2}\mid c_{0}3^{d}$. By (11) the value of $c_{0}$ is
squarefree, so $a=\pm 3^{r}$ for some non-negative integer $r$. Since
$\operatorname{\operatorname{Aut}}|\Psi_{n}|$ is a group, we may replace $M$
with

$M^{-1}=\begin{pmatrix}1/2&-a/(2b)\\\ 3b/(2a)&1/2\end{pmatrix},$

and so without loss of generality we may assume that $a=3^{r}$.

After plugging $a=3^{r}$ and $b=1$ into (12) we obtain

$2^{d}3^{(r-1)d}\Psi_{n}(x,y)=\pm\Psi_{n}\left(3^{r-1}x+3^{2r-1}y,-x+3^{r-1}y\right).$
(13)

Suppose that $r\geq 3$. Then

$\pm\Psi_{n}\left(3^{r-1}x+3^{2r-1}y,-x+3^{r-1}y\right)\equiv\Psi_{n}(0,-x)\equiv
c_{0}(-x)^{d}\equiv 0\pmod{9}.$

Since this congruence must hold for all $x$, it holds for those $x$ that are
not divisible by $3$, which means that $9$ divides $c_{0}$. However, this
result contradicts (11), which states that the value of $c_{0}$ is squarefree.
We conclude that the only possible values of $r$ are $0,1,2$, and so the only
possible values of $a=3^{r}$ are $1,3$ and $9$. We consider these three cases
separately:

     * •

For $r=0$, $x=0$, $y=1$ the equation (13) gives us $|\Psi_{n}(1)|=2^{d}$;

     * •

For $r=1$, $x=1$, $y=1$ the equation (13) gives us $|\Psi_{n}(1)|=2^{d}$;

     * •

For $r=2$, $x=1$, $y=1$ the equation (13) gives us
$|\Psi_{n}(1)|=3^{-d}|\Psi_{n}(15)|$. Since $15-2\cos(x)\geq 13$ for any
$x\in\mathbb{R}$,

$|\Psi_{n}(1)|=3^{-d}|\Psi_{n}(15)|=3^{-d}\prod\limits_{\begin{subarray}{c}1\leq
j<n/2\\\ \gcd(j,n)=1\end{subarray}}\left|15-2\cos\left(\frac{2\pi
j}{n}\right)\right|\geq 3^{-d}13^{d}>2^{d}.$

Thus, regardless of the value of $r$, we must have $|\Psi_{n}(1)|\geq 2^{d}$.
We claim that the opposite is always true, i.e., $|\Psi_{n}(1)|<2^{d}$.

To see that this is the case, first assume that $n\geq 14336$. Let
$\Phi_{n}(x)$ denote the $n$-th cyclotomic polynomial, and recall Lehmer’s
identity

$\Psi_{n}(z+z^{-1})=z^{-d}\Phi_{n}(z),$

which holds for every non-zero $z\in\mathbb{C}$. Then

$\Psi_{n}(1)=\Psi_{n}\left(2\cos\left(\frac{\pi}{3}\right)\right)=\Psi_{n}\left(e^{\frac{\pi
i}{3}}+e^{-\frac{\pi i}{3}}\right)=e^{-\frac{d\pi
i}{3}}\Phi_{n}\left(e^{\frac{\pi i}{3}}\right).$

As a consequence of this,

$|\Psi_{n}(1)|=\left|\Phi_{n}\left(e^{\frac{\pi i}{3}}\right)\right|\leq
L(\Phi_{n})\max\left\\{1,\left|e^{\frac{\pi
i}{3}}\right|\right\\}=L(\Phi_{n}),$

where $L(\Phi_{n})$ is the sum of absolute values of coefficients of
$\Phi_{n}$. By [4, Lemme 4.1], it is the case that $L(\Phi_{n})\leq
n^{\frac{\sigma_{0}(n)}{2}}$, where $\sigma_{0}(n)$ is the number of positive
divisors of $n$. We conclude that $|\Psi_{n}(1)|\leq
n^{\frac{\sigma_{0}(n)}{2}}$. By [13],

$\sigma_{0}(n)\leq n^{\frac{1.067}{\log\log n}}.$

By [16, Theorem 15],

$\varphi(n)>\frac{n}{5\log\log n}.$

Since $n^{\frac{1.067}{\log\log n}}\log n<\frac{\log 2}{2}\frac{n}{5\log\log
n}$ for all $n\geq 14336$, we find that

$\displaystyle|\Psi_{n}(1)|$ $\displaystyle\leq L(\Phi_{n})$
$\displaystyle\leq\exp\left(\frac{1}{2}\sigma_{0}(n)\log n\right)$
$\displaystyle\leq\exp\left(\frac{1}{2}n^{\frac{1.067}{\log\log n}}\log
n\right)$ $\displaystyle<\exp\left(\frac{\log 2}{2}\cdot\frac{n}{5\log\log
n}\right)$ $\displaystyle<\exp\left(\frac{\log 2}{2}\varphi(n)\right)$
$\displaystyle=2^{d}.$

It remains to check that $|\Psi_{n}(1)|<2^{d}$ for $16\leq n\leq 14335$. Since
$|\Psi_{n}(1)|=\left|\Phi_{n}\left(e^{\frac{\pi i}{3}}\right)\right|$, this
fact can be verified with the following PARI/GP code:

        for (n = 16, 14335,
         if ( abs(polcyclo(n, exp(Pi*I/3))) >= 2^(eulerphi(n)/2),
              print(n)
         )
    )

Since the above code does not print out any integers, we conclude that the
relation (12) is impossible, and so neither
$\operatorname{\operatorname{Aut}}\Psi_{n}$ nor
$\operatorname{\operatorname{Aut}}|\Psi_{n}|$ are isomorphic to
$\mathbf{D}_{6}$.

###  4.2 Case $d\geq 5$ and $n\not\equiv 0\pmod{4}$

If we let $S=\left(\begin{smallmatrix}-1&0\\\ 0&1\end{smallmatrix}\right)$,
then Part 2 of Lemma 3.2 tells us that
$\Psi_{2n}=(-1)^{\varphi(n)/2}(\Psi_{n})_{S}$ for any odd integer $n\geq 3$.
By Lemma 3.3,

$\operatorname{\operatorname{Aut}}\Psi_{2n}=S^{-1}(\operatorname{\operatorname{Aut}}\Psi_{n})S\quad\text{and}\quad\operatorname{\operatorname{Aut}}|\Psi_{2n}|=S^{-1}(\operatorname{\operatorname{Aut}}|\Psi_{n}|)S.$

In other words, we can derive $\operatorname{\operatorname{Aut}}\Psi_{2n}$ and
$\operatorname{\operatorname{Aut}}|\Psi_{2n}|$ from
$\operatorname{\operatorname{Aut}}\Psi_{n}$ and
$\operatorname{\operatorname{Aut}}|\Psi_{n}|$. Therefore, we may assume that
$n$ is odd.

Let $M=\frac{1}{m}\left(\begin{smallmatrix}s&u\\\ t&v\end{smallmatrix}\right)$
be an element of $\operatorname{\operatorname{Aut}}|\Psi_{n}|$, where
$s,t,u,v$ and $m=\sqrt{|sv-tu|}$ are integers such that $\gcd(s,t,u,v)=1$.
Then

$m^{d}\Psi_{n}(x,y)=\Psi_{n}(sx+uy,tx+vy),$

which means that the polynomials $m^{d}\Psi_{n}(x)$ and $\Psi_{n}(sx+u,tx+v)$
are equal. For an integer $\ell$, let
$\alpha_{\ell}=2\cos\left(\frac{2\pi\ell}{n}\right)$, and let
$\alpha=\alpha_{1}$. Then

$\displaystyle m^{d}\Psi_{n}(sx+u,tx+v)$
$\displaystyle=\pm\prod\limits_{\begin{subarray}{c}1\leq\ell<n/2\\\
\gcd(\ell,n)=1\end{subarray}}\left((sx+u)-\alpha_{\ell}(tx+v)\right)$
$\displaystyle=\pm\prod\limits_{\begin{subarray}{c}1\leq\ell<n/2\\\
\gcd(\ell,n)=1\end{subarray}}\left((-t\alpha_{\ell}+s)x-(v\alpha_{\ell}-u)\right)$
$\displaystyle=\pm\Psi_{n}(s,t)\prod\limits_{\begin{subarray}{c}1\leq\ell<n/2\\\
\gcd(\ell,n)=1\end{subarray}}\left(x-\frac{v\alpha_{\ell}-u}{-t\alpha_{\ell}+s}\right).$

Since $\Psi_{n}(x)$ and $\Psi_{n}(sx+u,tx+v)$ have the same roots, we conclude
that there exists some $j$ coprime to $n$ such that

$\alpha_{j}=\frac{v\alpha-u}{-t\alpha+s}.$

It follows from Lemma 3.5 that $s\neq 0$, $s=v$ and $t=u=0$. Since $s$, $t$,
$u$ and $v$ are integers such that $\gcd(s,t,u,v)=1$, we find that
$\gcd(s,v)=1$. This means that $s=v=\pm 1$ and $M\in\\{\pm
I\\}\subseteq\operatorname{\operatorname{Aut}}|\Psi_{n}|$. Therefore,
$\operatorname{\operatorname{Aut}}|\Psi_{n}|\subseteq\\{\pm I\\}$. Thus,

  * •

if $d$ is odd, then $\operatorname{\operatorname{Aut}}\Psi_{n}=\\{I\\}$ and
$\operatorname{\operatorname{Aut}}|\Psi_{n}|=\\{\pm I\\}$; and

  * •

if $d$ is even, then
$\operatorname{\operatorname{Aut}}\Psi_{n}=\operatorname{\operatorname{Aut}}|\Psi_{n}|=\\{\pm
I\\}$.

###  4.3 Case $d=3,4$ and $n\not\equiv 0\pmod{4}$

It remains to consider the cases $d=3,4$ and $n\not\equiv 0\pmod{4}$, which
correspond to $n\in\\{7,9,14,15,18,30\\}$.

The binary forms $\Psi_{7},\Psi_{9},\Psi_{14},\Psi_{18}$ have degree $3$ and
their discriminants are

$D_{\Psi_{7}}=D_{\Psi_{14}}=7^{2}\quad\text{and}\quad\quad
D_{\Psi_{9}}=D_{\Psi_{18}}=9^{2}.$

By Part 2 of [22, Theorem 3.1], if a binary cubic form
$F(x,y)=b_{3}x^{3}+b_{2}x^{2}y+b_{1}xy^{2}+b_{0}y^{3}$ is irreducible and
$D_{F}$ is a square of an integer, then
$\operatorname{\operatorname{Aut}}F=\langle\mathcal{N}_{q}\rangle$ is
isomorphic to $\bf C_{3}$. The matrix $\mathcal{N}_{q}$ which generates
$\operatorname{\operatorname{Aut}}F$ can be determined with the formula

$\mathcal{N}_{q}=\frac{1}{2D_{q}}\begin{pmatrix}b\sqrt{-3D_{q}}-D_{q}&2c\sqrt{-3D_{q}}\\\
-2a\sqrt{-3D_{q}}&-b\sqrt{-3D_{q}}-D_{q}\end{pmatrix},$

where $q(x,y)=ax^{2}+bxy+cy^{2}$ is the _Hessian_ of $F$ of discriminant
$D_{q}$, with coefficients

$a=b_{2}^{2}-3b_{3}b_{1},\quad b=b_{2}b_{1}-9b_{3}b_{0},\quad
c=b_{1}^{2}-3b_{2}b_{0}.$

In the case when $F=\Psi_{7}$, we have $b_{3}=1$, $b_{2}=1$, $b_{1}=-2$ and
$b_{0}=-1$. Thus $q(x,y)=7x^{2}+7xy+7y^{2}$, $D_{q}=-147$ and
$\mathcal{N}_{q}=\frac{1}{-294}\left(\begin{smallmatrix}294&294\\\
-294&0\end{smallmatrix}\right)=\left(\begin{smallmatrix}-1&-1\\\
1&0\end{smallmatrix}\right)$. Since
$\operatorname{\operatorname{Aut}}\Psi_{7}$ is a normal subgroup of
$\operatorname{\operatorname{Aut}}|\Psi_{7}|$ of index at most $2$,
$-I\in\operatorname{\operatorname{Aut}}|\Psi_{7}|$ and
$-I\notin\operatorname{\operatorname{Aut}}\Psi_{7}$, we find that
$\operatorname{\operatorname{Aut}}|\Psi_{7}|=\langle\mathcal{N}_{q},-I\rangle\cong\bf
D_{3}$. The automorphism groups of $\Psi_{9}$, $\Psi_{14}$ and $\Psi_{18}$ can
be determined analogously.

The binary forms $\Psi_{15}$ and $\Psi_{30}$ both have degree $4$. Using the
formula provided in [22, Section 4], we find that $\Psi_{15}$ has degree $6$
covariant

$F_{6}(x,y)=15(x^{2}-2xy+2y^{2})(x^{4}+6x^{3}y+6x^{2}y^{2}-4xy^{3}-4y^{4}).$

Now, we refer to a binary quadratic form $f$ as _rationally significant_ if it
is proportional over $\mathbb{C}$ to a quadratic form $g$ with integer
coefficients and $|D_{g}|$ is a square of an integer. Notice that
$f(x,y)=ax^{2}+bxy+cy^{2}$ with $a=1$, $b=-2$ and $c=2$ is a unique rationally
significant factor of $F_{6}$. Thus it follows from [22, Theorem 4.1] that
$\operatorname{\operatorname{Aut}}F$ is generated by $-I$ and

$U_{f}=\frac{1}{\sqrt{|D_{f}|}}\begin{pmatrix}b&2c\\\
-2a&-b\end{pmatrix}=\frac{1}{2}\begin{pmatrix}-2&4\\\
-2&2\end{pmatrix}=\begin{pmatrix}-1&2\\\ -1&1\end{pmatrix}.$

Since $U_{f}^{2}=-I$, we conclude that
$\operatorname{\operatorname{Aut}}\Psi_{15}=\langle U_{f}\rangle\cong\bf
C_{4}$.

It remains to determine $\operatorname{\operatorname{Aut}}|\Psi_{15}|$.
Suppose that there exists $A=\left(\begin{smallmatrix}s&u\\\
t&v\end{smallmatrix}\right)$ in $\operatorname{\operatorname{Aut}}|\Psi_{15}|$
such that $A\notin\operatorname{\operatorname{Aut}}\Psi_{15}$. If we let
$\alpha_{\ell}=2\cos\left(\frac{2\pi\ell}{n}\right)$ and put
$\alpha=\alpha_{1}$, then the roots of $\Psi_{15}(x)$ are $\alpha$,
$\alpha_{2}$, $\alpha_{4}$ and $\alpha_{7}$. Since
$A\in\operatorname{\operatorname{Aut}}|\Psi_{15}|$, there must exist
$j\in\\{2,4,7\\}$ such that $\alpha_{j}=\frac{v\alpha-u}{-t\alpha+s}$. Since
$U_{f}\in\operatorname{\operatorname{Aut}}\Psi_{15}$, we find that
$\alpha_{4}=\frac{\alpha-2}{\alpha-1}$ and
$\alpha_{7}=\frac{\alpha_{2}-2}{\alpha_{2}-1}$. We also know that
$\alpha_{2}=\alpha^{2}-2$, so $\alpha_{7}=\frac{\alpha^{2}-4}{\alpha^{2}-3}$.
Since $\deg\alpha=4$, it is straightforward to verify that there are no
rational $s,t,u,v$ such that
$\frac{v\alpha-u}{-t\alpha+s}=\alpha_{2}=\alpha^{2}-2$ or
$\frac{v\alpha-u}{-t\alpha+s}=\alpha_{7}=\frac{\alpha^{2}-4}{\alpha^{2}-3}$.
In turn, the rationals $s,t,u,v$ that satisfy
$\frac{v\alpha-u}{-t\alpha+s}=\alpha_{4}=\frac{\alpha-2}{\alpha-1}$ correspond
to $\pm U_{f}$. Thus it must be the case that
$\operatorname{\operatorname{Aut}}|\Psi_{15}|=\operatorname{\operatorname{Aut}}\Psi_{15}=\langle
U_{f}\rangle$. Since $\Psi_{30}(x,y)=\Psi_{15}(-x,y)$, we can easily determine
$\operatorname{\operatorname{Aut}}\Psi_{30}$ and
$\operatorname{\operatorname{Aut}}|\Psi_{30}|$.

##  5 Automorphisms of $T_{n}(x,y)$ and $U_{n}(x,y)$

In this section we prove Theorem 1.3. Let $T_{n}(x,y)$ and $U_{n}(x,y)$ denote
the homogenizations of the $n$-th Chebyshev polynomials of first and second
kinds, respectively. Define

$\tilde{U}_{n}(x,y)=U_{n-1}\left(\frac{x}{2},y\right)\quad\text{and}\quad\tilde{V}_{n}(x,y)=2T_{n}\left(\frac{x}{2},y\right).$

Then for $n\geq 1$ we have

$\tilde{U}_{n}(x,y)=x^{\frac{1-(-1)^{n-1}}{2}}\prod\limits_{\begin{subarray}{c}d\mid
2n\\\ d\notin\\{1,2,4\\}\end{subarray}}\Psi_{d}(x,y)$ (14)

and

$\tilde{V}_{n}(x,y)=x^{\frac{1-(-1)^{n}}{2}}\prod\limits_{\begin{subarray}{c}d\mid
n\\\ 1\leq d<n\\\ \textrm{$d$ is odd}\end{subarray}}\Psi_{4n/d}(x,y).$ (15)

Note that all binary forms in the above factorizations are irreducible. We
will prove the following lemma, which implies Theorem 1.3.

######  Lemma 5.1.

Let $n$ be an integer such that $n\geq 3$.

  1. 1.

If $n$ is odd, then

$\operatorname{\operatorname{Aut}}\tilde{V}_{n}=\left\langle\begin{pmatrix}1&0\\\
0&-1\end{pmatrix}\right\rangle\cong\bm{C}_{2},\quad\operatorname{\operatorname{Aut}}|\tilde{V}_{n}|=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}1&0\\\
0&-1\end{pmatrix}\right\rangle\cong\bm{D}_{2}.$

  2. 2.

If $n$ is even, then

$\operatorname{\operatorname{Aut}}\tilde{V}_{n}=\operatorname{\operatorname{Aut}}|\tilde{V}_{n}|=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}1&0\\\
0&-1\end{pmatrix}\right\rangle\cong\bm{D}_{2}.$

  3. 3.

If $n$ is odd, then

$\operatorname{\operatorname{Aut}}\tilde{U}_{n}=\operatorname{\operatorname{Aut}}|\tilde{U}_{n}|=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}1&0\\\
0&-1\end{pmatrix}\right\rangle\cong\bm{D}_{2}.$

  4. 4.

If $n$ is even, then

$\operatorname{\operatorname{Aut}}\tilde{U}_{n}=\left\langle\begin{pmatrix}1&0\\\
0&-1\end{pmatrix}\right\rangle\cong\bm{C}_{2},\quad\operatorname{\operatorname{Aut}}|\tilde{U}_{n}|=\left\langle\begin{pmatrix}-1&0\\\
0&1\end{pmatrix},\begin{pmatrix}1&0\\\
0&-1\end{pmatrix}\right\rangle\cong\bm{D}_{2}.$

Let us now see why Theorem 1.3 follows from Lemma 5.1. Note that

$\tilde{U}_{n}=(U_{n-1})_{S}\quad\text{and}\quad\tilde{V}_{n}=2(T_{n})_{S},$

where $S=\left(\begin{smallmatrix}1/2&0\\\ 0&1\end{smallmatrix}\right)$. Note
that $S\left(\begin{smallmatrix}-1&0\\\
0&1\end{smallmatrix}\right)S^{-1}=\left(\begin{smallmatrix}-1&0\\\
0&1\end{smallmatrix}\right)$ and $S\left(\begin{smallmatrix}1&0\\\
0&-1\end{smallmatrix}\right)S^{-1}=\left(\begin{smallmatrix}1&0\\\
0&-1\end{smallmatrix}\right)$. By Lemma 3.3,

$\begin{array}[]{r l
l}\operatorname{\operatorname{Aut}}U_{n-1}&=S(\operatorname{\operatorname{Aut}}\tilde{U}_{n})S^{-1}&=\operatorname{\operatorname{Aut}}\tilde{U}_{n},\\\
\operatorname{\operatorname{Aut}}|U_{n-1}|&=S(\operatorname{\operatorname{Aut}}|\tilde{U}_{n}|)S^{-1}&=\operatorname{\operatorname{Aut}}|\tilde{U}_{n}|,\\\
\operatorname{\operatorname{Aut}}T_{n}&=S(\operatorname{\operatorname{Aut}}\tilde{V}_{n})S^{-1}&=\operatorname{\operatorname{Aut}}\tilde{V}_{n},\\\
\operatorname{\operatorname{Aut}}|T_{n}|&=S(\operatorname{\operatorname{Aut}}|\tilde{V}_{n}|)S^{-1}&=\operatorname{\operatorname{Aut}}|\tilde{V}_{n}|.\end{array}$

This concludes the proof of Theorem 1.3. Before we proceed to the proof of
Lemma 5.1, we need to establish one supplementary result.

######  Lemma 5.2.

Let $F$ denote either $\tilde{U}_{n}$ or $\tilde{V}_{n}$, with $\deg F\geq 7$.
Suppose that $\Psi_{k}\mid F$ and $\deg\Psi_{k}\geq 5$. Then
$\operatorname{\operatorname{Aut}}|F|\subseteq\operatorname{\operatorname{Aut}}|\Psi_{k}|$.

###### Proof.

Let $M=\left(\begin{smallmatrix}s&u\\\ t&v\end{smallmatrix}\right)$ be an
element of $\operatorname{\operatorname{Aut}}|F|$. Then

$D(F_{M})=(\det M)^{\deg F(\deg F-1)}D(F).$

Since $F_{M}=\pm F$, $D(F)\neq 0$ and $\deg F>1$, we see that $\det M\neq 0$.

Suppose that the binary forms $(\Psi_{k})_{M}$ and $\Psi_{k}$ are distinct.
Since $\Psi_{k}$ is irreducible and $\det M\neq 0$, it must be the case that
$(\Psi_{k})_{M}$ is also irreducible. At this point, we consider two cases.

_Case 1._ Suppose that $(\Psi_{k})_{M}=r\Psi_{k}$ for some non-zero
$r\in\mathbb{Q}$. Then

$(\det
M)^{\deg\Psi_{k}(\deg\Psi_{k}-1)}D(\Psi_{k})=D\left((\Psi_{k})_{M}\right)=D(r\Psi_{k})=r^{2(\deg\Psi_{k}-1)}D(\Psi_{k}).$

Since $D(\Psi_{k})\neq 0$, it must be the case that

$(\det M)^{\deg\Psi_{k}(\deg\Psi_{k}-1)}=r^{2(\deg\Psi_{k}-1)}.$

Since $\det M=\pm 1$, we see that $r^{2(\deg\Psi_{k}-1)}=1$. Since
$r\in\mathbb{Q}$, we conclude that $r\in\\{\pm 1\\}$, so
$M\in\operatorname{\operatorname{Aut}}|\Psi_{k}|$.

_Case 2._ Suppose that $(\Psi_{k})_{M}$ is not a rational multiple of
$\Psi_{k}$. Since $\Psi_{k}\mid F$ and the greatest common divisors of the
coefficients of $F$ (known as the _content_ of $F$) is equal to $1$, there
exists some $H(x,y)\in\mathbb{Z}[x,y]$ such that $F=\Psi_{k}H$. Since
$M\in\operatorname{\operatorname{Aut}}|F|$, we have

$\pm F=F_{M}=(\Psi_{k}H)_{M}=(\Psi_{k})_{M}H_{M},$

which means that $(\Psi_{k})_{M}\mid F$ in $\mathbb{Q}[x,y]$. Since

  1. a)

$F$ factors as in (14) or in (15);

  2. b)

both $\Psi_{k}$ and $(\Psi_{k})_{M}$ are irreducible; and

  3. c)

$\deg(\Psi_{k})_{M}=\deg\Psi_{k}>1$,

we see that $(\Psi_{k})_{M}=r\Psi_{\ell}$ for some non-zero $r\in\mathbb{Q}$
and $\ell\in\mathbb{N}$. Furthermore, since $(\Psi_{k})_{M}$ is not a rational
multiple of $\Psi_{k}$, it must be the case that $k\neq\ell$. Since
$(\Psi_{\ell})_{M^{-1}}=r^{-1}\Psi_{k}$, without loss of generality we may
assume that $k<\ell$.

Now, since $(\Psi_{k})_{M}=r\Psi_{\ell}$, the polynomials
$rm^{\deg\Psi_{k}}\Psi_{\ell}(x)$ and $\Psi_{k}(sx+u,tx+v)$ are equal. In
particular, their roots are the same, which means that

$2\cos\left(\frac{2\pi
q}{\ell}\right)=\frac{2\cos\left(\frac{2\pi}{k}\right)v-u}{-2\cos\left(\frac{2\pi}{k}\right)t+s}$

for some integer $q$ coprime to $\ell$. Therefore, $2\cos\left(\frac{2\pi
q}{\ell}\right)\in\mathbb{Q}\left(2\cos\left(\frac{2\pi}{k}\right)\right)$. By
Lemma 3.1, the Galois group of
$\mathbb{Q}\left(2\cos\left(\frac{2\pi}{n}\right)\right)$ is Abelian.
Consequently, all the conjugates of $2\cos\left(\frac{2\pi q}{\ell}\right)$,
including $2\cos\left(\frac{2\pi}{\ell}\right)$, belong to
$\mathbb{Q}\left(2\cos\left(\frac{2\pi}{k}\right)\right)$, so
$\mathbb{Q}\left(2\cos\left(\frac{2\pi}{\ell}\right)\right)\subseteq\mathbb{Q}\left(2\cos(\frac{2\pi}{k})\right)$.
Since $M$ is invertible, we conclude that
$\mathbb{Q}\left(2\cos\left(\frac{2\pi}{k}\right)\right)=\mathbb{Q}\left(2\cos\left(\frac{2\pi}{\ell}\right)\right)$.
Since $k<\ell$ and $k,\ell\notin\\{1,2,3,4,6\\}$, it follows from Lemma 3.7
that $k$ is odd and $\ell=2k$. Therefore,

$2\cos\left(\frac{\pi
q}{k}\right)=\frac{2\cos\left(\frac{2\pi}{k}\right)v-u}{-2\cos\left(\frac{2\pi}{k}\right)t+s}.$

Since $k$ is odd and $q$ is coprime to $2k$, it must be the case that $q$ is
odd. Also,

$2\cos\left(\frac{\pi q}{k}\right)=-2\cos\left(\pi-\frac{\pi
q}{k}\right)=-2\cos\left(\frac{2\pi m}{k}\right),$

where $m=\frac{k-q}{2}$ is an integer coprime to $k$. Hence

$2\cos\left(\frac{2\pi
m}{k}\right)=\frac{-2\cos\left(\frac{2\pi}{k}\right)v+u}{2\cos\left(\frac{2\pi}{k}\right)t-s}.$

Since $\deg\Psi_{k}\geq 5$, it follows from Lemma 3.5 that $s\neq 0$, $s=v$
and $t=u=0$. Since $s$, $t$, $u$ and $v$ are integers such that
$\gcd(s,t,u,v)=1$, we find that $\gcd(s,v)=1$. This means that $s=v=\pm 1$ and
$M\in\\{\pm I\\}\subseteq\operatorname{\operatorname{Aut}}|\Psi_{k}|$. ∎

We will now turn our attention to the proof of the main result of this
section.

###### Proof of Lemma 5.1..

For $n\geq 7$ consider the binary form $\tilde{V}_{n}(x,y)$. Then
$\Psi_{4n}\mid\tilde{V}_{n}$ and $\deg\Psi_{4n}=\varphi(4n)/2\geq 5$. It
follows from Lemma 5.2 that
$\operatorname{\operatorname{Aut}}|\tilde{V}_{n}|\subseteq\operatorname{\operatorname{Aut}}|\Psi_{4n}|$.
By Part 7 of Theorem 1.1, $\operatorname{\operatorname{Aut}}|\Psi_{4n}|=\\{\pm
I,\pm M\\}$, where $M=\left(\begin{smallmatrix}1&0\\\
0&-1\end{smallmatrix}\right)$. We consider two cases.

_Case 1._ If $n$ is even, then there exists a binary form $G(x,y)$ such that
$\tilde{V}_{n}(x,y)=G(x^{2},y^{2})$. Then $(\tilde{V}_{n})_{A}=\tilde{V}_{n}$
for any $A\in\operatorname{\operatorname{Aut}}|\Psi_{4n}|$. Therefore,

$\operatorname{\operatorname{Aut}}\tilde{V}_{n}=\operatorname{\operatorname{Aut}}|\tilde{V}_{n}|=\\{\pm
I,\pm M\\}.$

_Case 2._ If $n$ is odd, then there exists a binary form $G(x,y)$ such that
$\tilde{V}_{n}(x,y)=xG(x^{2},y^{2})$. Then
$(\tilde{V}_{n})_{-I}=-\tilde{V}_{n}$, $(\tilde{V}_{n})_{M}=\tilde{V}_{n}$ and
$(\tilde{V}_{n})_{-M}=-\tilde{V}_{n}$. Therefore,

$\operatorname{\operatorname{Aut}}\tilde{V}_{n}=\\{I,M\\},\quad\operatorname{\operatorname{Aut}}|\tilde{V}_{n}|=\\{\pm
I,\pm M\\}.$

Next, for $n\notin\\{4,5,6,7,8,9,10,12,15\\}$ consider the binary form
$\tilde{U}_{n}(x,y)$. Then $\Psi_{2n}\mid\tilde{U}_{n}$ and
$\deg\Psi_{2n}=\varphi(2n)/2\geq 5$. It follows from Lemma 5.2 that
$\operatorname{\operatorname{Aut}}|\tilde{V}_{n}|\subseteq\operatorname{\operatorname{Aut}}|\Psi_{2n}|$.
By Part 7 of Theorem 1.1, $\operatorname{\operatorname{Aut}}|\Psi_{2n}|=\\{\pm
I,\pm M\\}$, where $M=\left(\begin{smallmatrix}1&0\\\
0&-1\end{smallmatrix}\right)$. We consider two cases.

_Case 1._ If $n$ is even, then there exists a binary form $G(x,y)$ such that
$\tilde{U}_{n}(x,y)=xG(x^{2},y^{2})$. Then
$(\tilde{U}_{n})_{-I}=-\tilde{U}_{n}$, $(\tilde{U}_{n})_{M}=\tilde{U}_{n}$ and
$(\tilde{U}_{n})_{-M}=-\tilde{U}_{n}$. Therefore,

$\operatorname{\operatorname{Aut}}\tilde{U}_{n}=\\{I,M\\},\quad\operatorname{\operatorname{Aut}}|\tilde{U}_{n}|=\\{\pm
I,\pm M\\}.$

_Case 2._ If $n$ is odd, then there exists a binary form $G(x,y)$ such that
$\tilde{U}_{n}(x,y)=G(x^{2},y^{2})$. Then $(\tilde{U}_{n})_{A}=\tilde{U}_{n}$
for any $A\in\operatorname{\operatorname{Aut}}|\Psi_{2n}|$. Therefore,

$\operatorname{\operatorname{Aut}}\tilde{U}_{n}=\operatorname{\operatorname{Aut}}|\tilde{U}_{n}|=\\{\pm
I,\pm M\\}.$

It remains to compute the automorphism groups for thirteen exceptional binary
forms whose factors have degree at most $4$:

$\tilde{V}_{3},\quad\tilde{V}_{4},\quad\tilde{V}_{5},\quad\tilde{V}_{6},\quad\tilde{U}_{4},\quad\tilde{U}_{5},\quad\tilde{U}_{6},\quad\tilde{U}_{7},\quad\tilde{U}_{8},\quad\tilde{U}_{9},\quad\tilde{U}_{10},\quad\tilde{U}_{12},\quad\tilde{U}_{15}.$

Notice that $\tilde{V}_{4}=\Psi_{16}$, so the result for this binary form
follows from Part 7 of Theorem 1.1. The remaining calculations can be done
manually. We will demonstrate them for $\tilde{V}_{3}$, $\tilde{U}_{4}$ and
$\tilde{U}_{15}$, as the other cases can be established analogously. In what
follows we implicitly use the fact that, for any invertible linear fractional
transformation $\mu(z)=\frac{vz-u}{-tz+s}$, with $s,t,u,v\in\mathbb{Z}$, it is
the case that $\deg\alpha=\deg\mu(\alpha)$ for any algebraic number $\alpha$.

Consider $\tilde{V}_{3}(x,y)=x^{3}-3xy^{2}$. Let
$A=\frac{1}{m}\left(\begin{smallmatrix}s&u\\\ t&v\end{smallmatrix}\right)$ be
an element of $\operatorname{\operatorname{Aut}}|\tilde{V}_{3}|$, where
$s,t,u,v$ and $m=\sqrt{|sv-tu|}$ are integers such that $\gcd(s,t,u,v)=1$.
Then

$m^{3}\tilde{V}_{3}(x,y)=\tilde{V}_{3}(sx+uy,tx+vy),$

which means that the polynomials $m^{3}\tilde{V}_{3}(x,1)$ and
$\tilde{V}_{3}(sx+u,tx+v)$ are equal. Since the roots of $\tilde{V}_{3}(x,1)$
are $0$, $\sqrt{3}$ and $-\sqrt{3}$, it must be the case that either

$0=\frac{v0-u}{-t0+s},\quad\sqrt{3}=\frac{v\sqrt{3}-u}{-t\sqrt{3}+s},\quad-\sqrt{3}=\frac{v(-\sqrt{3})-u}{-t(-\sqrt{3})+s}$

or

$0=\frac{v0-u}{-t0+s},\quad-\sqrt{3}=\frac{v\sqrt{3}-u}{-t\sqrt{3}+s},\quad\sqrt{3}=\frac{v(-\sqrt{3})-u}{-t(-\sqrt{3})+s}.$

In either case from the first equation we find that $u=0$, and from the second
equation we find that $t=0$. Finally, from the third equation we find that
$v/s=\pm 1$. Since $\gcd(s,t,u,v)=1$, the integers $s$ and $v$ are coprime, so
$s=\pm 1$ and $v=\pm 1$. Thus it must be the case that
$\operatorname{\operatorname{Aut}}|\tilde{V}_{3}|=\\{\pm I,\pm M\\}$, where
$M=\left(\begin{smallmatrix}1&0\\\ 0&-1\end{smallmatrix}\right)$. In view of
this we also have $\operatorname{\operatorname{Aut}}\tilde{V}_{3}=\\{I,M\\}$.
An analogous result can be established for the binary form
$\tilde{U}_{4}(x,y)=x^{3}-2xy^{2}$.

Finally, consider

$\tilde{U}_{15}(x,y)=(x-1)(x+1)(x^{2}-x-1)(x^{2}+x-1)(x^{4}-x^{3}-4x^{2}+4x+1)(x^{4}+x^{3}-4x^{2}-4x+1).$

Let $A=\frac{1}{m}\left(\begin{smallmatrix}s&u\\\ t&v\end{smallmatrix}\right)$
be an element of $\operatorname{\operatorname{Aut}}|\tilde{U}_{15}|$, where
$s,t,u,v$ and $m=\sqrt{|sv-tu|}$ are integers such that $\gcd(s,t,u,v)=1$.
Then

$m^{14}\tilde{U}_{15}(x,y)=\tilde{U}_{15}(sx+uy,tx+vy),$

which means that the polynomials $m^{14}\tilde{U}_{15}(x,1)$ and
$\tilde{U}_{15}(sx+u,tx+v)$ are equal. Notice that the rational roots of
$\tilde{U}_{15}(x,1)$ are given by $\pm 1$, while the roots of degree $2$ are
given by $\frac{\pm 1\pm\sqrt{5}}{2}$. Thus there are eight possible subcases
to consider.

  1. (i)

$1=\frac{v1-u}{-t1+s},\quad-1=\frac{v(-1)-u}{-t(-1)+s},\quad\frac{1+\sqrt{5}}{2}=\frac{v\left(\frac{1+\sqrt{5}}{2}\right)-u}{-t\left(\frac{1+\sqrt{5}}{2}\right)+s}$.

In this case, we obtain a homogeneous system of $3$ linear equations in $4$
unknowns:

$\begin{array}[]{r l}s-t+u-v&=0\\\ -s-t+u+v&=0\\\
\frac{1+\sqrt{5}}{2}s+\frac{-3-\sqrt{5}}{2}t+u+\frac{-1-\sqrt{5}}{2}v&=0\end{array}$

Since $s$, $t$, $u$ and $v$ are integers, this system is equivalent to

$\begin{array}[]{r l}s-t+u-v&=0\\\ -s-t+u+v&=0\\\
\frac{1}{2}s-\frac{3}{2}t+u-\frac{1}{2}v&=0\\\
\frac{1}{2}s-\frac{1}{2}t-\frac{1}{2}v&=0\end{array}$

Solving this system yields $s=v$ and $u=v=0$. Since $\gcd(s,v)=1$, we conclude
that $s=v=\pm 1$.

  2. (ii)

$-1=\frac{v1-u}{-t1+s},\quad
1=\frac{v(-1)-u}{-t(-1)+s},\quad\frac{1+\sqrt{5}}{2}=\frac{v\left(\frac{1+\sqrt{5}}{2}\right)-u}{-t\left(\frac{1+\sqrt{5}}{2}\right)+s}$.

In this case, $s=t=u=v=0$, which is impossible.

  3. (iii)

$1=\frac{v1-u}{-t1+s},\quad-1=\frac{v(-1)-u}{-t(-1)+s},\quad\frac{1-\sqrt{5}}{2}=\frac{v\left(\frac{1+\sqrt{5}}{2}\right)-u}{-t\left(\frac{1+\sqrt{5}}{2}\right)+s}$.

In this case, $s=t=u=v=0$, which is impossible.

  4. (iv)

$-1=\frac{v1-u}{-t1+s},\quad
1=\frac{v(-1)-u}{-t(-1)+s},\quad\frac{1-\sqrt{5}}{2}=\frac{v\left(\frac{1+\sqrt{5}}{2}\right)-u}{-t\left(\frac{1+\sqrt{5}}{2}\right)+s}$.

In this case, $s=v=0$ and $t=-u$. Since $\gcd(t,u)=1$, we conclude that $t=\pm
1$ and $u=\mp 1$.

  5. (v)

$1=\frac{v1-u}{-t1+s},\quad-1=\frac{v(-1)-u}{-t(-1)+s},\quad\frac{-1+\sqrt{5}}{2}=\frac{v\left(\frac{1+\sqrt{5}}{2}\right)-u}{-t\left(\frac{1+\sqrt{5}}{2}\right)+s}$.

In this case, $s=v=0$ and $t=u$. Since $\gcd(t,u)=1$, we conclude that
$t=u=\pm 1$.

  6. (vi)

$-1=\frac{v1-u}{-t1+s},\quad
1=\frac{v(-1)-u}{-t(-1)+s},\quad\frac{-1+\sqrt{5}}{2}=\frac{v\left(\frac{1+\sqrt{5}}{2}\right)-u}{-t\left(\frac{1+\sqrt{5}}{2}\right)+s}$.

In this case, $s=t=u=v=0$, which is impossible.

  7. (vii)

$1=\frac{v1-u}{-t1+s},\quad-1=\frac{v(-1)-u}{-t(-1)+s},\quad\frac{-1-\sqrt{5}}{2}=\frac{v\left(\frac{1+\sqrt{5}}{2}\right)-u}{-t\left(\frac{1+\sqrt{5}}{2}\right)+s}$.

In this case, $s=t=u=v=0$, which is impossible.

  8. (viii)

$-1=\frac{v1-u}{-t1+s},\quad
1=\frac{v(-1)-u}{-t(-1)+s},\quad\frac{-1-\sqrt{5}}{2}=\frac{v\left(\frac{1+\sqrt{5}}{2}\right)-u}{-t\left(\frac{1+\sqrt{5}}{2}\right)+s}$.

In this case, $s=-v$ and $t=u=0$. Since $\gcd(s,v)=1$, we conclude that $s=\pm
1$ and $v=\mp 1$.

We conclude that $A\in\left\\{\pm I,\pm M,\pm N,\pm MN\right\\}$, where
$M=\left(\begin{smallmatrix}1&0\\\ 0&-1\end{smallmatrix}\right)$ and
$N=\left(\begin{smallmatrix}0&1\\\ -1&0\end{smallmatrix}\right)$. By checking
each possible value of $A$ we find that

$\operatorname{\operatorname{Aut}}\tilde{U}_{15}=\operatorname{\operatorname{Aut}}|\tilde{U}_{15}|=\\{\pm
I,\pm M\\}.$

∎

## Acknowledgements

The author is grateful to his PhD advisor, Prof. Cameron L. Stewart, who
proposed to explore this exciting subject, to Prof. Étienne Fouvry for
correcting the statement of Lemma 3.7 and his other suggestions, as well as to
the anonymous reviewer for their excellent advice on how to improve the
article.

## References

  * [1] M. A. Bean, _The practical computation of areas associated with binary quartic forms_ , Math. Comp. 66 (219), pp. 1269–1293, 1997.
  * [2] M. Demirci and I. N. Cangül, _The constant term of the minimal polynomial $\cos\left(\frac{2\pi}{n}\right)$ over $\mathbb{Q}$_, Fixed Point Theory and Applications 2013:77, 2013.
  * [3] D. S. Dummit and R. M. Foote, _Abstract Algebra_. John Wiley and Sons, Inc., 2004.
  * [4] E. Fouvry and M. Waldschmidt, _Sur la représentation des entiers par les formes cyclotomiques de grand degré_. Bulletin de la SMF 148 (2), pp. 253–282, 2020.
  * [5] C. Hooley, _On binary cubic forms_ , J. Reine Angew. Math. 226, pp. 30–87, 1967.
  * [6] C. Hooley, _On binary cubic forms, II_ , J. Reine Angew. Math. 521, pp. 185–240, 2000.
  * [7] H. Iwaniec, _On the problem of Jacobsthal_ , Demonstration Mathematica 11 (1), pp. 225–231, 1978.
  * [8] H. Kanold, _Über eine zahlentheoretische Funktion von Jacobsthal._ Math. Ann., 170 (4), pp. 314–326, 1967.
  * [9] D. H. Lehmer, _An extended theory of Lucas’ functions_ , Annals of Math. 31, pp. 419–448, 1930.
  * [10] J. C. Mason and D. C. Handscomb, _Chebyshev Polynomials_. Chapman & Hal/CRC, 2003.
  * [11] A. Mosunov, _On the area of the fundamental region of a binary Form associated with algebraic trigonometric quantities_ , Mathematika 67 (2), pp. 532–551, 2021.
  * [12] M. Newman, _Integral Matrices_. Volume 45 of _Pure and Appl. Math._ Academic Press, New York, 1972.
  * [13] J. L. Nicolas and G. Robin, _Majorations explicites pour le nombre de diviseurs de $n$_. Bull. Can. Math. 26, pp. 485–492, 1983.
  * [14] I. Niven, _Irrational Numbers_. The Mathematical Association of America, New Jersey, 1956.
  * [15] G. Robin, _Estimation de la fonction de Tchebychef $\theta$ sur le $k$-ième nombre premier et grandes valeurs de la fonction $\omega(n)$ nombre de diviseurs premiers de $n$_, Acta Arith. 42 (4), pp. 367–389, 1983.
  * [16] J. B. Rosser and L. Schoenfeld, _Approximate formulas for some functions of prime numbers_ , Illinois J. Math 6 (1), pp. 64–94, 1962.
  * [17] C. L. Stewart, _On the number of solutions of polynomial congruences and Thue equations_ , J. of the Amer. Math. Soc. 4 (4), pp. 793–835, 1991.
  * [18] C. L. Stewart and S. Y. Xiao, _On the representation of integers by binary forms_ , Math. Ann. 375, pp. 133–163, 2019.
  * [19] A. Thue, _Über Annäherungwerte algebraischer Zahlen_ , Journal für die reine und angewandte Mathematik 135, pp. 284–305, 1909.
  * [20] R. C. Vaughan, _On the order of magnitude of Jacobsthal’s function_ , Proc. Edinburgh Math. Soc. 20, pp. 329–331, 1976–77.
  * [21] W. Watkins and J. Zeitlin, _The minimal polynomial of $\cos(2\pi/n)$_, the Amer. Math. Monthly, pp. 471–474, 1993.
  * [22] S. Y. Xiao, _On binary cubic and quartic forms_ , Journal de Théorie des Nombres de Bordeaux, 31 (2), pp. 323–341, 2019.


# On the differential operators of odd order with PT-symmetric periodic matrix
coefficients

O. A. Veliev
Department of Mechanical Engineering, Dogus University, Istanbul, Turkey
e-mail<EMAIL_ADDRESS>
###### Abstract

In this paper we investigate the spectrum of the differential operators
generated by the ordinary differential expression of odd order with PT-
symmertic periodic matrix coefficients.

Key Words: Differential operator, PT-symmetric coefficients, Real spectrum.

AMS Mathematics Subject Classification: 34L05, 34L20.

In this paper, we consider the spectrum $\sigma(T)$ of the differential
operator $T$ generated in the space $L_{2}^{m}(-\infty,\infty)$ by the
differential expression

$(i)^{n}y^{(n)}+(i)^{n-1}P_{1}y^{(n-1)}+(i)^{n-2}P_{2}y^{(n-2)}+...+P_{n}y,$
(1)

where $P_{k}$ for $k=1,2,...,n$ are the $m\times m$ matrices with the complex-
valued PT-symmetric periodic entries

$p_{k,i,j}\left(x+1\right)=p_{k,i,j}\left(x\right),\text{
}p_{k,i,j}\left(-x\right)=\overline{p_{k,i,j}\left(x\right)}$ (2)

and $y=(y_{1},y_{2},...,y_{m})^{T}$ is a vector-valued function. Here
$L_{2}^{m}(a,b)$ for $-\infty\leq a<b\leq\infty$ is the space of the vector-
valued functions $f=\left(f_{1},f_{2},...,f_{m}\right)^{T}$ with the norm
$\left\|\cdot\right\|_{(a,b)}$ and inner product $(\cdot,\cdot)_{(a,b)}$
defined by

$\left\|f\right\|_{(a,b)}^{2}=\int_{a}^{b}\left|f\left(x\right)\right|^{2}dx,\text{
}(f,g)_{(a,b)}=\int_{a}^{b}\left\langle
f\left(x\right),g\left(x\right)\right\rangle dx,$

where $\left|\cdot\right|$ and $\left\langle\cdot,\cdot\right\rangle$ are the
norm and inner product in $\mathbb{C}^{m}.$ We prove that if $m$ and $n$ are
the odd numbers, then $\mathbb{R}\subset\sigma(T).$

Note that there are a large number of papers for the scalar case $m=1$ and
$n=2$, namely for the Schrödinger operator (see the monographs [1, Chapters 4
and 6] and [5, Chapters 3 and 5] and the papers they refer to). However, as
far as I know, only paper [6] is devoted to the vector Schrödinger operator
with a PT-symmetric periodic matrix potential, where $n=2$. The main result of
this paper concerns the spectrum for the case when $n$ and $m$ are odd
numbers. Moreover, this result and the method used in this paper are
completely different from the results and methods of those papers. That is
why, and in order not to deviate from the purpose of this very short paper, we
do not discuss them here.

First, using (1) and (2) we prove the following theorem about the solutions of
the equation

$(i)^{n}y^{(n)}+(i)^{n-1}P_{1}y^{(n-1)}+(i)^{n-2}P_{2}y^{(n-2)}+...+P_{n}y=\lambda
y,$ (3)

where $n$ and $m$ are arbitrary positive integers.

###### Theorem 1

$(a)$ If $\Psi$ is a solution of (3), then the function $\Phi$ defined by
$\Phi(x,\lambda)=\overline{\Psi(-x,\lambda)}$ is a solution of

$(i)^{n}y^{(n)}+(i)^{n-1}P_{1}y^{(n-1)}+(i)^{n-2}P_{2}y^{(n-2)}+...+P_{n}y=\overline{\lambda}y.$
(4)

$(b)$ Suppose that $\lambda$ is a real number. Let $S_{+}(\lambda)$ and
$S_{-}(\lambda)$ be the spaces of the solutions of (3) belonging to
$L_{2}^{m}(0,\infty)$ and $L_{2}^{m}(-\infty,0)$ respectively. Then the
mapping $A$ defined by $A:f(x)\rightarrow\overline{f(-x)}$ is an antilinear
isometric bijection from $S_{+}(\lambda)$ onto $S_{-}(\lambda).$

Proof. $(a)$ If $\Psi(x,\lambda)$ is a solution of (3) then the equality

$(i)^{n}\Psi^{(n)}(x)+(i)^{n-1}P_{1}(x)\Psi^{(n-1)}(x)+...+P_{n}(x)\Psi(x)=\lambda\Psi(x)$

holds for all $x\in(-\infty,\infty).$ In this equality replacing $x$ by $-x$,
taking the conjugate and using (2) we obtain

$(-i)^{n}\overline{\Psi^{(n)}(-x)}+(-i)^{n-1}P_{1}(x)\overline{\Psi^{(n-1)}(-x)}+...+P_{n}(x)\overline{\Psi(-x)}=\overline{\lambda}\overline{\Psi(-x)}.$

On the other hand, it follows from the definition of $\Phi$ that
$\Phi^{(k)}(x)=(-1)^{k}\overline{\Psi^{(k)}(-x)}.$ Therefore, we have

$(i)^{n}\Phi^{(n)}(x)+(i)^{n-1}P_{1}(x)\Phi^{(n-1)}(x)+...+P_{n}(x)\Phi(x)=\overline{\lambda}\Phi(x).$

It means that $\Phi$ is the solution of (4).

$(b)$ If $\lambda$ is a real number, then it follows from $(a)$ that if
$\Psi(x,\lambda)$ is a solution of (3), then $\Phi(x,\lambda)$ is also a
solution of (3). Moreover, using the equality $\Phi(x)=\overline{\Psi(-x)}$
and the substituting $u=-x$ we get

${\displaystyle\int\limits_{-a}^{0}}\left|\Phi(x)\right|^{2}dx={\displaystyle\int\limits_{-a}^{0}}\left|\Psi(-x)\right|^{2}dx={\displaystyle\int\limits_{0}^{a}}\left|\Psi(u)\right|^{2}du$
(5)

for any $0<a<\infty.$ If $\Psi\in S_{+}(\lambda),$ then using (5) and letting
$a$ tend to infinity we obtain that $\Phi\in S_{-}(\lambda).$ Thus for any
solution of (3) belonging to $L_{2}^{m}(0,\infty)$ there corresponds a
solution $\Phi$ of (3) belonging to $L_{2}^{m}(-\infty,0).$ In the same way we
prove that, for any solution $\Psi$ of (3) belonging to $L_{2}^{m}(-\infty,0)$
there corresponds a solution $\Phi$ of (3) belonging to $L_{2}^{m}(0,\infty).$
Moreover, in the both cases these correspondences are made by the mapping $A.$
Thus the mapping $A$ is an antilinear isometric bijection from
$S_{+}(\lambda)$ onto $S_{-}(\lambda).$

Now using Theorem 1, the following well-known connection of $T$ with the
operators defined in $L_{2}^{m}[0,1]$ by the quasiperiodic boundary conditions
and the well-known Floquet theory we consider the spectrum of $T.$ It is well
known that (see for example [2, 4]) the spectrum $\sigma(T)$ of $T$ is the
union of the spectra $\sigma(T_{t})$ of the operators $T_{t}$ for
$t\in[0,2\pi)$ generated in $L_{2}^{m}[0,1]$ by (1) and the boundary
conditions

$y^{(\mathbb{\nu})}\left(1\right)=e^{it}y^{(\mathbb{\nu})}\left(0\right),\text{
}\mathbb{\nu}=0,1,...,(n-1).$ (6)

Thus

$\sigma(T)={\textstyle\bigcup\limits_{t\in[0,2\pi)}}\sigma(T_{t}).$ (7)

The spectrum of $T_{t}$ consists of the eigenvalues that are the roots of the
characteristic equation

$\det(Y_{j}^{(\nu-1)}(1,\lambda)-e^{it}Y_{j}^{(\nu-1)}(0,\lambda))_{j,\nu=1}^{n}=0$
(8)

of (3), where $Y_{1}(x,\lambda),Y_{2}(x,\lambda),\ldots,Y_{n}(x,\lambda)$ are
the solutions of the matrix equation

$(i)^{n}Y^{(n)}(x)+(i)^{n-1}P_{1}(x)Y^{(n-1)}(x)+...+P_{n}(x)Y=\lambda Y(x)$

satisfying $Y_{k}^{(j)}(0,\lambda)=0_{m}$ for $j\neq k-1$ and
$Y_{k}^{(k-1)}(0,\lambda)=I_{m}$, where $0_{m}$ and $I_{m}$ are $m\times m$
zero and identity matrices respectively (see [3, Chapter 3], and [7]).

Now, let us recall the well-known Floquet theory for the differential
equations with periodic coefficients. The $n$-th order equation (3) can be
reduced to the first-order equation

$\mathbf{x}^{{}^{\prime}}=A(x,\lambda)\mathbf{x}$ (9)

by setting $\mathbf{x=(x}_{1}\mathbf{,x}_{2}\mathbf{,...x}_{n}\mathbf{)}^{T},$
where
$\mathbf{x}_{1}=y\mathbf{,x}_{2}=y^{{}^{\prime}}\mathbf{,...x}_{n}=y^{(n-1)},$
$A(x,\lambda)$ is $mn\times mn$ matrix and $A(x+1,\lambda)=A(x,\lambda).$
According to the Floquet theory, any solution (3) is a linear combination of
solutions (3) having the form

$e^{it_{k}x}(p_{0,k}(x)+xp_{1,k}(x)+...+x^{s_{k}}p_{s_{k},k}(x)),$ (10)

where $p_{0,k}(x),p_{1,k}(x),...$ are the periodic functions,
$e^{it_{1}},e^{it_{2}},...e^{it_{nm}}$ are the multipliers of (3) and (9).
These multipliers are the roots of the characteristic equation

$\det(X(1,\lambda)-e^{it}I)=0$ (11)

of (9), where $X(x,\lambda)$ is a solution of the matrix equation
$X^{{}^{\prime}}(x,\lambda)=A(x,\lambda)X(x,\lambda\mathbf{)}$ satisfying the
initial condition $X(0,\lambda)=I_{mn}$ (see for example [8, Chapter 2]).
Using (6)-(9) one can easily verify that the characteristic equations (8) and
(11) of (3) and (9) are the same and

$\lambda\in\sigma(T)\Longleftrightarrow\exists t\in\mathbb{R}:e^{it}\in
M(\lambda)\Longleftrightarrow\exists\Psi\in
S(\lambda):\Psi(x,\lambda)=e^{itx}p(x),$ (12)

where $M(\lambda)$ and $S(\lambda)$ are respectively the sets of the
multipliers and solutions of (3) and $p(x+1)=p(x).$ Now, using this and
Theorem 1 we prove the following.

###### Theorem 2

$(a)$ The nonreal part $\sigma(T)\backslash\mathbb{R}$ of $\sigma(T)$ consists
of the pairs of curves symmetric with respect to the real line.

$(b)$ If $n$ and $m$ are the odd numbers, then $\mathbb{R}\subset\sigma(T)$.

Proof. $(a)$ Since the spectrum of $T$ consists of the union of the curves it
is enough to prove that if $\lambda\in\sigma(T),$ then
$\overline{\lambda}\in\sigma(T).$ If $\lambda\in\sigma(T),$ then by (12)
equation (3) has a solution $\Psi(x,\lambda)$ of the form $e^{itx}p(x)$, where
$t\in\mathbb{R}$ and $p(x+1)=p(x).$ Then, by Theorem 1$(a),$
$\Phi(x,\lambda)=\overline{\Psi(-x,\lambda)}$ is a solution of (4) and
$\Phi(x,\lambda)=e^{itx}\overline{p(-x)}$ . Thus, by (12),
$\overline{\lambda}\in\sigma(T).$

$(b)$ Suppose to the contrary that there exists a real number $\lambda$ such
that $\lambda\notin\sigma(T).$ Then, it follows from (12) that the absolute
values of all multipliers $e^{it_{1}},$ $e^{it_{2}},...,e^{it_{nm}}$ of (3)
differ from $1,$ that is, $t_{k}\neq\overline{t_{k}}$ for all $k=1,2,...,mn.$
It is clear that solution (10) belong to $L_{2}^{m}(0,\infty)$ and
$L_{2}^{m}(-\infty,0)$ respectively, if $\operatorname{Im}t_{k}>0$ and
$\operatorname{Im}t_{k}<0.$ Therefore, the set $S(\lambda)$ of all solution of
(3) is

$S(\lambda)=\left\\{y+z:y\in S_{+}(\lambda),\text{ }z\in
S_{-}(\lambda)\right\\},$

where $S_{+}(\lambda)$ and $S_{-}(\lambda)$ are defined in Theorem 1$(b)$ and
$S_{+}(\lambda)\cap S_{-}(\lambda)=\left\\{0\right\\}.$ This implies that
$\dim S(\lambda)=\dim S_{+}(\lambda)+\dim S_{-}(\lambda).$ On the other hand,
by Theorem 1$(b),$ $\dim S_{+}(\lambda)=\dim S_{-}(\lambda).$ Therefore, the
dimension $nm$ of the space of all solutions of (3) is $2\dim S_{+}(\lambda)$
which contradicts the assumption that $n$ and $m$ are odd numbers.

## References

  * [1] F. Bagarello, J. P. Gazeau, F. H. Szafraniec and M. Znojil, Non-self-adjoint Operators in Quantum Physics: Mathematical Aspects (John Wiley & Sons, Inc. Published, 2015).
  * [2] D. C. McGarvey, Differential operators with periodic coefficients in $L_{p}(-\infty,\infty)$, Journal of Mathematical Analysis and Applications, 11 (1965), 564-596.
  * [3] M. A. Naimark, Linear Differential Operators, Part 1 (George G. Harrap, London, 1967).
  * [4] F. S. Rofe-Beketov, The spectrum of nonself-adjoint differential operators with periodic coefficients. Sov. Math. Dokl. 4 (1963), 1563-1564.
  * [5] O. A. Veliev, Non-self-adjoint Schrödinger operator with a periodic potential (Springer Nature, Switzerland, 2021).
  * [6] O. A. Veliev, On the Schrödinger operator with a Periodic PT-symmetric Matrix Potential, J. Math. Phys. 62, 103501 (2021).
  * [7] O. A. Veliev, Uniform Convergence of the Spectral Expansion for a Differential Operator with Periodic Matrix Coefficients, Boundary Value Probl. 2008, 628973 (2008).
  * [8] V. A. Yakubovich and V. M. Starzhinskii, Linear Differential Equations with Periodic Coefficients (Wiley, New York, 1975).