# Some Remarks on Small Values of $\tau(n)$

Kaya Lakein and Anne Larsen  Department of Mathematics, Stanford University,
Stanford, CA 94305<EMAIL_ADDRESS>Department of Mathematics, Harvard
University, Cambridge, MA 02138<EMAIL_ADDRESS>
###### Abstract.

A natural variant of Lehmer’s conjecture that the Ramanujan $\tau$-function
never vanishes asks whether, for any given integer $\alpha$, there exist any
$n\in\mathbb{Z}^{+}$ such that $\tau(n)=\alpha$. A series of recent papers
excludes many integers as possible values of the $\tau$-function using the
theory of primitive divisors of Lucas numbers, computations of integer points
on curves, and congruences for $\tau(n)$. We synthesize these results and
methods to prove that if $0<\absolutevalue{\alpha}<100$ and $\alpha\notin
T:=\\{2^{k},-24,-48,-70,-90,92,-96\\}$, then $\tau(n)\neq\alpha$ for all
$n>1$. Moreover, if $\alpha\in T$ and $\tau(n)=\alpha$, then $n$ is square-
free with prescribed prime factorization. Finally, we show that a strong form
of the Atkin-Serre conjecture implies that $\absolutevalue{\tau(n)}>100$ for
all $n>2$.

##  1\. Introduction and Statement of Results

In his 1916 paper titled “On certain arithmetical functions,” Ramanujan
defined the function $\tau(n)$ to be the Fourier coefficients of the
normalized weight 12 cusp form for $\operatorname{SL}_{2}(\mathbb{Z})$ given
by

(1.1)
$\Delta(z)=\sum_{n=1}^{\infty}\tau(n)q^{n}:=q\prod_{n=1}^{\infty}(1-q^{n})^{24}=q-24q^{2}+252q^{3}-1472q^{4}+4830q^{5}-\dots,$

where $q:=e^{2\pi iz}$. The $\tau$-function plays an important role in the
theory of modular forms, yet some of its basic properties remain unknown. For
instance, Lehmer’s 1947 conjecture [12] that $\tau(n)$ never vanishes remains
open.

A natural variant of Lehmer’s conjecture asks whether, for any given integer
$\alpha$, there exist any $n\in\mathbb{Z}^{+}$ such that $\tau(n)=\alpha$.
Much is known for odd $\alpha$ due to the fact that

(1.2) $\Delta(z)\equiv\sum_{n=0}^{\infty}q^{(2n+1)^{2}}\allowbreak\mkern
8.0mu({\operator@font mod}\mkern 6.0mu2).$

In 1987, Murty, Murty, and Shorey [16] showed that for any odd integer
$\alpha$, we have $\tau(n)=\alpha$ for at most finitely many
$n\in\mathbb{Z}^{+}$. It was not until 2013, however, that the first integers
were ruled out as values of $\tau(n)$, when Lygeros and Rozier [13] proved
that $\tau(n)\neq\pm 1$ for all $n>1$. In 2020, Balakrishnan, Craig, Ono, and
Tsai [2, 3] introduced new methods to the problem, and showed that

$\tau(n)\notin\\{\pm 1,\pm 3,\pm 5,\pm 7,\pm 13,\pm 17,-19,\pm 23,\pm 37,\pm
691\\}.$

Later that year, Hanada and Madhukara [11] additionally proved that

$\tau(n)\notin\\{-9,\pm 15,\pm 21,-25,-27,-33,\pm 35,\pm 45,\pm 49,-55,\pm
63,\pm 77,-81,\pm 91\\},$

and Dembner and Jain [9] showed that $\tau(n)\neq\pm\ell$, where $\ell<100$ is
any odd prime. Shortly thereafter, Bennett, Gherga, Patel, and Siksek [5]
proved that the same is true for any positive power of $\ell$, and that
$\tau(n)\neq\pm 3^{a}5^{b}7^{c}11^{d}$ for any $a,b,c,d\in\mathbb{Z}_{\geq 0}$
and any $n>1$. Using some of these results concerning odd primes,
Balakrishnan, Ono, and Tsai [4] were able to make progress eliminating non-
zero even integers as values of the $\tau$-function. For odd primes $\ell$,
they showed that

$\tau(n)\notin\\{\pm 2\ell:\,\ell<100\\}\cup\\{\pm
2\ell^{2}:\,\ell<100\\}\cup\\{\pm 2\ell^{3}:\,\ell<100\text{ with }\ell\neq
59\\}.$

The methods used to obtain these results include the theory of primitive
divisors of Lucas numbers, the paucity of integer points on hyperelliptic
curves and Thue equations, and the congruences for the $\tau$-function (see
(3.2) and (4.2)) arising from the theory of 2-dimensional Galois
representations due to Serre and Swinnerton-Dyer [19]. The objective of this
paper is to synthesize these results and exclude as many additional non-zero
$\tau$-values with absolute value less than $100$ as possible given current
methods, as well as to explain the limitations of these methods. In
particular, we prove the following:

######  Theorem 1.1.

The following are true:

  1. (i)

If $\tau(n)\in\\{2,-24,-70,-90,92\\}$, then $n=p$ is prime.

  2. (ii)

If $\tau(n)=2^{k}<100$, then $n=p_{1}\cdots p_{k}$ for distinct primes $p_{i}$
with $\tau(p_{i})=2$ for all $i$.

  3. (iii)

If $\tau(n)=-48$, then $n=p_{1}p_{2}$ for primes $p_{1},p_{2}$ with
$\tau(p_{1})=2$ and $\tau(p_{2})=-24$.

  4. (iv)

If $\tau(n)=-96$, then $n=p_{1}p_{2}p_{3}$ for distinct primes with
$\tau(p_{1})=\tau(p_{2})=2$ and $\tau(p_{3})=-24$.

  5. (v)

For all $\alpha\in\mathbb{Z}$ with $0<\absolutevalue{\alpha}<100$ not listed
in (i)-(iv), we have that $\tau(n)\neq\alpha$ for all $n>1$.

######  Remark.

Assuming the Lang-Trotter conjecture [15], there are only finitely many primes
$p$ with $\tau(p)=\alpha$ for any integer $\alpha$, in which case Theorem 1.1
shows that there are only finitely many $n$ with
$0<\absolutevalue{\tau(n)}<100$.

The obstacle to proving that $\absolutevalue{\tau(n)}\geq 100$ for all $n>2$
is that the congruences given in [19], which are currently the only known
means for ruling out possible values of $\tau(p)$, do not eliminate the
exceptional values in Theorem 1.1 (i) (whose products give rise to the
exceptional values in (ii)-(iv)). New methods will therefore be needed to
better understand possible values of $\tau(p)$. Recent progress on the Atkin-
Serre conjecture provides hope that $\absolutevalue{\tau(p)}$ could be bounded
below for all primes. Using an effective form of the Sato-Tate conjecture,
Gafni, Thorner, and Wong [10] recently showed that

(1.3) $2p^{\frac{11}{2}}\,\frac{\log\log p}{\sqrt{\log
p}}<\absolutevalue{\tau(p)}$

for 100 percent of (but not all) primes $p$. We call the set $S$ of all primes
contradicting (1.3) the exceptional set. If $p\notin S\cup\\{2,3\\}$, then
(1.3) shows that $\absolutevalue{\tau(p)}>100$, and hence we immediately
obtain the following corollary of Theorem 1.1:

######  Corollary 1.2.

If $n>1$ and $0<\absolutevalue{\tau(n)}<100$, then every odd prime factor of
$n$ is in $S$.

Of course, to prove that $\absolutevalue{\tau(n)}\geq 100$ for $n>2$, we would
need a stronger statement, such as an effective version of the Atkin-Serre
conjecture for $\tau(n)$ [18].

######  Conjecture (Atkin-Serre $+\,\varepsilon$).

For any $\varepsilon>0$, there are constants $c(\varepsilon),d(\varepsilon)>0$
such that for all primes $p>d(\varepsilon)$, we have

(1.4) $\absolutevalue{\tau(p)}\geq\,c(\varepsilon)\cdot
p^{\frac{9}{2}-\varepsilon}.$

If one could find an $\varepsilon>0$ and corresponding constants
$c(\varepsilon)$ and $d(\varepsilon)$ satisfying this form of the Atkin-Serre
conjecture, then proving that $\absolutevalue{\tau(n)}<100$ if and only if
$n\leq 2$ would reduce to a finite calculation. Since such a statement is
currently out of reach, we instead carry out numerical experiments that
suggest an $\varepsilon>0$ and constants $c(\varepsilon),d(\varepsilon)$ which
could conceivably satisfy this condition. In particular, we let $P_{n}$ denote
the set of the first $n$ primes, and $p_{\mathrm{min}}(\varepsilon,n)\in
P_{n}$ denote the prime at which the expression
$\absolutevalue{\tau(p)}p^{-\frac{9}{2}+\varepsilon}$ is minimized. The values
of $p_{\mathrm{min}}(\varepsilon,n)$ and $\min_{p\in
P_{n}}\absolutevalue{\tau(p)}p^{-\frac{9}{2}+\varepsilon}$ for $n=30000$ and
different small values of $\varepsilon$ are given in the following table:

$\varepsilon$ | $0.001$ | $0.01$ | $0.1$ | $0.25$ | $0.5$
---|---|---|---|---|---
$p_{\mathrm{min}}(\varepsilon,30000)$ | $43$ | $43$ | $43$ | $2$ | $2$
$\min_{p\in P_{30000}}\absolutevalue{\tau(p)}p^{-\frac{9}{2}+\varepsilon}$ | $0.766\dots$ | $0.793\dots$ | $1.112\dots$ | $1.261\dots$ | $1.5$
Table 1. Numerics for the Atkin-Serre conjecture $+\,\varepsilon$ with the
first $30000$ primes

Assuming, for instance, that $\absolutevalue{\tau(p)}\geq
0.793p^{\frac{9}{2}-0.01}$ for all primes $p$ (so that $\varepsilon=0.01$
satisfies the Atkin-Serre conjecture with $c(\varepsilon)=0.793$,
$d(\varepsilon)=0$), we have $\absolutevalue{\tau(n)}<100$ if and only if
$n\leq 2$.

######  Remark.

It would be interesting to carry out further numerics with the goal of finding
explicit constants $c(\varepsilon)$ and $d(\varepsilon)$ for which some form
of the Atkin-Serre conjecture for arbitrary newforms of integer weight $k\geq
4$ without complex multiplication appears to hold.

The proof of Theorem 1.1 follows the approach of [3] and [4] for odd and even
values, respectively. For each $\alpha$, we use the fact that smaller values
have been excluded as values of the $\tau$-function to show that
$\tau(n)=\alpha$ implies $n=p^{a}$ for some prime $p$. Due to the
classification of primitive divisors of Lucas sequences in [6, 1], we are able
to restrict the list of possibilities for $a$, each of which we then eliminate
using the congruences for the $\tau$-function listed in [19]. (In contrast to
previous work, where $\tau(p^{a})=\alpha$ for $a>1$ is excluded by solving
Diophantine equations, we found these congruences to be sufficient in all
cases of interest.)

This paper is organized as follows: In Section 2, we recall some facts about
Lucas numbers and their primitive prime divisors, and connect these to values
of the $\tau$-function at prime powers. In Sections 3 and 4, we exclude the
odd and even integers indicated in Theorem 1.1 as possible values of
$\tau(n)$, respectively. Finally, we complete the proof of Theorem 1.1 in
Section 5.

## Acknowledgements

We would like to thank Ken Ono for suggesting and advising this project, and
for many valuable comments. We also thank William Craig, Badri Pandey, Wei-Lun
Tsai, and the referee for their helpful suggestions. Finally, we are grateful
for the generous support of the National Science Foundation (DMS 2002265 and
DMS 205118), the National Security Agency (H98230-21-1-0059), the Thomas
Jefferson Fund at the University of Virginia, and the Templeton World Charity
Foundation. This research was conducted as part of the Number Theory Research
Experience for Undergraduates at the University of Virginia.

##  2\. Preliminaries

In this section, we recall some facts about primitive prime divisors of Lucas
sequences, as well as several properties of the $\tau$-function, which we will
need to prove Theorem 1.1.

###  2.1. Lucas sequences

If $\alpha$ and $\beta$ are algebraic integers such that $\alpha+\beta$ and
$\alpha\beta$ are relatively prime non-zero integers and
$\frac{\alpha}{\beta}$ is not a root of unity, then we call $(\alpha,\beta)$ a
Lucas pair, and we define the Lucas sequence
$u_{n}(\alpha,\beta)=\\{u_{1}=1,\,u_{2}=\alpha+\beta,\dots\\}$ by

$u_{n}(\alpha,\beta):=\frac{\alpha^{n}-\beta^{n}}{\alpha-\beta}.$

Lucas sequences have the following divisibility property:

######  Proposition 2.1 (Proposition 2.1 (ii) of [6]).

If $d\mid n$, then $u_{d}(\alpha,\beta)\mid u_{n}(\alpha,\beta)$.

For any prime $\ell$, let $m_{\ell}(\alpha,\beta)$ denote the smallest $n\geq
2$ such that $\ell\mid u_{n}(\alpha,\beta)$. Then we have
$m_{\ell}(\alpha,\beta)=2$ if and only if $\alpha+\beta\equiv
0\allowbreak\mkern 8.0mu({\operator@font mod}\mkern 6.0mu\ell)$, and
$m_{\ell}(\alpha,\beta)$ fulfills the following condition:

######  Proposition 2.2 (Proposition 2.3 of [4]).

If $\ell$ is an odd prime with $2<m_{\ell}(\alpha,\beta)<\infty$, then the
following are true:

  1. (i)

If $\ell\mid(\alpha-\beta)^{2}$, then $m_{\ell}(\alpha,\beta)=\ell$.

  2. (ii)

If $\ell\nmid(\alpha-\beta)^{2}$, then $m_{\ell}(\alpha,\beta)\mid\ell-1$ or
$m_{\ell}(\alpha,\beta)\mid\ell+1$.

If $(\alpha,\beta)$ is a Lucas pair, then a prime number $p$ is called a
primitive prime divisor of $u_{n}(\alpha,\beta)$ if $p\mid
u_{n}(\alpha,\beta)$, but $p\nmid(\alpha-\beta)^{2}u_{1}(\alpha,\beta)\cdots
u_{n-1}(\alpha,\beta)$. If $n>2$ and $u_{n}(\alpha,\beta)$ does not have a
primitive prime divisor, then $u_{n}(\alpha,\beta)$ is called defective. Bilu,
Hanrot, and Voutier [6] showed that every Lucas number $u_{n}(\alpha,\beta)$
with $n>30$ has a primitive prime divisor. Their work, together with a
subsequent paper of Abouzaid [1], completely classifies defective Lucas
numbers.

###  2.2. Properties of $\tau(n)$

Important properties of the $\tau$-function include the Hecke multiplicativity
established by Mordell [14], and a deep theorem of Deligne [7, 8] that gives
an upper bound for $\absolutevalue{\tau(p)}$.

######  Theorem 2.3.

The following are true:

  1. (i)

If $\gcd(m,n)=1$, then $\tau(mn)=\tau(m)\tau(n)$.

  2. (ii)

If $p$ is prime and $a\geq 2$, then

$\tau(p^{a})=\tau(p)\tau(p^{a-1})-p^{11}\tau(p^{a-2}).$

  3. (iii)

If $p$ is prime and $\alpha_{p},\beta_{p}$ are roots of
$F_{p}(X):=X^{2}-\tau(p)X+p^{11},$ then

$\tau(p^{a})=u_{a+1}(\alpha_{p},\beta_{p})=\frac{\alpha_{p}^{a+1}-\beta_{p}^{a+1}}{\alpha_{p}-\beta_{p}}.$

Moreover, $\absolutevalue{\tau(p)}\leq 2p^{\frac{11}{2}}$, and $\alpha_{p}$
and $\beta_{p}$ are complex conjugates.

In particular, Theorem 2.3 (i) has the following corollary, which we use to
prove Theorem 1.1:

######  Corollary 2.4.

Suppose $\alpha\in\mathbb{Z}$ is such that for any factorization
$\alpha=\beta\gamma$, we have either $\tau(n)\neq\beta$ or $\tau(n)\neq\gamma$
for $n>1$. If $\tau(n)=\alpha$, then $n=p^{a}$ for some prime $p$.

If $p\mid\tau(p)$, then $p^{a}\mid\tau(p^{a})$ for all $a\in\mathbb{Z}^{+}$ by
the recursion in Theorem 2.3 (ii). Since $\absolutevalue{\tau(n)}>100$ for
$3\leq n\leq 100$, we have that $\alpha\neq\tau(p^{a})$ for any $p$ such that
$p\mid\tau(p)$ when $0<\absolutevalue{\alpha}<100$ (except for $\tau(2)=-24$).
Therefore, for the purposes of this paper, it suffices to assume that
$p\nmid\tau(p)$, in which case $\\{1,\tau(p),\tau(p^{2}),\dots\\}$ forms a
Lucas sequence by Theorem 2.3 (iii).

######  Lemma 2.5.

If $p\nmid\tau(p)$ and $\absolutevalue{\tau(p^{a})}<100$, then $\tau(p^{a})$
is not a defective term in the Lucas sequence
$\\{1,\tau(p),\tau(p^{2}),\dots\\}$.

###### Proof.

A complete classification of the defective Lucas numbers is given in Tables 1
(sporadic examples) and 2 (parametrized families) of [3]. For the particular
Lucas sequence $u_{n}=\tau(p^{n-1})$, the variables $(A,B)$ given in these
tables correspond to $(\tau(p),p^{11})$. Since none of the values of $B$
listed in Table 1 are $11$th powers, the sporadic examples listed here do not
apply to the Lucas sequence $u_{n}$. Now, by the results of Lygeros and Rozier
[13] as well as Bennett, Gherga, Patel, and Siksek [5] stated in Section 1, we
have that $\tau(p^{a})\neq-1,\pm 3^{r}$ as in rows 1 and 2 of Table 2. Rows 3,
5, and 7 give rise to a defective term only if $m=\tau(p)$ is odd; however, by
(1.2) we know that $\tau(p)$ is even for all primes. If row 4 of Table 2 gives
rise to a defective term $u_{4}=\tau(p^{3})$, then $(p,\pm
m)=(p,\pm\tau(p))\in B_{3,k}^{\pm}$ (as defined on p. 20 of [3]), meaning that
$\tau(p)^{2}=2p^{11}\pm 2$. Using Theorem 2.3 (ii), we see that then

$\displaystyle\absolutevalue{\tau(p^{3})}=\absolutevalue{\tau(p)(\tau(p)^{2}-2p^{11})}=2\absolutevalue{\tau(p)}=2(2p^{11}\pm
2)^{\frac{1}{2}}>100.$

Finally, row 6 yields a defective term $u_{6}$ only if
$\absolutevalue{\tau(p)}\geq 6$. However, if this is the case, then

$\absolutevalue{\tau(p^{5})}=\absolutevalue{u_{6}}=\absolutevalue{\tau(p)}(2\tau(p)^{2}\pm
3)>100.$

∎

##  3\. Odd Values of $\tau(n)$

The goal of this section is to prove the following theorem concerning odd
inadmissible values for the $\tau$-function:

######  Theorem 3.1.

If $\alpha$ is any odd integer with $\absolutevalue{\alpha}<100$, then
$\tau(n)\neq\alpha$ for any $n>1$.

###### Proof.

Using the previous results described in Section 1, it suffices to show that

(3.1) $\tau(n)\notin\\{\pm 39,\pm 51,\pm 57,\pm 65,\pm 69,\pm 85,\pm 87,\pm
93,\pm 95\\}$

for $n\in\mathbb{Z}^{+}$. We note that all the values $\alpha$ in (3.1)
satisfy the conditions of Corollary 2.4 by the results stated in Section 1.
Thus if $\tau(n)=\alpha$, we must have $n=p^{a}$ for some prime $p$ and $a\geq
1$, where $p$ is odd and $a$ is even by (1.2). As explained in Section 2, it
suffices to consider the case $p\nmid\tau(p)$, so that the values
$\\{1,\tau(p),\tau(p^{2}),\cdots\\}$ form a Lucas sequence. By Lemma 2.5, we
have that $\alpha=\tau(p^{a})$ is a non-defective term in this sequence, so
$\alpha$ has some prime divisor $\ell$ such that $m_{\ell}=a+1$. Hence by
Proposition 2.2, we have that $(a+1)\mid\ell(\ell^{2}-1)$. Moreover, we note
that $a+1$ must be prime. Indeed, if $a+1$ had a non-trivial divisor $b$, then
by Proposition 2.1, we would have $\tau(p^{b-1})\mid\tau(p^{a})$, but by non-
defectivity of $\tau(p^{a})$ and the odd $\tau$-values previously ruled out,
this is impossible. Considering each possible primitive prime divisor $\ell$
of $\alpha$ individually, we use these conditions on $a+1$ to obtain a finite
list of possible values for $a$ (which is given in Table 2 below for each
$\alpha$ in (3.1)).

To rule out each of these values of $a$, we use the following well-known
congruences [17, 19]:

(3.2)
$\displaystyle\begin{split}\tau(n)\equiv\begin{dcases}\sigma_{11}(n)\qquad\quad\allowbreak\mkern
8.0mu({\operator@font mod}\mkern 6.0mu691),\\\
0\qquad\,\qquad\quad\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu23)&\text{ if }n=p\text{ and }p^{11}\equiv-1\allowbreak\mkern
8.0mu({\operator@font mod}\mkern 6.0mu23),\\\
-1,2\,\,\,\qquad\quad\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu23)&\text{ if }n=p\text{ and }p^{11}\equiv 1\,\,\,\,\,\allowbreak\mkern
8.0mu({\operator@font mod}\mkern 6.0mu23),\\\
n^{-610}\sigma_{1231}(n)\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu3^{6})&\text{ if }n\not\equiv 0\allowbreak\mkern 8.0mu({\operator@font
mod}\mkern 6.0mu3).\end{dcases}\end{split}$

If $n=p^{a}$, then the mod $691$ congruence in (3.2) gives

$\tau(p^{a})\equiv\sigma_{11}(p^{a})=\sum_{i=0}^{a}p^{11i}\allowbreak\mkern
8.0mu({\operator@font mod}\mkern 6.0mu691).$

In particular, if the equation
$\sum_{i=0}^{a}x^{i}\equiv\alpha\allowbreak\mkern 8.0mu({\operator@font
mod}\mkern 6.0mu691)$ has no solutions, then $\alpha\neq\tau(p^{a})$ for any
prime $p$. Checking this for each $(\alpha,a)$ in Table 2, we eliminate the
pairs indicated in the table.

By the mod 23 congruence, we have $\tau(p)\equiv 0,-1,2\allowbreak\mkern
8.0mu({\operator@font mod}\mkern 6.0mu23)$ for all $p\neq 23$, where
$\tau(p)\equiv 0\allowbreak\mkern 8.0mu({\operator@font mod}\mkern 6.0mu23)$
exactly when $p^{11}\equiv-1\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu23)$. Combining this with the recurrence in Theorem 2.3 (ii), we can find
a list of possible values for $\tau(p^{a})$ mod 23, as long as $p\neq 23$.
Checking whether $\alpha$ is on this list for each pair in Table 2 not already
eliminated by the mod 691 congruence, and then checking numerically that
$|\tau(23^{a})|>10^{14}>\alpha$ for each $a$ in the table, we eliminate the
pairs shown in Table 2.

Finally, if for all possible equivalence classes of $3\nmid p\allowbreak\mkern
8.0mu({\operator@font mod}\mkern 6.0mu3^{6})$ we have
$p^{-610a}\sigma_{1231}(p^{a})\not\equiv\alpha\allowbreak\mkern
8.0mu({\operator@font mod}\mkern 6.0mu3^{6})$, then $\alpha\neq\tau(p^{a})$
for $p\neq 3$. Furthermore, for all $\alpha$ in the table and any $a\geq 1$,
we have $\alpha\not\equiv\tau(3^{a})\equiv 0\allowbreak\mkern
8.0mu({\operator@font mod}\mkern 6.0mu9)$. This eliminates all remaining pairs
$(\alpha,a)$ in Table 2.

$\absolutevalue{\alpha}$ $a$
$\mathrm{Congruences}\,\,\mathrm{for}\,\,\tau(p^{a})\neq\absolutevalue{\alpha}$
$\mathrm{Congruences}\,\,\mathrm{for}\,\,\tau(p^{a})\neq-\absolutevalue{\alpha}$
$39=3\cdot 13$ $2,6,12$ $2,6,12\allowbreak\mkern 8.0mu({\operator@font
mod}\mkern 6.0mu691)$ $2,6\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu691),12\allowbreak\mkern 8.0mu({\operator@font mod}\mkern 6.0mu23)$
$51=3\cdot 17$ $2,16$ $2,16\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu23)$ $2,16\allowbreak\mkern 8.0mu({\operator@font mod}\mkern 6.0mu23)$
$57=3\cdot 19$ $2,4,18$ $2,18\allowbreak\mkern 8.0mu({\operator@font
mod}\mkern 6.0mu23),4\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu691)$ $2\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu691),4,18\allowbreak\mkern 8.0mu({\operator@font mod}\mkern 6.0mu23)$
$65=5\cdot 13$ $2,4,6,12$ $2,4,6,12\allowbreak\mkern 8.0mu({\operator@font
mod}\mkern 6.0mu691)$ $2\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu691),4,6,12\allowbreak\mkern 8.0mu({\operator@font mod}\mkern 6.0mu23)$
$69=3\cdot 23$ $2,10,22$ $2,22\allowbreak\mkern 8.0mu({\operator@font
mod}\mkern 6.0mu3^{6}),10\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu23)$ $2,22\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu3^{6}),10\allowbreak\mkern 8.0mu({\operator@font mod}\mkern 6.0mu691)$
$85=5\cdot 17$ $2,4,16$ $2,4,16\allowbreak\mkern 8.0mu({\operator@font
mod}\mkern 6.0mu691)$ $2\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu691),4,16\allowbreak\mkern 8.0mu({\operator@font mod}\mkern 6.0mu23)$
$87=3\cdot 29$ $2,4,6,28$ $2,6,28\allowbreak\mkern 8.0mu({\operator@font
mod}\mkern 6.0mu23),4\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu691)$ $2\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu691),4\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu3^{6}),6,28\allowbreak\mkern 8.0mu({\operator@font mod}\mkern 6.0mu23)$
$93=3\cdot 31$ $2,4,30$ $2\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu691),4,30\allowbreak\mkern 8.0mu({\operator@font mod}\mkern 6.0mu3^{6})$
$2\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu23),4\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu3^{6}),30\allowbreak\mkern 8.0mu({\operator@font mod}\mkern 6.0mu691)$
$95=5\cdot 19$ $2,4,18$ $2,4,18\allowbreak\mkern 8.0mu({\operator@font
mod}\mkern 6.0mu691)$ $2\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu691),4,18\allowbreak\mkern 8.0mu({\operator@font mod}\mkern 6.0mu23)$

Table 2. Congruences showing that $\tau(p^{a})\neq\alpha$ for each pair
$(\alpha,a)$

∎

##  4\. Even Values of $\tau(n)$

The main result of this section is the following theorem concerning even
inadmissible values for the $\tau$-function:

######  Theorem 4.1.

If $\alpha$ is any even integer with $\absolutevalue{\alpha}<100$ such that

$\alpha\notin\\{2^{k}:\,k\in\mathbb{Z}^{+}\\}\cup\\{0,-24,-48,-70,-90,92,-96\\},$

then $\tau(n)\neq\alpha$ for any $n\in\mathbb{Z}^{+}$.

###### Proof.

Given the previous results stated in Section 1, it suffices to show that

(4.1) $\displaystyle\begin{split}\tau(n)\notin\\{&-2,-4,-8,\pm 12,-16,\pm
20,24,\pm 28,\pm 30,-32,\pm 36,\pm 40,\pm 42,\pm 44,48,\\\ &\pm 52,\pm 56,\pm
60,-64,\pm 66,\pm 68,70,\pm 72,\pm 76,\pm 78,\pm 80,\pm 84,\pm
88,90,-92,96\\},\end{split}$

for all $n\in\mathbb{Z}^{+}$. We generalize the proof of Theorem 1.1 in [4],
and apply the following argument to the values $\alpha$ in (4.1) inductively
starting with the $\alpha$ of smallest absolute value (for example, we need to
know that $12$ is not an admissible value for the $\tau$-function in order to
show that $\tau(n)\neq 24$ for all $n\in\mathbb{Z}^{+}$). Suppose for the sake
of contradiction that for some $\alpha$ in (4.1), there exists a positive
integer $n$ such that $\tau(n)=\alpha$. By the induction hypothesis, the
results in Section 1, and Theorem 3.1, Corollary 2.4 applies, and so it
follows that $n=p^{a}$ for some prime $p$ and some $a\in\mathbb{Z}^{+}$. As
explained in Section 2, we can assume that $p\nmid\tau(p)$, so in particular
$p\neq 2$.

Since $\tau(p^{a})=\alpha\equiv 0\allowbreak\mkern 8.0mu({\operator@font
mod}\mkern 6.0mu2)$, it follows by (1.2) that $a$ must be odd. We claim that
this implies $a=1$. Assume otherwise. By Lemma 2.5, we know
$\alpha=\tau(p^{a})$ is not a defective term in the Lucas sequence
$\\{1,\tau(p),\tau(p^{2}),\dots\\}$. Therefore, $\tau(p^{a})$ must have a
primitive prime divisor $\ell$, which must furthermore be odd since
$2\mid\tau(p)$. Suppose now that $a+1$ has a non-trivial divisor $b$. Then
$\tau(p^{b-1})$ divides $\tau(p^{a})=\alpha$ by Proposition 2.1.

Using Theorem 3.1, the results stated in Section 1, and the induction
hypothesis, we check that for any odd prime $\ell$ dividing $\alpha$, any
factor of $\alpha$ not divisible by $\ell$ which is not a power of 2 is not an
admissible value for the $\tau$-function. Thus, we must have
$\tau(p^{b-1})=2^{k}$ for some $1\leq k\leq 6$. By Lemma 2.5 and the fact that
$2\mid\tau(p)$, it follows that $b=2$. We conclude that $2$ is the only
possible non-trivial factor of $a+1$, so either $a+1$ is prime or $a+1=4$.

If $a+1$ is prime, then since $a$ is odd, we must have $a=1$. On the other
hand, if $a+1=4$, then $\tau(p)=2^{k}$ for some $1\leq k\leq 6$, and by the
recursive relation in Theorem 2.3 (ii), we then get
$\alpha=\tau(p^{3})=2^{k+1}(2^{2k-1}-p^{11})$. However, for any odd prime $p$,
we have $\absolutevalue{2^{2k-1}-p^{11}}>175000>\absolutevalue{\alpha}$, so
this is impossible. We conclude that $n=p$ for some odd prime $p$.

Finally, the congruences for $\tau(n)$ given in [19, p. 4] imply that for any
prime $p\neq 23$,

(4.2) $\displaystyle\begin{split}\tau(p)\equiv 0&\allowbreak\mkern
8.0mu({\operator@font mod}\mkern 6.0mu2),\quad\tau(p)\equiv
0,2\allowbreak\mkern 8.0mu({\operator@font mod}\mkern
6.0mu3),\quad\tau(p)\equiv 0,1,2\allowbreak\mkern 8.0mu({\operator@font
mod}\mkern 6.0mu5),\\\ \tau(p)&\equiv 0,1,2,4\allowbreak\mkern
8.0mu({\operator@font mod}\mkern 6.0mu7),\quad\tau(p)\equiv
0,-1,2\allowbreak\mkern 8.0mu({\operator@font mod}\mkern 6.0mu23).\end{split}$

However, we check that each $\alpha$ in (4.1) fails to satisfy one of these
congruences and $\absolutevalue{\tau(23)}>100$, so we cannot have
$\tau(p)=\alpha$. ∎

##  5\. Proof of Theorem 1.1

Theorems 3.1 and 4.1 prove Theorem 1.1 (v). Moreover, the argument used in the
proof of Theorem 4.1 shows that if $\tau(n)\in\\{2,-24,-70,-90,92\\}$, then
$n=p$ is prime, which proves (i). To show (ii), we proceed by induction on
$k$. From (i), we know that if $\tau(n)=2$, then $n=p$ is prime. Now, if
$\tau(n)=2^{k}$ and $n=p_{1}^{a_{1}}\cdots p_{j}^{a_{j}}$, then since
$\tau(m)\notin\\{-2^{k}:0\leq k\leq 6\\}\cup\\{1\\}$ for all $m>1$ by (v), we
must have $\tau(p_{i}^{a_{i}})=2^{c_{i}}$ for some $c_{i}>0$ such that
$c_{1}+\cdots+c_{j}=k$. If $j>1$, then by the induction hypothesis, we have
$a_{i}=1$ and $\tau(p_{i})=2$ for all $i$, so that in particular $j=k$. Hence,
it suffices to consider the case $n=p^{a}$. As in the proof of Theorem 4.1, we
can assume $p\nmid\tau(p)$, so that $\\{1,\tau(p),\tau(p^{2}),\cdots\\}$ forms
a Lucas sequence. By Lemma 2.5, $\tau(p^{a})=2^{k}$ is not a defective term.
Thus since $2\mid\tau(p)$, we must have $a=1$. However, $\tau(p)\neq 2^{k}$ by
the congruences in (4.2), hence we cannot have $n=p^{a}$.

The proofs of (iii) and (iv) are quite similar: Suppose first that
$\tau(n)=-48$. Then given the values ruled out for $\tau(n)$ in (v), we find
that either $n=p^{a}$ or $n=p_{1}^{a_{1}}p_{2}^{a_{2}}$ for primes
$p_{1},p_{2}$ such that $\tau(p_{1}^{a_{1}})=2$ and $\tau(p_{2}^{a_{2}})=-24$
(in which case $a_{1},a_{2}=1$ by part (i)). Therefore, it suffices to exclude
the possibility that $\tau(p^{a})=-48$. By the argument used in the proof of
Theorem 4.1, we need only to consider the case $a=1$, at which point
$\tau(p)=-48$ is again ruled out by the congruences in (4.2). Finally, if
$\tau(n)=-96$, then given the values ruled out for $\tau(n)$ in (v) and the
cases of $2,4,-24,-48$ handled above, we find that either $n=p^{a}$ or
$n=p_{1}p_{2}p_{3}$ with $\tau(p_{1})=\tau(p_{2})=2$ and $\tau(p_{3})=-24$. As
before, to rule out the case $\tau(p^{a})=-96$, we note that the proof of
Theorem 4.1 implies that we must have $a=1$, which is impossible by the
congruences in (4.2). ∎

## References

  * [1] M. Abouzaid, _Les Nombres de Lucas et Lehmer sans Diviseur Primitif_ , J. Théor. Nombres Bordeaux 18 (2006), 299-313.
  * [2] J. S. Balakrishnan, W. Craig, and K. Ono, _Variants of Lehmer’s Conjecture for Ramanujan’s Tau-Function_ , https://arxiv.org/abs/2005.10345, 2020.
  * [3] J. S. Balakrishnan, W. Craig, K. Ono, and W.-L. Tsai, _Variants of Lehmer’s Speculation for Newforms_ , https://arxiv.org/abs/2005.10354, 2020.
  * [4] J. S. Balakrishnan, K. Ono, and W.-L. Tsai, _Even Values of Ramanujan’s Tau-Function_ , https://arxiv.org/abs/2102.00111, 2021.
  * [5] M. Bennett, A. Gherga, V. Patel, and S. Siksek, _Odd Values of the Ramanujan Tau Function_ , https://arxiv.org/abs/2101.02933, 2021.
  * [6] Y. Bilu, G. Hanrot and P. M. Voutier, _Existence of Primitive Divisors of Lucas and Lehmer Numbers_ , J. Reine Angew. Math. 539 (2001), 75-122.
  * [7] P. Deligne, _La conjecture de Weil I_ , Publ. Math. de IHES 43 (1974), 273-307.
  * [8] P. Deligne, _La conjecture de Weil II_ , Publ. Math. de IHES 52 (1980), 137-252.
  * [9] S. Dembner and V. Jain, _Hyperelliptic Curves and Newform Coefficients_ , https://arxiv.org/abs/2007.08358, J. Number Th., accepted for publication.
  * [10] A. Gafni, J. Thorner, and P.-J. Wong, _Almost All Primes Satisfy the Atkin–Serre Conjecture and are Not Extremal_ , Res. Number Theory 7, no. 2 (2021), paper no. 31.
  * [11] M. Hanada and R. Madhukara, _Fourier Coefficients of Level 1 Hecke Eigenforms_ , https://arxiv.org/abs/2007.08683, Acta Arithmetica, accepted for publication.
  * [12] D. H. Lehmer, _The Vanishing of Ramanujan’s $\tau(n)$_, Duke Math J. 14 (1947), 429-433.
  * [13] N. Lygeros and O. Rozier, _Odd Prime Values of the Ramanujan Tau Function_ , Bull. Soc. Math. France 32 (2013), 269-280.
  * [14] L. J. Mordell, _On Mr. Ramanujan’s $\tau(n)$_, Duke Math. J. 14 (1947), 429-433.
  * [15] M. R. Murty, V. K. Murty, and N. Saradha, _Modular Forms and the Chebotarev Density Theorem_ , Am. J. Math. 110 (1988), 253-281.
  * [16] M. R. Murty, V. K. Murty, and T. N. Shorey, _Odd Values of the Ramanujan Tau Function_ , Bull. Soc. Math. France 115 (1987), 391-395.
  * [17] S. Ramanujan, _On Certain Arithmetical Functions_ , Trans. Camb. Philos. Soc. 22 (1916), 159-184.
  * [18] J.-P. Serre, _Divisibilité de Certaines Fonctions Arithmétiques_ , Enseign. Math. (2), 22 (1976), 227-260.
  * [19] H. P. F. Swinnerton-Dyer, _On $\ell$-Adic Representations and Congruences for Coefficients of Moular Forms_, in Modular Functions of One Variable III, Springer (1973), 1-55.


Fig. 14(b) | 0.09 | 0.05 | 0.28 | 0.89 | 0.40 | 0.34
Fig. 14(c) | 0.06 | 0.08 | 0.65 | 0.78 | 0.39 | 0.56
Fig. 14(d) | 0.10 | 0.09 | 0.60 | 0.94 | 0.43 | 0.55
Fig. 14(e) | 0.10 | 0.07 | 0.48 | 0.86 | 0.28 | 0.43
Fig. 14(f) | 0.08 | 0.07 | 0.63 | 1.14 | 0.34 | 0.29

From the test of learned weights, as seen from Fig. 14(a) and Table II, CL
trajectory takes a larger detour on flying through gate $2$ than OL
trajectory. The average final distance to the goal is slightly smaller. Under
the longer horizon, new initial positions, new desired position, and new gate
position, Fig. 14(b)-14(f) show that the generalized CL trajectories still fly
through two gates sequentially, and arrive at the vicinity of the desired
position. However, generalized OL trajectories fail to fly through gate $2$.
Specifically, as seen in Fig. 14(b) and Table II, with a longer planning
horizon, both OL and CL trajectories will be closer to gate $1$ center. Then,
both of them take a larger detour towards the center of gate $2$. In this
case, the apex of the CL trajectory gets closer to the center of gate $2$ and
results in a large drop in terms of minimum distance to gate $2$ center.
However, this is not the case for OL trajectory, since the increase of horizon
only reshapes its segment near gate $1$. Nevertheless, this detour changes the
velocity profile of OL trajectory and results in a smaller terminal velocity
and overshoot.

Note that for the generalization of gate position in Fig. 14(f), the movement
of gate $2$ enlarges the width of the curve (see the top view) due to the
attraction force from its center. However, this (discrete) change of
environment is too significant to be followed by a continuous adaptation of
the trajectory, which results in an increase of minimum distance to gate $2$
center, especially for OL trajectory as it does not reshape in $Y$-direction
but the gate moves further away in $X$-direction. We also report a failure
case where we further move gate $2$ away from the initial position, i.e.,
$\mathbf{c}_{\mathrm{g},2}=[-0.2,1,1]^{\top}$, as visualized in Fig. 15. It
can be seen that the CL trajectory fails to reshape itself to fly through gate
$2$. Moreover, the change of gate position completely alters the landscape of
the cost function, enlarging the minimum distance to gate $1$ and failing to
arrive at the vicinity of the desired position. This result clearly shows the
bound of the generalizability, i.e., the learned cost function can only be
applied to some unseen scenarios that are close to the training setting.

Figure 15: Failure case of generalization of the CL learned cost function on
trajectory planning with a new gate position. The legend is the same as that
in Fig. 14.

##  VII Conclusion

In this work, we have proposed a DDP-based framework for IRL with general
constraints, where the DDP was exploited to compute the gradient required in
the outer loop. We have established the equivalence between DDP-based and PDP-
based methods in terms of computation. In addition, inspired by the DDP
condition, we have proposed the closed-loop IRL with the closed-loop loss
function to capture the nature of collected demonstrations. Moreover, we have
shown that this new formulation can be reduced to a general constrained IOC
problem under certain conditions, which leads to a generalized recoverability
condition. Simulations and experiments demonstrated the superiority of the
closed-loop algorithm. Future work can be on the extension of this framework
to the multi-agent systems and stochastic systems.

## References

  * [1] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot _et al._ , “Mastering the game of go with deep neural networks and tree search,” _Nature_ , vol. 529, no. 7587, pp. 484–489, 2016.
  * [2] J. Kober, J. A. Bagnell, and J. Peters, “Reinforcement learning in robotics: A survey,” _The International Journal of Robotics Research_ , vol. 32, no. 11, pp. 1238–1274, 2013.
  * [3] B. Hambly, R. Xu, and H. Yang, “Recent advances in reinforcement learning in finance,” _Mathematical Finance_ , vol. 33, no. 3, pp. 437–503, 2023.
  * [4] M. Zhang, P. I. Gómez, Q. Xu, and T. Dragicevic, “Review of online learning for control and diagnostics of power converters and drives: Algorithms, implementations and applications,” _Renewable and Sustainable Energy Reviews_ , p. 113627, 2023.
  * [5] D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman, and D. Mané, “Concrete problems in AI safety,” _arXiv preprint arXiv:1606.06565_ , 2016\.
  * [6] P. Abbeel and A. Y. Ng, “Apprenticeship learning via inverse reinforcement learning,” in _Proceedings of the 21st International Conference on Machine Learning_ , 2004, p. 1.
  * [7] B. D. Ziebart, A. L. Maas, J. A. Bagnell, and A. K. Dey, “Maximum entropy inverse reinforcement learning.” in _AAAI_ , vol. 8. Chicago, IL, USA, 2008, pp. 1433–1438.
  * [8] N. D. Ratliff, J. A. Bagnell, and M. A. Zinkevich, “Maximum margin planning,” in _Proceedings of the 23rd International Conference on Machine Learning_ , 2006, pp. 729–736.
  * [9] W. Jin, Z. Wang, Z. Yang, and S. Mou, “Pontryagin differentiable programming: An end-to-end learning and control framework,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 7979–7992, 2020.
  * [10] W. Jin, S. Mou, and G. J. Pappas, “Safe pontryagin differentiable programming,” _Advances in Neural Information Processing Systems_ , vol. 34, pp. 16 034–16 050, 2021.
  * [11] B. Amos, I. Jimenez, J. Sacks, B. Boots, and J. Z. Kolter, “Differentiable mpc for end-to-end planning and control,” _Advances in Neural Information Processing Systems_ , vol. 31, 2018.
  * [12] W. Jin, T. D. Murphey, D. Kulić, N. Ezer, and S. Mou, “Learning from sparse demonstrations,” _IEEE Transactions on Robotics_ , vol. 39, no. 1, pp. 645–664, 2022.
  * [13] H. Von Stackelberg, _Market structure and equilibrium_. Springer Science & Business Media, 2010.
  * [14] J. Bracken and J. T. McGill, “Mathematical programs with optimization problems in the constraints,” _Operations Research_ , vol. 21, no. 1, pp. 37–44, 1973\.
  * [15] P. Hansen, B. Jaumard, and G. Savard, “New branch-and-bound rules for linear bilevel programming,” _SIAM Journal on Scientific and Statistical Computing_ , vol. 13, no. 5, pp. 1194–1217, 1992.
  * [16] L. S. Pontryagin, _Mathematical theory of optimal processes_. Routledge, 2018.
  * [17] H. G. Bock and K.-J. Plitt, “A multiple shooting algorithm for direct solution of optimal control problems,” _IFAC Proceedings Volumes_ , vol. 17, no. 2, pp. 1603–1608, 1984.
  * [18] M. Patyerson and A. V. G. I. Rao, “a matlab software for solving multiple-phase optimal control problems using hp-adaptive gaussian quadrature collocation methods and sparse nonlinear programming,” _ACM Transactions on Mathematical Software_ , vol. 41, no. 1, pp. 1–41, 2014.
  * [19] R. Bellman, “The theory of dynamic programming,” _Bulletin of the American Mathematical Society_ , vol. 60, no. 6, pp. 503–515, 1954.
  * [20] D. Mayne, “A second-order gradient method for determining optimal trajectories of non-linear discrete-time systems,” _International Journal of Control_ , vol. 3, no. 1, pp. 85–95, 1966.
  * [21] J. De O. Pantoja, “Differential dynamic programming and newton’s method,” _International Journal of Control_ , vol. 47, no. 5, pp. 1539–1553, 1988\.
  * [22] B. Plancher, Z. Manchester, and S. Kuindersma, “Constrained unscented dynamic programming,” in _2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2017, pp. 5674–5680.
  * [23] Z. Xie, C. K. Liu, and K. Hauser, “Differential dynamic programming with nonlinear constraints,” in _2017 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2017, pp. 695–702.
  * [24] Y. Aoyama, G. Boutselis, A. Patel, and E. A. Theodorou, “Constrained differential dynamic programming revisited,” _arXiv preprint arXiv:2005.00985_ , 2020.
  * [25] A. Pavlov, I. Shames, and C. Manzie, “Interior point differential dynamic programming,” _IEEE Transactions on Control Systems Technology_ , vol. 29, no. 6, pp. 2720–2727, 2021.
  * [26] A. Keshavarz, Y. Wang, and S. Boyd, “Imputing a convex objective function,” in _2011 IEEE International Symposium on Intelligent Control_. IEEE, 2011, pp. 613–619.
  * [27] P. Englert, N. A. Vien, and M. Toussaint, “Inverse KKT: Learning cost functions of manipulation tasks from demonstrations,” _The International Journal of Robotics Research_ , vol. 36, no. 13-14, pp. 1474–1488, 2017.
  * [28] M. Johnson, N. Aghasadeghi, and T. Bretl, “Inverse optimal control for deterministic continuous-time nonlinear systems,” in _52nd IEEE Conference on Decision and Control_. IEEE, 2013, pp. 2906–2913.
  * [29] T. L. Molloy, J. J. Ford, and T. Perez, “Finite-horizon inverse optimal control for discrete-time nonlinear systems,” _Automatica_ , vol. 87, pp. 442–446, 2018.
  * [30] W. Jin, D. Kulić, S. Mou, and S. Hirche, “Inverse optimal control from incomplete trajectory observations,” _The International Journal of Robotics Research_ , vol. 40, no. 6-7, pp. 848–865, 2021.
  * [31] T. L. Molloy, J. J. Ford, and T. Perez, “Online inverse optimal control for control-constrained discrete-time systems on finite and infinite horizons,” _Automatica_ , vol. 120, p. 109109, 2020.
  * [32] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in pytorch,” 2017.
  * [33] J. Nocedal and S. J. Wright, _Numerical optimization_. Springer, 1999.
  * [34] A. Forsgren, P. E. Gill, and M. H. Wright, “Interior methods for nonlinear optimization,” _SIAM review_ , vol. 44, no. 4, pp. 525–597, 2002.
  * [35] S. Ghadimi and M. Wang, “Approximation methods for bilevel programming,” _arXiv preprint arXiv:1802.02246_ , 2018.
  * [36] J. R. Magnus and H. Neudecker, _Matrix differential calculus with applications in statistics and econometrics_. John Wiley & Sons, 2019.
  * [37] W. Xue, P. Kolaric, J. Fan, B. Lian, T. Chai, and F. L. Lewis, “Inverse reinforcement learning in tracking control based on inverse optimal control,” _IEEE Transactions on Cybernetics_ , vol. 52, no. 10, pp. 10 570–10 581, 2021.
  * [38] A. Pinosky, I. Abraham, A. Broad, B. Argall, and T. D. Murphey, “Hybrid control for combining model-based and model-free reinforcement learning,” _The International Journal of Robotics Research_ , vol. 42, no. 6, pp. 337–355, 2023.
  * [39] M. Lutter and J. Peters, “Combining physics and deep learning to learn continuous-time dynamics models,” _The International Journal of Robotics Research_ , vol. 42, no. 3, pp. 83–107, 2023.
  * [40] R. E. Allen and M. Pavone, “A real-time framework for kinodynamic planning in dynamic environments with application to quadrotor obstacle avoidance,” _Robotics and Autonomous Systems_ , vol. 115, pp. 174–193, 2019.
  * [41] P. Sharma, B. Sundaralingam, V. Blukis, C. Paxton, T. Hermans, A. Torralba, J. Andreas, and D. Fox, “Correcting robot plans with natural language feedback,” _arXiv preprint arXiv:2204.05186_ , 2022.