# Evolution from quantum anomalous Hall insulator to heavy-fermion semimetal
in magic-angle twisted bilayer graphene

Cheng Huang  Department of Physics and HKU-UCAS Joint Institute of Theoretical
and Computational Physics, The University of Hong Kong, Pokfulam Road, Hong
Kong SAR, China  Xu Zhang  Department of Physics and HKU-UCAS Joint Institute
of Theoretical and Computational Physics, The University of Hong Kong,
Pokfulam Road, Hong Kong SAR, China  Gaopei Pan  Beijing National Laboratory
for Condensed Matter Physics and Institute of Physics, Chinese Academy of
Sciences, Beijing 100190, China  School of Physical Sciences, University of
Chinese Academy of Sciences, Beijing 100049, China  Heqiu Li  Department of
Physics, University of Toronto, Toronto, Ontario M5S 1A7, Canada  Kai Sun
<EMAIL_ADDRESS>Department of Physics, University of Michigan, Ann Arbor,
Michigan 48109, USA  Xi Dai<EMAIL_ADDRESS>Department of Physics, Hong Kong
University of Science and Technology, Kowloon, Hong Kong SAR, China  Zi Yang
Meng<EMAIL_ADDRESS>Department of Physics and HKU-UCAS Joint Institute of
Theoretical and Computational Physics, The University of Hong Kong, Pokfulam
Road, Hong Kong SAR, China

###### Abstract

The ground states of twisted bilayer graphene (TBG) at chiral and flat-band
limit with integer fillings are known from exact solutions, while their
dynamical and thermodynamical properties are revealed by unbiased quantum
Monte Carlo (QMC) simulations. However, to elucidate experimental observations
of correlated metallic, insulating and superconducting states and their
transitions, investigations on realistic, or non-chiral cases are vital. Here
we employ momentum-space QMC method to investigate the evolution of correlated
states in magic-angle TBG away from chiral limit at charge neutrality with
polarized spin/valley, which approximates to an experimental case with filling
factor $\nu=-3$. We find that the ground state evolves from quantum anomalous
Hall insulator into an intriguing correlated semi-metallic state possessing
heavy-fermion features as AA hopping strength reaches experimental values.
Such a state resembles the recently proposed heavy-fermion representations
with localized electrons residing at AA stacking regions and delocalized
electrons itinerating via AB/BA stacking regions. The spectral signatures of
the localized and itinerant electrons in the heavy-fermion semimetal phase are
revealed, with the connection to experimental results being discussed.

Introduction.— Flat bands of non-interacting magic-angle twisted bilayer
graphene (TBG) can be readily characterized by tight-binding [1, 2] and
continuous Bistritzer-MacDonald [3] models. Yet, in such deceptively simple
systems, it is the interplay of long-range Coulomb interaction, quantum metric
of flat-band wavefunction and the extremely high tunability, by twisting
angles, gating and dielectric environment, that gives rise to a plethora of
novel quantum states including correlated insulator [4, 5, 6, 7, 8, 9, 10, 11,
12, 13, 14], unconventional metal with linear-$T$ resistivity, superconductor
and beyond [15, 16, 17, 18, 19, 20], the underlying principles and mechanisms
of which are still under intensive investigations.

Among these efforts, exact solutions [21, 22, 23, 14, 24], mean-field analyses
[25, 23, 26, 27, 28, 29, 30, 31, 32], density matrix renormalization group
(DMRG) investigations [33, 34], and momentum-space quantum Monte Carlo (QMC)
simulations [35, 36, 37, 38, 39] reveal that the TBG model of the first magic
angle ($1.08^{\circ}$) with Coulomb interaction projected onto the flat bands
possesses a quantum anomalous Hall (QAH) ground state with Chern number $C=1$,
at chiral (AA stacking hopping $u_{0}=0$) and flat-band limit (ignoring
kinetic energy) with filling factor $\nu=\pm 3$ (which means one of the eight
spin/valley degenerate bands are occupied for $\nu=-3$, or seven considering
particle-hole symmetry for $\nu=3$). For the same system away from chiral
limit, which means $u_{0}>0$ (can be varied experimentally by twisting angle,
temperature, strain, magnetic field, or pressure, etc [16, 18]), a DMRG
investigation suggests its ground state remains Chern number polarized until
$u_{0}/u_{1}\approx 0.8$ (AB/BA stacking hopping $u_{1}\approx 110$ meV [40,
41, 42]), beyond which competing states such as gapless $C_{2}\mathcal{T}$
nematic and gapped $C_{2}\mathcal{T}$ stripe phases emerge [33]. Nevertheless,
another DMRG work indicates the ground state a nematic $C_{2}\mathcal{T}$
semimetal when $u_{0}/u_{1}>0.8$ [34], while a recent Hatree-Fock mean-field
study suggests the corresponding state to be insulating and monolayer charge
density modulated with $C_{2}\mathcal{T}$ symmetry [29].

Experimentally, magic-angle (hereafter magic-angle refers to the fisrt magic
angle) TBG at $\nu=-3$ is found to be metallic (or semi-metallic considering
carrier concentration) with $C_{3}$ symmetric electron localization,
neighboring to a superconducting phase, while that of $\nu=3$ is
semiconducting or insulating [43, 11, 44, 8, 9, 45, 6, 45], when $u_{1}$ is
around 110 meV [40, 41, 42] and $u_{0}$ around 0.8 of $u_{1}$ [8, 46, 47] due
to lattice relaxation. The discrepancies between the experimentally
observations (especially for $\nu=-3$) and the above theoretical proposals are
intriguing.

To understand and clarify the discrepancies between theories and experiments,
in this work, we apply the unbiased momentum-space QMC simulation [35, 36, 38,
37, 39, 48], supplemented by exact diagonalization (ED) [49], to
systematically investigate the evolution of the correlated states of magic-
angle TBG at $\nu=-3$ and away from chiral limit (increasing $u_{0}$), with
the Coulomb interaction projected onto flat bands. Our results show the
critical transition temperature ($T_{\mathrm{c}}$) of the QAH state decreases
with increasing $u_{0}$, and eventually vanish at $u_{0}\approx$ 90 meV, the
approximate value of realistic $u_{0}$, where a semimetallic (SM) state
probably preserving $C_{3}$ rotation symmetry emerges, as meanwhile the quasi-
particle gap reduces to zero. This semimetallic state is gapless in the
vicinity of $\Gamma$ point of the moiré Brilliouin zone (mBZ). In real space
it has localized states at the AA stacking region while itinerant states at
the AB/BA stacking regions of the moiré unit cell (mUC), and the localized
states and itinerant states are well separated in energy – an analog to the
experiments [8, 9] and recently found heavy-fermion representations of the TBG
system, with AA stacking regions proposed to hold states of localized flat
bands ($f$ electrons) and AB/BA stacking regions bridging itinerant states
($c$ electrons) [50, 51, 52, 53, 54, 55, 32, 56].

Since our model has no prior heavy-fermion representation in the first place
but mainly the approximation of projecting the Coulomb interaction into the
flat bands, our observation, the evolution from QAH insulator to heavy-fermion
semimetal at $\nu=-3$, provides an unbiased and unifying interpretation of the
ground state transition of magic-angle TBG from topological insulator (at
chiral limit) to the SM state (non-chiral) with topological heavy-fermion
features.

Figure 1: QAH state to heavy-fermion semimetal transition away from chiral
limit in the magic-angle TBG of $\nu=-3$. (a) The phase diagram, where
$T_{\mathrm{c}}$ refers to the transition temperature of the QAH state,
marking the time-reversal symmetry breaking and being determinded by finite-
size scaling crossings in (c)-(f), and the dased line is a fit to the data.
(b) Chern number polarization, $S$, obtained from the $L=4$ system, where the
simulation $T$ for each $u_{0}$ reaches as low temperature as possible.
(c)-(f) Finite-size effect crossings with 2D Ising exponents to derive
$T_{\mathrm{c}}$. Figure 2: Quasiparticle gap of $\Gamma$, $\Delta_{\Gamma}$,
and charge compressibility, $\kappa$, as functions of $u_{0}$ or $T$. (a) The
simulation temperature is 3 meV considering the sign problem and the
consistency to those of ED. (b) The system size is $L=3$.

Model and Method.— The real space moiré superlattice generated by TBG is shown
in Fig. S2. In principle, the Hamiltonian of TBG contains kenetics and
interaction, $H=H_{0}+H_{I}$. The kenetic part is

$H_{0}=\sum_{\mathbf{k},{\mathbf{G}},{\mathbf{G}}^{\prime},\eta,s}\mathbf{c}^{\dagger}_{\mathbf{k},{\mathbf{G}},\eta,s}H^{\eta,s}(\mathbf{k})_{{\mathbf{G}},{\mathbf{G}}^{\prime}}\mathbf{c}_{\mathbf{k},{\mathbf{G}}^{\prime},\eta,s},$
(1)

where

$\displaystyle H^{\eta,s}(\mathbf{k})_{{\mathbf{G}},{\mathbf{G}}^{\prime}}=$
$\displaystyle\delta_{{\mathbf{k}}_{1},{\mathbf{k}}_{2}}\hbar\nu_{\mathrm{F}}\left(\begin{array}[]{cc}-({\mathbf{k}}_{1}-{\mathbf{K}}_{1}^{\eta})\cdot\boldsymbol{\sigma}^{\eta}&0\\\
0&-({\mathbf{k}}_{1}-{\mathbf{K}}_{2}^{\eta})\cdot\boldsymbol{\sigma}^{\eta}\end{array}\right)$
(2) $\displaystyle+\left(\begin{array}[]{cc}0&T^{\eta}_{1}\\\
{T^{\eta}_{2}}^{\dagger}&0\end{array}\right)$

is the BM model [3] with

$\displaystyle T^{\eta}_{l}=$
$\displaystyle\left(\begin{array}[]{cc}u_{0}&u_{1}\\\
u_{1}&u_{0}\end{array}\right)\delta_{{\mathbf{k}}_{1},{\mathbf{k}}_{2}}+\left(\begin{array}[]{cc}u_{0}&u_{1}\mathrm{e}^{-i\frac{2\pi}{3}\eta}\\\
u_{1}\mathrm{e}^{i\frac{2\pi}{3}\eta}&u_{0}\end{array}\right)\delta_{{\mathbf{k}}_{1},{\mathbf{k}}_{2}+(-1)^{l}\eta{\mathbf{G}}_{1}}$
(3)
$\displaystyle+\left(\begin{array}[]{cc}u_{0}&u_{1}\mathrm{e}^{i\frac{2\pi}{3}\eta}\\\
u_{1}\mathrm{e}^{-i\frac{2\pi}{3}\eta}&u_{0}\end{array}\right)\delta_{{\mathbf{k}}_{1},{\mathbf{k}}_{2}+(-1)^{l}\eta\left({\mathbf{G}}_{1}+{\mathbf{G}}_{2}\right)}.$

Here ${\mathbf{k}}$ is a momentum in mBZ, ${\mathbf{G}}$ is a momentum
difference for ${\mathbf{k}}$, as ${\mathbf{k}}_{1}={\mathbf{k}}+{\mathbf{G}}$
and ${\mathbf{k}}_{2}={\mathbf{k}}+{\mathbf{G}}^{\prime}$, to account for the
contribution from extended Brillouin zones.
${\mathbf{G}},{\mathbf{G}}^{\prime}\in\\{N_{1}{\mathbf{G}}_{1}+N_{2}{\mathbf{G}}_{2}\\}$,
the combinations of lattices of mBZ, with $N_{1}$ and $N_{2}$ being integers,
and a cutoff, $|{\mathbf{G}}|$, $|{\mathbf{G}}^{\prime}|\leq
4|{\mathbf{G}}_{1}|$, is applied.
$\mathbf{c}^{\dagger}_{\mathbf{k},{\mathbf{G}},\eta,s}$ is a $1\times 4$
vector of creation operators,
$c^{\dagger}_{\mathbf{k},{\mathbf{G}},\eta,s,X}$, where $X$ is the index
refering to layers and sublattices, as
$X\in\\{\mathrm{A}_{1},\mathrm{B}_{1},\mathrm{A}_{2},\mathrm{B}_{2}\\}$ for
layers (1, 2) and sublattices (A, B). In addition, $s$ specifies spins
($\uparrow,\downarrow$) and $\eta=\pm$ labels the two valleys.
$\nu_{\mathrm{F}}$ is the Fermi velocity and we set $\hbar
v_{\mathrm{F}}/(\sqrt{3}a)=2.37745$ eV corresponding to flat bands, with $a$
the distance of nearest carbon atoms.
$\boldsymbol{\sigma}^{\eta}=\left(\eta\sigma_{x},\sigma_{y}\right)$, where
$\sigma_{x}$ and $\sigma_{y}$ are Pauli matrices refer to sublattices. The
first part of $H^{\eta,s}({\mathbf{k}})_{{\mathbf{G}},{\mathbf{G}}^{\prime}}$
stands for intra-layer hopping of the top (1) layer and the bottom (2) layer,
respectively, and the second part stands for inter-layer hopping, which is
actually the morié potential.

The interaction part is of density-density, as

$H_{\mathrm{I}}=\frac{1}{2\Omega}\sum_{{\mathbf{q}}\in\mathrm{mBZ},{\mathbf{G}}}V({\mathbf{q}}+{\mathbf{G}})\delta\rho_{{\mathbf{q}}+{\mathbf{G}}}\delta\rho_{-{\mathbf{q}}-{\mathbf{G}}},$
(4)

where

$\delta\rho_{{\mathbf{q}}+{\mathbf{G}}}=\sum_{{\mathbf{k}},{\mathbf{G}}^{\prime},\eta,s,X}\left(c^{\dagger}_{{\mathbf{k}},{\mathbf{G}}^{\prime},\eta,s,X}c_{{\mathbf{k}}+{\mathbf{q}}+{\mathbf{G}},{\mathbf{G}}^{\prime},\eta,s,X}-\frac{\nu+4}{8}\delta_{{\mathbf{q}},0}\delta_{{\mathbf{G}},0}\right).$
(5)

Here $\delta\rho_{\mathbf{q}}$ is the electron density operator with the
reference to $(\nu+4)/8$, where $\nu$ is the filling parameter with $\nu=0$ as
the charge neutrality. $V({\mathbf{q}})$ is the long-range screened Coulombic
interaction decaying as
$V(\mathbf{q})=\frac{\mathrm{e}^{2}}{2\varepsilon}\frac{1}{|{\mathbf{q}}|}\left(1-\mathrm{e}^{-|{\mathbf{q}}|d}\right)$
[38], where $d/2$ is the distance between TBG and a bottom gate [57] (here we
set $d=40$ nm) and $\varepsilon$ (=7 $\varepsilon_{0}$) is the dielectric
constant. The fermion momentum transfer, ${\mathbf{q}}+{\mathbf{G}}$, is cut
off at the amplitude of $|{\mathbf{G}}_{1}|$.
$\Omega=N_{k}|\mathbf{a}_{\mathrm{M1}}||\mathbf{a}_{\mathrm{M2}}|\sqrt{3}/2$
is the total area of morié superlattice that we consider in real space, where
$N_{k}=L\times L$ is the number of ${\mathbf{k}}$ points sampling in mBZ and
$L$ is the linear size, and $\mathbf{a}_{\mathrm{M1}}$ and
$\mathbf{a}_{\mathrm{M2}}$ are the lattice vectors of mUC as shown in Fig. S2,
with $|\mathbf{a}_{\mathrm{M1/M2}}|=\sqrt{3}a/[2\mathrm{sin}(\theta/2)]$ and
$\theta=1.08^{\circ}$, the twisting angle.

As shown in Fig. S3 in the appendix, even in non-chiral cases ($u_{0}>0$), the
two low-energy bands of $H_{0}$ are quite flat and isolated from the remote
bands, except when $u_{0}\approx u_{1}$ and $u_{0}>160$ meV. In these isolated
flat-band cases, therefore, the low-energy physics of $H=H_{0}+H_{\mathrm{I}}$
can be well captured by the two flat bands, and $H$ can thus be projected onto
them. Setting the eigenvalues and eigenvectors of $H^{\eta,s}({\mathbf{k}})$
to be $\epsilon^{\eta,s}_{m}({\mathbf{k}})$ and
$\left|u_{{\mathbf{k}},m,\eta,s}\right>$, where $m=\pm$ is the index for the
flat bands, and projecting $H$ onto the flat bands, as
$c^{\dagger}_{{\mathbf{k}},m,\eta,s}=\sum_{{\mathbf{G}},X}c^{\dagger}_{{\mathbf{k}},{\mathbf{G}},\eta,s,X}\left|u_{{\mathbf{k}},m,\eta,s}\right>_{{\mathbf{G}},X}$,
yields

$\displaystyle
H_{0}=\sum_{{\mathbf{k}},m,\eta,s}\epsilon^{\eta,s}_{m}({\mathbf{k}})c^{\dagger}_{{\mathbf{k}},m,\eta,s}c_{{\mathbf{k}},m,\eta,s},$
(6) $\displaystyle
H_{\mathrm{I}}=\frac{1}{2\Omega}\sum_{|{\mathbf{q}}+{\mathbf{G}}|\neq
0}V({\mathbf{q}}+{\mathbf{G}})\delta\rho_{{\mathbf{q}}+{\mathbf{G}}}\delta\rho_{-{\mathbf{q}}-{\mathbf{G}}},$

where
$\delta\rho_{{\mathbf{q}}+{\mathbf{G}}}=\sum_{{\mathbf{k}},m_{1},m_{2},\eta,s}\lambda_{m_{1},m_{2},\eta,s}({\mathbf{k}},{\mathbf{k}}+{\mathbf{q}}+{\mathbf{G}})\left(c_{{\mathbf{k}},m_{1},\eta,s}^{\dagger}c_{{\mathbf{k}}+{\mathbf{q}}+{\mathbf{G}},m_{2},\eta,s}-\frac{\nu+4}{8}\delta_{{\mathbf{q}},0}\delta_{m_{1},m_{2}}\right)$,
and the form factor
$\lambda_{m_{1},m_{2},\eta,s}(\mathbf{k},\mathbf{k}+\mathbf{q}+{\mathbf{G}})\equiv\left\langle
u_{{\mathbf{k}},m_{1},\eta,s}\mid
u_{{\mathbf{k}}+{\mathbf{q}}+{\mathbf{G}},m_{2},\eta,s}\right\rangle$. To
efficiently simulate the system at filling $\nu=-3$, same as earlier studies
[33, 34], we take two half-filled flat bands of $\eta=+$ and $s=\uparrow$ into
consideration due to its spin and valley polarized ground state [33, 34, 29],
while other flat bands of the rest combinations of spin and valley, which
remain empty, can be safely ignored in the simulations. Note that we strip the
indices of valley and spin, $\eta$ and $s$, in the followings. The above $H$
is tackled by determinant QMC method, the detail of which is depicted in
Appendix A. We would like to note that our simulation ingnores the kinetic
term $H_{0}$, due to its flat dispersion, after the projection onto flat
bands, thus corresponding to a perfect flat band limit. Moreover, our model
tends to predict the nature of pristine TBG and does not take any heterostrain
into account, and therefore the Incommensurate Kekule Spiral state found in
experiments [58, 59, 60, 61] will not be detected.

Figure 3: $LDOS$ and quasiparticle spectra with $L=6$ and $T=3$ meV. (a)
Normalized local density of states ($LDOS$) versus energy $\omega$ and
$u_{0}$. (b), (c), and (d) Single-particle spectra $A(\mathbf{k},\omega)$
along a high-symmetry path when $u_{0}=0$, $u_{0}=40$ meV, and $u_{0}=90$ meV,
respectively, where the blue lines denote the spectra of $\Gamma$, while the
orange lines are $A(\mathbf{k},\omega)$ of other momentum points. Please note
that the vertical axis in (d) is broken, due to the sharp peak of $\Gamma$, to
make the drawing compact. Mind that $T=3$ meV is blow $T_{\mathrm{c}}$ when
$u_{0}=0$ but above it when $u_{0}=40$ meV.

Results.— The ground state of this system at chiral limit is found to be a
Chern-number polarized QAH state [38]. To investigate if the QAH state
sustains in non-chairal cases, the correlation of its QAH order is calculated.
The QAH order parameter is defined as $P_{\mathrm{C}}/N_{\mathbf{k}}$, where
$P_{\mathrm{C}}=N_{+}-N_{-}$ and
$N_{m}=\sum_{\mathbf{k}}c^{\dagger}_{{\mathbf{k}},m}c_{{\mathbf{k}},m}$, the
occupation number operator of two flat bands. The correlation of this QAH
order, or Chern-number polarization, is
$S\equiv\left\langle\left(N_{+}-N_{-}\right)^{2}\right\rangle/N_{\mathbf{k}}^{2}$.
Its $T$ and $u_{0}$ dependence for $L=4$ system is shown in Fig. 1 (b). For
specific $u_{0}$, $S$ increases with decreasing $T$, which manifests
spontaneous time-reversal symmetry breaking and a QAH ground state (a
topological insulator) at low temperature [38, 62, 63]. For each fixed
$u_{0}$, finite-size scaling is carried out to derive the $T_{\mathrm{c}}$ of
the spontaneous time-reversal symmetry breaking of the QAH state, as shown in
Fig. 1 (c)-(f), where the 2D Ising exponents $\beta=1/8$ and $\nu=1$ for the
scaling form $S(T,L)L^{2\beta/\nu}=f((T-T_{\mathrm{c}})L^{1/\nu})$, with $f$
being the scaling function, are used. The obtained $T_{\mathrm{c}}$ reduces
with increasing $u_{0}$, as shown in Fig. 1 (a), and it can be extrapolated
that, when $u_{0}\approx 90$ meV, $T_{\mathrm{c}}$ is zero within our
resolution as indicated by the dashed fitting line with an error of 5 meV.

Such vanishing of the QAH phase as a function of $u_{0}$ is consistent with
the evolution of the quasiparticle gap of $\Gamma$, $\Delta_{\Gamma}$, with
different system sizes ($N_{k}$) at the temperature 3 meV, as shown in Fig. 2
(a). The counterparts from ED also yield the same tendency, as shown in Fig.
S6. The values of $\Delta_{\Gamma}$ are obtained from fitting imaginary time
($\tau$) Green’s functions (GFs) to a range with linear $\tau$-dependence of
logarithmic GFs, see Figs. S4 and S5 in the appendix. As a system should be
gapless if there is at least one kind of gapless quasiparticles, therefore,
the marginal and converged values of $\Delta_{\Gamma}$ of different system
sizes indicate the system gapless when $u_{0}>0.8u_{1}$, as shown in Fig. 2
(a) and Fig. S6, although quasiparticle gaps of other ${\mathbf{k}}$ points of
mBZ remain significant even for large $u_{0}$, as shown in Fig. S5, suggesting
an SM phase. In addition, as also depicted in Fig. 2 (a), due to finite-size
effect, the value of $\Delta_{\Gamma}$ with a small $u_{0}$ increases but
converges as the system size enlarges. It is found as well that the values of
$T_{\mathrm{c}}$ are much smaller than those of the quasiparticle gaps, due to
particle-hole excitons with smaller energies above the QAH phase, which
restore the broken time-reversal symmetry above $T_{\mathrm{c}}$ and has been
evidenced in our previous works [63, 38]. Moreover, the spectra (will be
discussed below) of quasiparticles of the Dirac points in mBZ, $K_{1}$ and
$K_{2}$ (see Fig. S2 for their positions), from the 6$\times$6 system are
always and well gapped through the $u_{0}$ range investigated, as shown in
Fig. 3 (b)-(d), which thus prove that the Dirac points do not play key roles
in the transition towards the SM phase.

The charge compressibility, $\kappa=d\langle
N\rangle/d\mu/n_{0}/N_{k}=\beta\left(\langle N^{2}\rangle-\langle
N\rangle^{2}\right)/n_{0}/N_{k}$, where $N=\sum_{m}N_{m}$ and $n_{0}$ is the
particle density, is also calculated to to evidence the transition from the
QAH state to the SM state, whose values are shown in Fig. 2 (b). The vanishing
of $\kappa$ indicates the state incompressible and insulating, and
compressibly conducting otherwise. Therefore, a small $u_{0}$ yields an
insulating ground state, which is the QAH phase, while a large $u_{0}$ leads
to a conducting ground state, corresponding to the SM phase. Such that the
Chern-number polorization, the quasiparticle gap, and the charge
compressibility yield the same physics.

To further reveal the evolution from the QAH phase to the SM phase with
increasing $u_{0}$, we derive the quasiparticle spectra,
$A(\mathbf{k},\omega)$, and the correspoding local density of state,
$LDOS=\sum_{\mathbf{k}}A({\mathbf{k}},\omega)/N_{\mathbf{k}}$, from the
stochastic analytic continuation scheme [64, 65], the results of $L=6$ system
at $T=3$ meV of which are shown in Fig. 3. As is consistent to
$T_{\mathrm{c}}$ of the QAH phase at a specific $u_{0}$,
$A(\mathbf{k},\omega)$ is gapped for all ${\mathbf{k}}$-points when $T$ is
below $T_{\mathrm{c}}$ as shown in Fig. 3 (b), and gapless around $\Gamma$
when $T$ is slightly above $T_{\mathrm{c}}$ as shown in Fig. 3 (c), suggesting
the sate above QAH a semimetal and the quasiparticles high energy exitons.
Therefore, $T_{\mathrm{c}}$s set the phases boundary between QAH state and SM,
indicating again the ground state of magic-angle TBG is semimetallic when
$u_{0}>0.8u_{1}$, as being further evidenced by Fig. 3 (d), which is
consistent to the experimentally found SM phases [43, 44, 45]. The gap of
$A(\Gamma,\omega)$ is the least and gradually decreases with growing $u_{0}$
(Fig. 3 (b), (c) and (d), where two peaks gradually merge into a single one.
The gap totally closes when $u_{0}=90$ meV (Fig. 3 (d)). The $LDOS$ remains
gapped except a peak around Fermi level arises as $u_{0}$ increases (Fig. 3
(a)). This peak comes from the closing of $\Delta_{\Gamma}$, as shown in Fig.
3 (c) and (d), while the spectra of other quasiparticles remain well gapped.

Figure 4: The decaying of average Green’s function (a) and the reducing of the
peak at Fermi level of $LDOS$ (c) with the system size up to $L=8$ when
$u_{0}=90$ meV. The extrapolations of $G(\beta/2)$ and the intensity of the
peak of $LDOS$ at Fermi level to larger sizes are shown in (b) and (d),
respectively, where the fitting is quadratic due to that it is versus $1/L$,
except the horizontal part in (d).

Average Green’s function summing over ${\mathbf{k}}$ points and $LDOS$ with
different linear sizes are demonstrated in Fig. 4, where $u_{0}=90$ meV. The
convergence of decaying of $G(\tau)$ can be seen, and $G(\beta/2)$ reduces
probably to a minimum but finite value at the large $L$ limit, as shown in (a)
and (b) respectively, suggesting that there are finite electrons at the Fermi
level. Meanwhile, the peak of $LDOS$ at Fermi level decreases probably to a
minimum but finite value as the system size is larger than 6, as depicted in
Fig. 4 (d). The probable non-zero extropaltions suggest quadratic band
touching of the semimetallic phase of the magic-angle TBG when $u_{0}=90$ meV.
However, a linear fit extrapolating to 0, indicating degenerate Dirac cone
touching, cannot be totally ruled out in both Fig. 4 (b) and (d). To further
verify that the probable semimetallic state is of quadratic band touching,
here we check if the $C_{3}$ rotation symmetry is broken by scrutinizing the
correlation of a nematic order parameter, which is defined as
$\mathcal{N}=\frac{1}{N_{k}}\sum_{k}c^{\dagger}_{{\mathbf{k}},+}c_{{\mathbf{k}},-}$.
The correlation of $\mathcal{N}$,
$S_{\mathcal{N}}=\langle\mathcal{N}^{\dagger}\mathcal{N}\rangle$, is
calculated and illustrated in Fig. S7, as the system is in the SM phase. Since
the value of $S_{\mathcal{N}}$ extrapolates to zero towards the thermodynamic
limit, the $C_{3}$ rotation symmetry is probably not broken. Therefore, our
found SM phase is probably not nematic, enhancing the probability of the SM
state being a quadratic band touching semimetal.

As degenerate Dirac cone touching cannot be ruled out, we argue that it is
less probable in our case. Based on the Nielsen–Ninomiya theorem of fermion
doubling [66], Dirac cones shall appear in pairs. The only way obeying the
fermion doubling theorem is to have two degenerate Dirac cones at $\Gamma$.
However, considering that we only have one valley/spin degree of freedom (no
symmetry to enforce two-fold degeneracy), there is no obvious reason why the
system would stabilize such a two-fold degenerate energy spectrum.
Nevertheless, it should be emphasized that this is not impossible, and it
would imply that this semimetal phase is highly unconventional. Aside from the
unconventional pathway towards Dirac semimetal, there is another less exotic
possibility producing Dirac points: nematic Dirac semimetal. The system has a
pair of Dirac points near $\Gamma$, one at a momentum point ${\bf q}$ and the
other at ${\bf-q}$. However, such a scenario requires the breaking of
rotational symmetry, which is probably preserved in our case, as indicated by
$S_{\mathcal{N}}$. For quadratic band touching, as there is no constraint of
doubling theorem, it is allowed to have a single quadratic band crossing at
$\Gamma$.

The closing of the quasiparitcle gap of $\Gamma$ is consistent with the
recently proposed heavy-fermion picture of magic-angle TBG [50, 51, 52, 53,
54, 55, 32, 56]. In the heavy-fermion representations, the flat bands are
mostly composed of localized electrons, except for the tiny vicinity of
$\Gamma$, which is mainly contributed by delocalized electronic states. In the
flat-band limit, strong interactions between localized electrons ought to
induce energy gaps in the quasiparticle spectra. However, these interaction-
induced gaps are expected to be much smaller in the narrow surrounding of
$\Gamma$, since the fermion states there are delocalized with weaker
interaction, and it is indeed so at chiral limit and when $u_{0}=40$ meV, as
shown in Fig. 3 (b) and (c). Moreover, the interactions among fermions around
$\Gamma$ can be further weaker as the system deviates from chiral limit, and
the corresponding quasiparitcle gaps decline and eventually close when $u_{0}$
goes beyond a threshold value ($u_{0}\approx 0.8u_{1}$ judged from
$T_{\mathrm{c}}$), as also illustrated by Fig. 3 (d).

Figure 5: Real-space spectra $A(r,\omega)$ at various $u_{0}$ and $\omega$
when $T=3$ meV. Here $u_{0}$ equals to (a) 0 meV, (b) 40 meV, and (c) 90 meV
respectively, while $\omega$ is chosed to be 0, 7.5, 15, and 22.5 meV in each
figure. The intensity of blue stands for the value of $A(r,\omega)$, with a
color bar locating at the right of (c). Positions of AA, AB, and BA stacking
areas are labeled in (c).

To further scrutinize the heavy-fermion features of the SM phase, here we
directly probe the localized and delocalized electron states via real-space
spectra by projecting $A({\mathbf{k}},\omega)$ of the $L=6$ system at $T=3$
meV with the real-space wavefunction $\psi_{\mathbf{k}}(\mathbf{r},X)$ of
$H_{0}$,
$A(r,\omega)=\sum_{{\mathbf{k}},X}\psi^{\dagger}_{\mathbf{k}}(\mathbf{r},X)A({\mathbf{k}},\omega)\psi_{\mathbf{k}}(\mathbf{r},X)$,
as a function of $u_{0}$ and shown in Fig. 5. It can be clearly seen that the
density of low energy electrons ($\omega=0$ meV, first row of panels (a) -
(c)) increases in AB/BA stacking regions (or is lifted as the maximum values
are around AA stacking center, where however the minimum values locate),
marking the emergence of the SM state. Meanwhile, the localized electrons
($\omega=15,22.5$ meV, bottom two rows of panels (a) - (c)) always and further
concentrate at AA centers as $u_{0}$ grows, which is consistent with the
heavy-fermion picture [50, 51] and experiments [8, 9]. Furthermore, the
density of electrons with intermediate energy ($\omega=7.5$ meV, second row in
panel (c)) manifests a minimum value, separating the itinerant and localized
eletrons when $u_{0}=90$ meV. These observations suggest that the realistic
(when $u_{0}\approx 0.8u_{1}$) magic-angle TBG of $\nu=-3$ can be preliminary
viewed as a heavy-fermion system. Besides, the itinerant electrons of
$A(r,\omega=0)$ in panel (c) come from $A(\Gamma,\omega=0)$ in Fig. 3 (d).

Discussions.— In this study, an evolution from the Chern insulator to a heavy-
fermion semimetal probably preserving $C_{3}$ symmetry is found for $\nu=-3$
filling magic-angle TBG by increasing AA hopping strength, $u_{0}$, which can
be experimentally altered. The ground state phase transition happens at
$u_{0}\approx 0.8u_{1}$ or 90 meV, where the Chern-number polorization and the
quasiparticle gap of $\Gamma$ vanish, the system becomes compressible, and a
peak in $LDOS$ at Fermi level emerges due to the gap’s closing. The intensity
of this peak rises as $u_{0}$ increases for a specific system size, but
reduces probably to a minimum but finite value as $L$ enlarges for a specific
$u_{0}$, which manifests a semimetal state as the gaps of other momenta remain
significant. The momentum and real-space distributions of the localized
electrons are well separated from those of itinerant electrons, which are of
the $\Gamma$ point. The nature of the found semimetal state implies that the
magic-angle TBG can be preliminarily, based on the numerics here, treated as a
heavy-fermion system when $\nu=-3$. Our findings shed new lights on the
correlated nature of TBG and provide critical information for understanding
this fascinating system.

Acknowledgments — We thank DongKeun Ki and Wang Yao for useful discussions.
CH, XZ, GPP and ZYM acknowledge the support from the Research Grants Council
(RGC) of Hong Kong SAR of China (Projects No. 17302223, No. 17301721, No.
AoE/P-701/20, No. 17309822, and No. C7037-22GF), the ANR/RGC Joint Research
Scheme sponsored by RGC of Hong Kong and French National Research Agency
(Project No. A_HKU703/22), the Strategic Priority Research Program of Chinese
Academy of Sciences (Grant No. XDB33000000), and K. C. Wong Education
Foundation (Grant No. GJTD-2020-01). We thank the HPC2021 system of
Information Technology Services and the Blackbody HPC system at the Department
of Physics, of the University of Hong Kong, and Beijing PARATERA Tech Co.,
Ltd. (https://cloud.paratera.com) for providing computational resources that
have contributed to the research results in this paper.

##  I Appendix A: The momentum-space determinant QMC method

Due to the flatness of the lowest energy bands, as shown in Fig. S3, $H_{0}$
is ignored in our simulations. The partition function thus reads
$Z=\mathrm{Tr}\left(\mathrm{e}^{-\beta H_{\mathrm{I}}}\right)$, where
$\beta=1/k_{\mathrm{B}}T$ and $T$ is the temperature. $Z$ can be Trotter-
decoupled after spliting inverse temperature into $N_{\tau}$ pieces,
$\beta=\triangle\tau N_{\tau}$, with an error of order $\triangle\tau^{2}$.
That is

$\displaystyle Z$ $\displaystyle=\mathrm{Tr}\left(\mathrm{e}^{-\beta
H_{\mathrm{I}}}\right)$ (7)
$\displaystyle=\mathrm{Tr}\left(\left(\mathrm{e}^{-\triangle\tau
H_{\mathrm{I}}}\right)^{N_{\tau}}\right)$
$\displaystyle=\mathrm{Tr}\left(\prod^{\beta}_{\tau=\triangle\tau}\mathrm{e}^{-\triangle\tau
H_{\mathrm{I}}}\right)+\mathcal{O}(\triangle\tau^{2}).$

Denoting ${\mathbf{Q}}={\mathbf{q}}+{\mathbf{G}}$, $H_{\mathrm{I}}$ can be
rewritten as

$\displaystyle H_{\mathrm{I}}$ $\displaystyle=\sum_{|{\mathbf{Q}}|\neq
0}\frac{1}{2\Omega}V({\mathbf{Q}})\delta\rho_{{\mathbf{Q}}}\delta\rho_{-{\mathbf{Q}}}$
(8)
$\displaystyle=\sum_{\mathcal{Q}}\frac{1}{2\Omega}V(\mathcal{Q})(\delta\rho_{\mathcal{Q}}\delta\rho_{-\mathcal{Q}}+\delta\rho_{-\mathcal{Q}}\delta\rho_{\mathcal{Q}})$
$\displaystyle=\sum_{\mathcal{Q}}\frac{1}{4\Omega}V(\mathcal{Q})\left(\left(\delta\rho_{-\mathcal{Q}}+\delta\rho_{\mathcal{Q}}\right)^{2}-\left(\delta\rho_{-\mathcal{Q}}-\delta\rho_{\mathcal{Q}}\right)^{2}\right),$

where $\mathcal{Q}$ is the half of $|{\mathbf{Q}}|\neq 0$ [35]. Thus,
$\mathrm{e}^{-\triangle\tau H_{\mathrm{I}}}$ can be further decoupled to an
auxiliary field by the Hubbard-Stratonovich transformation, with an error of
order $\triangle\tau^{4}$, as

$\displaystyle\mathrm{e}^{-\triangle\tau H_{\mathrm{I}}}=$ (9)
$\displaystyle\prod_{\mathcal{Q}}\frac{1}{16}\sum_{l_{1\mathcal{Q}},l_{2\mathcal{Q}}}\gamma(l_{1\mathcal{Q}})\gamma(l_{2\mathcal{Q}})\mathrm{e}^{i\sqrt{\alpha}\eta(l_{1\mathcal{Q}})\left(\delta\rho_{-\mathcal{Q}}+\delta\rho_{\mathcal{Q}}\right)}\mathrm{e}^{\sqrt{\alpha}\eta(l_{2\mathcal{Q}})\left(\delta\rho_{-\mathcal{Q}}-\delta\rho_{\mathcal{Q}}\right)}$
$\displaystyle+\mathcal{O}(\triangle\tau^{4}),$

where $l=\pm 1,\pm 2$, is the auxilariy field, and $\gamma(\pm
1)=1+\sqrt{6}/3$, $\gamma(\pm 2)=1-\sqrt{6}/3$, $\eta(\pm
1)=\pm\sqrt{6-2\sqrt{6}}$, and $\eta(\pm 2)=\pm\sqrt{6+2\sqrt{6}}$.
$\alpha=\triangle\tau V(\mathcal{Q})/4\Omega$.

The partition function can be finally written as $Z=\sum_{\\{l_{\tau
1\mathcal{Q}},l_{\tau 2\mathcal{Q}}\\}}\mathrm{Tr}U_{C}$, with

$\displaystyle U_{C}=\prod^{\beta}_{\tau=\triangle\tau}\prod_{\mathcal{Q}}$
(10) $\displaystyle\frac{1}{16}\gamma(l_{\tau 1\mathcal{Q}})\gamma(l_{\tau
2\mathcal{Q}})\mathrm{e}^{i\sqrt{\alpha}\eta(l_{\tau
1\mathcal{Q}})\left(\delta\rho_{-\mathcal{Q}}+\delta\rho_{\mathcal{Q}}\right)}\mathrm{e}^{\sqrt{\alpha}\eta(l_{\tau
2\mathcal{Q}})\left(\delta\rho_{-\mathcal{Q}}-\delta\rho_{\mathcal{Q}}\right)},$

a configuration of $\\{l_{\tau 1\mathcal{Q}},l_{\tau 2\mathcal{Q}}\\}$ labeled
by $C$. Physical quantities, $O$, can thus be obtained by sampling the
auxiliary field, as

$\displaystyle\langle O\rangle$
$\displaystyle=\frac{\mathrm{Tr}\left(\mathrm{e}^{-\beta
H_{\mathrm{I}}}O\right)}{Z}$ (11)
$\displaystyle=\sum_{C}\frac{\mathrm{Tr}\left(U_{C}O\right)}{Z}$
$\displaystyle=\sum_{C}\frac{\mathrm{Tr}\left(U_{C}\right)}{Z}\frac{\mathrm{Tr}\left(U_{C}O\right)}{\mathrm{Tr}\left(U_{C}\right)},$

with $\mathrm{Tr}\left(U_{C}\right)/Z$ or $\mathrm{Tr}\left(U_{C}\right)$ the
sampling weight and denoted as $P_{C}$.

However, $P_{C}$ of the magic-anlge TBG with $\nu=-3$ is not constantly
positive, or sign-problematic, as shown in Fig. S1. Nevertheless, the
symmetries of TBG make $P_{C}$ real [35] and the average of its sign, $\langle
sign\rangle=\sum_{C}sgn(P_{C})$, is always positive and ranges from 0 to 1.
Here 1 means no sign problem while 0 means the simulation is valueless. It is
quite intuitive how to avoid negitive weight. Denoting
$O_{C}=\mathrm{Tr}\left(U_{C}O\right)/\mathrm{Tr}\left(U_{C}\right)$, $\langle
O\rangle$ can be rewritten as

$\displaystyle\langle O\rangle$
$\displaystyle=\frac{\sum_{C}O_{C}P_{C}}{\sum_{C}P_{C}}$ (12)
$\displaystyle=\frac{\sum_{C}O_{C}\mathrm{sgn}(P_{C})|P_{C}|/\sum_{C}|P_{C}|}{\sum_{C}\mathrm{sgn}(P_{C})|P_{C}|/\sum_{C}|P_{C}|}$

Thus, the numerator and the denominator can be sampling separately by
$|P_{C}|/\sum_{C}|P_{C}|$, with $O_{C}\mathrm{sgn}(P_{C})$ and
$\mathrm{sgn}(P_{C})$ being the measured quantities, and $\langle O\rangle$
can be further obtained.

Our stable and reproducible results show that a QMC simulation can still be
efficiently and reliably done when $\langle sign\rangle>0.01$ with reasonable
computing resources. In the obtaining of reliable imaginary time Green’s
function, the simulation temperature is set to be $T=3$ meV considering the
$\langle sign\rangle$ of the $L=6$ system, while the $T$ can be lowered when
measuring Chern-number polarization. With the reliable imaginary time Green’s
function, we further employ the stochastic analytic continuation (SAC) [36,
67, 68, 69, 70, 38] method to extract the real frequency spectra.

Figure S1: $\langle sign\rangle$ of magic-angle TBG with $\nu=-3$, when (a)
$L=4$ varying $T$ and $u_{0}$, and (b) $L=5,6$ and $T=3$ meV varying $u_{0}$.
$\langle sign\rangle$ reaches 0 when $u_{0}\geq 40$ meV as $T=1$ meV in (a),
while $\langle sign\rangle$ of $L=6$ reaches 0.01 when $u_{0}=40$ meV in (b).

##  II Appendix B: Moiré superlattice and single-particle dispersions

The real space moiré superlattice generated by TBG is shown in Fig. S2, where
a mUC and its riciprocal first Brillouin zone (mBZ) are drawn. The module of
the lattice vectors of mBZ is
$|{\mathbf{G}}_{1}|,|{\mathbf{G}}_{2}|=8\pi\mathrm{sin}(\theta/2)/(3a)$.
${\mathbf{K}}_{1}^{\eta}$ and ${\mathbf{K}}_{2}^{\eta}$ are Dirac points from
the top (1) and bottom (2) layers, respectively. Schematics of hopping
patterns are also shown in the right panel of Fig. S2. In this work, we
consider the system with the first magic angle ($\theta=1.08^{\circ}$).

Figure S2: Moiré unit cell (blue hexagon) and its Brillouin zone (black
hexagon), as well as three kinds of stackings and hoppings (AA, AB, and BA) of
TBG. $u_{0}$ and $u_{1}$ are hopping strengths of AA- and AB/BA-stacking
areas, respectively. The top (bottom) layer of TBG are denoted by cyan (red)
color. $\\{\mathbf{a}_{\mathrm{M1}},\mathbf{a}_{\mathrm{M2}}\\}$ and
$\\{\mathbf{G}_{1},\mathbf{G}_{2}\\}$ are the lattice vectors of mUC and mBZ,
respectively. $\Gamma$ is the origin of mBZ, and $\mathbf{K}_{1}$ and
$\mathbf{K}_{2}$ are the two Dirac points of the same vellay and different
layers (1, 2). The twisted angle $\theta$ in this schematic is $10^{\circ}$,
while that of the case we investigate in the main text is $1.08^{\circ}$.

From this moiré supperlattice, a momentum space single-particle Hamiltonian,
$H_{0}$, considering intralayer hopping and interlayer hopping, can be
constructed, as descriped in the main text. The dispersions of $H_{0}$ at the
magic angle with different $u_{0}$ are shown in Fig. S3, where two low energy
bands are quite flat and isolated from remote bands when $u_{0}\leq 90$ meV.
It implies that the low energy physics can be well captured by these two
bands. The projection of $H$ into flat bands, as depicted in the main text,
makes the description of $H$ easier since only two bands are considered.
However, we suppose that the projections when $u_{0}=110$ (meV) and
$u_{0}>160$ (meV) fail as remote bands are too close to flat bands.

Figure S3: Single-particle dispersions of $H_{0}$ at the first magic angle and
along a high-symmetry line. The remote bands are beyond the scale when
$u_{0}<70$ meV, and approches the flat bands around $\Gamma$ point as $u_{0}$
further increases. Note that the lowest remote bands touch the flat bands at
$\Gamma$ point only when $u_{0}=110$ meV ($u_{0}=u_{1}$). The band width of
the flat bands enlarge as $u_{0}$ increases, but remainds a quite small value
(less than 5 meV when $u_{0}=90$ meV).

##  III Appendix C: The Calculation of Chern-number polorization

A perturbation term, which reads

$\displaystyle\delta H=\left(\begin{array}[]{cc}0&0\\\
0&\delta\sigma_{\mathrm{z}}\end{array}\right),$ (13)

with $\delta=1$ meV and $\sigma_{\mathrm{z}}$ being the third Pauli matrix, is
added to the bottom layer of $H_{0}$ to break inversion and an isolated Chern
bands basis $|\psi\rangle$ is obtained. This perturbation term breaks
$C_{\mathrm{2z}}\mathcal{T}$ but $\mathcal{T}$, which means the spontaneous
time-revesal symmetry breaking of the TBG system is still alowed. Denoting the
original basis as $|\phi\rangle$, the imaginary-time Green’s function in the
isolated Chern bands basis can thus be related to the original basis, by
$\langle\psi|G({\mathbf{k}})|\psi^{\prime}\rangle=\sum_{\phi,\phi^{\prime}}\langle\psi|\phi\rangle\langle\phi|G({\mathbf{k}})|\phi^{\prime}\rangle\langle\phi^{\prime}|\psi^{\prime}\rangle$.
Thus, the correlation function of the QAH order, or Chern-number polarization,
can be calculated in the isolated Chern band basis. As this perturbation term
resembles an h-BN potential, therefore the strength, $\delta$, which can be as
low as 0.3 meV, is choosed to be 1 meV to minimize the relavance. This
perturbation term is only added when calculating Chern-number polarization.

##  IV Appendix D: Imaginary time Green’s funtion and derived gaps

The decaying of $G(\Gamma,\tau)$ is depicted in Fig. S4, where the convergence
of decaying can be seen with increasing $L$. The minimum single-particle gap
of each momentum can be obtained by fitting
$G(\tau)=\mathrm{exp}(-\Delta\tau)/2+\mathrm{exp}\left(-\Delta(\beta-\tau)\right)/2$
to the data in a range with linear $\tau$-dependence of logarithmic GFs as
shown in Fig. S4, and the obtained gaps are demonstrated in Fig. 2 (a) and
Fig. S5, where those of other ${\mathbf{k}}$ points are also shown.

Figure S4: Decaying of imaginary time Green’s function of $\Gamma$ point with
$L=3,4,5,6$, when (a) $u_{0}=0$ meV, (b) $u_{0}=60$ meV, and (c) $u_{0}=90$
meV, where the log scale is used. Mind that the vertical scales of (b) and (c)
is different from that of (a). The fitting dashed lines are also shown in each
panel, the slopes of which give the quasiparticle gaps in Fig. 2 (a) and Fig.
S5. Figure S5: Quasiparticle gaps of two bands, $\pm$, (a) and (b), of
${\mathbf{k}}$ points in mBZ with $L=4$ and varing $u_{0}$. Here $\Gamma$ and
$M$ refer to the red circle-line and the black circle-line, respectively, and
the rest circle-lines or star-lines are those of the rest momentum points.
Error bars of many data points are smaller than corresponding markers’ sizes.
Note that the gaps of - band remain quite large values ($>8$ meV) when
$u_{0}=150$ meV, as shown in (b).

The gaps of $L=3$ from ED, considering the computational complexity, is
calculated as well and shown in Fig. S6 (green squares), where counterparts
from QMC (red squares), with the simulation temperature being 3 meV, are also
shown. As the result from ED is of ground state, gaps from QMC at 3 meV are
slightly lower than those from ED, and the lowering of simulation $T$ can
reduce the deviation. Besides, the same tendency is obtained that the gap
closes at $u_{0}\approx 0.8u_{1}$. In addition, the detail of implementaion of
ED is depicted in Appendix G.

Figure S6: Quasiparticle gaps of $\Gamma$ from ED (green squares) when $L=3$
compared with counterparts from QMC (red squres) with $T=3$ meV.

##  V Appendix E: $C_{3}$ rotation symmetry of the semimetallic phase

The correlation of the nematic order, $\mathcal{N}$, as a function of reversed
linear size, $1/L$, with $u_{0}=$ 90 meV is shown in Fig. S7. As there might
be some correlation in a small system, there is none in the thermodynamic
limit, suggesting the absence of this nematic order and the presence of
$C_{3}$ rotation symmetry.

Figure S7: Correlation, $S_{\mathcal{N}}$, of the nematic order parameter,
$\mathcal{N}$, as a function of reversed linear size, $1/L$.

##  VI Appendix F: Implementation of the exact diagonalization

Exact diagonalization (ED) is implemented to obtain the charge gap of the
system in comparison to those from QMC. For $L\times L$ ($\equiv N_{k}$)
systems with the two nearly flat bands taken into account, there are $2L^{2}$
single particle states in total which are labeled by momentum and band index.
Here we use an integer $i\in[0,2L^{2}-1]$ to label each single particle state,
$d^{\dagger}_{i}\equiv d^{\dagger}_{\mathbf{k}m}$, whose momentum $\mathbf{k}$
and band index $m$ are determined by $i$. The Hamiltonian in Eq. (6) can be
rewritten in the following form by moving all electron creation operator to
the left:

$H=\sum_{i,i^{\prime},j^{\prime},j}V_{2}(i,i^{\prime},j^{\prime},j)d_{i}^{\dagger}d_{i^{\prime}}^{\dagger}d_{j^{\prime}}d_{j}+\sum_{i,j}V_{1}(i,j)d_{i}^{\dagger}d_{j}+V_{0}.$
(14)

The matrix element between arbitrary manybody states can be obtained from Eq.
(14). Let $\lvert\psi\rangle$ be a simple direct product state
$\lvert\psi\rangle=C^{\dagger}_{i_{N-1}}...C^{\dagger}_{i_{0}}\lvert 0\rangle$
whose creation operators are ordered by $i_{N-1}>...>i_{1}>i_{0}$. Then
$\lvert\psi\rangle$ can also be represented by a string of bits with length
$2L^{2}$ such as $\lvert 1101001...\rangle$. The Hamiltonian can be written in
the basis made of these distinct direct product states. For a given product
state $|\psi\rangle$, if there is another product state
$|\psi^{\prime}\rangle$ such that $\langle\psi^{\prime}|H|\psi\rangle\neq 0$,
then either of the two situations occurs: (1) there exist distinct numbers
$i\neq i^{\prime}$ such that $|\psi_{1}^{\prime}\rangle\propto
d^{\dagger}_{i^{\prime}}d_{i}|\psi\rangle$; (2) there exist distinct numbers
$i,j,i^{\prime},j^{\prime}$ with $i>j,i^{\prime}>j^{\prime}$ such that
$|\psi_{2}^{\prime}\rangle\propto
d^{\dagger}_{j^{\prime}}d^{\dagger}_{i^{\prime}}d_{i}d_{j}|\psi\rangle$. Then,
according to Eq.(14), we have

$\displaystyle\langle\psi_{2}^{\prime}\rvert H\lvert\psi\rangle=$
$\displaystyle(-1)^{\mu(i,j)}(-1)^{\mu^{\prime}(i^{\prime},j^{\prime})}[V_{2}(j^{\prime},i^{\prime},i,j)$
(15) $\displaystyle-
V_{2}(j^{\prime},i^{\prime},j,i)-V_{2}(i^{\prime},j^{\prime},i,j)+V_{2}(i^{\prime},j^{\prime},j,i)].$

Here $\mu(i,j)$ is the number of "1" between the $i$-th and $j$-th bits in the
bit-representation of $\lvert\psi\rangle$. This sign factor comes from the
Fermi anti-commutation of creation operators. Similarly, for
$|\psi_{1}^{\prime}\rangle\propto d^{\dagger}_{i^{\prime}}d_{i}|\psi\rangle$,
we have

$\displaystyle\langle\psi_{1}^{\prime}\rvert H\lvert\psi\rangle=\sum_{j}$
$\displaystyle(-1)^{\mu(i,j)}(-1)^{\mu^{\prime}(i^{\prime},j)}\left[V_{2}(j,i^{\prime},i,j)\right.$
(16) $\displaystyle-$
$\displaystyle\left.V_{2}(i^{\prime},j,i,j)-V_{2}(j,i^{\prime},j,i)+V_{2}(i^{\prime},j,j,i)\right]$
$\displaystyle+V_{1}(i^{\prime},i).$

The diagonal matrix elements are

$\displaystyle\langle\psi\rvert H\lvert\psi\rangle=$ $\displaystyle
V_{0}+\sum_{i>j,\psi_{i}=\psi_{j}="1"}[V_{2}(j,i,i,j)-V_{2}(j,i,j,i)$ (17)
$\displaystyle-
V_{2}(i,j,i,j)+V_{2}(i,j,j,i)]+\sum_{i,\psi_{i}="1"}V_{1}(i,i).$

Then $H$ can be diagonalized in the basis of $|\psi\rangle$ to obtain its
eigenvalues.

## References

  * Trambly de Laissardière  _et al._ [2010] G. Trambly de Laissardière, D. Mayou, and L. Magaud, Localization of dirac electrons in rotated graphene bilayers, Nano Letters 10, 804 (2010).
  * Trambly de Laissardière  _et al._ [2012] G. Trambly de Laissardière, D. Mayou, and L. Magaud, Numerical studies of confined states in rotated bilayers of graphene, Phys. Rev. B 86, 125413 (2012).
  * Bistritzer and MacDonald [2011] R. Bistritzer and A. H. MacDonald, Moiré bands in twisted double-layer graphene, Proceedings of the National Academy of Sciences 108, 12233 (2011).
  * Cao  _et al._ [2018a] Y. Cao, V. Fatemi, A. Demir, S. Fang, S. L. Tomarken, J. Y. Luo, J. D. Sanchez-Yamagishi, K. Watanabe, T. Taniguchi, E. Kaxiras, _et al._ , Correlated insulator behaviour at half-filling in magic-angle graphene superlattices, Nature 556, 80 (2018a).
  * Cao  _et al._ [2020] Y. Cao, D. Rodan-Legrain, O. Rubies-Bigorda, J. M. Park, K. Watanabe, T. Taniguchi, and P. Jarillo-Herrero, Tunable correlated states and spin-polarized phases in twisted bilayer–bilayer graphene, Nature 583, 215 (2020).
  * Polshyn  _et al._ [2019] H. Polshyn, M. Yankowitz, S. Chen, Y. Zhang, K. Watanabe, T. Taniguchi, C. R. Dean, and A. F. Young, Large linear-in-temperature resistivity in twisted bilayer graphene, Nature Physics 15, 1011 (2019).
  * Liu  _et al._ [2021a] X. Liu, Z. Wang, K. Watanabe, T. Taniguchi, O. Vafek, and J. Li, Tuning electron correlation in magic-angle twisted bilayer graphene using coulomb screening, Science 371, 1261 (2021a).
  * Xie  _et al._ [2019] Y. Xie, B. Lian, B. Jäck, X. Liu, C.-L. Chiu, K. Watanabe, T. Taniguchi, B. A. Bernevig, and A. Yazdani, Spectroscopic signatures of many-body correlations in magic-angle twisted bilayer graphene, Nature 572, 101 (2019).
  * Choi  _et al._ [2019] Y. Choi, J. Kemmer, Y. Peng, A. Thomson, H. Arora, R. Polski, Y. Zhang, H. Ren, J. Alicea, G. Refael, _et al._ , Electronic correlations in twisted bilayer graphene near the magic angle, Nature physics 15, 1174 (2019).
  * Nuckolls  _et al._ [2020] K. P. Nuckolls, M. Oh, D. Wong, B. Lian, K. Watanabe, T. Taniguchi, B. A. Bernevig, and A. Yazdani, Strongly correlated chern insulators in magic-angle twisted bilayer graphene, Nature 588, 610 (2020).
  * Saito  _et al._ [2021] Y. Saito, J. Ge, L. Rademaker, K. Watanabe, T. Taniguchi, D. A. Abanin, and A. F. Young, Hofstadter subband ferromagnetism and symmetry-broken chern insulators in twisted bilayer graphene, Nature Physics 17, 478 (2021).
  * Das  _et al._ [2021] I. Das, X. Lu, J. Herzog-Arbeitman, Z.-D. Song, K. Watanabe, T. Taniguchi, B. A. Bernevig, and D. K. Efetov, Symmetry-broken chern insulators and rashba-like landau-level crossings in magic-angle bilayer graphene, Nature Physics 17, 710 (2021).
  * Wu  _et al._ [2021] S. Wu, Z. Zhang, K. Watanabe, T. Taniguchi, and E. Y. Andrei, Chern insulators, van hove singularities and topological flat bands in magic-angle twisted bilayer graphene, Nature materials 20, 488 (2021).
  * Potasz  _et al._ [2021] P. Potasz, M. Xie, and A. H. MacDonald, Exact diagonalization for magic-angle twisted bilayer graphene, Phys. Rev. Lett. 127, 147203 (2021).
  * Jaoui  _et al._ [2022] A. Jaoui, I. Das, G. Di Battista, J. Díez-Mérida, X. Lu, K. Watanabe, T. Taniguchi, H. Ishizuka, L. Levitov, and D. K. Efetov, Quantum critical behaviour in magic-angle twisted bilayer graphene, Nature Physics 18, 633 (2022).
  * Cao  _et al._ [2018b] Y. Cao, V. Fatemi, S. Fang, K. Watanabe, T. Taniguchi, E. Kaxiras, and P. Jarillo-Herrero, Unconventional superconductivity in magic-angle graphene superlattices, Nature 556, 43 (2018b).
  * Cao  _et al._ [2021] Y. Cao, D. Rodan-Legrain, J. M. Park, N. F. Yuan, K. Watanabe, T. Taniguchi, R. M. Fernandes, L. Fu, and P. Jarillo-Herrero, Nematicity and competing orders in superconducting magic-angle graphene, science 372, 264 (2021).
  * Yankowitz  _et al._ [2019] M. Yankowitz, S. Chen, H. Polshyn, Y. Zhang, K. Watanabe, T. Taniguchi, D. Graf, A. F. Young, and C. R. Dean, Tuning superconductivity in twisted bilayer graphene, Science 363, 1059 (2019).
  * Diez-Merida  _et al._ [2021] J. Diez-Merida, A. Díez-Carlón, S. Yang, Y.-M. Xie, X.-J. Gao, K. Watanabe, T. Taniguchi, X. Lu, K. T. Law, and D. K. Efetov, Magnetic josephson junctions and superconducting diodes in magic angle twisted bilayer graphene, arXiv preprint arXiv:2110.01067 (2021).
  * Di Battista  _et al._ [2022] G. Di Battista, P. Seifert, K. Watanabe, T. Taniguchi, K. C. Fong, A. Principi, and D. K. Efetov, Revealing the thermal properties of superconducting magic-angle twisted bilayer graphene, Nano Letters 22, 6465 (2022).
  * Lian  _et al._ [2021] B. Lian, Z.-D. Song, N. Regnault, D. K. Efetov, A. Yazdani, and B. A. Bernevig, Twisted bilayer graphene. iv. exact insulator ground states and phase diagram, Phys. Rev. B 103, 205414 (2021).
  * Xie  _et al._ [2021] F. Xie, A. Cowsik, Z.-D. Song, B. Lian, B. A. Bernevig, and N. Regnault, Twisted bilayer graphene. vi. an exact diagonalization study at nonzero integer filling, Phys. Rev. B 103, 205416 (2021).
  * Bultinck  _et al._ [2020] N. Bultinck, E. Khalaf, S. Liu, S. Chatterjee, A. Vishwanath, and M. P. Zaletel, Ground state and hidden symmetry of magic-angle graphene at even integer filling, Phys. Rev. X 10, 031034 (2020).
  * Wilhelm  _et al._ [2021] P. Wilhelm, T. C. Lang, and A. M. Läuchli, Interplay of fractional chern insulator and charge density wave phases in twisted bilayer graphene, Phys. Rev. B 103, 125406 (2021).
  * Xie and MacDonald [2020] M. Xie and A. H. MacDonald, Nature of the correlated insulator states in twisted bilayer graphene, Phys. Rev. Lett. 124, 097601 (2020).
  * Zhang  _et al._ [2020] Y. Zhang, K. Jiang, Z. Wang, and F. Zhang, Correlated insulating phases of twisted bilayer graphene at commensurate filling fractions: A hartree-fock study, Phys. Rev. B 102, 035136 (2020).
  * Liu and Dai [2021] J. Liu and X. Dai, Theories for the correlated insulating states and quantum anomalous hall effect phenomena in twisted bilayer graphene, Phys. Rev. B 103, 035427 (2021).
  * Hejazi  _et al._ [2021] K. Hejazi, X. Chen, and L. Balents, Hybrid wannier chern bands in magic angle twisted bilayer graphene and the quantized anomalous hall effect, Phys. Rev. Res. 3, 013242 (2021).
  * Xie  _et al._ [2023] F. Xie, J. Kang, B. A. Bernevig, O. Vafek, and N. Regnault, Phase diagram of twisted bilayer graphene at filling factor $\nu=\pm{}3$, Phys. Rev. B 107, 075156 (2023).
  * Lin and Ni [2020] X. Lin and J. Ni, Symmetry breaking in the double moiré superlattices of relaxed twisted bilayer graphene on hexagonal boron nitride, Phys. Rev. B 102, 035441 (2020).
  * Kwan  _et al._ [2021a] Y. H. Kwan, G. Wagner, N. Chakraborty, S. H. Simon, and S. A. Parameswaran, Domain wall competition in the chern insulating regime of twisted bilayer graphene, Phys. Rev. B 104, 115404 (2021a).
  * Datta  _et al._ [2023] A. Datta, M. J. Calderón, A. Camjayi, and E. Bascones, Heavy quasiparticles and cascades without symmetry breaking in twisted bilayer graphene, arXiv e-prints , arXiv:2301.13024 (2023), arXiv:2301.13024 [cond-mat.str-el] .
  * Kang and Vafek [2020] J. Kang and O. Vafek, Non-abelian dirac node braiding and near-degeneracy of correlated phases at odd integer filling in magic-angle twisted bilayer graphene, Phys. Rev. B 102, 035161 (2020).
  * Soejima  _et al._ [2020] T. Soejima, D. E. Parker, N. Bultinck, J. Hauschild, and M. P. Zaletel, Efficient simulation of moiré materials using the density matrix renormalization group, Phys. Rev. B 102, 205111 (2020).
  * Zhang  _et al._ [2021] X. Zhang, G. Pan, Y. Zhang, J. Kang, and Z. Y. Meng, Momentum space quantum monte carlo on twisted bilayer graphene, Chinese Physics Letters 38, 077305 (2021).
  * Pan  _et al._ [2022] G. Pan, X. Zhang, H. Li, K. Sun, and Z. Y. Meng, Dynamical properties of collective excitations in twisted bilayer graphene, Phys. Rev. B 105, L121110 (2022).
  * Hofmann  _et al._ [2022] J. S. Hofmann, E. Khalaf, A. Vishwanath, E. Berg, and J. Y. Lee, Fermionic monte carlo study of a realistic model of twisted bilayer graphene, Phys. Rev. X 12, 011061 (2022).
  * Pan  _et al._ [2023] G. Pan, X. Zhang, H. Lu, H. Li, B.-B. Chen, K. Sun, and Z. Y. Meng, Thermodynamic characteristic for a correlated flat-band system with a quantum anomalous hall ground state, Phys. Rev. Lett. 130, 016401 (2023).
  * Zhang  _et al._ [2023] X. Zhang, G. Pan, B.-B. Chen, H. Li, K. Sun, and Z. Y. Meng, Polynomial sign problem and topological mott insulator in twisted bilayer graphene, Phys. Rev. B 107, L241105 (2023).
  * Kuzmenko  _et al._ [2009] A. B. Kuzmenko, I. Crassee, D. van der Marel, P. Blake, and K. S. Novoselov, Determination of the gate-tunable band gap and tight-binding parameters in bilayer graphene using infrared spectroscopy, Phys. Rev. B 80, 165406 (2009).
  * Andrei and MacDonald [2020] E. Y. Andrei and A. H. MacDonald, Graphene bilayers with a twist, Nature materials 19, 1265 (2020).
  * Xu  _et al._ [2020] Y. Xu, S. Liu, D. A. Rhodes, K. Watanabe, T. Taniguchi, J. Hone, V. Elser, K. F. Mak, and J. Shan, Correlated insulating states at fractional fillings of moiré superlattices, Nature 587, 214 (2020).
  * Saito  _et al._ [2020] Y. Saito, J. Ge, K. Watanabe, T. Taniguchi, and A. F. Young, Independent superconductors and correlated insulators in twisted bilayer graphene, Nature Physics 16, 926 (2020).
  * Lu  _et al._ [2019] X. Lu, P. Stepanov, W. Yang, M. Xie, M. A. Aamir, I. Das, C. Urgell, K. Watanabe, T. Taniguchi, G. Zhang, _et al._ , Superconductors, orbital magnets and correlated states in magic-angle bilayer graphene, Nature 574, 653 (2019).
  * Stepanov  _et al._ [2020] P. Stepanov, I. Das, X. Lu, A. Fahimniya, K. Watanabe, T. Taniguchi, F. H. Koppens, J. Lischner, L. Levitov, and D. K. Efetov, Untying the insulating and superconducting orders in magic-angle graphene, Nature 583, 375 (2020).
  * Nam and Koshino [2017] N. N. T. Nam and M. Koshino, Lattice relaxation and energy band modulation in twisted bilayer graphene, Phys. Rev. B 96, 075311 (2017).
  * Koshino  _et al._ [2018] M. Koshino, N. F. Q. Yuan, T. Koretsune, M. Ochi, K. Kuroki, and L. Fu, Maximally localized wannier orbitals and the extended hubbard model for twisted bilayer graphene, Phys. Rev. X 8, 031087 (2018).
  * Xi [2022] D. Xi, Quantum monte carlo simulations in momentum space, Chinese Physics Letters 39, 50101 (2022).
  * Li  _et al._ [2021] H. Li, U. Kumar, K. Sun, and S.-Z. Lin, Spontaneous fractional chern insulators in transition metal dichalcogenide moiré superlattices, Phys. Rev. Research 3, L032070 (2021).
  * Song and Bernevig [2022] Z.-D. Song and B. A. Bernevig, Magic-angle twisted bilayer graphene as a topological heavy fermion problem, Phys. Rev. Lett. 129, 047601 (2022).
  * Shi and Dai [2022] H. Shi and X. Dai, Heavy-fermion representation for twisted bilayer graphene systems, Phys. Rev. B 106, 245129 (2022).
  * Călugăru  _et al._ [2023] D. Călugăru, M. Borovkov, L. L. Lau, P. Coleman, Z.-D. Song, and B. A. Bernevig, Tbg as topological heavy fermion: Ii. analytical approximations of the model parameters, arXiv preprint arXiv:2303.03429 (2023).
  * Hu  _et al._ [2023a] H. Hu, B. A. Bernevig, and A. M. Tsvelik, Kondo lattice model of magic-angle twisted-bilayer graphene: Hund’s rule, local-moment fluctuations, and low-energy effective theory, Phys. Rev. Lett. 131, 026502 (2023a).
  * Hu  _et al._ [2023b] H. Hu, G. Rai, L. Crippa, J. Herzog-Arbeitman, D. Călugăru, T. Wehling, G. Sangiovanni, R. Valenti, A. M. Tsvelik, and B. A. Bernevig, Symmetric kondo lattice states in doped strained twisted bilayer graphene, arXiv preprint arXiv:2301.04673 (2023b).
  * Chou and Das Sarma [2023a] Y.-Z. Chou and S. Das Sarma, Kondo lattice model in magic-angle twisted bilayer graphene, Phys. Rev. Lett. 131, 026501 (2023a).
  * Chou and Das Sarma [2023b] Y. Chou and S. Das Sarma, Scaling theory of intrinsic kondo and hund’s rule interactions in magic-angle twisted bilayer graphene, arXiv preprint arXiv:2306.03121 (2023b).
  * Liu  _et al._ [2021b] S. Liu, E. Khalaf, J. Y. Lee, and A. Vishwanath, Nematic topological semimetal and insulator in magic-angle bilayer graphene at charge neutrality, Phys. Rev. Research 3, 013033 (2021b).
  * Nuckolls  _et al._ [2023] K. P. Nuckolls, R. L. Lee, M. Oh, D. Wong, T. Soejima, J. P. Hong, D. Călugăru, J. Herzog-Arbeitman, B. A. Bernevig, K. Watanabe, _et al._ , Quantum textures of the many-body wavefunctions in magic-angle graphene, Nature 620, 525 (2023).
  * Hong  _et al._ [2022] J. P. Hong, T. Soejima, and M. P. Zaletel, Detecting symmetry breaking in magic angle graphene using scanning tunneling microscopy, Phys. Rev. Lett. 129, 147001 (2022).
  * Kwan  _et al._ [2021b] Y. H. Kwan, G. Wagner, T. Soejima, M. P. Zaletel, S. H. Simon, S. A. Parameswaran, and N. Bultinck, Kekulé spiral order at all nonzero integer fillings in twisted bilayer graphene, Phys. Rev. X 11, 041063 (2021b).
  * Wang  _et al._ [2023] T. Wang, D. E. Parker, T. Soejima, J. Hauschild, S. Anand, N. Bultinck, and M. P. Zaletel, Ground-state order in magic-angle graphene at filling $\nu=-3$: A full-scale density matrix renormalization group study, Phys. Rev. B 108, 235128 (2023).
  * Chen  _et al._ [2021] B.-B. Chen, Y. D. Liao, Z. Chen, O. Vafek, J. Kang, W. Li, and Z. Y. Meng, Realization of topological mott insulator in a twisted bilayer graphene lattice model, Nature Communications 12, 5480 (2021).
  * Lin  _et al._ [2022] X. Lin, B.-B. Chen, W. Li, Z. Y. Meng, and T. Shi, Exciton proliferation and fate of the topological mott insulator in a twisted bilayer graphene lattice model, Phys. Rev. Lett. 128, 157201 (2022).
  * Sandvik [1998] A. W. Sandvik, Stochastic method for analytic continuation of quantum monte carlo data, Phys. Rev. B 57, 10287 (1998).
  * Shao and Sandvik [2023] H. Shao and A. W. Sandvik, Progress on stochastic analytic continuation of quantum monte carlo data, Physics Reports 1003, 1 (2023).
  * Nielsen and Ninomiya [1981] H. B. Nielsen and M. Ninomiya, A no-go theorem for regularizing chiral fermions, Physics Letters B 105, 219 (1981).
  * Yan and Meng [2021] Z. Yan and Z. Y. Meng, Relating entanglement spectra and energy spectra via path-integral on replica manifold, arXiv e-prints , arXiv:2112.05886 (2021), arXiv:2112.05886 [cond-mat.str-el] .
  * Zhang  _et al._ [2022] X. Zhang, K. Sun, H. Li, G. Pan, and Z. Y. Meng, Superconductivity and bosonic fluid emerging from moiré flat bands, Phys. Rev. B 106, 184517 (2022).
  * Zhou  _et al._ [2021] C. Zhou, Z. Yan, H.-Q. Wu, K. Sun, O. A. Starykh, and Z. Y. Meng, Amplitude mode in quantum magnets via dimensional crossover, Phys. Rev. Lett. 126, 227201 (2021).
  * Yan  _et al._ [2021] Z. Yan, Y.-C. Wang, N. Ma, Y. Qi, and Z. Y. Meng, Topological phase transition and single/multi anyon dynamics of $z_{2}$ spin liquid, npj Quantum Materials 6, 39 (2021).


# A simple proof of logarithmic Sobolev inequality on the Heisenberg groups

A. S. Üstünel

Abstract: In this note we give a simple, dimension independent, proof of the
logarithmic Sobolev inequality on the Heisenberg groups $H_{n}={\rm
I\\!R}^{2n+1}$ using the measure preserving transformations of the Brownian
motion.

2020 Mathematics Subject Classification. Primary 60H; Secondary 22E30, 22E66,
35A23, 35K08, 35R03, 60J65.

Keywords: Entropy, degenerate diffusions, relative entropy, Poincaré
inequality, logarithmic Sobolev inequality, intertwining, Heisenberg group,
left and right invariant vector fields, opératur carré du champs.

##  1\. Introduction

In this work we give a very simple and short proof of the logarithmic Sobolev
inequality for Heisenberg group $H_{n}$, for any $n\geq 1$. This inequality is
very interesting since this is the simplest non-linear (i.e., non-gaussian)
case with degenerate diffusion coefficients, hence outside the reach of the
standart methods and t has been suggested to us by Leonard Gross. For $n=1$,
the proof is done by using the expression of the density of the law of $x_{1}$
([5]) which is strongly dimension dependent (cf., also [1]). Here we give a
totally different proof which is dimension independent with the best constant
since it is the same as in the flat Gaussian case, cf. [4]. This is done with
the help of the invariance properties of the Gauss law under Euclidean
symmetries. The key point in our approach is the fact that the difficulties
due to the nonlinearity of the problem can be avoided by using spontanenously
the left Markov processes generated by the left invariant vector fields
generating the Lie algebra of $H_{n}$ and its mirror image. This approach
reduces the problem to a linear problem, cf., [4] and avoids all the
difficulties due to the nonlinearity of the problem.

Last but not the least we are grateful to  Leonard Gross who has indicated the
problem as well as the related references.

##  2\.  Poincaré and Log-Sobolev inequalities on the Heisenberg Group

We denote ${\rm I\\!R}^{2n}\times{\rm I\\!R}$ by $H_{n}$ which is equipped by
the non-commutative product defined as

$(v,z)\star(v^{\prime},z^{\prime})=(v+v^{\prime},z+z^{\prime}+\frac{1}{2}\>\omega(v,v^{\prime}))\,,$

$v=(x,y),\,v^{\prime}=(x^{\prime},y^{\prime})$ and
$x,y,x^{\prime},y^{\prime}\in{\rm I\\!R}^{n}$. $\omega$ is an antisymmetric
bilinear form defined as

$\omega(v,v^{\prime})=\sum_{i=1}^{n}(x_{i}y^{\prime}_{i}-x^{\prime}_{i}y_{i})$

Under the $\star$ operation defined above $H_{n}$ becomes a Lie group with its
Lie algebra, $\mathcal{H}$, generated by the left invariant vector fields

$X_{i}=\partial_{x_{i}}-\frac{y_{i}}{2}\partial_{z},\,\,Y_{i}=\partial_{y_{i}}+\frac{x_{i}}{2}\partial_{z},\,\,Z=\partial_{z}\,,$

$i=1,\ldots,n$. We define also the right invariant vector fields

$\displaystyle\hat{X}_{i}$ $\displaystyle=$
$\displaystyle\partial_{x_{i}}+\frac{y_{i}}{2}\partial_{z}$
$\displaystyle\hat{Y}_{i}$ $\displaystyle=$
$\displaystyle\partial_{y_{i}}-\frac{x_{i}}{2}\partial_{z}$
$\displaystyle\hat{Z}$ $\displaystyle=$ $\displaystyle Z=\partial_{z}\,.$

Let $b$ and $w$ be two ${\rm I\\!R}^{n}$-valued, independent Brownian motions,
the diffusion process with which we are concerned is the unique solution of
the following SDE which is written in terms of differential geometric
language:

$f(x_{t})=\int_{0}^{t}X_{i}f(x_{s})\cdot
db^{i}_{s}+\int_{0}^{t}Y_{i}f(x_{s})\cdot dw^{i}_{s}\,,$

for any smooth function $f$ on $H_{n}$ and the integrals are taken in
Stratonovitch sense111 denoted with a “dot”. The solution of this equation is
given explicitly as $(x_{t}=(b_{t},w_{t},\frac{1}{2}\>z_{t}(b,w)),t\in{\rm
I\\!R}_{+})$, where $(z_{t})$ is the Lévy area process (with Itô integrals):

$z_{t}=\sum_{i=1}^{n}\int_{0}^{t}(b^{i}_{s}dw^{i}_{s}-w^{i}_{s}db^{i}_{s})\,.$

We shall denote by $(Q_{t},t\geq 0)$ the (left) semi-group corresponding to
the Markov process defined by $Q_{t}f(p)=E[f(p\star x_{t})]$, $p\in H_{n}$.
Note the hypoelliptic infinitesimal generator ${\rm I\\!L}$ of $(Q_{t})$ is
given by

${\rm I\\!L}=\frac{1}{2}\>\sum_{i=1}^{n}(X_{i}^{2}+Y_{i}^{2})\,.$

Let us note that $[X_{i},Y_{i}]=Z,\,[X_{i},Z]=[Y_{i},Z]=0$ for $n=1,\ldots,n$,
consequently the Lie algebra generated by $\\{X_{i},Y_{i},Z;\,i=1,\ldots n\\}$
fulfills the Hörmander condition, which assures the hypoellipticity of the
operator ${\rm I\\!L}$.

It follows from direct calculation that $\hat{X_{i}}$ and $\hat{Y_{i}}$
commute with ${\rm I\\!L}$ hence they commute also with the semigroup
$(Q_{t})$ generated by ${\rm I\\!L}$, which is constructed through the
stochastic differential equation222we use the summation convention

$dx_{t}=X_{i}(x_{t})db^{i}_{t}+Y_{i}(x_{t})dw^{i}_{t},$

written with Itô integrals. Let now $A$ be the isometry defined by

$-A=\left[{\begin{array}[]{ccc}I_{{\rm I\\!R}^{n}}&0&0\\\ 0&I_{{\rm
I\\!R}^{n}}&0\\\ 0&0&-1\\\ \end{array}}\right]$

and let us denote by $(y_{t},t\geq 0)$ the process defined by
$y_{t}=(-b_{t},-w_{t},\frac{1}{2}\>z_{t}(b,w))=Ax_{t}$.

The following elementary property of $A$ is important for the sequel:

######  Lemma 1.

For any $p,q\in H_{n}$, we have

$A(p\star q)=Ap\star Aq\,.$

Proof: Let $p=(x,y,z),\,q=(x^{\prime},y^{\prime},z^{\prime})$, then

$\displaystyle A(p\star q)$ $\displaystyle=$ $\displaystyle
A\left((x+x^{\prime}),(y+y^{\prime}),z+z^{\prime}+\frac{1}{2}\>\sum_{i}(x_{i}y_{i}^{\prime}-x_{i}^{\prime}y_{i})\right)$
$\displaystyle=$
$\displaystyle\left(-(x+x^{\prime}),-(y+y^{\prime}),z+z^{\prime}+\frac{1}{2}\>\sum_{i}(x_{i}y_{i}^{\prime}-x_{i}^{\prime}y_{i})\right)\,.$

On the other hand

$\displaystyle Ap\star Aq$ $\displaystyle=$
$\displaystyle\left(-x-x^{\prime},-y-y^{\prime},z+z^{\prime}+\frac{1}{2}\>\sum_{i}((-x_{i})(-y_{i}^{\prime})-(-x_{i}^{\prime})(-y_{i}))\right)$
$\displaystyle=$
$\displaystyle\left(-(x+x^{\prime}),-(y+y^{\prime}),z+z^{\prime}+\frac{1}{2}\>\sum_{i}(x_{i}y_{i}^{\prime}-x_{i}^{\prime}y_{i})\right)$
$\displaystyle=$ $\displaystyle A(p\star q)\,.$

######  Lemma 2.

We have $y_{t}=Ax_{t}$ and the law of $y$ is equal to the law of $x$.

Proof: We have explicitly $x_{t}=(b_{t},w_{t},\frac{1}{2}\>z_{t}(b,w))$ and
$y_{t}=(-b_{t},-w_{t},\frac{1}{2}\>z_{t}(b,w))$. From the Paul Lévy theorem,
${\mbox{law}}(b,w)={\mbox{law}}(-b,-w)$, hence also

${\mbox{law}}\left(b,w,\frac{1}{2}\>z(b,w)\right)={\mbox{law}}\left(-b,-w,\frac{1}{2}\>z(-b,-w)\right)\,,$

but $z(-b,-w)=z(b,w)$, consequently the laws of $x$ and $y$ on the path space
are identical.

######  Proposition 1.

We have the following commutation results:

(2.1) $\displaystyle(X_{i}f)(Ay_{t})$ $\displaystyle=$
$\displaystyle-\hat{X}_{i}(f\circ A)(y_{t})$ (2.2)
$\displaystyle(Y_{i}f)(Ay_{t})$ $\displaystyle=$
$\displaystyle-\hat{Y}_{i}(f\circ A)(y_{t})$

or equivalently

(2.3) $\displaystyle(X_{i}f)(x_{t})$ $\displaystyle=$
$\displaystyle-\hat{X}_{i}(f\circ A)(Ax_{t})=-\hat{X}_{i}(f\circ A)(y_{t})$
(2.4) $\displaystyle(Y_{i}f)(x_{t})$ $\displaystyle=$
$\displaystyle-\hat{Y}_{i}(f\circ A)(Ax_{t})=-\hat{Y}_{i}(f\circ A)(y_{t})$

and in particular

$({\rm I\\!L}f)(Ay_{t})=\hat{{\rm I\\!L}}(f\circ A)(y_{t})$

almost surely for any smooth function $f$.

Proof: These results follow from the Itô formula and the unique decomposition
of a continuous semimartingale as a sum of a local martingale and a finite
variation process, cf. [2]. In fact, for any smooth function $f$ on ${\rm
I\\!R}^{2n+1}$, we can write

$\displaystyle f(x_{t})$ $\displaystyle=$ $\displaystyle
f(0)+\int_{0}^{t}X_{i}f(x_{s})\cdot db^{i}_{s}+\int_{0}^{t}Y_{i}(x_{s})\cdot
dw^{i}_{s}$ $\displaystyle=$ $\displaystyle f(Ay_{t})=(f\circ A)(y_{t})$
$\displaystyle=$ $\displaystyle f(0)-\int_{0}^{t}\hat{X}_{i}(f\circ
A)(y_{s})\cdot db^{i}_{s}-\int_{0}^{t}\hat{Y}_{i}(f\circ A)(y_{s})\cdot
dw^{i}_{s}\,.$

Hence the integrands in front of $b^{i}$’s of the first and third lines should
be equal, the same reasoning holds for the integrands in front of $w^{i}$’s in
the first and third lines and the proof of the relations 2.1, 2.2 follows. The
remaining part of the proof follows by changing just the notations.

######  Lemma 3.

For any $t\geq 0$, $f\in C^{\infty}_{b}(H_{n})$, we have

(2.5) $\displaystyle E[f(y_{T})|{\mathcal{F}}_{t}(x)]$ $\displaystyle=$
$\displaystyle Q_{T-t}f(y_{t})$ (2.6) $\displaystyle
E[f(x_{T})|{\mathcal{F}}_{t}(x)]$ $\displaystyle=$ $\displaystyle
Q_{T-t}f(x_{t})\,,$

for any $T\geq t$, P-almost surely.

Proof: Let $\theta(b,w)$ be any bounded and ${\mathcal{F}}_{t}(x)$–measurable
function. As ${\mathcal{F}}_{t}(x)=\sigma(b_{s},w_{s},s\leq
t)=\sigma(-b_{s},-w_{s},s\leq t)$ and as the law of $(b,w)$ is equal to the
law of $(-b,-w)$, we get

$\displaystyle E[f(y_{T}(b,w))\,\theta(b,w)]$ $\displaystyle=$ $\displaystyle
E[f(y_{T}(-b,-w))\theta(-b,-w)]$ $\displaystyle=$ $\displaystyle
E\left[f(x_{T}(b,w))\theta(-b,-w)\right]$ $\displaystyle=$ $\displaystyle
E[E[f(x_{T})|{\mathcal{F}}_{t}(x)]\theta(-b,-w)]$ $\displaystyle=$
$\displaystyle E\left[Q_{T-t}f(x_{t}(b,w))\theta(-b,-w)\right]$
$\displaystyle=$ $\displaystyle E[Q_{T-t}f(x_{t}(-b,-w))\theta(b,w)]$
$\displaystyle=$ $\displaystyle E[Q_{T-t}f(y_{t}(b,w))\theta(b,w)]\,,$

by the invariance of the law of the Brownian motion under the isometries,
i.e., by the relation

${\mbox{law}(b,w)}={\mbox{law}(-b,-w)}$

thanks to the Paul Lévy theorem.

######  Lemma 4.

For any positive function $f$ on $H_{n}$, we have

$(Q_{t}f)(Ap)=Q_{t}(f\circ A)(p)$

for any $p\in H_{n}$, $t\geq 0$.

Proof: We have

(2.7) $\displaystyle(Q_{t}f)(Ap)$ $\displaystyle=$ $\displaystyle
E[f(A(p)\star x_{t})]$ (2.8) $\displaystyle=$ $\displaystyle E[f(A(p)\star
A(x_{t}))]$ (2.9) $\displaystyle=$ $\displaystyle E[f(A(p\star x_{t}))]$
(2.10) $\displaystyle=$ $\displaystyle Q_{t}(f\circ A)(p)\,,$

where the equality 2.8 follows from the equality of the laws of $x$ and $Ax$
and equality2.9 follows from Lemma 1.

######  Proposition 2.

For any smooth $f$, we have

$\displaystyle X_{i}Q_{1-t}f(x_{t})$ $\displaystyle=$ $\displaystyle-
Q_{1-t}(\hat{X}_{i}(f\circ A))(y_{t})$ $\displaystyle Y_{i}Q_{1-t}f(x_{t})$
$\displaystyle=$ $\displaystyle-Q_{1-t}(\hat{Y}_{i}(f\circ A))(y_{t})\,,$

for almost surely. In particular, the processes
$(X_{i}Q_{1-t}f(x_{t}),t\in[0,1])$ and $(Y_{i}Q_{1-t}f(x_{t}),t\in[0,1])$ are
martingales for $i=1,\ldots,n$.

Proof: Let $f$ be a smooth (hence bounded) function on ${\rm I\\!R}^{2n+1}$.
We have, for $i=1,\ldots,n$, the following equalities:

(2.11) $\displaystyle X_{i}Q_{1-t}f(x_{t})$ $\displaystyle=$
$\displaystyle-\hat{X}_{i}((Q_{1-t}f)\circ A)(y_{t})$ (2.12) $\displaystyle=$
$\displaystyle-\hat{X}_{i}((Q_{1-t}(f\circ A))(y_{t})$ (2.13) $\displaystyle=$
$\displaystyle-Q_{1-t}(\hat{X}_{i}(f\circ A))(y_{t})$ (2.14) $\displaystyle=$
$\displaystyle-E[\hat{X}_{i}(f\circ A)(y_{1})|{\mathcal{F}}_{t}]\,,$

where the equality 2.11 follows from Proposition 1, the relation 2.12 is
proven in Lemma 4, the equality 2.14 is a consequece of the commutativity of
${\rm I\\!L}$ and $\hat{X}_{i}$ and the last one follows from Lemma 3. The
proof with the vector fields $(Y_{i},i=1,\ldots,n)$ is the same and omitted.

We have all we need to prove the Poincaré and log-Sobolev inequalities:

######  Theorem 1.

For any smooth $f$ on $H_{n}$, we have

$E[|f(x_{1})-E[f(x_{1})]|^{2}]\leq\sum_{i=1}^{n}E[|X_{i}f(x_{1})|^{2}+|Y_{i}f(x_{1})|^{2}]\,.$

Proof: From the Itô formula from the commutation results of Proposition 2

(2.15) $\displaystyle Q_{1-t}f(x_{t})-Q_{1}f(0)$ $\displaystyle=$
$\displaystyle\int_{0}^{t}X_{i}Q_{1-s}f(x_{s})db^{i}_{s}+\int_{0}^{t}Y_{i}f(x_{s})dw^{i}_{s}$
$\displaystyle=$ $\displaystyle-\int_{0}^{t}Q_{1-s}\hat{X}_{i}(f\circ
A)(y_{s})db^{i}_{s}+\int_{0}^{t}Q_{1-s}\hat{Y}_{i}(f\circ
A)(y_{s})dw^{i}_{s}\,.$

Again from Proposition 2, the integrands in the equality 2.15 are square
integrable martingales provided that $f$ has bounded first and second order
derivatives. Hence all these terms as well as $(Q_{1-t}f(x_{t}),t\in[0,1])$
converge in $L^{2}$ as $t\to 1$. Consequently

(2.16) $\displaystyle E[|f(x_{1})-E[f(x_{1})]|^{2}]$ $\displaystyle=$
$\displaystyle\lim_{t\to
1}\sum_{i=1}^{n}E[\int_{0}^{1}[|Q_{1-s}\hat{X}_{i}(f\circ
A)(y_{s})|^{2}+|Q_{1-s}\hat{Y}_{i}(f\circ A)(y_{s})|^{2}]ds$
$\displaystyle\leq$ $\displaystyle\sum_{i=1}^{n}E[|\hat{X}_{i}(f\circ
A)(y_{1})|^{2}+|\hat{Y}_{i}(f\circ A)(y_{1})|^{2}]$ $\displaystyle=$
$\displaystyle\sum_{i=1}^{n}E[|X_{i}f(x_{1})|^{2}+|Y_{i}f(x_{1})|^{2}]\,.$

where the line 2.16 follows again from Proposition 2.

The last theorem will be the justification of this manuscript:

######  Theorem 2.

For any smooth $f$ on $H_{n}$ with bounded derivatives upto the second order,
the following inequality holds true:

$E\left[f^{2}(x_{1})\log\frac{f^{2}(x_{1})}{E[f^{2}(x_{1})]}\right]\leq
2\sum_{i=1}^{n}E[|X_{i}f(x_{1})|^{2}+|Y_{i}f(x_{1})|^{2}]\,.$

Proof: We can assume that $f$ is positive and $E[f(x_{1})]=1$. In this case it
suffices to prove, by replacing $f$ with its square root, that

$E[f(x_{1})\log
f(x_{1})]\leq\frac{1}{2}\>\sum_{i}E[\frac{1}{f(x_{1})}(|X_{i}\log
f(x_{1})|^{2}+|Y_{i}\log f(x_{1})|^{2}]\,.$

Thanks to our choice of $f$, we may look at $f(x_{1})$ ias a probability
density and we write $d\nu=f(x_{1})dP$ for the new probability. From the Itô
formula and from the commutation relations given by Proposition 2, employed as
in the proof of Theorem 1, we have

$Q_{1-t}f(x_{t})=1-\int_{0}^{t}Q_{1-s}\hat{X}_{i}(f\circ
A)(y_{s})db^{i}_{s}-\int_{0}^{t}Q_{1-s}\hat{Y}_{i}(f\circ
A)(y_{s})dw^{i}_{s}\,.$

By dividing and multiplying each integrand above with $Q_{1-s}(f\circ
A)(y_{s})$, we can represent $Q_{1-t}f(x_{t})=l_{t}$ as an exponential
martingale: in fact let

$u^{i}_{t}=\frac{1}{l_{t}}\left(Q_{1-t}\hat{X}_{i}(f\circ
A)(y_{t}),Q_{1-t}\hat{Y}_{i}(f\circ A)(y_{t})\right),\,i=1,\ldots,n\,$

and let

$W^{i}_{t}=(b^{i}_{t},w^{i}_{t}),\,i=1,\ldots,n\,.$

Then we can represent $l_{t}$ as

(2.17) $l_{t}=Q_{1-t}(f\circ
A)(y_{t})=\exp\left(-\int_{0}^{t}(u_{s},dW_{s})_{{\rm
I\\!R}^{2n}}-\frac{1}{2}\>\int_{0}^{t}|u_{s}|_{{\rm
I\\!R}^{2n}}^{2}ds\right)\,.$

From the Girsanov theorem, $(W_{t}+\int_{0}^{t}u_{s}ds,t\in[0,1])$ is a
$\nu$-Brownian motion, consequently

$\displaystyle E[l_{t}\log l_{t}]$ $\displaystyle=$ $\displaystyle
E_{\nu}[\log l_{t}]=\frac{1}{2}\>E_{\nu}\int_{0}^{t}|u_{s}|^{2}_{{\rm
I\\!R}^{2n}}ds$ $\displaystyle=$
$\displaystyle\frac{1}{2}\>\sum_{i}E_{\nu}\int_{0}^{t}\frac{1}{l_{s}^{2}}(|Q_{1-s}\hat{X}_{i}(f\circ
A)(y_{s})|^{2}+|Q_{1-s}\hat{Y}_{i}(f\circ A)(y_{s})|^{2})ds$ $\displaystyle=$
$\displaystyle\frac{1}{2}\>\sum_{i}\big{(}E_{\nu}\int_{0}^{t}|E_{\nu}[\hat{X}_{i}\log(f\circ
A)(y_{1})|{\mathcal{F}}_{s}]|^{2}ds$
$\displaystyle+E_{\nu}\int_{0}^{t}|E_{\nu}[\hat{Y}_{i}\log(f\circ
A)(y_{1})|{\mathcal{F}}_{s}]|^{2}ds\big{)}$ $\displaystyle\leq$
$\displaystyle\frac{1}{2}\>\sum_{i}E_{\nu}[|\hat{X}_{i}\log(f\circ
A)(y_{1})|^{2}+|\hat{Y}_{i}\log(f\circ A)(y_{1})|^{2}]$ $\displaystyle=$
$\displaystyle\frac{1}{2}\>\sum_{i}E\left[\frac{1}{f(x_{1})}(|\hat{X}_{i}(f\circ
A)(y_{1})|^{2}+|\hat{Y}_{i}(f\circ A)(y_{1})|^{2})\right]$ $\displaystyle=$
$\displaystyle\frac{1}{2}\>\sum_{i}E\left[\frac{1}{f(x_{1})}(|X_{i}f(x_{1})|^{2}+|Y_{i}f(x_{1})|^{2}\right]\,,$

where in the fourth and fifth lines we have used the fact that the map $t\to
E[M_{t}^{2}]$ is increasing for a square integrable martingale
$M=(M_{t},t\in[0,1])$ and the equality between the two last lines follow from
the commutation relations proved in Proposition 2. This inequality holds for
any $t\in[0,1]$, in particular it is also true when we replace the left hand
side with $E[l_{1}\log l_{1}]$.

## References

  * [1] D. Bakry, F. Baudoin , M. Bonnefont, D. Chafai: “On gradient bounds for the heat kernel on the Heisenberg group”. Jour. Functional Analysis, 255, 1905-1938 (2008).
  * [2] C. Dellacherie et P.-A. Meyer:“Probabilités et Potentiel, Processus de Markov, Chapitres 17 à 24”. Collection Enseignement des Sciences, Hermann, 1992.
  * [3] B.K. Driver, T. Melcher:“ Hypoelliptic heat kernel inequalities on the Heisenberg group”. Jour. Funct. Anal. 221 (2) 340–365 (2005).
  * [4] L. Gross: “Logarithmic Sobolev inequalities”. Amer. Jour. Math.97, p.1061-1083 (1975).
  * [5] H.-Q. Li:“ Estimation optimale du gradient du semi-groupe de la chaleur sur le groupe de Heisenberg”. Jour. Funct. Anal. 236 (2) 369–394 (2006).

A. S. Üstünel, Bilkent University, Math. Dept., Ankara, Turkey
<EMAIL_ADDRESS>