# Analyticity of the solutions to degenerate Monge-Ampère equations

Genggeng Huang***Email address<EMAIL_ADDRESS>and Yingshu
Lü†††Corresponding author. Email address<EMAIL_ADDRESS>
###### Abstract

This paper is devoted to study the following degenerate Monge-Ampère equation:

$\displaystyle\begin{cases}\det
D^{2}u=\Lambda_{q}(-u)^{q}\quad\text{in}\quad\Omega,\\\
u=0\quad\text{on}\quad\partial\Omega\end{cases}$ (0.1)

for some positive constant $\Lambda_{q}$. Suppose
$\Omega\subset\subset\mathbb{R}^{n}$ is uniformly convex and analytic. Then
the solution of (0.1) is analytic in $\bar{\Omega}$ provided
$q\in\mathbb{Z}^{+}$.

School of Mathematical Sciences, Fudan University, Shanghai, China

Keywords: Analyticity; degenerate elliptic; Monge-Ampère equations

Mathematics Subject Classification: 35A20, 35J70, 35J96

##  1 Introduction

In this paper, we focus on the analyticity of the solution of the following
Monge-Ampère equation:

$\displaystyle\begin{cases}\det
D^{2}u=\Lambda_{q}(-u)^{q}\quad\text{in}\quad\Omega,\\\
u=0\quad\text{on}\quad\partial\Omega,\end{cases}$ (1.1)

where $q>0$ and $\Omega$ is a bounded convex domain in $\mathbb{R}^{n}$.

This problem was first studied by Lions[12]. In [12], Lions proved that for
$q=n$, (1.1) admits a unique eigenvalue $\Lambda_{n}$ and eigenfunction $u\in
C^{1,1}(\bar{\Omega})\cap C^{\infty}(\Omega)$(up to multiplications of
positive constants) provided $\Omega$ is smooth and uniformly convex.

Later, Chou [18] studied the problem (1.1) for $q>0$. His approach is based on
the Monge-Ampère functional

$\displaystyle J(u)=\frac{1}{n+1}\int_{\Omega}(-u)\det
D^{2}udx-\frac{1}{q+1}\int_{\Omega}|u|^{q+1}dx$ (1.2)

and the following logarithmic gradient flow

$\left\\{\begin{array}[]{ll}\frac{\partial u}{\partial
t}=\ln\det(D_{x}^{2}u)-q\ln(-u),\quad(x,t)\in\Omega\times(0,+\infty),\\\
u(x,0)=u_{0}(x),\quad x\in\Omega,\\\
u=0\quad\text{on}\quad\partial\Omega\times(0,+\infty).\end{array}\right.$
(1.3)

Then Chou obtained the existence of non-trivial solutions $u\in
C^{0,1}(\bar{\Omega})\cap C^{\infty}(\Omega)$ provided $\Omega$ is smooth and
uniformly convex. Also the uniqueness of non-trivial solution was established
in [18] for $0<q<n$ and $q=n$(up to multiplications of positive constants).
Later, Hartenstine [2] extended it to bounded and strictly convex domain
$\Omega$ for $0<q<n$. Recently, Le [10] proved the existence($q>0$) of
solutions of (1.1) and the uniqueness($0<q\leq n$) of solutions of (1.1) for
general bounded convex domain $\Omega$. For $q>n$, the uniqueness of non-
trivial solutions of (1.1) is less known. Recently, the first named author [7]
proved that there exists a constant $\varepsilon(n)>0$ such that (1.1) admits
a unique non-trivial solution for $q\in(n,n+\varepsilon(n))$.

The higher order global regularity of the solutions of (1.1) remains unknown
until rencent 10 years. The main difficulty arises from the degeneracy of the
equation (1.1) on the boundary $\partial\Omega$. In a survey paper, Trudinger
and Wang [[17],P21] proposed the problem that whether $u$ is smooth up to the
boundary for $q=n$ when $\Omega$ is smooth and uniformly convex. Later, Hong,
Huang and Wang [4] gave an affirmative answer to this problem in dimension 2.
Their approach relies on an auxiliary function

$H=u_{22}u_{1}^{2}-2u_{12}u_{1}u_{2}+u_{11}u_{2}^{2}$ (1.4)

which is related to the Gauss curvature of the level set of $u$. For arbitrary
dimensions, Savin [16] made a first contribution on the global $C^{2}$
regularity of solutions of (1.1). Later, Le and Savin[11] completely solved
the above problem in arbitrary dimensions. The key observation of the work
[16] and [11] is that near the boundary,

$u(x)\sim\frac{1}{2}|x^{\prime}|^{2}+\frac{1}{(q+1)(q+2)}x_{n}^{q+2}$ (1.5)

which allows them to use blow-up and perturbation arguments to show $u\in
C^{2,\alpha}$. Then they can raise the regularity up to $C^{\infty}$ by
investigating a linear degenerate elliptic equation.

It is natural to ask that whether the solution of (1.1) is analytic up to the
boundary provided the domain $\Omega$ is analytic and uniformly convex. The
analyticity of the solutions of uniformly elliptic equations(systems) are well
studied. We refer readers to [15] for linear equations(systems), [14, 13] for
non-linear equations(systems) and [1] for more general regularity results.
There seems no unified results for degenerate elliptic cases. In the present
case, the model degenerate equation is

$\displaystyle\begin{cases}\mathcal{L}(u)=u_{nn}+x_{n}^{m}\Delta_{x^{\prime}}u=f\quad\text{in}\quad\mathbb{R}^{n}_{+},\\\
u(x^{\prime},0)=g(x^{\prime}),\quad x^{\prime}\in\mathbb{R}^{n-1},\end{cases}$
(1.6)

where $x=(x_{1},\cdots,x_{n-1},x_{n})=(x^{\prime},x_{n})$. Usually, people
call (1.6) a Grushin type degenerate elliptic equation. To the authors’s best
knowledge, there seems no results considering the analyticity of solutions of
fully non-linear elliptic equations with degeneracy as in (1.6).

Now we state our main results in the present paper.

######  Theorem 1.1.

Suppose $u$ is a non-trivial solution of (1.1) and
$\Omega\subset\subset\mathbb{R}^{n}$ is analytic and uniformly convex. Then
$u$ is analytic in $\bar{\Omega}$ provided $q\in\mathbb{Z}^{+}$, i.e. $u\in
C^{\omega}(\bar{\Omega})$.

######  Remark 1.1.

The idea of the proof of Theorem 1.1 originated from [8] for semi-linear
elliptic equation. For the fully non-linear uniform elliptic case, we refer
the readers to [3].

The present paper is organized as follows. In Section 2, we will collect some
basic estimates for the linear degenerate elliptic equation (1.6). Then we
apply the ideas of [8] to show that $u$ is analytic up to the boundary in
Section 3.

##  2 Estimates for linear model equation

In the present section, we consider the following linear degenerate elliptic
equations:

$\displaystyle\begin{cases}u_{nn}+x_{n}^{m}\Delta_{x^{\prime}}u=f\quad\text{in}\quad\mathbb{R}^{n}_{+},\\\
u(x^{\prime},0)=g(x^{\prime}),\quad x^{\prime}\in\mathbb{R}^{n-1},\end{cases}$
(2.1)

where $x=(x_{1},\cdots,x_{n-1},x_{n})=(x^{\prime},x_{n})$.

Firstly, we introduce some notations. Let $m$ be a positive integer. For a
multi-index $\alpha=(\alpha_{1},\alpha_{2},\cdots,\alpha_{n})$, we denote
$\partial_{x}^{\alpha}=\partial_{1}^{\alpha_{1}}\partial_{2}^{\alpha_{2}}\cdots\partial_{n}^{\alpha_{n}}$
with $|\alpha|=\alpha_{1}+\alpha_{2}+\cdots+\alpha_{n}$ and
$u_{i}=\partial_{i}u=\frac{\partial u}{\partial{x_{i}}}$($i=1,2,\cdots,n$).

For $k\geq 0$, we define the following weighted Sobolev space
$\widetilde{W}^{k,2}(G_{1})$ as

$\begin{split}\|u\|_{\widetilde{W}^{k,2}(G_{1})}=&\sum_{|\alpha|=k}\|\partial_{n}^{2}\partial_{x}^{\alpha}u\|_{L^{2}(G_{1})}+\sum_{|\alpha|=k+2,\alpha_{n}\leq
1}\|x_{n}^{m}\partial_{x}^{\alpha}u\|_{L^{2}(G_{1})}+\sum_{|\alpha|=k}\|\partial_{n}\partial^{\alpha}_{x}u\|_{L^{2}(G_{1})}\\\
+&\sum_{|\alpha|=k+1,\alpha_{n}=0}\|x_{n}^{m-1}\partial_{x}^{\alpha}u\|_{L^{2}(G_{1})}+\|\partial_{n}u(x^{\prime},0)\|_{H^{k}(\mathbb{R}^{n-1})}+\|u\|_{H^{k}(G_{1})},\end{split}$

where $G_{1}=\mathbb{R}^{n-1}\times[0,1)$.

In the following, we introduce two lemmas which are the key ingredients to
prove the analyticity of the solutions to Monge-Ampère equations.

######  Lemma 2.1.

Suppose $u\in C_{c}^{\infty}(\overline{\mathbb{R}^{n}_{+}})$ solves (2.1) with
supp $u\subset G_{1}$. Then there holds

$\displaystyle\|u\|_{\widetilde{W}^{0,2}(G_{1})}\leq
C(\|f\|_{L^{2}(G_{1})}+\|g\|_{H^{1}(\mathbb{R}^{n-1})})$ (2.2)

for some constant $C$ depending only on $n$.

###### Proof.

We divide the proof of the present lemma into two cases.

  * Case 1.

For $m=1$, we follow the steps of standard $W^{2,2}$-estimates as Laplacian
equation. By (2.1), one gets

$\displaystyle\int_{\mathbb{R}^{n}_{+}}(u_{nn}+x_{n}\Delta_{x^{\prime}}u)^{2}dx=\int_{\mathbb{R}^{n}_{+}}f^{2}dx.$
(2.3)

Integrating by parts the crossproduct term yields

$\begin{split}&2\int_{\mathbb{R}^{n}_{+}}x_{n}u_{nn}\Delta_{x^{\prime}}udx=-2\int_{\mathbb{R}^{n}_{+}}x_{n}\nabla_{x^{\prime}}u_{nn}\cdot\nabla_{x^{\prime}}udx\\\
=&2\int_{\mathbb{R}^{n}_{+}}x_{n}|\nabla_{x^{\prime}}u_{n}|^{2}dx+2\int_{\mathbb{R}^{n}_{+}}\nabla_{x^{\prime}}u_{n}\cdot\nabla_{x^{\prime}}udx\\\
=&2\int_{\mathbb{R}^{n}_{+}}x_{n}|\nabla_{x^{\prime}}u_{n}|^{2}dx-\int_{\mathbb{R}^{n-1}}|\nabla_{x^{\prime}}g|^{2}dx^{\prime}.\end{split}$
(2.4)

Integrating by parts the other terms of (2.3), one gets

$\displaystyle\begin{split}\int_{\mathbb{R}^{n}_{+}}u^{2}_{nn}+x_{n}^{2}|\nabla_{x^{\prime}}^{2}u|^{2}+2\int_{\mathbb{R}^{n}_{+}}x_{n}|\nabla_{x^{\prime}}u_{n}|^{2}dx=\int_{\mathbb{R}^{n}_{+}}f^{2}dx+\int_{\mathbb{R}^{n-1}}|\nabla_{x^{\prime}}g|^{2}dx^{\prime}.\end{split}$
(2.5)

Multiplying $-u$ on both sides of (2.1) and integrating by parts, one obtains

$\displaystyle\begin{split}\int_{\mathbb{R}^{n}_{+}}u_{n}^{2}+x_{n}|\nabla_{x^{\prime}}u|^{2}dx=-\int_{\mathbb{R}^{n-1}}u(x^{\prime},0)u_{n}(x^{\prime},0)dx^{\prime}-\int_{\mathbb{R}^{n}_{+}}fudx.\end{split}$
(2.6)

Suppose

$\displaystyle\int_{\mathbb{R}^{n-1}}u^{2}_{n}(x^{\prime},t)dx^{\prime}=\inf_{x_{n}\in[0,1]}\int_{\mathbb{R}^{n-1}}u^{2}_{n}(x^{\prime},x_{n})dx^{\prime}.$
(2.7)

Then we know

$\begin{split}&\left|\int_{\mathbb{R}^{n-1}}u(x^{\prime},0)u_{n}(x^{\prime},0)dx^{\prime}\right|\leq
C_{\varepsilon}\int_{\mathbb{R}^{n-1}}g^{2}dx^{\prime}+\varepsilon\int_{\mathbb{R}^{n-1}}u_{n}^{2}(x^{\prime},0)dx^{\prime}\\\
\leq&C_{\varepsilon}\int_{\mathbb{R}^{n-1}}g^{2}dx^{\prime}+C\varepsilon(\int_{\mathbb{R}^{n-1}}u_{n}^{2}(x^{\prime},t)dx^{\prime}+\int_{\mathbb{R}^{n}_{+}}u_{nn}^{2}dx)\\\
\leq&C(\|g\|_{H^{1}(\mathbb{R}^{n-1})}^{2}+\|f\|_{L^{2}(\mathbb{R}^{n}_{+})}^{2})+\frac{1}{4}\int_{\mathbb{R}^{n}_{+}}u_{n}^{2}dx.\end{split}$
(2.8)

In getting the last inequality, we use (2.5), (2.7) and take suitable
$\varepsilon>0$ small. Also, one has

$\begin{split}\int_{\mathbb{R}^{n}_{+}}u^{2}dx\leq
C\int_{\mathbb{R}^{n-1}}g^{2}dx^{\prime}+\int_{\mathbb{R}^{n}_{+}}u_{n}^{2}dx\end{split}$
(2.9)

since $u$ has compact support. Noticing that
$\nabla_{x^{\prime}}u=\partial_{n}(x_{n}\nabla_{x^{\prime}}u)-x_{n}\nabla_{x^{\prime}}u_{n}$,
one knows

$\displaystyle\begin{split}&\int_{\mathbb{R}^{n}_{+}}|\nabla_{x^{\prime}}u|^{2}dx\leq
2\int_{\mathbb{R}^{n}_{+}}|\partial_{n}(x_{n}\nabla_{x^{\prime}}u)|^{2}+x_{n}^{2}|\nabla_{x^{\prime}}u_{n}|^{2}dx\\\
=&2\int_{\mathbb{R}^{n}_{+}}x_{n}^{2}|\nabla_{x^{\prime}}u_{n}|^{2}dx-2\int_{\mathbb{R}^{n}_{+}}x_{n}\nabla_{x^{\prime}}u\cdot(2\nabla_{x^{\prime}}u_{n}+x_{n}\nabla_{x^{\prime}}u_{nn})dx\\\
=&4\int_{\mathbb{R}^{n}_{+}}x_{n}^{2}|\nabla_{x^{\prime}}u_{n}|^{2}dx.\end{split}$
(2.10)

Combining estimates (2.5)-(2.10) and the assumption supp $u\subset G_{1}$
yield the present lemma for $m=1$.

  * Case 2.

For $m\geq 2$, the above method fails. We need employ some previous estimates
for (2.1). One may view the solutions of (2.1) as $u=v+w$ where $v$ solves

$\displaystyle\begin{cases}\partial_{nn}v+x_{n}^{m}\Delta_{x^{\prime}}v=0\quad\text{in}\quad\mathbb{R}^{n}_{+},\\\
v(x^{\prime},0)=g(x^{\prime}),\quad x^{\prime}\in\mathbb{R}^{n-1},\end{cases}$
(2.11)

and $w$ solves

$\displaystyle\begin{cases}\partial_{nn}w+x_{n}^{m}\Delta_{x^{\prime}}w=f\quad\text{in}\quad\mathbb{R}^{n}_{+},\\\
w(x^{\prime},0)=0,\quad x^{\prime}\in\mathbb{R}^{n-1}.\end{cases}$ (2.12)

Since $g(x^{\prime})$ is compactly supported in $\mathbb{R}^{n-1}$, Hong-Wang
[Lemma 3.1,[6]] proved that there exists an operator $\mathcal{B}$ such that
$v(x)=\mathcal{B}(g)(x)$ solves (2.11) and satisfies the following estimates

$\displaystyle\begin{split}&\|\partial_{nn}(\mathcal{B}(g))(x_{n})\|_{L^{2}_{x^{\prime}}(\mathbb{R}^{n-1})}+\|x_{n}^{\frac{m}{2}}\Lambda_{1}\partial_{n}(\mathcal{B}(g))(x_{n})\|_{L^{2}_{x^{\prime}}(\mathbb{R}^{n-1})}\\\
&+\|x_{n}^{m}\Lambda_{1}^{2}(\mathcal{B}(g))(x_{n})\|_{L^{2}_{x^{\prime}}(\mathbb{R}^{n-1})}+\|\partial_{n}(\mathcal{B}(g))(x^{\prime},0)\|_{L^{2}_{x^{\prime}}(\mathbb{R}^{n-1})}\\\
&+\|x_{n}^{\frac{m}{2}}\Lambda_{1}^{\frac{m+4}{m+2}}(\mathcal{B}(g))(x_{n})\|_{L^{2}_{x^{\prime}}(\mathbb{R}^{n-1})}\leq
C\|\Lambda^{\frac{4}{m+2}}g\|_{L^{2}(\mathbb{R}^{n-1})},\end{split}$ (2.13)

where $C$ is a universal constant depending only on $n$, $\Lambda_{1}$ and
$\Lambda$ represent the singular integral operators with symbols $|\xi|$ and
$(1+|\xi|^{2})^{\frac{1}{2}}$, $\xi=(\xi_{1},\cdots,\xi_{n-1})$. Also the norm
$L^{2}_{x^{\prime}}(\mathbb{R}^{n-1})$ represents the $L^{2}-$norm over
$\mathbb{R}^{n-1}$ in $x^{\prime}$ variable. Then by a simple integration over
$x_{n}$, one obtains

$\displaystyle\begin{split}&\|\partial^{2}_{n}(\mathcal{B}(g))\|_{L^{2}(G_{1})}+\|x_{n}^{\frac{m}{2}}\Lambda_{1}\partial_{n}(\mathcal{B}(g))\|_{L^{2}(G_{1})}+\|x_{n}^{m}\Lambda_{1}^{2}(\mathcal{B}(g))\|_{L^{2}(G_{1})}\\\
+&\|\partial_{n}(\mathcal{B}(g))\|_{L^{2}(G_{1})}+\|x_{n}^{\frac{m}{2}}\Lambda_{1}^{\frac{m+4}{m+2}}(\mathcal{B}(g))\|_{L^{2}(G_{1})}+\|\partial_{n}(\mathcal{B}(g))(x^{\prime},0)\|_{L^{2}(\mathbb{R}^{n-1})}\\\
+&\|\mathcal{B}(g)\|_{L^{2}(G_{1})}\leq
C\|\Lambda^{\frac{4}{m+2}}g\|_{L^{2}(\mathbb{R}^{n-1})}.\end{split}$ (2.14)

Since $f$ is compactly supported in $\overline{\mathbb{R}^{n}_{+}}$. Hong-Li
[Theorem 3.2,[5]](also see Lemma 3.4 in [6]) proved that there exists an
operator $\mathcal{T}$ such that $w=\mathcal{T}(f)$ solves (2.12) and
satisfies the following estimates

$\displaystyle\begin{split}&\|\partial_{n}^{2}(\mathcal{T}f)\|_{L^{2}(G_{1})}+\|x_{n}^{m}\Lambda\partial_{n}(\mathcal{T}f)\|_{L^{2}(G_{1})}+\|x_{n}^{m}\Lambda^{2}(\mathcal{T}f)\|_{L^{2}(G_{1})}+\|\partial_{n}\mathcal{T}f\|_{L^{2}(G_{1})}\\\
+&\|x_{n}^{\frac{m}{2}}\Lambda(\mathcal{T}f)\|_{L^{2}(G_{1})}+\|\partial_{n}\mathcal{T}f(x^{\prime},0)\|_{L^{2}(\mathbb{R}^{n-1})}+\|\mathcal{T}f\|_{L^{2}(G_{1})}\leq
C\|f\|_{L^{2}(G_{1})}.\end{split}$ (2.15)

Then combining (2.14) and (2.15), one gets

$\displaystyle\begin{split}&\|\partial_{n}^{2}u\|_{L^{2}(G_{1})}+\sum_{i=1}^{n-1}\|x_{n}^{m}\partial_{in}u\|_{L^{2}(G_{1})}+\sum_{i,j=1}^{n-1}\|x_{n}^{m}\partial_{ij}u\|_{L^{2}(G_{1})}+\|\partial_{n}u\|_{L^{2}(G_{1})}\\\
+&\sum_{i=1}^{n-1}\|x_{n}^{\frac{m}{2}}\partial_{i}u\|_{L^{2}(G_{1})}+\|\partial_{n}u(x^{\prime},0)\|_{L^{2}(\mathbb{R}^{n-1})}+\|u\|_{L^{2}(G_{1})}\\\
\leq&C\left(\|f\|_{L^{2}(G_{1})}+\|g\|_{H^{1}(\mathbb{R}^{n-1})}\right).\end{split}$
(2.16)

The last inequality of (2.16) comes from

$\|\Lambda^{\frac{4}{m+2}}g\|_{L^{2}(\mathbb{R}^{n-1})}=\|(1+|\xi|^{2})^{\frac{2}{m+2}}\hat{g}\|_{L^{2}(\mathbb{R}^{n-1})}\leq\|(1+|\xi|^{2})^{\frac{1}{2}}\hat{g}\|_{L^{2}(\mathbb{R}^{n-1})}\leq\|g\|_{H^{1}(\mathbb{R}^{n-1})}$

because of $m\geq 2$. Noticing that $\frac{m}{2}\leq m-1$ for $m\geq 2$, one
knows

$\|x_{n}^{\frac{m}{2}}\partial_{x^{\prime}}u\|_{L^{2}(G_{1})}\geq\|x_{n}^{m-1}\partial_{x^{\prime}}u\|_{L^{2}(G_{1})}$

which implies the present lemma for $m\geq 2$.

∎

In fact, differentiating the linear equation (2.1), one can get the following
lemma.

######  Lemma 2.2.

Suppose $u\in C_{c}^{\infty}(\overline{\mathbb{R}^{n}_{+}})$ solves (2.1) with
supp $u\subset G_{1}$. Then there holds

$\displaystyle\|u\|_{\widetilde{W}^{k,2}(G_{1})}\leq
C_{k}\left(\|f\|_{H^{k}(G_{1})}+\|g\|_{H^{k+1}(\mathbb{R}^{n-1})}\right),\quad
k\geq 0$ (2.17)

for some positive constant $C_{k}$ depending only on $k,m,n$.

###### Proof.

For $k=0$, this is just (2.2). We now prove the present lemma by induction on
$k$. Suppose the estimate (2.17) is true for $k$, we need to prove (2.17)
holds for $k+1$.

First, we consider $\partial_{x}^{\alpha}u=\partial_{x^{\prime}}^{\alpha}u$,
$|\alpha|=k+1$, $\alpha_{n}=0$. Differentiating (2.1) with respect to
$x^{\prime}$ for $k+1$ times, one gets

$\displaystyle\begin{cases}\partial^{2}_{n}\partial_{x^{\prime}}^{\alpha}u+x_{n}^{m}\Delta_{x^{\prime}}\partial_{x^{\prime}}^{\alpha}u=\partial_{x^{\prime}}^{\alpha}f\quad\text{in}\quad\mathbb{R}^{n}_{+},\\\
\partial_{x^{\prime}}^{\alpha}u(x^{\prime},0)=\partial_{x^{\prime}}^{\alpha}g(x^{\prime}),\quad
x^{\prime}\in\mathbb{R}^{n-1}.\end{cases}$ (2.18)

Applying (2.2) to (2.18), one gets

$\displaystyle\|\partial_{x^{\prime}}^{\alpha}u\|_{\widetilde{W}^{0,2}(G_{1})}\leq
C(\|\partial_{x^{\prime}}^{\alpha}f\|_{L^{2}(G_{1})}+\|\partial_{x^{\prime}}^{\alpha}g\|_{H^{1}(\mathbb{R}^{n-1})}).$
(2.19)

Especially, we obtain

$\displaystyle\begin{split}&\sum_{|\beta|=k+1,\beta_{n}=0}\|\partial_{n}\partial_{x^{\prime}}^{\beta}u(x^{\prime},0)\|_{L^{2}(\mathbb{R}^{n-1})}+\sum_{|\beta|=k+2,\beta_{n}=0}\|x_{n}^{m-1}\partial_{x^{\prime}}^{\beta}u\|_{L^{2}(G_{1})}\\\
&\leq C(\|f\|_{H^{k+1}(G_{1})}+\|g\|_{H^{k+2}(\mathbb{R}^{n-1})}).\end{split}$
(2.20)

Then we consider the case
$\partial_{x}^{\alpha}u=\partial_{n}\partial_{x^{\prime}}^{\alpha^{\prime}}u$
with $|\alpha^{\prime}|=k$. Differentiating (2.1) with respect to $x^{\prime}$
for $k$ times and then with respect to $x_{n}$, one can derive

$\displaystyle\partial_{n}^{2}\partial_{n}\partial_{x^{\prime}}^{\alpha^{\prime}}u+x_{n}^{m}\Delta_{x^{\prime}}\partial_{n}\partial_{x^{\prime}}^{\alpha^{\prime}}u=\partial_{n}\partial_{x^{\prime}}^{\alpha^{\prime}}f-mx_{n}^{m-1}\Delta_{x^{\prime}}\partial_{x^{\prime}}^{\alpha^{\prime}}u\quad\text{in}\quad\mathbb{R}^{n}_{+}.$
(2.21)

For the boundary term, by (2.20), we know

$\displaystyle\|\partial^{\alpha^{\prime}}_{x^{\prime}}u_{n}(x^{\prime},0)\|_{H^{1}({\mathbb{R}^{n-1}})}\leq
C(\|f\|_{H^{k+1}(G_{1})}+\|g\|_{H^{k+2}(\mathbb{R}^{n-1})}).$ (2.22)

Applying (2.2) to (2.21) and using (2.20) and (2.22), one gets

$\begin{split}\|\partial_{n}\partial_{x^{\prime}}^{\alpha^{\prime}}u\|_{\widetilde{W}^{0,2}(G_{1})}\leq&C(\|f\|_{H^{k+1}(G_{1})}+\|x_{n}^{m-1}\Delta_{x^{\prime}}\partial_{x^{\prime}}^{\alpha^{\prime}}u\|_{L^{2}(G_{1})}+\|\partial^{\alpha^{\prime}}_{x^{\prime}}u_{n}(x^{\prime},0)\|_{H^{1}(\mathbb{R}^{n-1})})\\\
\leq&C(\|f\|_{H^{k+1}(G_{1})}+\|g\|_{H^{k+2}(\mathbb{R}^{n-1})}).\end{split}$

The last case is
$\partial^{\alpha}_{x}u=\partial_{n}^{l}\partial^{\beta}_{x^{\prime}}u$,
$l\geq 2$, $|\beta|=k+1-l$. Differentiating (2.1) with respect to $x^{\prime}$
for $k+1-l$ times and then with respect to $x_{n}$ for $l$ times, one can
derive

$\displaystyle\begin{split}&\partial_{n}^{2}(\partial_{n}^{l}\partial^{\beta}_{x^{\prime}}u)+x_{n}^{m}\Delta_{x^{\prime}}(\partial_{n}^{l}\partial^{\beta}_{x^{\prime}}u)\\\
=&\partial_{n}^{l}\partial^{\beta}_{x^{\prime}}f-\sum_{r=1}^{\min(m,l)}\frac{l!}{r!(l-r)!}\partial_{n}^{r}(x_{n}^{m})\partial_{n}^{l-r}\Delta_{x^{\prime}}\partial_{x^{\prime}}^{\beta}u\quad\text{in}\quad\mathbb{R}^{n}_{+}.\end{split}$
(2.23)

We first consider the boundary term. Applying
$\partial_{n}^{l-2}\partial_{x^{\prime}}^{\beta}$ to (2.1), one gets

$\displaystyle\partial_{n}^{l}\partial^{\beta}_{x^{\prime}}u=-\partial_{n}^{l-2}\partial_{x^{\prime}}^{\beta}(x_{n}^{m}\Delta_{x^{\prime}}u)+\partial_{n}^{l-2}\partial_{x^{\prime}}^{\beta}f.$
(2.24)

Restricting (2.24) on $x_{n}=0$, then we obtain

$\displaystyle\|\partial_{n}^{l-2}\partial_{x^{\prime}}^{\beta}f(x^{\prime},0)\|_{H^{1}(\mathbb{R}^{n-1})}\leq
C\|f\|_{H^{k+1}(G_{1})}$ (2.25)

by the trace theorem of Sobolev space. For the term
$\partial_{n}^{l-2}\partial_{x^{\prime}}^{\beta}(x_{n}^{m}\Delta_{x^{\prime}}u)$,
it matters only if $l\geq m+2$ and equals to
$m!C_{l-2}^{m}\partial_{n}^{l-2-m}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u(x^{\prime},0)$.

If $l=m+2$, then we know

$\|\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u(x^{\prime},0)\|_{H^{1}(\mathbb{R}^{n-1})}\leq\|g\|_{H^{k+1}(\mathbb{R}^{n-1})}.$

If $l\geq m+3$, then we know
$\partial_{n}^{l-2-m}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u(x^{\prime},0)=\partial_{n}(\partial_{n}^{l-3-m}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u)(x^{\prime},0)$.
This implies

$\displaystyle\|\partial_{n}^{l-2-m}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u(x^{\prime},0)\|_{H^{1}(\mathbb{R}^{n-1})}$
$\displaystyle\leq$ $\displaystyle
C\|\partial_{n}\partial_{x}^{\tilde{\beta}}u(x^{\prime},0)\|_{H^{1}(\mathbb{R}^{n-1})}$
$\displaystyle\leq$ $\displaystyle
C_{k}(\|f\|_{H^{k}(G_{1})}+\|g\|_{H^{k+1}(\mathbb{R}^{n-1})})$

by the induction assumption for some $\tilde{\beta}$ with
$|{\tilde{\beta}}|=k-m\leq k-1$.

Combining (2.25)-(2), one obtains

$\displaystyle\|\partial_{n}^{l}\partial^{\beta}_{x^{\prime}}u(x^{\prime},0)\|_{H^{1}(\mathbb{R}^{n-1})}\leq
C_{k}(\|f\|_{H^{k}(G_{1})}+\|g\|_{H^{k+1}(\mathbb{R}^{n-1})}).$ (2.27)

According to (2.2), the remaining thing is to analysis the term
$\displaystyle\sum_{r=1}^{\min(m,l)}\partial_{n}^{r}(x_{n}^{m})\partial_{n}^{l-r}\Delta_{x^{\prime}}\partial_{x^{\prime}}^{\beta}u$.
We only need to take care of
$x_{n}^{m-1}\partial_{n}^{l-1}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u$,
$x_{n}^{m-2}\partial_{n}^{l-2}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u(m\geq
2)$, $l\geq 2$, $|\beta|=k+1-l$.

For the term
$x_{n}^{m-1}\partial_{n}^{l-1}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u$:

  * (1)

If $l=2$, i.e.
$x_{n}^{m-1}\partial_{n}^{l-1}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u=x_{n}^{m-1}\partial_{n}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u$,
$|\beta|=k-1$. Then by estimate (2.19), this term can be controlled by
$\|f\|_{H^{k+1}(G_{1})}+\|g\|_{H^{k+2}(\mathbb{R}^{n-1})}$.

  * (2)

If $l\geq 3$, then by induction assumption on $k$, we have

$\begin{split}&\|x_{n}^{m-1}\partial_{n}^{l-1}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u\|_{L^{2}(G_{1})}\leq\|\partial_{n}^{2}(\partial_{n}^{l-3}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u)\|_{L^{2}(G_{1})}\\\
\leq&\|u\|_{\widetilde{W}^{k,2}(G_{1})}\leq
C_{k}(\|f\|_{H^{k}(G_{1})}+\|g\|_{H^{k+1}(\mathbb{R}^{n-1})}).\end{split}$

Similarly, for the term
$x_{n}^{m-2}\partial_{n}^{l-2}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u(m\geq
2)$:

  * (1)

If $l=2$, i.e.
$x_{n}^{m-2}\partial_{n}^{l-2}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u=x_{n}^{m-2}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u$,
$|\beta|=k-1$. Then by estimate (2.19), this term can be controlled by
$\|f\|_{H^{k+1}(G_{1})}+\|g\|_{H^{k+2}(\mathbb{R}^{n-1})}$.

  * (2)

If $l\geq 3$, then by induction assumption on $k$, we have

$\begin{split}&\|x_{n}^{m-2}\partial_{n}^{l-2}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u\|_{L^{2}(G_{1})}\leq\|\partial_{n}(\partial_{n}^{l-3}\partial_{x^{\prime}}^{\beta}\Delta_{x^{\prime}}u)\|_{L^{2}(G_{1})}\\\
\leq&\|u\|_{\widetilde{W}^{k,2}(G_{1})}\leq
C_{k}(\|f\|_{H^{k}(G_{1})}+\|g\|_{H^{k+1}(\mathbb{R}^{n-1})}).\end{split}$

Overall, we obtain

$\|\partial_{n}^{l}\partial^{\beta}_{x^{\prime}}u\|_{\widetilde{W}^{0,2}(G_{1})}\leq
C(\|f\|_{H^{k+1}(G_{1})}+\|g\|_{H^{k+2}(\mathbb{R}^{n-1})}),\quad\forall l\geq
2,|\beta|=k+1-l.$

This ends the proof of present lemma. ∎

######  Lemma 2.3.

For any $u,v\in\widetilde{W}^{k,2}(G_{1})$, there exists a constant $C_{k}$
such that

$\|uv\|_{\widetilde{W}^{k,2}(G_{1})}\leq
C_{k}\|u\|_{\widetilde{W}^{k,2}(G_{1})}\|v\|_{\widetilde{W}^{k,2}(G_{1})}$
(2.28)

provided $k\geq n+3$.

###### Proof.

For any multi-index $\alpha\in\mathbb{N}^{n}$, one has

$\partial_{x}^{\alpha}(uv)=\sum_{\beta+\gamma=\alpha}\frac{\alpha!}{\beta!\gamma!}\partial_{x}^{\beta}u\partial_{x}^{\gamma}v.$

Since the highest order of derivative in $\widetilde{W}^{k,2}(G_{1})$ is
$k+2$, we know $\min(|\beta|,|\gamma|)\leq\frac{k+2}{2}$. Then for the product
$\partial_{x}^{\beta}u\partial_{x}^{\gamma}v$, at least one term is in
$H^{k-\frac{k+2}{2}}(\mathbb{R}^{n}_{+})\hookrightarrow
L^{\infty}(\mathbb{R}^{n}_{+})$ provided $k>n+2$ by Sobolev embedding theorem.
This implies the present lemma. ∎

In the following, we give a lemma which is essentially Lemma 1 in [1].

######  Lemma 2.4.

Let $B_{1}\times\mathbb{B}_{R}$ be the domain in
$\mathbb{R}^{n}\times\mathbb{R}^{L}$. Assume that $\Phi(x,y)$ is a polynomial
and $p$ is a positive integer. Let $\eta\in C_{c}^{\infty}(B_{1})$ is a cut-
off function. Then, there exist positive constants $A_{0}$,
$\widetilde{A}_{0}$ and $A_{1}$, depending only on $n$, $L$, $k$, $\eta$ and
the polynomial $\Phi(x,y)$, such that, for any $C^{p}$-function
$y=(y_{1},\cdots,y_{L}):B_{1}\rightarrow\mathbb{B}_{R}$, if for any $x\in
B_{1}$ and any non-negative integer $l\leq p$,

$\sum_{i=1}^{L}\|\eta^{l}\partial_{x}^{l}y_{i}(x)\|_{\widetilde{W}^{k,2}(B_{1}^{+})}\leq
A_{0}A_{1}^{(l-2)^{+}}{(l-2)^{+}}!,$

for some $k>n+2$. Then, for any $x\in B_{1}^{+}$,

$\|\eta^{p}\partial_{x}^{p}[\Phi(x,y(x))]\|_{\widetilde{W}^{k,2}(B_{1}^{+})}\leq\widetilde{A}_{0}A_{1}^{(p-2)^{+}}(p-2)^{+}!.$

Here, if no confusion occurs, the meaning of $l$ and $p$ can be vary from
multi-index to pure integer.

###### Proof.

By our assumptions, we know

$\Phi(x,y(x))=\sum_{|\alpha|+|\beta|\leq
d}C_{\alpha\beta}x^{\alpha}y(x)^{\beta}.$

Here $d$ is the degree of the polynomial $\Phi(x,y)$ and
$x^{\alpha}=x_{1}^{\alpha_{1}}\cdots x_{n}^{\alpha_{n}}$. Then for
$x^{\alpha}y(x)^{\beta}=x^{\alpha}y_{i_{1}}(x)\cdots y_{i_{|\beta|}}(x)$,
$1\leq i_{1},\cdots,i_{|\beta|}\leq L$, one knows

$\displaystyle\partial_{x}^{p}(x^{\alpha}y(x)^{\beta})=\sum_{k_{0}+k_{1}+\cdots+k_{|\beta|}=p}\frac{p!}{k_{0}!k_{1}!\cdots
k_{|\beta|}!}\partial_{x}^{k_{0}}(x^{\alpha})\partial_{x}^{k_{1}}(y_{i_{1}}(x))\cdots\partial_{x}^{k_{|\beta|}}(y_{i_{|\beta|}}(x)).$
(2.29)

Let $C_{0},C_{1}$ be the positive constants such that

$\begin{split}\sum_{|\alpha|\leq
d}\|\eta^{l}\partial_{x}^{l}(x^{\alpha})\|_{\widetilde{W}^{k,2}(B_{1}^{+}(0))}\leq
C_{0}C_{1}^{(l-2)^{+}}(l-2)^{+}!,\quad l\geq 0.\end{split}$

Let $\|\cdot\|$ be $\|\cdot\|_{\widetilde{W}^{k,2}(B_{1}^{+}(0))}$. Then by
Lemma 2.3 and the assumptions, we know

$\displaystyle\begin{split}&\|\eta^{p}\partial_{x}^{p}(x^{\alpha}y(x)^{\beta})\|\\\
\leq&C_{k}^{|\beta|+1}\sum_{k_{0}+k_{1}+\cdots+k_{|\beta|}=p}\frac{p!}{k_{0}!k_{1}!\cdots
k_{|\beta|}!}C_{0}C_{1}^{(k_{0}-2)^{+}}(k_{0}-2)^{+}!\Pi_{j=1}^{|\beta|}A_{0}A_{1}^{(k_{j}-2)^{+}}(k_{j}-2)!\\\
\leq&(A_{0}C_{k})^{|\beta|+1}A_{1}^{(p-2)^{+}}p!\sum_{k_{0}+k_{1}+\cdots+k_{|\beta|}=p}\frac{(k_{0}-2)^{+}!(k_{1}-2)^{+}!\cdots(k_{|\beta|}-2)^{+}!}{k_{0}!k_{1}!\cdots
k_{|\beta|}!}\end{split}$ (2.30)

provided $A_{0}\geq C_{0},A_{1}\geq C_{1}$. In the following, we will show
that there exists a constant $C_{2}>0$ such that

$\displaystyle\sum_{k_{0}+k_{1}+\cdots+k_{|\beta|}=p}\frac{(k_{0}-2)^{+}!(k_{1}-2)^{+}!\cdots(k_{|\beta|}-2)^{+}!}{k_{0}!k_{1}!\cdots
k_{|\beta|}!}\leq\frac{C_{2}(8\pi^{2}(d+1))^{|\beta|+1}}{(p+1)^{2}},\quad\forall
p\in\mathbb{N}$ (2.31)

by induction for all $|\beta|\leq d$. It is easy to see that, we can choose
$C_{2}$ large enough such that (2.31) holds for all $0\leq p\leq 10$. Suppose
(2.31) holds for $p\geq 10$. Then for $p+1$, one has

$\begin{split}&\sum_{k_{0}+k_{1}+\cdots+k_{|\beta|}=p+1}\frac{(k_{0}-2)^{+}!(k_{1}-2)^{+}!\cdots(k_{|\beta|}-2)^{+}!}{k_{0}!k_{1}!\cdots
k_{|\beta|}!}\\\
\leq&(d+1)\sum_{k_{|\beta|}=1}^{p+1}\frac{(k_{|\beta|}-2)^{+}!}{k_{|\beta|}!}\sum_{k_{0}+\cdots+k_{|\beta|-1}=p+1-k_{|\beta|}}\frac{(k_{0}-2)^{+}!(k_{1}-2)^{+}!\cdots(k_{|\beta|-1}-2)^{+}!}{k_{0}!k_{1}!\cdots
k_{|\beta|-1}!}\\\
\leq&(d+1)C_{2}(8\pi^{2}(d+1))^{|\beta|}\sum_{k_{|\beta|}=1}^{p+1}\frac{(k_{|\beta|}-2)^{+}!}{k_{|\beta|}!(p+2-k_{|\beta|})^{2}}\\\
\leq&4(d+1)C_{2}(8\pi^{2}(d+1))^{|\beta|}\sum_{k_{|\beta|}=1}^{p+1}\frac{1}{k_{|\beta|}^{2}(p+2-k_{|\beta|})^{2}}\\\
\leq&4(d+1)C_{2}(8\pi^{2}(d+1))^{|\beta|}\left(\frac{9}{4(p+1)^{2}}\left(\sum_{k_{|\beta|}=1}^{[\frac{p+1}{3}]}\frac{1}{k_{|\beta|}^{2}}+\sum_{k_{|\beta|}=2[\frac{p+1}{3}]}^{p+1}\frac{1}{(p+2-k_{|\beta|})^{2}}\right)\right.\\\
+&\left.\sum_{k_{|\beta|}=[\frac{p+1}{3}]}^{2[\frac{p+1}{3}]}\frac{1}{k_{|\beta|}^{2}(p+2-k_{|\beta|})^{2}}\right)\leq\frac{C_{2}(8\pi^{2}(d+1))^{|\beta|+1}}{(p+2)^{2}}.\end{split}$

In getting the last inequality of the above, we used

$\sum_{l=1}^{+\infty}\frac{1}{l^{2}}=\frac{\pi^{2}}{6},\quad\frac{27}{p+1}\leq\pi^{2},\quad\frac{(p+2)^{2}}{(p+1)^{2}}\leq
2.$

This ends the proof of (2.31). By (2.31), one knows

$\begin{split}\|\eta^{p}\partial_{x}^{p}(x^{\alpha}y(x)^{\beta})\|\leq(8\pi^{2}A_{0}C_{k}(d+1))^{d+1}A_{1}^{(p-2)^{+}}(p-2)^{+}!.\end{split}$

This implies

$\begin{split}\|\eta^{p}\partial_{x}^{p}[\Phi(x,y(x))]\|\leq(8\pi^{2}A_{0}C_{k}(d+1))^{d+1}A_{1}^{(p-2)^{+}}(p-2)^{+}!\sum_{|\alpha|+|\beta|\leq
d}|C_{\alpha\beta}|.\end{split}$

By letting
$\tilde{A}_{0}=(8\pi^{2}A_{0}C_{k}(d+1))^{d+1}\sum_{|\alpha|+|\beta|\leq
d}|C_{\alpha\beta}|$, we finish the proof of present lemma. ∎

##  3 Analyticity of the solutions

Through out this section, we let $q=m\in\mathbb{Z}^{+}$ in (1.1). Then $u$
solves

$\displaystyle\begin{cases}\det
D^{2}u=\Lambda_{m}(-u)^{m}\quad\text{in}\quad\Omega,\\\
u=0\quad\text{on}\quad\partial\Omega.\end{cases}$ (3.1)

In the following, we omit the constant $\Lambda_{m}$. Since (3.1) is uniformly
elliptic in the interior of $\Omega$, then applying the results of [1, 14], we
can get $u\in C^{\omega}(\Omega)$. The remaining thing is to show $u$ is
analytic up to the boundary.

Suppose $\Omega\subset\mathbb{R}^{n}_{+}$, $0\in\partial\Omega$ and $x_{n}=0$
is the supporting plane of $\Omega$ at $0$. Then we know the non-trivial
solution $u$ of (3.1) satisfies

  * (I)

$u\in C^{\infty}(\overline{\Omega})$ and $u_{n}(0)<0$,
$\nabla_{x^{\prime}}u(0)=0$.

  * (II)

$\lambda I\leq D^{2}_{x^{\prime}}u(0)\leq\Lambda I$ for two positive constants
$\lambda,\Lambda$.

For the proof of (I) and (II), we refer the readers to [Theorem B, [4]] for
$n=2$ and [Theorem 1.4, [11]] for arbitrary dimension.

Then as in [11], we take the following Hodo-graph transformation near $0$,

$y_{n}=-x_{n+1},\quad y_{n+1}=x_{n},\quad y_{k}=x_{k}(1\leq k\leq n-1).$

In the new coordinates, the graph of $u$ near $0$ can be represented as

$y_{n+1}=v(y)\quad\text{in}\quad\\{y\in\mathbb{R}^{n}:\quad y_{n}>0\\}\cap
B_{\delta}(0)$

for some $\delta>0$ small enough. This implies

$\displaystyle
u(y^{\prime},v(y))+y_{n}=0\quad\text{in}\quad\\{y\in\mathbb{R}^{n}:\quad
y_{n}>0\\}\cap B_{\delta}(0).$ (3.2)

Then, differentiating the above equation directly yields

$u_{\alpha}+u_{n}v_{\alpha}=0,\quad u_{n}v_{n}+1=0,\quad\alpha=1,\cdots,n-1.$

Since the Gauss curvature will not change, one has

$\displaystyle\begin{cases}\det D^{2}v=K(1+|\nabla
v|^{2})^{\frac{n+2}{2}}=\det D^{2}u\frac{(1+|\nabla
v|^{2})^{\frac{n+2}{2}}}{(1+|\nabla
u|^{2})^{\frac{n+2}{2}}}=y_{n}^{m}v_{n}^{n+2}\quad\text{in}\quad
B^{+}_{\delta}(0),\\\
v(y^{\prime},0)=\phi(y^{\prime}),\quad|y^{\prime}|\leq\delta,\end{cases}$
(3.3)

where $y_{n}=\phi(y^{\prime})$ represents the boundary $\partial\Omega$ near
$0$. Moreover, the Hodo-graph transformation preserves the analyticity.

######  Lemma 3.1.

Let $v$ be given by (3.2) where $u$ is the non-trivial solution of (3.1). Then
$v$ is analytic near $0$ implies $u$ is analytic near $0$.

###### Proof.

Let $F(y,y_{n+1})=v(y)-y_{n+1}$ and $F(0)=0$. Then by the assumption of the
present lemma, we know $F(y,y_{n+1})$ is analytic in $B_{\delta}(0)$ for some
$\delta>0$ small. Also by (I), one has

$F_{n}(0,0)=v_{n}(0)=-\frac{1}{u_{n}(0)}\neq 0.$

By the real analytic implicit function theorem(Theorem 1.8.3,[9]), we know
there exists $u$ analytic near $0$ such that

$F(y_{1},\cdots,y_{n-1},u(y_{1},\cdots,y_{n-1},y_{n+1}),y_{n+1})=0,$

which implies the present lemma. ∎

Differentiating (3.2) with respect to $y^{\prime}$ for two times, one obtains

$u_{\alpha\beta}+u_{\alpha
n}v_{\beta}+u_{n\beta}v_{\alpha}+u_{nn}v_{\alpha}v_{\beta}+u_{n}v_{\alpha\beta}=0,\quad
1\leq\alpha,\beta\leq n-1.$

This implies

$v_{\alpha\beta}(0)=-\frac{u_{\alpha\beta}(0)}{u_{n}(0)}$ (3.4)

is a positive definite matrix. Then we can perform the partial Legendre
transformation to the solutions of (3.3):

$z_{i}=v_{i}(y)\quad(i\leq n-1),\quad z_{n}=y_{n},\quad
v^{*}(z)=y^{\prime}\cdot\nabla_{y^{\prime}}v-v(y).$ (3.5)

Set the partial Legendre transformation by $z=T(y)$. Then $v^{*}$ should
satisfy

$\displaystyle\left\\{\begin{array}[]{ll}z_{n}^{m}(-v_{n}^{*})^{n+2}\det
D^{2}_{z^{\prime}}v^{*}+v^{*}_{nn}=0\quad\text{in}\quad\Sigma^{+}=T(B^{+}_{\delta}(0))\subset\mathbb{R}^{n}_{+},\\\
v^{*}=\phi^{*}\quad\text{on}\quad\partial\Sigma^{+}\cap\\{z_{n}=0\\},\end{array}\right.$
(3.8)

where $\phi^{*}$ is the Legendre transformation of $\phi$. Also the partial
Legendre transformation preserves the analyticity.

######  Lemma 3.2.

Let $v^{*}$ be the partial Legendre transformation of $v$. Then $v^{*}$ is
analytic near $0$ implies $v$ is analytic near $0$.

###### Proof.

Consider

$\begin{split}&F_{0}(y^{\prime},z,t)=v^{*}(z)-y^{\prime}\cdot
z^{\prime}+t,\quad F_{i}(y^{\prime},z,t)=y_{i}-\partial_{z_{i}}v^{*},\quad
i=1,\cdots,n-1,\end{split}$ (3.9)

where $y^{\prime}\in\mathbb{R}^{n-1},z\in\mathbb{R}^{n},t\in\mathbb{R}$ and
$z^{\prime}=(z_{1},\cdots,z_{n-1})$. Then $F_{0}(0)=F_{i}(0)=0$,
$i=1,\cdots,n-1$. By assumptions, we know $F_{0}(y^{\prime},z,t)$,
$F_{i}(y^{\prime},z,t)$ are real analytic functions and

$\det\left(\frac{\partial(F_{0},\cdots,F_{n-1})}{\partial(t,z^{\prime})}\right)(0)=(-1)^{n-1}\det
D_{z^{\prime}}^{2}v^{*}(0)\neq 0.$ (3.10)

Again by the real analytic implicit function theorem(Theorem 1.8.3,[9]), we
know $F_{0}(\tilde{y}^{\prime},\tilde{z},\tilde{t})=0$,
$F_{i}(\tilde{y}^{\prime},\tilde{z},\tilde{t})=0$ determine real analytic
functions $\tilde{t}(y^{\prime},z_{n})$,
$\tilde{z}^{\prime}(y^{\prime},z_{n})$. This implies the present lemma. ∎

Hence, by the conclusions of Lemma 3.1 and Lemma 3.2, we only need to consider
the following type of equation:

$\displaystyle\begin{cases}x_{n}^{m}(-u_{n})^{n+2}\det
D^{2}_{x^{\prime}}u+u_{nn}=0\quad\text{in}\quad B_{1}^{+},\\\
u=\varphi\quad\text{on}\quad\\{x_{n}=0\\}\cap
B_{1}(=B^{\prime}_{1}).\end{cases}$ (3.11)

######  Theorem 3.1.

Suppose $u\in C^{\infty}(\overline{B_{1}^{+}})$ and $\varphi\in
C^{\omega}(B^{\prime}_{1})$ solve (3.11). Moreover, $u$ satisfies

$\lambda_{\min}(D^{2}_{x^{\prime}}u)\geq
c_{0}>0,\quad|u_{n}(x^{\prime},0)|\geq
c_{0}>0\quad\text{in}\quad\overline{B_{1}^{+}}.$

Then $u\in C^{\omega}(\overline{B^{+}_{\frac{1}{2}}})$.

######  Remark 3.1.

Combining Theorem 3.1, Lemma 3.1 and Lemma 3.2, we know Theorem 1.1 holds.

Since analyticity is a local property, we restrict our discussion on
$B_{2r}^{+}$ for some $r$ small enough to be determined later. Then our aim of
the remaining paragraph is to show that there exist two positive constants
$A_{0},A_{1}$ such that

$\displaystyle\|\eta^{N-2}\partial_{x}^{N}u\|_{\widetilde{W}^{k-i,2}}\leq
A_{0}A_{1}^{(N-4)^{+}}(N-4-i)^{+}!,\quad i=0,1,2.$ (3.12)

Here, we fix a large enough integer $k$ such that Lemma 2.3 holds. $\eta$ is a
cut-off function and has the following form

$\eta(x)=\chi(x_{1})\cdots\chi(x_{n}),$

where $0\leq\chi\leq 1$ is a cut-off function satisfying

$\chi(t)\equiv 1,\quad\text{in}\quad[-r,r],\quad\chi\equiv
0,\quad\text{in}\quad[-2r,2r]^{c}.$ (3.13)

In fact, we just need to consider $\eta$ in $\\{x\in B_{2r}|x_{n}>0\\}$. And
if no confusion occurs, the meaning of $N$ can vary from multi-index to pure
positive integer.

In the following, we will prove (3.12) via induction. Suppose (3.12) is true
for $0,1,2,\cdots,N$. We need to show it holds for $N+1$.

Firstly, differentiating equation (3.11) with respect to $x_{l}$,
$l=1,\cdots,n-1$, one has

$\displaystyle\begin{cases}\displaystyle\sum_{i,j=1}^{n-1}x_{n}^{m}(-u_{n})^{n+2}U^{ij}\partial_{ij}(\partial_{l}u)+\partial_{nn}\partial_{l}u=-\partial_{l}(x_{n}^{m}(-u_{n})^{n+2})\det
D^{2}_{x^{\prime}}u\quad\text{in}\quad B_{1}^{+},\\\
\partial_{l}u=\partial_{l}\varphi\quad\text{on}\quad\\{x_{n}=0\\}\cap
B_{1},\end{cases}$ (3.14)

where $U^{ij}$ is the cofactor matrix of $D_{x^{\prime}}^{2}u$. Set

$G=-\partial_{l}(x_{n}^{m}(-u_{n})^{n+2})\det D^{2}_{x^{\prime}}u.$

Then we know $G$ is a polynomial with arguments $x,u,\nabla u,\nabla^{2}u$.
Without loss of generality(after a transformation of coordinates), we may
assume

$(-u_{n})^{n+2}U^{ij}(0)=\delta_{ij}.$

Then we can rewrite the equation of (3.14) as

$\displaystyle
x_{n}^{m}\Delta_{x^{\prime}}(\partial_{l}u)+\partial_{nn}\partial_{l}u=G+\sum_{i,j=1}^{n-1}a^{ij}x_{n}^{m}\partial_{ij}\partial_{l}u\quad\text{in}\quad
B_{1}^{+},$ (3.15)

where

$a^{ij}=\delta_{ij}-(-u_{n})^{n+2}U^{ij},\quad a^{ij}(x)=O(|x|),\quad|x|<<1.$

Differentiating (3.15) for $N$ times with respect to $x^{\prime}$, one gets

$\displaystyle
x_{n}^{m}\Delta_{x^{\prime}}\partial_{x^{\prime}}^{N+1}u+\partial_{nn}(\partial_{x^{\prime}}^{N+1}u)=\partial_{x^{\prime}}^{N}G+\sum_{i,j=1}^{n-1}\partial_{x^{\prime}}^{N}(a^{ij}x_{n}^{m}\partial_{ij}\partial_{l}u)\quad\text{in}\quad
B_{1}^{+}.$ (3.16)

Multiplying (3.16) with $\eta^{N-1}$, one obtains

$\displaystyle\begin{split}&x_{n}^{m}\Delta_{x^{\prime}}(\eta^{N-1}\partial_{x^{\prime}}^{N+1}u)+\partial^{2}_{n}(\eta^{N-1}\partial_{x^{\prime}}^{N+1}u)\\\
=&\quad\eta^{N-1}\partial_{x^{\prime}}^{N}G+\sum_{i,j=1}^{n-1}\eta^{N-1}\partial_{x^{\prime}}^{N}(a^{ij}x_{n}^{m}\partial_{ij}\partial_{l}u)+[\mathcal{L},\eta^{N-1}]\partial_{x^{\prime}}^{N+1}u\quad\text{in}\quad
B_{1}^{+},\end{split}$ (3.17)

where

$\displaystyle\begin{split}[\mathcal{L},\eta^{N-1}]\partial_{x^{\prime}}^{N+1}u&=[x_{n}^{m}\Delta_{x^{\prime}},\eta^{N-1}]\partial_{x^{\prime}}^{N+1}u+[\partial_{n}^{2},\eta^{N-1}]\partial_{x^{\prime}}^{N+1}u\\\
&=x_{n}^{m}(\Delta_{x^{\prime}}\eta^{N-1})\partial_{x^{\prime}}^{N+1}u+2x_{n}^{m}\sum_{i=1}^{n-1}\partial_{i}(\eta^{N-1})\partial_{i}(\partial_{x}^{N+1}u)\\\
&\quad\quad+(\partial_{n}^{2}\eta^{N-1})\partial_{x^{\prime}}^{N+1}u+2\partial_{n}\eta^{N-1}\partial_{n}\partial_{x^{\prime}}^{N+1}u.\end{split}$
(3.18)

As previous, we consider three cases stated in the following three lemmas. In
the following proof, the constants $C,C_{1}...$ may vary from line to line,
and can depend on $r$ but is independent of $A_{0},A_{1}$. In addition, we use
the constant $c$ to denote the quantities which are independent of $r$.

We first prove (3.12) holds for $\eta^{N-1}\partial_{x^{\prime}}^{N+1}u$ case.

######  Lemma 3.3.

Suppose the assumptions in Theorem 3.1 are fulfilled. Suppose (3.12) holds for
sufficiently large $A_{0},A_{1}$. Then there holds

$\|\eta^{N-1}\partial_{x^{\prime}}^{N+1}u\|_{\widetilde{W}^{k-i,2}}\leq
C_{1}\tilde{A}_{0}A_{0}A_{1}^{N-4}(N-3-i)!,\quad i=0,1,2$

for some constant $C_{1}>0$ and $\tilde{A}_{0}$ is the constant in Lemma 2.4.

###### Proof.

For $i=1$, we know

$\begin{split}\|\eta^{N-1}\partial_{x^{\prime}}^{N+1}u\|_{\widetilde{W}^{k-1,2}}=&\|\partial_{x^{\prime}}(\eta^{N-1}\partial_{x^{\prime}}^{N}u)-(N-1)\eta^{N-2}\partial_{x^{\prime}}\eta\partial_{x^{\prime}}^{N}u\|_{\widetilde{W}^{k-1,2}}\\\
\leq&C_{1}\|\eta^{N-2}\partial_{x^{\prime}}^{N}u\|_{\widetilde{W}^{k,2}}+C_{1}(N-1)\|\eta^{N-2}\partial_{x^{\prime}}^{N}u\|_{\widetilde{W}^{k-1,2}}\\\
\leq&CA_{0}A_{1}^{N-4}(N-4)!.\end{split}$

In getting the last inequality, we used (3.12). By a similar argument, we can
prove the case $i=2$.

It remains to prove the case $i=0$. Recall that
$\eta^{N-1}\partial_{x^{\prime}}^{N+1}u$ satisfies

$\displaystyle\begin{split}&x_{n}^{m}\Delta_{x^{\prime}}(\eta^{N-1}\partial_{x^{\prime}}^{N+1}u)+\partial^{2}_{n}(\eta^{N-1}\partial_{x^{\prime}}^{N+1}u)\\\
=&\quad\eta^{N-1}\partial_{x^{\prime}}^{N}G+\sum_{i,j=1}^{n-1}\eta^{N-1}\partial_{x^{\prime}}^{N}(a^{ij}x_{n}^{m}\partial_{ij}\partial_{x^{\prime}}u)+[\mathcal{L},\eta^{N-1}]\partial_{x^{\prime}}^{N+1}u\\\
=&\quad\mathcal{L}(\eta^{N-1}\partial_{x^{\prime}}^{N+1}u)\quad\text{in}\quad\mathbb{R}^{n}_{+},\\\
&\eta^{N-1}\partial_{x^{\prime}}^{N+1}u(x^{\prime},0)=\hat{\eta}^{N-1}\partial_{x^{\prime}}^{N+1}\varphi(x^{\prime}),\quad
x^{\prime}\in\mathbb{R}^{n-1}.\end{split}$ (3.19)

Here $\hat{\eta}(x^{\prime})=\eta(x^{\prime},0)$. By Lemma 2.2, we know

$\|\eta^{N-1}\partial_{x^{\prime}}^{N+1}u\|_{\widetilde{W}^{k,2}}\leq
C(\|\mathcal{L}(\eta^{N-1}\partial_{x^{\prime}}^{N+1}u)\|_{H^{k}}+\|\hat{\eta}^{N-1}\partial_{x^{\prime}}^{N+1}\varphi\|_{H^{k+1}}).$
(3.20)

By the analyticity of $\varphi$, we may assume

$\|\hat{\eta}^{N-1}\partial_{x^{\prime}}^{N+1}\varphi\|_{H^{k+1}}\leq
A_{0}A_{1}^{N-4}(N-3)!.$

In the following, we only need to estimate the terms in
$\mathcal{L}(\eta^{N-1}\partial_{x^{\prime}}^{N+1}u)$. By the definition of
$G$, we may denote $G$ as $G=x_{n}^{m}\tilde{G}$ where $\tilde{G}$ is a
polynomial of $\nabla u,\nabla^{2}u$.

Firstly, we rewrite $\eta^{N-1}\partial_{x^{\prime}}^{N}G$ as

$\begin{split}\eta^{N-1}\partial_{x^{\prime}}^{N}G=&\partial_{x^{\prime}}^{2}(x_{n}^{m}\eta^{N-1}\partial_{x^{\prime}}^{N-2}\tilde{G})-2(N-1)\partial_{x^{\prime}}((\partial_{x^{\prime}}\eta)x_{n}^{m}\eta^{N-2}\partial_{x^{\prime}}^{N-2}\tilde{G})\\\
&+(N-1)(N-2)(\partial_{x^{\prime}}\eta)(\partial_{x^{\prime}}\eta)x_{n}^{m}\eta^{N-3}\partial_{x^{\prime}}^{N-2}\tilde{G}\\\
&+(N-1)(\partial_{x^{\prime}}^{2}\eta)x_{n}^{m}\eta^{N-2}\partial_{x^{\prime}}^{N-2}\tilde{G}.\end{split}$

Then we estimate the terms in $\eta^{N-1}\partial_{x^{\prime}}^{N}G$ one by
one.

$\displaystyle\begin{split}&\|\partial_{x^{\prime}}^{2}(x_{n}^{m}\eta^{N-1}\partial_{x^{\prime}}^{N-2}\tilde{G})\|_{H^{k}}\leq\|x_{n}^{m}\eta^{N-1}\partial_{x^{\prime}}^{N-2}\tilde{G}\|_{H^{k+2}}\\\
\leq&C\|\eta^{N-2}\partial_{x^{\prime}}^{N-2}\tilde{G}\|_{\widetilde{W}^{k,2}}\leq
C\tilde{A}_{0}A_{1}^{N-4}(N-4)!.\end{split}$ (3.21)

In getting the above inequality, we used Lemma 2.4 and induction assumption
(3.12). And also

$\begin{split}&\|\partial_{x^{\prime}}((\partial_{x^{\prime}}\eta)x_{n}^{m}\eta^{N-2}\partial_{x^{\prime}}^{N-2}\tilde{G})\|_{H^{k}}\leq\|(\partial_{x^{\prime}}\eta)x_{n}^{m}\eta^{N-2}\partial_{x^{\prime}}^{N-2}\tilde{G}\|_{H^{k+1}}\\\
\leq&C\|\eta^{N-2}\partial_{x^{\prime}}^{N-2}\tilde{G}\|_{\widetilde{W}^{k-1,2}}\leq
C\tilde{A}_{0}A_{1}^{N-4}(N-5)!.\end{split}$ (3.22)

Similar arguments yield that

$\displaystyle\|(\partial_{x^{\prime}}\eta)(\partial_{x^{\prime}}\eta)x_{n}^{m}\eta^{N-3}\partial_{x^{\prime}}^{N-2}\tilde{G}\|_{H^{k}}\leq
C\tilde{A}_{0}A_{1}^{N-4}(N-6)!$ (3.23)

if we notice that $\frac{\eta_{j}\eta_{l}}{\eta}\in
C_{c}^{\infty}(\overline{\mathbb{R}^{n}_{+}})$, $j,l=1,2,\cdots,n-1$.
Combining the estimates (3.21)-(3.23), one gets

$\|\eta^{N-1}\partial_{x^{\prime}}^{N}G\|_{H^{k}}\leq
C\tilde{A}_{0}A_{1}^{N-4}(N-4)!.$ (3.24)

In the following, we estimate
$\eta^{N-1}\partial_{x^{\prime}}^{N}(a^{ij}x_{n}^{m}\partial_{ij}\partial_{x^{\prime}}u)$.
As previous, we rewrite this term as

$\begin{split}&\eta^{N-1}\partial_{x^{\prime}}^{N}(a^{ij}x_{n}^{m}\partial_{ij}\partial_{x^{\prime}}u)\\\
=&\partial_{x^{\prime}}^{2}(x_{n}^{m}\eta^{N-1}\partial_{x^{\prime}}^{N-2}(a^{ij}\partial_{ij}\partial_{x^{\prime}}u))-2(N-1)\partial_{x^{\prime}}((\partial_{x^{\prime}}\eta)x_{n}^{m}\eta^{N-2}\partial_{x^{\prime}}^{N-2}(a^{ij}\partial_{ij}\partial_{x^{\prime}}u))\\\
&+(N-1)(N-2)(\partial_{x^{\prime}}\eta)(\partial_{x^{\prime}}\eta)x_{n}^{m}\eta^{N-3}\partial_{x^{\prime}}^{N-2}(a^{ij}\partial_{ij}\partial_{x^{\prime}}u)\\\
&+(N-1)(\partial_{x^{\prime}}^{2}\eta)x_{n}^{m}\eta^{N-2}\partial_{x^{\prime}}^{N-2}(a^{ij}\partial_{ij}\partial_{x^{\prime}}u).\end{split}$
(3.25)

Then one knows

$\displaystyle\begin{split}&\|\partial_{x^{\prime}}^{2}(x_{n}^{m}\eta^{N-1}\partial_{x^{\prime}}^{N-2}(a^{ij}\partial_{ij}\partial_{x^{\prime}}u))\|_{H^{k}}\leq\|\eta^{N-1}\partial_{x^{\prime}}^{N-2}(a^{ij}\partial_{ij}\partial_{x^{\prime}}u)\|_{\widetilde{W}^{k,2}}\\\
\leq&cr\|\eta^{N-1}\partial_{x^{\prime}}^{N+1}u\|_{{\widetilde{W}^{k,2}}}+\sum_{l=1}^{N-2}\frac{(N-2)!}{l!(N-2-l)!}\|\eta^{N-1}\partial_{x^{\prime}}^{l}a^{ij}\partial_{x^{\prime}}^{N-2-l}\partial_{ij}\partial_{x^{\prime}}u\|_{\widetilde{W}^{k,2}}\\\
\leq&cr\|\eta^{N-1}\partial_{x^{\prime}}^{N+1}u\|_{{\widetilde{W}^{k,2}}}+C\tilde{A}_{0}A_{0}A_{1}^{N-4}(N-3)!.\end{split}$
(3.26)

The constant $c$ in the above inequality is independent of $r$. In getting the
last inequality of (3.26), we need to use the induction assumption (3.12) and
do calculations as in Lemma 2.4. Since only the order of derivative in
$\eta^{N-1}\partial_{x^{\prime}}^{l}a^{ij}\partial_{x^{\prime}}^{N-2-l}\partial_{ij}\partial_{x^{\prime}}u$
matters to the estimate, we make a convention that all the terms
$\eta^{N-1}\partial_{x^{\prime}}^{l}a^{ij}\partial_{x^{\prime}}^{N-2-l}\partial_{ij}\partial_{x^{\prime}}u$
are the same for a fixed $l$.

Similarly, we estimate the other terms in
$\eta^{N-1}\partial_{x^{\prime}}^{N}(a^{ij}x_{n}^{m}\partial_{ij}\partial_{x^{\prime}}u)$
and then get

$\|\eta^{N-1}\partial_{x^{\prime}}^{N}(a^{ij}x_{n}^{m}\partial_{ij}\partial_{x^{\prime}}u)\|_{H^{k}}\leq
cr\|\eta^{N-1}\partial_{x^{\prime}}^{N+1}u\|_{{\widetilde{W}^{k,2}}}+C\tilde{A}_{0}A_{0}A_{1}^{N-4}(N-3)!.$
(3.27)

For the last term $[\mathcal{L},\eta^{N-1}]\partial_{x^{\prime}}^{N+1}u$, the
part $[x_{n}^{m}\Delta_{x^{\prime}},\eta^{N-1}]\partial_{x^{\prime}}^{N+1}u$
always contains the factor $x_{n}^{m}$, we can deduce the $H^{k}$-norm for
$[x_{n}^{m}\Delta_{x^{\prime}},\eta^{N-1}]\partial_{x^{\prime}}^{N+1}u$
exactly the same as previous two terms in
$\mathcal{L}(\eta^{N-1}\partial_{x^{\prime}}^{N+1}u)$.

In the following, we estimate the remaining term
$[\partial_{n}^{2},\eta^{N-1}]\partial_{x^{\prime}}^{N+1}u$.

$\displaystyle\begin{split}&[\partial_{n}^{2},\eta^{N-1}]\partial_{x^{\prime}}^{N+1}u=2(N-1)\eta_{n}\eta^{N-2}\partial_{n}\partial_{x^{\prime}}^{N+1}u\\\
&\quad\quad+(N-1)(N-2)\eta_{n}^{2}\eta^{N-3}\partial_{x^{\prime}}^{N+1}u+(N-1)\eta_{nn}\eta^{N-2}\partial_{x^{\prime}}^{N+1}u.\end{split}$
(3.28)

By the definition of $\eta$, we know

$\text{supp }\eta_{n}\subset[-2r,2r]^{n-1}\times[r,2r](:=Q_{r}).$

From the discussion at the beginning of this section, we know $u$ is analytic
in $\\{x_{n}>0\\}$. Hence, in the following, we can always assume

$\displaystyle\|\eta^{\tilde{N}-2}\partial_{x}^{\tilde{N}+2}u\|_{H^{k}(Q_{r})}\leq
A_{0}A_{1}^{(\tilde{N}-4)^{+}}(\tilde{N}-4)^{+}!,\quad\forall\tilde{N}=0,1,2,\cdots.$
(3.29)

Then from (3.28) and (3.29), one obtains

$\|[\partial_{n}^{2},\eta^{N-1}]\partial_{x^{\prime}}^{N+1}u\|_{H^{k}}\leq
CA_{0}A_{1}^{N-4}(N-3)!.$ (3.30)

Combining all the above estimates, one gets

$\|\eta^{N-1}\partial_{x^{\prime}}^{N+1}u\|_{\widetilde{W}^{k,2}}\leq
cr\|\eta^{N-1}\partial_{x^{\prime}}^{N+1}u\|_{\widetilde{W}^{k,2}}+C\tilde{A}_{0}A_{0}A_{1}^{N-4}(N-3)!.$

By taking $r$ small enough, one proves the present lemma. ∎

######  Lemma 3.4.

Suppose the assumptions in Theorem 3.1 are fulfilled. Suppose (3.12) holds for
sufficiently large $A_{0},A_{1}$. Then there holds

$\|\eta^{N-1}\partial_{n}\partial_{x^{\prime}}^{N}u\|_{\widetilde{W}^{k-i,2}}\leq
C_{1}\tilde{A}_{0}A_{0}A_{1}^{N-4}(N-3-i)!,\quad i=0,1,2.$

###### Proof.

As the proof in Lemma 3.3, we only need to show it for $i=0$. Recall that
$\eta^{N-1}\partial_{n}\partial_{x^{\prime}}^{N}u$ satisfies

$\displaystyle\begin{split}&x_{n}^{m}\Delta_{x^{\prime}}(\eta^{N-1}\partial_{n}\partial_{x^{\prime}}^{N}u)+\partial^{2}_{n}(\eta^{N-1}\partial_{n}\partial_{x^{\prime}}^{N}u)\\\
=&\quad\eta^{N-1}\partial_{n}\partial_{x^{\prime}}^{N-1}G+\sum_{i,j=1}^{n-1}\eta^{N-1}\partial_{n}\partial_{x^{\prime}}^{N-1}(a^{ij}x_{n}^{m}\partial_{ij}\partial_{x^{\prime}}u)\\\
&\quad\quad+[\mathcal{L},\eta^{N-1}]\partial_{n}\partial_{x^{\prime}}^{N}u-mx_{n}^{m-1}\eta^{N-1}\Delta_{x^{\prime}}\partial_{x^{\prime}}^{N}u\quad\text{in}\quad\mathbb{R}^{n}_{+}.\end{split}$
(3.31)

We first consider the boundary term
$\eta^{N-1}\partial_{n}\partial_{x^{\prime}}^{N}u(x^{\prime},0)$.

$\displaystyle\begin{split}&\|\eta^{N-1}\partial_{n}\partial_{x^{\prime}}^{N}u(x^{\prime},0)\|_{H^{k+1}}\\\
=&\|\partial_{n}\partial_{x^{\prime}}(\eta^{N-1}\partial_{x^{\prime}}^{N}u(x^{\prime},0))\|_{H^{k}}\\\
\leq&\|\partial_{n}(\eta^{N-1}\partial_{x^{\prime}}^{N+1}u(x^{\prime},0))\|_{H^{k}}+(N-1)\|\eta_{x^{\prime}}\partial_{n}(\eta^{N-2}\partial_{x^{\prime}}^{N}u(x^{\prime},0))\|_{H^{k}}\\\
\leq&C\tilde{A}_{0}A_{0}A_{1}^{N-4}(N-3)!.\end{split}$ (3.32)

In the calculation of (3.32), the property of
$\eta(x)=\chi(x_{1})\cdots\chi(x_{n})$ is used in the commutation of
$\partial_{n}$ and $\eta^{N-1}$ on $x_{n}=0$. In getting the last inequality
of (3.32), we used Lemma 3.3.

In the following, we estimate the $H^{k}$-norm for
$\eta^{N-1}\partial_{n}\partial_{x^{\prime}}^{N-1}G,\eta^{N-1}\partial_{n}\partial_{x^{\prime}}^{N-1}(a^{ij}x_{n}^{m}\partial_{ij}\partial_{x^{\prime}}u)$,
$[\mathcal{L},\eta^{N-1}]\partial_{n}\partial_{x^{\prime}}^{N}u,mx_{n}^{m-1}\eta^{N-1}\Delta_{x^{\prime}}\partial_{x^{\prime}}^{N}u$
due to Lemma 2.2 and (3.31).

As in the proof of Lemma 3.3, we rewrite
$\eta^{N-1}\partial_{n}\partial_{x^{\prime}}^{N-1}G$ as

$\displaystyle\begin{split}\eta^{N-1}\partial_{n}\partial_{x^{\prime}}^{N-1}G=&\eta^{N-1}\partial_{n}(x_{n}^{m}\partial_{x^{\prime}}^{N-1}\tilde{G})\\\
=&\partial_{nx^{\prime}}(\eta^{N-1}x_{n}^{m}\partial_{x^{\prime}}^{N-2}\tilde{G})-(N-1)\partial_{n}(\eta_{x^{\prime}}\eta^{N-2}x_{n}^{m}\partial_{x^{\prime}}^{N-2}\tilde{G})\\\
&-(N-1)\partial_{x^{\prime}}(\eta_{n}\eta^{N-2}x_{n}^{m}\partial_{x^{\prime}}^{N-2}\tilde{G})\\\
&+(N-1)(N-2)\eta_{n}\eta_{x^{\prime}}\eta^{N-3}x_{n}^{m}\partial_{x^{\prime}}^{N-2}\tilde{G}+(N-1)\eta_{nx^{\prime}}\eta^{N-2}x_{n}^{m}\partial_{x^{\prime}}^{N-2}\tilde{G}.\end{split}$
(3.33)

Then all the terms on the right hand-side of (3.33) can be estimated as in the
proof of Lemma 3.3 due to the presence of $x_{n}^{m}$. Thus we have

$\|\eta^{N-1}\partial_{n}(x_{n}^{m}\partial_{x^{\prime}}^{N-1}\tilde{G})\|_{H^{k}}\leq
C\tilde{A}_{0}A_{1}^{N-4}(N-4)!.$

Similarly,
$\eta^{N-1}\partial_{n}\partial_{x^{\prime}}^{N-1}(a^{ij}x_{n}^{m}\partial_{ij}u_{x^{\prime}})$,
$[\mathcal{L},\eta^{N-1}]\partial_{n}\partial_{x^{\prime}}^{N}u$ can be
estimated exactly as in Lemma 3.3. We only need to take care of
$x_{n}^{m-1}\eta^{N-1}\Delta_{x^{\prime}}\partial_{x^{\prime}}^{N}u$.

$\displaystyle\begin{split}&x_{n}^{m-1}\eta^{N-1}\Delta_{x^{\prime}}\partial_{x^{\prime}}^{N}u\\\
=&\partial_{x^{\prime}}(x_{n}^{m-1}\eta^{N-1}\partial_{x^{\prime}}^{N-1}\Delta_{x^{\prime}}u)-(N-1)x_{n}^{m-1}\partial_{x^{\prime}}(\eta_{x^{\prime}}\eta^{N-2}\partial_{x^{\prime}}^{N-2}\Delta_{x^{\prime}}u)\\\
+&(N-1)(N-2)x_{n}^{m-1}\eta_{x^{\prime}}\eta_{x^{\prime}}\eta^{N-3}\partial_{x^{\prime}}^{N-2}\Delta_{x^{\prime}}u+(N-1)\eta_{x^{\prime}x^{\prime}}\eta^{N-2}x_{n}^{m-1}\partial_{x^{\prime}}^{N-2}\Delta_{x^{\prime}}u.\end{split}$
(3.34)

For the first term
$\partial_{x^{\prime}}(x_{n}^{m-1}\eta^{N-1}\partial_{x^{\prime}}^{N-1}\Delta_{x^{\prime}}u)$,
we have

$\begin{split}&\|\partial_{x^{\prime}}(x_{n}^{m-1}\eta^{N-1}\partial_{x^{\prime}}^{N-1}\Delta_{x^{\prime}}u)\|_{H^{k}}\leq\|x_{n}^{m-1}\eta^{N-1}\partial_{x^{\prime}}^{N-1}\Delta_{x^{\prime}}u\|_{H^{k+1}}\\\
\leq&C_{1}\|\eta^{N-1}\partial_{x^{\prime}}^{N+1}u\|_{\widetilde{W}^{k,2}}\leq
C_{2}\tilde{A}_{0}A_{0}A_{1}^{N-4}(N-3)!.\end{split}$ (3.35)

In getting the above inequality, we used Lemma 3.3 and the definition of
$\widetilde{W}^{k,2}$. All the remaining terms in (3.34) can be estimated in a
similar way.

Combining all the above estimates together yield the present lemma. ∎

######  Lemma 3.5.

Suppose the assumptions in Theorem 3.1 are fulfilled. Suppose (3.12) holds for
sufficiently large $A_{0},A_{1}$. For $2\leq l\leq N+1$, there holds

$\|\eta^{N-1}\partial^{l}_{n}\partial_{x^{\prime}}^{N+1-l}u\|_{\widetilde{W}^{k-i,2}}\leq
C_{1}\tilde{A}_{0}A_{0}A_{1}^{N-4}(N-3-i)!,\quad i=0,1,2.$

###### Proof.

As previous, we only need to show the case for $i=0$. Recall that
$\eta^{N-1}\partial^{l}_{n}\partial_{x^{\prime}}^{N+1-l}u$ satisfies

$\displaystyle\begin{split}&x_{n}^{m}\Delta_{x^{\prime}}(\eta^{N-1}\partial^{l}_{n}\partial_{x^{\prime}}^{N+1-l}u)+\partial^{2}_{n}(\eta^{N-1}\partial^{l}_{n}\partial_{x^{\prime}}^{N+1-l}u)\\\
=&\quad\eta^{N-1}\partial^{l-1}_{n}\partial_{x^{\prime}}^{N+1-l}G+\sum_{i,j=1}^{n-1}\eta^{N-1}\partial^{l-1}_{n}\partial_{x^{\prime}}^{N-1+l}(a^{ij}x_{n}^{m}\partial_{ij}\partial_{n}u)+[\mathcal{L},\eta^{N-1}]\partial^{l}_{n}\partial_{x^{\prime}}^{N+1-l}u\\\
&\quad-\eta^{N-1}\sum_{l^{\prime}=1}^{\min(l-1,m)}\frac{(l-1)!}{l^{\prime}!(l-1-l^{\prime})!}\partial_{n}^{l^{\prime}}(x_{n}^{m})\partial_{n}^{l-l^{\prime}-1}\partial_{x^{\prime}}^{N+1-l}(\Delta_{x^{\prime}}\partial_{n}u)\quad\text{in}\quad\mathbb{R}^{n}_{+}\end{split}$
(3.36)

where

$\displaystyle[\mathcal{L},\eta^{N-1}]\partial^{l}_{n}\partial_{x^{\prime}}^{N+1-l}u=[x_{n}^{m}\Delta_{x^{\prime}},\eta^{N-1}]\partial^{l}_{n}\partial_{x^{\prime}}^{N+1-l}u+[\partial_{n}^{2},\eta^{N-1}]\partial^{l}_{n}\partial_{x^{\prime}}^{N+1-l}u.$

For the boundary term
$\eta^{N-1}\partial^{l}_{n}\partial_{x^{\prime}}^{N+1-l}u(x^{\prime},0)$, we
distinguish with two cases:

  * (1)

$2\leq l\leq m+1$. Then by the equation (3.11), we know
$\eta^{N-1}\partial^{l}_{n}\partial_{x^{\prime}}^{N+1-l}u(x^{\prime},0)\equiv
0$.

  * (2)

$m+2\leq l\leq N+1$. Differentiating (3.11) with respect to $x_{n}$ for $l-2$
times and then with respect to $x^{\prime}$ for $N+1-l$ times, one gets

$\begin{split}&\eta^{N-1}\partial^{l}_{n}\partial_{x^{\prime}}^{N+1-l}u(x^{\prime},0)\\\
=&-\eta^{N-1}\partial_{n}^{l-2}\partial_{x^{\prime}}^{N+1-l}(x_{n}^{m}(-u_{n})^{n+2}\det
D^{2}_{x^{\prime}}u)\bigg{|}_{x_{n}=0}\\\
=&-\eta^{N-1}\frac{(l-2)!}{(l-2-m)!}\partial^{l-2-m}_{n}\partial_{x^{\prime}}^{N+1-l}(x_{n}^{m}(-u_{n})^{n+2}\det
D^{2}_{x^{\prime}}u)(x^{\prime},0).\end{split}$ (3.37)

Then following the same calculations as in Lemma 2.4 and using the induction
assumptions, one obtains

$\|\eta^{N-1}\partial^{l}_{n}\partial_{x^{\prime}}^{N+1-l}u(x^{\prime},0)\|_{H^{k+1}}\leq
C\tilde{A}_{0}A_{1}^{N-4}(N-4)!.$ (3.38)

For (3.36), we can deduce the $H^{k}$-norm for
$\eta^{N-1}\partial^{l-1}_{n}\partial_{x^{\prime}}^{N+1-l}G$,
$[\mathcal{L},\eta^{N-1}]\partial^{l}_{n}\partial_{x^{\prime}}^{N+1-l}u$ and
$\eta^{N-1}\partial^{l-1}_{n}\partial_{x^{\prime}}^{N-1+l}(a^{ij}x_{n}^{m}\partial_{ij}\partial_{n}u)$
exactly the same as previous lemmas.

Afterwards, we only need to take care of the term
$\eta^{N-1}\partial_{n}^{l^{\prime}}(x_{n}^{m})\partial_{n}^{l-l^{\prime}-1}\partial_{x^{\prime}}^{N+1-l}(\Delta_{x^{\prime}}\partial_{n}u)$.
This term can be discussed in the following different cases:

  * (1)

$l=2,l^{\prime}=1$. We have

$\eta^{N-1}\partial_{n}^{l^{\prime}}(x_{n}^{m})\partial_{n}^{l-l^{\prime}-1}\partial_{x^{\prime}}^{N+1-l}(\Delta_{x^{\prime}}\partial_{n}u)=mx_{n}^{m-1}\eta^{N-1}\partial_{x^{\prime}}^{N-1}\Delta_{x^{\prime}}\partial_{n}u.$

Then similar arguments as (3.34) and by the results of Lemma 3.4, one knows

$\|\eta^{N-1}x_{n}^{m-1}\partial_{x^{\prime}}^{N-1}\Delta_{x^{\prime}}\partial_{n}u\|_{H^{k}}\leq
C\tilde{A}_{0}A_{0}A_{1}^{N-4}(N-3)!.$

  * (2)

$l\geq 3$, $1\leq l^{\prime}\leq\min(l-1,m)$. We can rewrite
$\eta^{N-1}\partial_{n}^{l-l^{\prime}-1}\partial_{x^{\prime}}^{N+1-l}(\Delta_{x^{\prime}}\partial_{n}u)$
as

$\begin{split}&\eta^{N-1}\partial_{n}^{l-l^{\prime}-1}\partial_{x^{\prime}}^{N+1-l}(\Delta_{x^{\prime}}\partial_{n}u)=\partial_{n}^{2}(\eta^{N-1}\partial_{n}^{l-l^{\prime}-2}\partial_{x^{\prime}}^{N+1-l}\Delta_{x^{\prime}}u)\\\
-&2(N-1)\partial_{n}(\eta_{n}\eta^{N-2}\partial_{n}^{l-l^{\prime}-2}\partial_{x^{\prime}}^{N+1-l}\Delta_{x^{\prime}}u)+(N-1)\eta_{nn}\eta^{N-2}\partial_{n}^{l-l^{\prime}-2}\partial_{x^{\prime}}^{N+1-l}\Delta_{x^{\prime}}u\\\
+&(N-1)(N-2)\eta_{n}^{2}\eta^{N-3}\partial_{n}^{l-l^{\prime}-2}\partial_{x^{\prime}}^{N+1-l}\Delta_{x^{\prime}}u.\end{split}$

By the induction assumption (3.12), we know

$\begin{split}&\|\partial_{n}^{2}(\eta^{N-1}\partial_{n}^{l-l^{\prime}-2}\partial_{x^{\prime}}^{N+1-l}\Delta_{x^{\prime}}u)\|_{H^{k}}\\\
\leq&\|\eta^{N-1}\partial_{n}^{l-l^{\prime}-2}\partial_{x^{\prime}}^{N+1-l}\Delta_{x^{\prime}}u)\|_{\widetilde{W}^{k,2}}\\\
\leq&A_{0}A_{1}^{(N-3-l^{\prime})^{+}}(N-3-l^{\prime})^{+}!.\end{split}$

Similar estimates also hold for the remaining terms of
$\eta^{N-1}\partial_{n}^{l-l^{\prime}-1}\partial_{x^{\prime}}^{N+1-l}(\Delta_{x^{\prime}}\partial_{n}u)$
by noticing that these terms contain $\eta_{n}$.

Then

$\|\eta^{N-1}\partial_{n}^{l-l^{\prime}-1}\partial_{x^{\prime}}^{N+1-l}(\Delta_{x^{\prime}}\partial_{n}u)\|_{H^{k}}\leq
CA_{0}A_{1}^{(N-3-l^{\prime})^{+}}(N-3-l^{\prime})^{+}!.$

This implies

$\begin{split}&\sum_{l^{\prime}=1}^{\min(l-1,m)}\frac{(l-1)!}{l^{\prime}!(l-1-l^{\prime})!}\|\eta^{N-1}\partial_{n}^{l^{\prime}}(x_{n}^{m})\partial_{n}^{l-l^{\prime}-1}\partial_{x^{\prime}}^{N+1-l}(\Delta_{x^{\prime}}\partial_{n})\|_{H^{k}}\\\
\leq&CA_{0}A_{1}^{N-4}(N-3)!.\end{split}$ (3.39)

Thus, applying Lemma 2.2, one proves the present lemma. ∎

######  Remark 3.2.

Combining the estimates of Lemma 3.3-3.5 and choosing $A_{1}\geq
C_{1}\tilde{A}_{0}$ large enough, we can get the estimate (3.12) for $N+1$.

##  4 Acknowledgments

The work of the first author is sponsored by Shanghai Rising-Star Program
19QA1400900.

## References

  * [1] A. Friedman.  On the regularity of the solutions of nonlinear elliptic and parabolic systems of partial differential equations.  J. Math. Mech., 7:43–59, 1958.
  * [2] D. Hartenstine.  Brunn-Minkowski-type inequalities related to the Monge-Ampère equation.  Adv. Nonlinear Stud., 9(2):277–294, 2009.
  * [3] Y. Hashimoto.  A remark on the analyticity of the solutions for non-linear elliptic partial differential equations.  Tokyo J. Math., 29(2):271–281, 2006.
  * [4] J. Hong, G. Huang, and W. Wang.  Existence of global smooth solutions to Dirichlet problem for degenerate elliptic Monge-Ampère equations.  Comm. Partial Differential Equations, 36(4):635–656, 2011.
  * [5] J. Hong and G. Li.  $L^{p}$ estimates for a class of integral operators.  J. Partial Differential Equations, 9(4):343–364, 1996.
  * [6] J. Hong and W. Wang.  The regularity of a class of degenerate elliptic Monge-Ampère equations.  J. Partial Differ. Equ., 22(3):234–265, 2009.
  * [7] G. Huang.  Uniqueness of least energy solutions for Monge-Ampère functional.  Calc. Var. Partial Differential Equations, 58(2):Paper No. 73, 20, 2019.
  * [8] K. Kato.  New idea for proof of analyticity of solutions to analytic nonlinear elliptic equations.  SUT J. Math., 32(2):157–161, 1996.
  * [9] S. G. Krantz and H. R. Parks.  A primer of real analytic functions, volume 4 of Basler Lehrbücher [Basel Textbooks].  Birkhäuser Verlag, Basel, 1992.
  * [10] N. Q. Le.  The eigenvalue problem for the Monge-Ampère operator on general bounded convex domains.  Ann. Sc. Norm. Super. Pisa Cl. Sci. (5), 18(4):1519–1559, 2018\.
  * [11] N. Q. Le and O. Savin.  Schauder estimates for degenerate Monge-Ampère equations and smoothness of the eigenfunctions.  Invent. Math., 207(1):389–423, 2017.
  * [12] P.-L. Lions.  Two remarks on Monge-Ampère equations.  Annali di matematica pura ed applicata, 142(1):263–275, 1985.
  * [13] C. B. Morrey, Jr.  On the analyticity of the solutions of analytic non-linear elliptic systems of partial differential equations. I. Analyticity in the interior.  Amer. J. Math., 80:198–218, 1958.
  * [14] C. B. Morrey, Jr.  On the analyticity of the solutions of analytic non-linear elliptic systems of partial differential equations. II. Analyticity at the boundary.  Amer. J. Math., 80:219–237, 1958.
  * [15] C. B. Morrey, Jr. and L. Nirenberg.  On the analyticity of the solutions of linear elliptic systems of partial differential equations.  Comm. Pure Appl. Math., 10:271–290, 1957.
  * [16] O. Savin.  A localization theorem and boundary regularity for a class of degenerate Monge-Ampere equations.  J. Differential Equations, 256(2):327–388, 2014.
  * [17] N. S. Trudinger and X.-J. Wang.  The Monge-Ampère equation and its geometric applications.  In Handbook of geometric analysis. No. 1, volume 7 of Adv. Lect. Math. (ALM), pages 467–524. Int. Press, Somerville, MA, 2008.
  * [18] K. Tso.  On a real Monge-Ampère functional.  Invent. Math., 101(2):425–448, 1990.


# Almost invariant subspaces and operators

David Kazhdan  and  Alexander Polishchuk  To Victor Guillemin, with admiration
Einstein Institute of Mathematics, The Hebrew University of Jerusalem,
Jerusalem 91904, Israel<EMAIL_ADDRESS>Department of Mathematics,
University of Oregon, Eugene, OR 97403, USA; National Research University
Higher School of Economics; and Korea Institute for Advanced Study
<EMAIL_ADDRESS>
###### Abstract.

We give an elementary proof of an efficient version of the Wagner’s theorem on
almost invariant subspaces (see [5]) and deduce some consequences in the
context of Galois extensions.

A.P. is partially supported by the NSF grant DMS-2001224, and within the
framework of the HSE University Basic Research Program and by the Russian
Academic Excellence Project ‘5-100’.

##  1\. Introduction

The goal of this note is to present an elementary proof of the following
linear algebra result and to consider its consequences in the context of
Galois extensions (see Theorem B). In [3] this result is applied to the study
of the Schmidt rank of quartic polynomials.

Theorem A. Let $V$ be a vector space over a field $\mathbf{k}$, $X$ a
collection of subspaces of $V$, $G$ a group acting linearly on $V$ and
preserving $X$ setwise. For $x\in X$ we denote by $A_{x}\subset V$ the
corresponding subspace. Assume that for some $r\geq 1$ the following condition
holds: for any $x,y\in X$ one has $\dim A_{x}/(A_{x}\cap A_{y})\leq r$. Then
there exists a $G$-invariant subspace $W\subset V$, which is a finite sum of
some finite intersections of subspaces from $X$, such that

$\dim W/(W\cap A_{x})\leq r,\ \ \dim A_{x}/(W\cap A_{x})\leq
r\cdot(r+1)^{r+1}.$

Originally, this theorem was proved by Wagner [5], using model theory and
without an explicit bound on $\dim A_{x}/(W\cap A_{x})$. To get an explicit
bound we combine Wagner’s proof with the idea of the proof of Neumann’s
explicit bound in Bergman-Lenstra’s theorem on almost normal subgroups (see
[1]). We conjecture that it should be possible to improve the bound
$r\cdot(r+1)^{r+1}$ to a polynomial (possibly linear) function of $r$, at
least in the case when $G$ is finite.

In section 3 we show that in the special case when $G$ is a finite group
interchanging vectors of a basis, and subspaces are spanned by subsets of this
bases, one can replace $r\cdot(r+1)^{r+1}$ by $2r$. This reduces to the known
set-theoretic result (see [4], [2]), for which we give a new short proof
assuming that $G$ is finite (see Theorem 3.1). Our version of this result
suggests a formula for the approximating $G$-invariant subspace, for which a
better bound (polynomial or even linear in $r$) might hold, see Sec. 3 for a
discussion.

As a corollary of Theorem A, we obtain the following result about linear
subspaces and linear operators that are almost invariant under the action of
the Galois group. This result will be used in our forthcoming study of the
Schmidt rank of quartics over non-closed fields.

Theorem B. Let $E/\mathbf{k}$ be a finite Galois extension with the Galois
group $G$, and let $V_{0}$ be a finite dimensional $\mathbf{k}$-vector space.
Let us set $V=V_{0}\otimes_{\mathbf{k}}E$. We consider the natural action of
$G$ on $V$ and the induced action of $G$ on the set of $E$-linear subspaces of
$V$ and on ${\operatorname{End}}_{E}(V)$.

(i) Suppose $A\subset V$ is an $E$-linear subspace such that for each
$\sigma\in G$, one has

$\dim_{E}(A/(A\cap\sigma A))\leq r,$

for some $r\geq 0$. Then there exists a $\mathbf{k}$-linear subspace
$W_{0}\subset V_{0}$ such that for $W=W_{0}\otimes E\subset V$ one has

$\dim_{E}(W/(W\cap A))\leq r,\ \ \dim_{E}(A/(W\cap A))\leq r\cdot(r+1)^{r+1}.$

(ii) Suppose $V^{\prime}_{0}$ is another finite dimensional
$\mathbf{k}$-vector space, $V^{\prime}=V^{\prime}_{0}\otimes_{\mathbf{k}}E$,
and $T:V\to V^{\prime}$ is an $E$-linear operator such that for any $\sigma\in
G$, one has

${\operatorname{rk}}_{E}(\sigma(T)-T)\leq r,$

for some $r\geq 0$. Then there exists a $\mathbf{k}$-linear operator
$T_{0}:V_{0}\to V^{\prime}_{0}$, such that

${\operatorname{rk}}_{E}(T-(T_{0})_{E})\leq 2r+r\cdot(r+1)^{r+1},$

where $(T_{0})_{E}:V\to V^{\prime}$ is obtained from $T_{0}$ by the extension
of scalars.

Acknowledgment. We are grateful to Ehud Hrushovski for pointing out the
references and for useful discussions.

##  2\. Linear algebra results

###  2.1. Proof of Theorem A

Notation: for a subset $S\subset X$ we set $A_{S}:=\cap_{x\in S}A_{x}$.

For each $m$, $0\leq m\leq r$, let $\SS_{m}$ denote the set of all nonempty
subsets $S\subset X$ such that

$\dim(A_{S}+A_{x})/A_{x}\leq m\ \text{ for any }x\in X.$

Note that $S=X$ is contained in $\SS_{0}$ and $\SS_{m-1}\subset\SS_{m}$. Set

$h(m)=\min_{S\in\SS_{m}}|S|$

which is either a natural number or $\infty$. Note that by assumption
$h(r)=1$. We also have $h(m-1)\geq h(m)$.

We consider separately two cases.

Case 1. There exists $m$, $1\leq m\leq r$, such that $h(m)$ is finite and
$h(m-1)>(r+1)h(m)+1$. Let us take the maximal $m$ with this property and set

$W:=\sum_{S\in\SS_{m}:|S|=h(m)}A_{S}.$

Since for all $m^{\prime}>m$ we have
$h(m^{\prime}-1)\leq(r+1)h(m^{\prime})+1$, we get

$h(m)\leq(r+1)^{r-m}+\ldots+(r+1)^{2}+(r+1)+1\leq(r+1)^{r}.$

Note that since $A_{S}\subset W$ for some $S\in\SS_{m}$ with $|S|=h(m)$, we
have for any $x\in X$,

$\dim A_{x}/A_{x}\cap W\leq\dim A_{x}/A_{x}\cap A_{S}\leq r\cdot h(m)\leq
r\cdot(r+1)^{r}.$

Next, we claim that for any $x\in X$ one has $\dim W/W\cap A_{x}\leq r$.
Indeed, suppose there exists $x\in X$ such that $\dim(W+A_{x})/A_{x}\geq r+1$.
Then there exist $r+1$ subsets $S_{1},\ldots,S_{r+1}\in\SS_{m}$ with
$|S_{i}|=h(m)$, such that

$\dim(\sum_{i=1}^{r+1}A_{S_{i}}+A_{x})/A_{x}\geq r+1.$

Let us consider the subset

$T:=\\{x\\}\cup S_{1}\cup\ldots\cup S_{r+1}\subset X.$

Then we have $|T|\leq(r+1)h(m)+1<h(m-1)$. Hence, $T\not\in\SS_{m-1}$, so there
exists an element $y\in X$ such that

$\dim(A_{T}+A_{y})/A_{y}\geq m.$

But for any $i=1\ldots,r+1$, we have

$(A_{T}+A_{y})/A_{y}\subset(A_{S_{i}}+A_{y})/A_{y},$

and the latter space has dimension $\leq m$ by the definition of $\SS_{m}$.
Hence, we have $A_{T}+A_{y}=A_{S_{i}}+A_{y}$ for any $i$. Therefore,

$\sum_{i=1}^{r+1}A_{S_{i}}\subset A_{T}+A_{y}\subset A_{x}+A_{y},$

so

$\dim(\sum_{i=1}^{r+1}A_{S_{i}}+A_{x})/A_{x}\leq\dim(A_{x}+A_{y})/A_{x}\leq
r,$

which is a condtradiction.

Finally, we note that since $W/(W\cap A_{S})$ is finite dimensional, for any
$S\in\SS_{m}$, $W$ can be written as a finite sum of some intersections
$A_{S}$.

Case 2. For each $m=1,\ldots,r$ one has $h(m-1)\leq(r+1)h(m)+1$. This implies
that

$h(0)\leq(r+1)^{r}+\ldots+(r+1)^{2}+(r+1)+1\leq(r+1)^{r+1}.$

Note that $\SS_{0}$ consists of $S$ such that $A_{S}=\cap_{x\in X}A_{x}$. In
this case we set

$W:=\cap_{x\in X}A_{x}.$

Then since there exists a subset $S\in\SS_{0}$ with $|S|=h(0)$, we get

$\dim A_{x}/A_{x}\cap W=\dim A_{x}/A_{x}\cap A_{S}\leq r\cdot h(0)\leq
r(r+1)^{r+1}.$

This ends the proof.

###  2.2. Almost invariant operators

In this section we use Theorem A to approximate almost $G$-invariant operators
by $G$-invariant ones by considering their graphs.

######  Lemma 2.1.

Let $V$ and $V^{\prime}$ be linear representation of a group $G$, such that
any $G$-invariant subspace $W\subset V$ (resp., $W^{\prime}\subset
V^{\prime}$) of finite codimension (resp., dimension) admits a $G$-invariant
complement. Assume that $T:V\to V^{\prime}$ is a linear operator such that for
any $g\in G$, one has ${\operatorname{rk}}(gTg^{-1}-g)\leq r$, for some $r\geq
0$. Then there exists a $G$-invariant operator $T_{0}:V\to V^{\prime}$ such
that

${\operatorname{rk}}(T-T_{0})\leq 2r+r\cdot(r+1)^{r+1}.$

###### Proof.

Let $A=A_{T}\subset V\oplus V^{\prime}$ denote the graph of $T$, i.e.,
$A=\\{(v,Tv)\\}$. Note that if $A$ and $A^{\prime}$ are graphs of $T$ and
$T^{\prime}$ then $T\cap T^{\prime}=\\{(v,Tv)\ |\ v\in\ker(T-T^{\prime})\\}$,
so

$\dim T/T\cap T^{\prime}={\operatorname{rk}}(T-T^{\prime}).$

Thus, the assumption on $T$ implies that $\dim(A/(A\cap gA))\leq r$ for all
$g\in G$. Applying Theorem A, we obtain a $G$-invariant subspace $A_{0}\subset
V\oplus V^{\prime}$ such that

$\dim A_{0}/(A_{0}\cap A)\leq r,\ \ \dim A/(A_{0}\cap A)\leq r(r+1)^{r+1}.$

Let $p_{1}:V\oplus V^{\prime}\to V$ and $p_{2}:V\oplus V^{\prime}\to
V^{\prime}$ be the projections. Set $K:=\\{v^{\prime}\in V^{\prime}\ |\
(0,v^{\prime})\in A_{0}\\}\subset V^{\prime}$ and $I:=p_{1}(A_{0})\subset V$.
Note that the subspaces $K$ and $I$ are $G$-invariant, and $p_{2}:A_{0}\to
V^{\prime}$ induces a well defined $G$-invariant linear map

$\overline{T}_{0}:I\to V^{\prime}/K,$

such that $A_{0}$ is the pull-back of the graph $\overline{A}_{0}\subset
I\oplus V^{\prime}/K$ of $\overline{T}_{0}$ under the projection $\pi:I\oplus
V^{\prime}\to I\oplus V^{\prime}/K$. Furthermore, we have

$\dim K\leq r,\ \ \dim V/I\leq r(r+1)^{r+1}.$

Hence, we can find a $G$-linear projector $p_{I}:V\to I$ and a $G$-linear
projector $p_{C}:V^{\prime}\to C\subset V^{\prime}$, where $C$ is a
$G$-invariant complement to $K$, such that $p_{C}(K)=0$. Now we set

$T_{0}=p_{C}\circ\overline{T}_{0}\circ p_{I}.$

Note that $\overline{T}_{0}$ is obtained as the composition of $T_{0}|_{I}$
with the projection $V^{\prime}\to V^{\prime}/K$. Let $\overline{T}:I\to
V^{\prime}/K$ denote the composition of $T|_{I}$ with the projection
$V^{\prime}\to V^{\prime}/K$, and let $\overline{A}\subset I\oplus
V^{\prime}/K$ denote the graph of $\overline{T}$. It is easy to see that $\pi$
induces a surjection

$A_{0}/(A_{0}\cap A)\to\overline{A}_{0}/(\overline{A}_{0}\cap\overline{A}),$

so we get

${\operatorname{rk}}(\overline{T}-\overline{T}_{0})=\dim\overline{A}_{0}/(\overline{A}_{0}\cap\overline{A})\leq
r.$

But $\overline{T}-\overline{T}_{0}$ is obtained from $T-T_{0}$ by restricting
to the subspace $I\subset V$ and composing with the projection $V^{\prime}\to
V^{\prime}/K$, so we get

${\operatorname{rk}}(T-T_{0})\leq\dim K+\dim
V/I+{\operatorname{rk}}(\overline{T}-\overline{T}_{0})\leq 2r+r(r+1)^{r+1}.$

∎

###  2.3. Proof of Theorem B

(i) Let us apply Theorem A to the collection $(\sigma A)$ of all Galois
conjugates of $A$. Note that the action of $G$ is only $\mathbf{k}$-linear, so
we should view this is a collection of $\mathbf{k}$-linear subspaces, and use
the dimension function $\dim_{\mathbf{k}}=[E:\mathbf{k}]\cdot\dim_{E}$.
However, the resulting $G$-invariant $\mathbf{k}$-subspace $W$ is in the
lattice generated by $(\sigma A)$, so it is actually an $E$-linear subspace.
But $G$-invariant $E$-linear subspaces of $V$ are precisely subspaces obtained
by extension of scalars from $\mathbf{k}$-linear subspaces of $V_{0}$. This
gives the statement.

(ii) We apply the same strategy as in the proof of Lemma 2.1. First, we find a
$G$-invariant $E$-subspace $W$ approximating the graph $A$ of $T$. Then we use
the fact that $W$ is an extension of scalars from $W_{0}\subset V_{0}\oplus
V^{\prime}_{0}$ and construct $\mathbf{k}$-linear projectors
$p_{C}:V^{\prime}_{0}\to C\subset V^{\prime}_{0}$ and $p_{I}:V_{0}\to I$, as
in the proof of Lemma 2.1, where $I=p_{1}(W_{0})$ and $C$ is a complement in
$V^{\prime}_{0}$ to $(0\oplus V^{\prime}_{0})\cap W_{0}$. This gives the
required operator defined over $\mathbf{k}$.

##  3\. Almost invariant subsets

The following result is a more concrete version of a theorem of Neumann [4]
(see also [2] for related results), stating that if a subset $A$ in a $G$-set
$X$ satisfies $|A\setminus gA|\leq r$ for every $g\in G$ and some $r>0$, then
there exists a $G$-invariant subset $A_{0}\subset X$ such that

$|A\setminus A_{0}|+|A_{0}\setminus A|<2r.$

We make an extra assumption that $G$ is finite but as a bonus we get an
explicit construction of $A_{0}$.

######  Theorem 3.1.

Let $X$ be a set with action of a finite group $G$. Let $A\subset X$ be a
subset such that for any $g\in G$ one has $|A\setminus gA|\leq r$ for some
$r>0$. For every subset $S\subset G$ we denote $A_{S}:=\cap_{g\in S}gA.$ Now
consider the $G$-invariant subset

$A_{0}:=\cup_{|S|>|G|/2}A_{S}.$

Then

$|A\setminus A_{0}|+|A_{0}\setminus A|\leq 2r.$

###### Proof.

We have

$A\setminus A_{0}=\cup_{|S|>|G|/2}A\setminus A_{S}=\\{a\in A\ |\ |\\{g\in G\
|\ ga\in A\\}|\leq|G|/2\\}.$

Now let us consider the set

$P:=\\{(a\in A,g\in G)\ |\ ga\not\in A\\}.$

Considering the fibers of the projection of $P$ to $G$ and using the
assumption, we see that

$|P|\leq r\cdot|G|.$

Now, let us consider the projection $p_{A}:P\to A$. For every $a\in A\setminus
A_{0}$, we have

$|p_{A}^{-1}(a)|=|\\{g\in G\ |\ ga\not\in A\\}|\geq\frac{|G|}{2}.$

Hence, setting

$P_{1}:=p_{A}^{-1}(A\setminus A_{0}),$

we get

$\frac{|G|}{2}\cdot|A\setminus A_{0}|\leq|P_{1}|.$ (1)

Next, setting $B=X\setminus A$, we observe that

$A_{0}\setminus A=A_{0}\cap B=\cup_{|S|>|G|/2}A_{S}\cap B=\\{b\in B\ |\
|\\{g\in G\ |\ gb\in A\\}|>|G|/2\\}.$

Let us consider the projection

$p_{B}:P\to B:(a,g)\mapsto ga$

and set

$P_{2}:=p_{B}^{-1}(A_{0}\cap B).$

Since for every $b\in A_{0}\cap B$, we have

$|p_{B}^{-1}(b)|=|\\{g\in G\ |\ g^{-1}b\in A\\}|>\frac{|G|}{2},$

we deduce that

$\frac{|G|}{2}\cdot|A_{0}\cap B|\leq|P_{2}|.$ (2)

Finally, we claim that $P_{1}$ and $P_{2}$ do not intersect. Indeed, suppose
$(a,g_{0})\in P_{1}\cap P_{2}$. Then

$|\\{g\in G\ |\ ga\not\in A\\}|\geq\frac{|G|}{2}$

and

$|\\{g\in G\ |\ gg_{0}a\in A\\}|>\frac{|G|}{2},$

which is impossible since

$|\\{g\in G\ |\ gg_{0}a\in A\\}|=|\\{g\in G\ |\ ga\in A\\}|=|G|-|\\{g\in G\ |\
gg_{0}a\in A\\}|.$

Thus, combining (1) and (2), we get

$\frac{|G|}{2}\cdot(|A\setminus A_{0}|+|A_{0}\setminus
A|)\leq|P_{1}|+|P_{2}|\leq|P|\leq r\cdot|G|,$

which gives

$|A\setminus A_{0}|+|A_{0}\setminus A|\leq 2r.$

∎

######  Remark 3.2.

One can see from the proof that the only case when we possibly do not get a
strict inequality

$|A\setminus A_{0}|+|A_{0}\setminus A|<2r$

is when $A_{0}\subset A$ and $P_{1}=P$, which is equivalent to
$A_{S}=\cap_{g\in G}gA$ whenever $|S|>|G|/2$. In this case we can replace
$A_{0}$ with

$A^{\prime}_{0}:=\cup_{|S|\geq|G|/2}A_{S}.$

Since for any $S\subset G\setminus\\{1\\}$ with $|S|\geq|G|/2$, we have $A\cap
A_{S}=\cap_{g\in G}gA$, assuming that $A$ is not $G$-invariant, we get
$A\setminus A^{\prime}_{0}\neq\emptyset$. In this case, running the similar
argument to the above proof, we get that

$|A\setminus A^{\prime}_{0}|+|A^{\prime}_{0}\setminus A|<2r.$

Theorem 3.1 suggests that in the linear algebra setup with $A$ a linear
subspace of a $G$-representation $V$, such that $\dim(A/A\cap gA)\leq r$, for
finite $G$, one can try to define the approximating $G$-invariant subspace as

$A_{0}:=\sum_{|S|>|G|/2}A_{S},$ (3)

where $A_{S}=\cap_{g\in S}gA$.

Question. Does there exist a polynomial (or even a linear) function $c(r)$
such that for $(V,G,A)$ as above and $A_{0}$ given by (3) we have

$\dim A/(A_{0}\cap A)\leq c(r)\ \text{ and }\ \dim A_{0}/(A_{0}\cap A)\leq
c(r)?$

## References

  * [1] G. M. Bergman, H. W. Lenstra, Jr., Subgroups close to normal subgroups, J. Algebra 127 (1989), 80–97.
  * [2] L. Brailovsky, D. V. Pasechnik, C. E. Praeger, Subsets close to invariant subsets for group actions, Proc. AMS 123 (1995), 2283–2295.
  * [3] D. Kazhdan, A. Polishchuk, Linear subspaces in hypersurfaces over perfect fields, arXiv preprint.
  * [4] P. Neumann, Permutation groups and close-knit families of sets, Arch. Math (Basel) 67 (1996), 265–274.
  * [5] F. O. Wagner, Almost invariant families, Bull. LMS 30 (1998), 235–240.