# On the Scaling Limits of Determinantal Point Processes with Kernels Induced
by Sturm–Liouville Operators

Folkmar Bornemann  Zentrum Mathematik – M3, Technische Universität München,
80290 München, Germany<EMAIL_ADDRESS>
###### Abstract.

By applying an idea of ?, we study various scaling limits of determinantal
point processes with trace class projection kernels given by spectral
projections of selfadjoint Sturm–Liouville operators. Instead of studying the
convergence of the kernels as functions, the method directly addresses the
strong convergence of the induced integral operators. We show that, for this
notion of convergence, the Dyson, Airy, and Bessel kernels are universal in
the bulk, soft-edge, and hard-edge scaling limits. This result allows us to
give a short and unified derivation of the known formulae for the scaling
limits of the classical unitary random matrix ensembles (GUE, LUE/Wishart,
JUE/MANOVA).

###### 2010 Mathematics Subject Classification:

15B52, 34B24, 33C45

##  1\. Introduction

We consider determinantal point processes on an interval $\Lambda=(a,b)$ with
trace class projection kernel

$K_{n}(x,y)=\sum_{j=0}^{n-1}\phi_{j}(x)\phi_{j}(y),$ (1.1)

where $\phi_{0},\phi_{1},\ldots,\phi_{n-1}$ are orthonormal in
$L^{2}(\Lambda)$; each $\phi_{j}$ may have some dependence on $n$ that we
suppress from the notation. We recall (see, e.g., ?, §4.2) that for such
processes the joint probability density of the $n$ points is given by

$p_{n}(x_{1},\ldots,x_{n})=\frac{1}{n!}\det_{i,j=1}^{n}K_{n}(x_{i},x_{j}),$

the mean counting probability is given by the density

$\rho_{n}(x)=n^{-1}K_{n}(x,x),$

and the gap probabilities are given, by the inclusion-exclusion principle, in
terms of a Fredholm determinant, namely

$E_{n}(J)={\mathbb{P}}(\\{x_{1},\ldots,x_{n}\\}\cap
J=\emptyset)=\det(I-{\mathbb{1}}_{J}K_{n}{\mathbb{1}}_{J}).$

The various scaling limits are usually derived from a pointwise convergence of
the kernel function $K_{n}(x,y)$ obtained from considering the large $n$
asymptotic of the eigenfunctions $\phi_{j}$, which can be technically _very_
involved.111Based on the two-scale Plancherel–Rotach asymptotic of classical
orthogonal polynomials or, methodically more general, on the asymptotic of
Riemann–Hilbert problems; see, e.g., ?, ?, ?, ?, ?, and ?. ? suggested, for
discrete point processes, a different, conceptually and technically much
simpler approach based on selfadjoint difference operators. We will show that
their method, generalized to selfadjoint Sturm–Liouville operators, allows us
to give a short and unified derivation of the various scaling limits for the
unitary random matrix ensembles (GUE, LUE/Wishart, JUE/MANOVA) that are based
on the classical orthogonal polynomials (Hermite, Laguerre, Jacobi).

### The Borodin–Olshanski Method

The method proceeds along three steps: First, we identify the induced integral
operator $K_{n}$ as the spectral projection

$K_{n}={\mathbb{1}}_{(-\infty,0)}(L_{n})$

of some selfadjoint ordinary differential operator $L_{n}$ on
$L^{2}(\Lambda)$. Any scaling of the point process by
$x=\sigma_{n}\xi+\mu_{n}$ ($\sigma_{n}\neq 0$) yields, in turn, the scaled
objects

$\tilde{E}_{n}(J)=\det(I-{\mathbb{1}}_{J}\tilde{K}_{n}{\mathbb{1}}_{J}),\quad\tilde{K}_{n}={\mathbb{1}}_{(-\infty,0)}(\tilde{L}_{n}),$

where $\tilde{L}_{n}$ is a selfadjoint differential operator on
$L^{2}(\tilde{\Lambda}_{n})$,
$\tilde{\Lambda}_{n}=(\tilde{a}_{n},\tilde{b}_{n})$.

Second, if $\tilde{\Lambda}_{n}\subset\tilde{\Lambda}=(\tilde{a},\tilde{b})$
with $\tilde{a}_{n}\to\tilde{a}$, $\tilde{b}_{n}\to\tilde{b}$, we aim for a
selfadjoint operator $\tilde{L}$ on $L^{2}(\tilde{\Lambda})$ with a core $C$
such that eventually $C\subset D(\tilde{L}_{n})$ and

$\tilde{L}_{n}u\,\to\,\tilde{L}u\qquad(u\in C).$ (1.2)

The point is that, if the test functions from $C$ are particularly nice, such
a convergence is just a simple consequence of the _locally uniform convergence
of the coefficients_ of the differential operators $\tilde{L}_{n}$—a
convergence that is, typically, an easy calculus exercise. Now, given (1.2),
the concept of _strong resolvent convergence_ (see Theorem 2) immediately
yields,222By “$\overset{s}{\longrightarrow}$” we denote the strong convergence
of operators acting on $L^{2}$. if $0\not\in\sigma_{pp}(\tilde{L})$,

$K_{n}{\mathbb{1}}_{\tilde{\Lambda}_{n}}={\mathbb{1}}_{(-\infty,0)}(\tilde{L}_{n}){\mathbb{1}}_{\tilde{\Lambda}_{n}}\,\overset{s}{\longrightarrow}\,{\mathbb{1}}_{(-\infty,0)}(\tilde{L}).$

Third, we take an interval $J\subset\tilde{\Lambda}$, eventually satisfying
$J\subset\tilde{\Lambda}_{n}$, such that the operator
${\mathbb{1}}_{(-\infty,0)}(\tilde{L}){\mathbb{1}}_{J}$ is trace class with
kernel $\tilde{K}(x,y)$ (which can be obtained from the generalized
eigenfunction expansion of $\tilde{L}$, see §A.2). Then, we immediately get
the strong convergence

$\tilde{K}_{n}{\mathbb{1}}_{J}\,\overset{s}{\longrightarrow}\,\tilde{K}\,{\mathbb{1}}_{J}.$

######  Remark 1.1.

? sketches the Borodin–Olshanski method, applied to the bulk and edge scaling
of GUE, as a heuristic device. Because of the microlocal methods that he uses
to calculate the projection ${\mathbb{1}}_{(-\infty,0)}(\tilde{L})$, he puts
his sketch under the headline “The Dyson and Airy kernels of GUE via
semiclassical analysis”.

### Scaling Limits and Other Modes of Convergence

Given that one just has to establish the convergence of the coefficients of a
differential operator (instead of an asymptotic of its eigenfunctions), the
Borodin–Olshanski method is an extremely simple device to determine all the
scalings $x=\sigma_{n}\xi+\mu_{n}$ that would yield some meaningful limit
$\tilde{K}_{n}{\mathbb{1}}_{J}\,\to\,\tilde{K}\,{\mathbb{1}}_{J}$, namely in
the strong operator topology. Other modes of convergence have been studied in
the literature, ranging from some weak convergence of $k$-point correlation
functions over pointwise convergence of the kernel functions to the
convergence of gap probabilities, that is,

$\tilde{E}_{n}(J)=\det(I-{\mathbb{1}}_{J}\tilde{K}_{n}{\mathbb{1}}_{J})\to\det(I-{\mathbb{1}}_{J}\tilde{K}\,{\mathbb{1}}_{J})=\tilde{E}(J).$

From a probabilistic point of view, the latter convergence is of particular
interest and has been shown in at least three ways:

  1. (1)

By Hadamard’s inequality, convergence of the determinants follows directly
from the locally uniform convergence of the kernels $K_{n}$ (?, Lemma 3.4.5)
and, for unbounded $J$, from further large deviation estimates (?, Lemma
3.3.2). This way, the limit gap probabilities in the bulk and soft edge
scaling limit of GUE can rigorously be established (see, e.g., ?, §§3.5 and
3.7).

  2. (2)

Since $A\mapsto\det(I-A)$ is continuous with respect to the trace class norm
(?, Thm. 3.4),
$\tilde{K}_{n}{\mathbb{1}}_{J}\,\to\,\tilde{K}\,{\mathbb{1}}_{J}$ in trace
class norm would generally suffice. Such a convergence can be proved by
factorizing the trace class operators into Hilbert–Schmidt operators and
obtaining the $L^{2}$-convergence of the factorized kernels once more from
locally uniform convergence, see the work of ? on the scaling limits of the
LUE/Wishart ensembles (?) and that on the limits of the JUE/MANOVA ensembles
(?).

  3. (3)

Since ${\mathbb{1}}_{J}\tilde{K}_{n}{\mathbb{1}}_{J}$ and
${\mathbb{1}}_{J}\tilde{K}\,{\mathbb{1}}_{J}$ are selfadjoint and positive
semi-definite, yet another way is by observing that the convergence
$\tilde{K}_{n}{\mathbb{1}}_{J}\,\to\,\tilde{K}\,{\mathbb{1}}_{J}$ in trace
class norm is, for continuous kernels, equivalent (?, Thm. 2.20) to the
combination of both, the convergence
$\tilde{K}_{n}{\mathbb{1}}_{J}\,\to\,\tilde{K}\,{\mathbb{1}}_{J}$ in the weak
operator topology and the convergence of the traces

$\displaystyle\int_{J}\tilde{K}_{n}(\xi,\xi)\,d\xi\to\int_{J}\tilde{K}(\xi,\xi)\,d\xi.$
(T)

Once again, these convergences follow from locally uniform convergence of the
kernels; see ? for an application of this method to the bulk scaling limit of
GUE.

Since convergence in the strong operator topology implies convergence in the
weak one, the Borodin–Olshanski method would thus establish the convergence of
gap probabilities if we were only able to show condition (T) by some
additional, similarly short and simply argument. Note that, by the ideal
property of the trace class, condition (T) implies the same condition for all
$J^{\prime}\subset J$. We fall, however, short of conceiving a proof strategy
for condition (T) that would be independent of all the laborious proofs of
locally uniform convergence of the kernels.

######  Remark 1.2.

Contrary to the discrete case originally considered by Borodin and Olshanski,
it is also not immediate to infer from the strong convergence of the induced
integral operators the pointwise convergence of the kernels,

$\tilde{K}_{n}(\xi,\eta)\to\tilde{K}(\xi,\eta).$

For instance, in §2, we will need just a single such instance, namely

$\displaystyle\tilde{K}_{n}(0,0)\to\tilde{K}(0,0),$ (K0)

to prove a limit law
$\tilde{\rho}_{n}(t)\,dt\overset{w}{\longrightarrow}\tilde{\rho}(t)\,dt$ for
the mean counting probability. Using mollified Dirac deltas, pointwise
convergence would generally follow, for continuously differentiable
$\tilde{K}_{n}(\xi,\eta)$, if were able to bound, locally uniform, the
gradient of $\tilde{K}_{n}(\xi,\eta)$. Then, by dominated convergence,
criterion (T) would already be satisfied if we established an integrable bound
of $\tilde{K}_{n}(\xi,\xi)$ on $J$. Since the scalings laws are, however,
maneuvering just at the edge between trivial cases (i.e., zero limits) and
divergent cases, it is conceivable that a proof of such bounds might not be
significantly simpler than a proof of convergence of the gap probabilities
itself.

### The Main Result

Using the Borodin–Olshanski method, we will prove the following general result
on selfadjoint Sturm–Liouville operators; a result that adds a further class
of problems to the _universality_ (?) of the Dyson, Airy, and Bessel kernel in
the bulk, soft-edge, and hard-edge scaling limits.

######  Theorem 1.

Consider a selfadjoint realization $L_{n}$ of the formally selfadjoint
Sturm–Liouville operator333Since, in this paper, we always consider a
particular selfadjoint realization of a formal differential operator, we will
use the same letter to denote both.

$L_{n}=-\frac{d}{dx}\left(p(x)\frac{d}{dx}\right)+q_{n}(x)-\lambda_{n}$

on $\Lambda=(-\infty,\infty)$, $\Lambda=(0,\infty)$, or $\Lambda=(0,1)$ with
$p,q_{n}\in C^{\infty}(\Lambda)$ and $p(x)>0$ for all $x\in\Lambda$ such that,
for $t\in\Lambda$ and $n\to\infty$,

$n^{-2\kappa^{\prime}}\lambda_{n}\sim\omega,\qquad
n^{-2\kappa^{\prime}}q_{n}(n^{\kappa}t)\sim\tilde{q}(t),\qquad
n^{2\kappa^{\prime\prime}}p(n^{\kappa}t)\sim\tilde{p}(t)>0,$ (1.3)

asymptotically up to an error $O(n^{-1})$; the scaling exponents should
satisfy

$\kappa+\kappa^{\prime}+\kappa^{\prime\prime}=1.$ (1.4)

We assume that these expansions can be differentiated444We say that an
expansion $f_{n}(t)-f(t)=O(1/n)$ can be differentiated if
$f_{n}^{\prime}(t)-f^{\prime}(t)=O(1/n)$. at least twice, that the roots of
$\tilde{q}(t)-\omega$ are simple and that the spectral projection
$K_{n}={\mathbb{1}}_{(-\infty,0)}(L_{n})$ satisfies ${\operator@font
tr}\,K_{n}=n$. Let $x=\sigma_{n}\xi+\mu_{n}$ induce the transformed projection
$\tilde{K}_{n}$ and let $x=n^{\kappa}t$ induce the transformed mean counting
probability density $\tilde{\rho}_{n}$. Then the following scaling limits
hold.

  * •

Bulk scaling limit: given $t\in\Lambda$ with $\tilde{q}(t)<\omega$, the
scaling

$\mu_{n}=n^{\kappa}t,\qquad\sigma_{n}=\frac{n^{\kappa-1}}{\tilde{\rho}(t)},\qquad\tilde{\rho}(t)=\frac{1}{\pi}\sqrt{\frac{(\omega-\tilde{q}(t))_{+}}{\tilde{p}(t)}},$

yields, for any bounded interval $J$, the strong operator limit
$\tilde{K}_{n}{\mathbb{1}}_{J}\,\overset{s}{\longrightarrow}\,K_{\text{\rm
Dyson}}\,{\mathbb{1}}_{J}$. Under condition (K0) and if $\tilde{\rho}$ has
unit mass on $\Lambda$, there is the limit law

$\tilde{\rho}_{n}(t)\,dt\overset{w}{\longrightarrow}\tilde{\rho}(t)\,dt.$

  * •

Soft-edge scaling limit: given $t_{*}\in\Lambda$ with
$\tilde{q}(t_{*})=\omega$, the scaling

$\mu_{n}x=n^{\kappa}t_{*},\qquad\sigma_{n}=n^{\kappa-\frac{2}{3}}\left(\frac{\tilde{p}(t_{*})}{\tilde{q}^{\prime}(t_{*})}\right)^{1/3},$

yields, for $s>-\infty$ and any interval $J\subset(s,\infty)$, the limit
$\tilde{K}_{n}{\mathbb{1}}_{J}\,\overset{s}{\longrightarrow}\,K_{\text{\rm
Airy}}\,{\mathbb{1}}_{J}$.

  * •

Hard-edge scaling limit: given that $\Lambda=(0,\infty)$ or $\Lambda=(0,1)$
with

$p(0)=0,\quad p^{\prime}(0)>0,\quad
q_{n}(x)=q(x)=\gamma^{2}x^{-1}+O(1)\;\,(x\to 0),$ (1.5)

the scaling

$\mu_{n}=0,\qquad\sigma_{n}=\frac{p^{\prime}(0)}{4\omega
n^{2\kappa^{\prime}}},\qquad\alpha=\frac{2\gamma}{\sqrt{p^{\prime}(0)}},$

yields,555Here, if $0\leqslant\alpha<1$, the selfadjoint realization $L_{n}$
is defined by means of the boundary condition  $2xu^{\prime}(x)-\alpha
u(x)=o(x^{-\alpha/2})\qquad(x\to 0).$ (1.6) for any bounded interval
$J\subset(0,\infty)$, the limit
$\tilde{K}_{n}{\mathbb{1}}_{J}\,\overset{s}{\longrightarrow}\,K_{\text{\rm
Bessel}}^{(\alpha)}\,{\mathbb{1}}_{J}$.

######  Remark 1.3.

Whether the intervals $J$ can be chosen unbounded or not depends on whether
the limit operator $K\,{\mathbb{1}}_{J}$ would then be trace class or not, see
the explicit formulae for the traces in the appendix: only in the former case
we get the representation of the scaling limit in terms of a particular
integral kernel. Note that we can never use $J=\Lambda$ since ${\operator@font
tr}\,K_{n}=n\to\infty$.

### Outline of the paper

The proof of Theorem 1 is subject of §2. In §3 we apply it to the classical
orthogonal polynomials, which yields a short and unified derivation of the
known formulae for the scaling limits for the classical unitary random matrix
ensembles (GUE, LUE/Wishart, JUE/MANOVA). In fact, by a result of Tricomi, the
only input needed is the weight function $w$ of the orthogonal polynomials;
from there one gets in a purely formula based fashion (by simple manipulations
which can easily be coded in any computer algebra system), first, to the
coefficients $p$ and $q_{n}$ as well as to the eigenvalues $\lambda_{n}$ of
the Sturm–Liouville operator $L_{n}$ and next, by applying Theorem 1, to the
particular scalings limits.

To emphasize that our main result and its application is largely independent
of concretely identifying the kernel $\tilde{K}$ of the spectral projection
${\mathbb{1}}_{(-\infty,0)}(\tilde{L})$, we postpone this identification to
the appendix: there, using generalized eigenfunction expansions, we calculate
the Dyson, Airy, and Bessel kernels directly from the limit differential
operators $\tilde{L}$.

##  2\. Proof of the Main Result for Sturm–Liouville Operators

To begin with, we first state the expansions of the coefficients, which also
motivate the particular regularity assumptions made in Theorem 1.

### Expansions

Since $L_{n}$ is a selfadjoint realization of

$L_{n}=-\frac{d}{dx}\left(p(x)\frac{d}{dx}\right)+q_{n}(x)-\lambda_{n}$

with $p,q_{n}\in C^{\infty}(\Lambda)$ and $p(x)>0$ for $x\in\Lambda$, we have
$C^{\infty}_{0}(\Lambda)\subset D(L_{n})$. The scaling

$x=\sigma_{n}\xi+n^{\kappa}t\qquad(\sigma_{n}\neq 0)$ (2.1)

(with $t\in\Lambda$ a _parameter_) yields $\tilde{\Lambda}=(-\infty,\infty)$
in each of the following cases:

  * •

$\Lambda=(-\infty,\infty)$;

  * •

$\Lambda=(0,\infty)$, $\kappa\geqslant 0$, $\sigma_{n}\to 0$;

  * •

$\Lambda=(0,1)$, $\kappa=0$, $\sigma_{n}\to 0$.

By virtue of $\tilde{K}_{n}(\xi,\eta)\,d\eta=K_{n}(x,y)\,dy$ (with
$y=\sigma_{n}\eta+n^{\kappa}t$), such a scaling induces

$\tilde{K}_{n}(\xi,\eta)=\sigma_{n}K_{n}(\sigma_{n}\xi+n^{\kappa}t,\sigma_{n}\eta+n^{\kappa}t).$

Because of $\sigma_{n}^{2}>0$ and $p(n^{\kappa}t)>0$, we see that
$\tilde{K}_{n}={\mathbb{1}}_{(-\infty,0)}(\tilde{L}_{n})$ with

$\tilde{L}_{n}=-\frac{1}{p(n^{\kappa}t)}\frac{d}{d\xi}\left(p(\sigma_{n}\xi+n^{\kappa}t)\frac{d}{d\xi}\right)+\frac{\sigma_{n}^{2}}{p(n^{\kappa}t)}\left(q_{n}(\sigma_{n}\xi+n^{\kappa}t)-\lambda_{n}\right).$

Now, if $\sigma_{n}=o(n^{\kappa-1/2})$, we get from (1.3) and (1.4) by Taylor
expansion that666If $\sigma_{n}=o(n^{\kappa-1/2})$, the error term is
$O(\max(\sigma_{n}n^{-\kappa},\sigma_{n}^{2}n^{1-2\kappa}))$, which amounts
for $O(n^{-1})$ in the bulk scaling limit and for $O(n^{-1/3})$ in the soft-
edge scaling limit.

$\tilde{L}_{n}u(\xi)\,=\,-u^{\prime\prime}(\xi)+\frac{\sigma_{n}^{2}n^{2-2\kappa}}{\tilde{p}(t)}\Big{(}\tilde{q}(t)-\omega+\sigma_{n}n^{-\kappa}\tilde{q}^{\prime}(t)\cdot\xi\Big{)}u(\xi)+o(1).$

### Bulk Scaling Limit

If $\tilde{q}(t)\neq\omega$, by choosing

$\sigma_{n}=\sigma_{n}(t)=\pi\,n^{\kappa-1}\sqrt{\frac{\tilde{p}(t)}{|\omega-\tilde{q}(t)|}},$

we get $\tilde{L}_{n}u(\xi)\,=\,-u^{\prime\prime}(\xi)-s\pi^{2}u(\xi)+o(1)$;
that is, $\tilde{L}_{n}u\to\tilde{L}u$ with the limit

$\tilde{L}=-\frac{d^{2}}{d\xi^{2}}-s\pi^{2},\qquad s={\operator@font
sign}(\omega-\tilde{q}(t)).$

We recall that $\tilde{L}$ is essentially selfadjoint on
$C^{\infty}_{0}(\tilde{\Lambda})$ and that its unique selfadjoint extension
has absolutely continuous spectrum:
$\sigma(\tilde{L})=\sigma_{\text{ac}}(\tilde{L})=[-s\pi^{2},\infty)$. Thus,
for $s=-1$, the spectral projection ${\mathbb{1}}_{(-\infty,0)}(\tilde{L})$ is
zero. For $s=1$, the spectral projection can be calculated by a generalized
eigenfunction expansion, yielding the _Dyson kernel_ (A.3); see Lemma A.1.

### Limit Law

The result for the bulk scaling limit allows, in passing, to calculate a limit
law of the mean counting probability $\rho_{n}(x)\,dx$. We observe that, by
virtue of $\rho_{n}(x)\,dx=\tilde{\rho}_{n}(t)\,dt$, the scaling
$x=n^{\kappa}t$ transforms $\rho_{n}(x)$ into777It is the cancellation of the
powers of $n$ in the last equality, which called for assumption (1.4).

$\tilde{\rho}_{n}(t)=n^{\kappa}\rho_{n}(n^{\kappa}t)=n^{\kappa-1}K_{n}(n^{\kappa}t,n^{\kappa}t)=\frac{n^{\kappa-1}}{\sigma_{n}(t)}\tilde{K}_{n}(0,0)=\frac{1}{\pi}\sqrt{\frac{|\omega-\tilde{q}(t)|}{\tilde{p}(t)}}\tilde{K}_{n}(0,0).$

Thus, to get to a limit, we have to _assume_ condition (K0), that
is,888Following Knuth, the bracketed notation $[S]$ stands for $1$ if the
statement $S$ is true, $0$ otherwise.

$\tilde{K}_{n}(0,0)\to[\tilde{q}(t)<\omega]\,K_{\text{\rm
Dyson}}(0,0)=[\tilde{q}(t)<\omega].$

We then get, for $\tilde{q}(t)\neq\omega$ (that is, almost everywhere in
$\Lambda$),

$\tilde{\rho}_{n}(t)\to\tilde{\rho}(t)=\frac{1}{\pi}\sqrt{\frac{(\omega-\tilde{q}(t))_{+}}{\tilde{p}(t)}}.$

Hence, by Helly’s selection theorem, the probability measure
$\tilde{\rho}_{n}(t)\,dt$ converges vaguely to $\tilde{\rho}(t)\,dt$, which
is, in general, just a sub-probability measure. If, however, it is checked
that $\tilde{\rho}(t)\,dt$ has unit mass, the convergence is weak.

### Soft-Edge Scaling Limit

If $\tilde{q}(t_{*})=\omega$, by choosing999Note that, by the assumption made
on the simplicity of the roots of $\tilde{q}(t)-\omega$, we have
$\tilde{q}^{\prime}(t_{*})\neq 0$.

$\sigma_{n}=\sigma_{n}(t_{*})=n^{\kappa-2/3}\left(\frac{\tilde{p}(t_{*})}{\tilde{q}^{\prime}(t_{*})}\right)^{1/3},$

we get $\tilde{L}_{n}u(\xi)\,=\,-u^{\prime\prime}(\xi)+\xi u(\xi)+o(1)$; that
is, $\tilde{L}_{n}u\to\tilde{L}u$ with the limit

$\tilde{L}=-\frac{d^{2}}{d\xi^{2}}+\xi.$

We recall that $\tilde{L}$ is essentially selfadjoint on
$C^{\infty}_{0}(\tilde{\Lambda})$ and that its unique selfadjoint extension
has absolutely continuous spectrum:
$\sigma(\tilde{L})=\sigma_{\text{ac}}(\tilde{L})=(-\infty,\infty)$. The
spectral projection ${\mathbb{1}}_{(-\infty,0)}(\tilde{L})$ can
straightforwardly be calculated by a generalized eigenfunction expansion,
yielding the _Airy kernel_ (A.4); see Lemma A.2.

### Hard-Edge Scaling Limit

In the case $a=0$, we take the scaling

$x=\sigma_{n}\xi\qquad(\sigma_{n}>0),$

with $\sigma_{n}=o(1)$ appropriately chosen, to explore the vicinity of this
“hard edge”; note that such a scaling yields $\tilde{\Lambda}=(0,\infty)$. To
simplify we make the assumptions stated in (1.5). We see that
$\tilde{K}_{n}={\mathbb{1}}_{(-\infty,0)}(\tilde{L}_{n})$ with

$\tilde{L}_{n}=-\frac{4}{p^{\prime}(0)\sigma_{n}}\frac{d}{d\xi}\left(p(\sigma_{n}\xi)\frac{d}{d\xi}\right)+\frac{4\sigma_{n}}{p^{\prime}(0)}\left(q(\sigma_{n}\xi)-\lambda_{n}\right).$

If we choose

$\sigma_{n}=n^{-2\kappa^{\prime}}\frac{p^{\prime}(0)}{4\omega},$

we get101010The error term is $O(n^{-\min(1,2\kappa^{\prime})})$, which
amounts for $O(n^{-1})$ if $\kappa^{\prime}\geqslant\frac{1}{2}$ (as
throughout §3). $\tilde{L}_{n}u(\xi)\,=\,-4(\xi
u^{\prime}(\xi))^{\prime}+\left(\alpha^{2}\xi^{-1}-1\right)u(\xi)+o(1)$; that
is, $\tilde{L}_{n}u\to\tilde{L}u$,

$\tilde{L}=-4\frac{d}{d\xi}\left(\xi\frac{d}{d\xi}\right)+\alpha^{2}\xi^{-1}-1,\qquad\alpha=\frac{2\gamma}{\sqrt{p^{\prime}(0)}}.$

We recall that, if $\alpha\geqslant 1$, the limit $\tilde{L}$ is essentially
selfadjoint on $C^{\infty}_{0}(\tilde{\Lambda})$ and that the spectrum of its
unique selfadjoint extension is absolutely continuous:
$\sigma(\tilde{L})=\sigma_{\text{ac}}(\tilde{L})=[-1,\infty)$. The spectral
projection can be calculated by a generalized eigenfunction expansion,
yielding the _Bessel kernel_ (A.5); see Lemma A.3.

######  Remark 2.1.

The theorem also holds in the case $0\leqslant\alpha<1$ if the particular
selfadjoint realization $L_{n}$ is defined by the boundary condition (1.6),
see Remark A.1.

##  3\. Projection Kernels Associated to Classical Orthogonal Polynomials

In this section we apply Theorem 1 to the kernels associated with the
classical orthogonal polynomials, that is, the Hermite, Laguerre, and Jacobi
polynomials. In random matrix theory, the thus induced determinantal processes
are modeled by the spectra of the Gaussian Unitary Ensemble (GUE), the
Laguerre Unitary Ensemble (LUE) or Wishart ensemble, and the Jacobi Unitary
Ensemble (JUE) or MANOVA111111MANOVA = multivariate analysis of variance
ensemble.

To prepare the study of the individual cases, we first discuss their common
structure. Let $P_{n}(x)$ be the sequence of classical orthogonal polynomials
belonging to the weight function $w(x)$ on the interval $(a,b)$. We normalize
$P_{n}(x)$ such that $\langle\phi_{n},\phi_{n}\rangle=1$, where
$\phi_{n}(x)=w(x)^{1/2}P_{n}(x)$. The functions $\phi_{n}$ form a complete
orthogonal set in $L^{2}(a,b)$; conceptual proofs of the completeness can be
found, e.g., in ? (§5.7 for the Jacobi polynomials, §6.5 for the Hermite and
Laguerre polynomials). By a result of Tricomi (see ?, §10.7), the $P_{n}(x)$
satisfy the eigenvalue problem

$-\frac{1}{w(x)}\frac{d}{dx}\left(p(x)w(x)\frac{d}{dx}P_{n}(x)\right)=\lambda_{n}P_{n}(x),\quad\lambda_{n}=-n(r^{\prime}+\tfrac{1}{2}(n+1)p^{\prime\prime}),$

where $p(x)$ is a _quadratic_ polynomial121212With the sign chosen that
$p(x)>0$ for $x\in(a,b)$. and $r(x)$ a _linear_ polynomial such that

$\frac{w^{\prime}(x)}{w(x)}=\frac{r(x)}{p(x)}.$

In terms of $\phi_{n}$, a simple calculation shows that

$-\frac{d}{dx}\left(p(x)\frac{d}{dx}\phi_{n}(x)\right)+q(x)\phi_{n}(x)=\lambda_{n}\phi_{n}(x),\quad
q(x)=\frac{r(x)^{2}}{4p(x)}+\frac{r^{\prime}(x)}{2}.$

Therefore, by the completeness of the $\phi_{n}$, the formally selfadjoint
Sturm–Liouville operator $L=-\frac{d}{dx}p(x)\frac{d}{dx}+q(x)$ has a
particular selfadjoint realization (which we continue to denote by the letter
$L$) with spectrum

$\sigma(L)=\\{\lambda_{0},\lambda_{1},\lambda_{2},\ldots\\}$

and corresponding eigenfunctions $\phi_{n}$. Hence, the projection kernel
(1.1) induces an integral operator $K_{n}$ with ${\operator@font tr}\,K_{n}=n$
that satisfies

$K_{n}={\mathbb{1}}_{(-\infty,0)}(L_{n}),\qquad L_{n}=L-\lambda_{n}.$

Note that this relation remains true if we choose to make some parameters of
the weight $w$ (and, therefore, of the functions $\phi_{j}$) to depend on $n$.
For the scaling limits of $K_{n}$, we are now in the realm of Theorem 1: given
the weight $w(x)$ as the only input, all the other quantities can now simply
be obtained by routine calculations.

### Hermite Polynomials

The weight is $w(x)=e^{-x^{2}}$ on $\Lambda=(-\infty,\infty)$; hence

$p(x)=1,\qquad r(x)=-2x,\qquad q(x)=x^{2}-1,\qquad\lambda_{n}=2n,$

and, therefore,

$\kappa=\kappa^{\prime}=\tfrac{1}{2},\qquad\kappa^{\prime\prime}=0,\qquad\tilde{p}(t)=1,\qquad\tilde{q}(t)=t^{2},\qquad\omega=2.$

Theorem 1 is applicable and we directly read off the following well-known
scaling limits of the GUE (see, e.g., ?, Chap. 3):

  * •

bulk scaling limit: if $-\sqrt{2}<t<\sqrt{2}$, the transformation

$x=\frac{\pi\,\xi}{n^{1/2}\sqrt{2-t^{2}}}+n^{1/2}t$

induces $\tilde{K}_{n}$ with a strong limit given by the _Dyson kernel_ ;

  * •

limit law: the transformation $x=n^{1/2}t$ induces $\tilde{\rho}_{n}$ with a
weak limit given by the _Wigner semicircle law_

$\tilde{\rho}(t)=\frac{1}{\pi}\sqrt{(2-t^{2})_{+}};$

  * •

soft-edge scaling limit: the transformation

$x=\pm(2^{-1/2}n^{-1/6}\xi+\sqrt{2n})$

induces $\tilde{K}_{n}$ with a strong limit given by the _Airy kernel_.

### Laguerre Polynomials

The weight is $w(x)=x^{\alpha}e^{-x}$ on $\Lambda=(0,\infty)$; hence

$p(x)=x,\qquad r(x)=\alpha-x,\qquad
q(x)=\frac{(\alpha-x)^{2}}{4x}-\frac{1}{2},\qquad\lambda_{n}=n.$

In random matrix theory, the corresponding determinantal point process is
modeled by the spectra of complex $n\times n$ Wishart matrices with a
dimension parameter $m\geqslant n$; the Laguerre parameter $\alpha$ is then
given by $\alpha=m-n\geqslant 0$. Of particular interest in statistics (?) is
the simultaneous limit $m,n\to\infty$ with

$\frac{m}{n}\to\theta\geqslant 1,$

for which we get

$\kappa=1,\quad\kappa^{\prime}=\frac{1}{2},\quad\kappa^{\prime\prime}=-\frac{1}{2},\quad\tilde{p}(t)=t,\quad\tilde{q}(t)=\frac{(\theta-1-t)^{2}}{4t},\quad\omega=1.$

Note that

$\omega-\tilde{q}(t)=\frac{(t_{+}-t)(t-t_{-})}{4t},\qquad
t_{\pm}=(\sqrt{\theta}\pm 1)^{2}.$

Theorem 1 is applicable and we directly read off the following well-known
scaling limits of the Wishart ensemble (?):

  * •

bulk scaling limit: if $t_{-}<t<t_{+}$,

$x=\frac{2\pi t\,\xi}{\sqrt{(t_{+}-t)(t-t_{-})}}+nt$

induces $\tilde{K}_{n}$ with a strong limit given by the _Dyson kernel_ ;

  * •

limit law: the scaling $x=nt$ induces $\tilde{\rho}_{n}$ with a weak limit
given by the _Marčenko–Pastur law_

$\tilde{\rho}(t)=\frac{1}{2\pi t}\sqrt{((t_{+}-t)(t-t_{-}))_{+}};$

  * •

soft-edge scaling limit: with signs chosen consistently as either $+$ or $-$,

$x=\pm n^{1/3}\theta^{-1/6}t_{\pm}^{2/3}\xi+nt_{\pm}$ (3.1)

induces $\tilde{K}_{n}$ with a strong limit given by the _Airy kernel_.

######  Remark 3.1.

The scaling (3.1) is better known in the asymptotically equivalent form

$x=\sigma\xi+\mu,\quad\mu=(\sqrt{m}\pm\sqrt{n})^{2},\quad\sigma=(\sqrt{m}\pm\sqrt{n})\left(\frac{1}{\sqrt{m}}\pm\frac{1}{\sqrt{n}}\right)^{1/3},$

which is obtained from (3.1) by replacing $\theta$ with $m/n$ (see ?, p. 305).

In the case $\theta=1$, which implies $t_{-}=0$, the lower soft-edge scaling
(3.1) breaks down and has to be replaced by a scaling at the _hard_ edge:

  * •

hard-edge scaling limit: if $\alpha=m-n$ is a constant,131313By Remark 2.1,
there is no need to restrict ourselves to $\alpha\geqslant 1$: since
$\phi_{n}(x)=x^{\alpha}\chi_{n}(x)$ with $\chi_{n}(x)$ extending smoothly to
$x=0$, we have, for $\alpha\geqslant 0$,
$x^{\alpha/2}(2x\phi_{n}^{\prime}(x)-\alpha\phi_{n}(x))=2x^{1+\alpha}\chi_{n}^{\prime}(x)=O(x)\qquad\qquad(x\to
0).$ Hence, the selfadjoint realization $L_{n}$ is compatible with the
boundary condition (1.6). $x=\xi/(4n)$ induces $\tilde{K}_{n}$ with a strong
limit given by the _Bessel kernel_ $K_{\text{\rm Bessel}}^{(\alpha)}$.

### Jacobi Polynomials

The weight is $w(x)=x^{\alpha}(1-x)^{\beta}$ on $\Lambda=(0,1)$; hence

$p(x)=x(1-x),\quad r(x)=\alpha-(\alpha+\beta)x,\quad
q(x)=\frac{(\alpha-(\alpha+\beta)x)^{2}}{4x(1-x)}-\frac{\alpha+\beta}{2},$

and

$\lambda_{n}=n(n+\alpha+\beta+1).$

In random matrix theory, the corresponding determinantal point process is
modeled by the spectra of complex $n\times n$ MANOVA matrices with dimension
parameters $m_{1},m_{2}\geqslant n$; the Jacobi parameters $\alpha$, $\beta$
are then given by $\alpha=m_{1}-n\geqslant 0$ and $\beta=m_{2}-n\geqslant 0$.
Of particular interest in statistics (?) is the simultaneous limit
$m_{1},m_{2},n\to\infty$ with

$\frac{m_{1}}{m_{1}+m_{2}}\to\theta\in(0,1),\qquad\frac{n}{m_{1}+m_{2}}\to\tau\in(0,1/2],$

for which we get

$\kappa=\kappa^{\prime\prime}=0,\;\kappa^{\prime}=1,\;\tilde{p}(t)=t(1-t),\;\tilde{q}(t)=\frac{(\theta-\tau-(1-2\tau)t)^{2}}{4\tau^{2}t(1-t)},\;\omega=\frac{1-\tau}{\tau}.$

Note that

$\omega-\tilde{q}(t)=\frac{(t_{+}-t)(t-t_{-})}{4\tau^{2}t(1-t)},\qquad
t_{\pm}=\left(\sqrt{\theta(1-\tau)}\pm\sqrt{\tau(1-\theta)}\right)^{2}.$

Theorem 1 is applicable and we directly read off the following (less well-
known) scaling limits of the MANOVA ensemble (?, ?):

  * •

bulk scaling limit: if $t_{-}<t<t_{+}$,

$x=\frac{2\pi\tau t(1-t)\,\xi}{n\sqrt{(t_{+}-t)(t-t_{-})}}+t$

induces $\tilde{K}_{n}$ with a strong limit given by the _Dyson kernel_ ;

  * •

limit law: (because of $\kappa=0$ there is no transformation here) $\rho_{n}$
has a weak limit given by the law (?)

$\rho(t)=\frac{1}{2\pi\tau t(1-t)}\sqrt{((t_{+}-t)(t-t_{-}))_{+}};$

  * •

soft-edge scaling limit: with signs chosen consistently as either $+$ or $-$,

$x=\pm n^{-2/3}\frac{(\tau
t_{\pm}(1-t_{\pm}))^{2/3}}{(\tau\theta(1-\tau)(1-\theta))^{1/6}}\,\xi+t_{\pm}$
(3.2)

induces $\tilde{K}_{n}$ with a strong limit given by the _Airy kernel_.

######  Remark 3.2.

? gives the soft-edge scaling in terms of a trigonometric parametrization of
$\theta$ and $\tau$. By putting

$\theta=\sin^{2}\frac{\phi}{2},\qquad\tau=\sin^{2}\frac{\psi}{2},$

we immediately get

$t_{\pm}=\sin^{2}\frac{\phi\pm\psi}{2}$

and (3.2) becomes

$x=\pm\sigma_{\pm}\xi+t_{\pm},\qquad\sigma_{\pm}=n^{-2/3}\left(\frac{\tau^{2}\sin^{4}(\phi\pm\psi)}{4\sin\phi\sin\psi}\right)^{1/3}.$

In the case $\theta=\tau=1/2$, which is equivalent to $m_{1}/n,m_{2}/n\to 1$,
we have $t_{-}=0$ and $t_{+}=1$. Hence, the lower and the upper soft-edge
scaling (3.2) break down and have to be replaced by a scaling at the _hard_
edges:

  * •

hard-edge scaling limit: if $\alpha=m_{1}-n$, $\beta=m_{2}-n$ are
constants,141414For the cases $0\leqslant\alpha<1$ and $0\leqslant\beta<1$,
see the justification of the limit given in Footnote 13. $x=\xi/(4n^{2})$
induces $\tilde{K}_{n}$ with a strong limit given by the _Bessel kernel_
$K_{\text{\rm Bessel}}^{(\alpha)}$; by symmetry, the _Bessel kernel_
$K_{\text{\rm Bessel}}^{(\beta)}$ is obtained for $x=1-\xi/(4n^{2})$.

##  A. Appendices

###  A.1. Generalized Strong Convergence

The notion of strong resolvent convergence (?, §9.3) links the convergence of
differential operators, pointwise for an appropriate class of smooth test
functions, to the strong convergence of their spectral projections. We recall
a slight generalization of that concept, which allows the underlying Hilbert
space to vary.

Specifically we consider, on an interval $(a,b)$ (not necessarily bounded) and
on a sequence of subintervals $(a_{n},b_{n})\subset(a,b)$ with $a_{n}\to a$
and $b_{n}\to b$, selfadjoint operators

$L:D(L)\subset L^{2}(a,b)\to L^{2}(a,b),\qquad L_{n}:D(L_{n})\subset
L^{2}(a_{n},b_{n})\to L^{2}(a_{n},b_{n}).$

By means of the natural embedding (that is, extension by zero) we take
$L^{2}(a_{n},b_{n})\subset L^{2}(a,b)$; the multiplication operator induced by
the characteristic function ${\mathbb{1}}_{(a_{n},b_{n})}$, which we continue
to denoted by ${\mathbb{1}}_{(a_{n},b_{n})}$, constitutes the orthogonal
projection of $L^{2}(a,b)$ onto $L^{2}(a_{n},b_{n})$. Following ?, we say that
$L_{n}$ converges to $L$ in the sense of _generalized strong convergence_
(gsc), if for some $z\in{\mathbb{C}}\setminus{\mathbb{R}}$, and hence, a
forteriori, for all such $z$,

$R_{z}(L_{n}){\mathbb{1}}_{(a_{n},b_{n})}\overset{s}{\longrightarrow}R_{z}(L)\qquad(n\to\infty)$

in the strong operator topology of $L^{2}(a,b)$.151515We denote by
$R_{z}(L)=(L-z)^{-1}$ the resolvent of an operator $L$.

######  Theorem 2 (?).

Let the selfadjoint operators $L_{n}$ and $L$ satisfy the assumptions stated
above and let $C$ be a core of $L$ such that, eventually, $C\subset D(L_{n})$.

  * (i)

If $L_{n}u\to Lu$ for all $u\in C$, then
$L_{n}\overset{gsc}{\longrightarrow}L$.

  * (ii)

If $L_{n}\overset{gsc}{\longrightarrow}L$ and if the endpoints of the interval
$\Delta\subset{\mathbb{R}}$ do not belong to the pure point spectrum
$\sigma_{\text{pp}}(L)$ of $L$, the spectral projections to $\Delta$ converge
as

${\mathbb{1}}_{\Delta}(L_{n}){\mathbb{1}}_{(a_{n},b_{n})}\overset{s}{\longrightarrow}{\mathbb{1}}_{\Delta}(L).$

###  A.2. Generalized Eigenfunction Expansion of Sturm–Liouville Operators

Let $L$ be a formally selfadjoint Sturm–Liouville operator on the interval
$(a,b)$,

$Lu=-(pu^{\prime})^{\prime}+qu,$

with smooth coefficient functions $p>0$ and $q$. We have the _limit point
case_ (LP) at the boundary point $a$ if there is some $c\in(a,b)$ and some
$z\in{\mathbb{C}}$ such that there exists at least one solution of $(L-z)u=0$
in $(a,b)$ for which $u\not\in L^{2}(a,c)$; otherwise, we have the limit
circle case (LC) at $a$. According to the Weyl alternative (?, Thm. 8.27) this
is then true for all such $c$ and $z$ and, moreover, in the LP case there
exists, for every $z\in{\mathbb{C}}\setminus{\mathbb{R}}$, a one-dimensional
space of solutions $u$ of the equation $(L-z)u=0$ for which $u\in L^{2}(a,c)$.
The same structure and notion holds for the boundary point $b$.

######  Theorem 3.

Let $L$ be a formally selfadjoint Sturm–Liouville operator on the interval
$(a,b)$ as defined above. If there is the LP case at $a$ and $b$, then $L$ is
essentially self-adjoint on the domain $C^{\infty}_{0}(a,b)$ and, for
$z\in{\mathbb{C}}\setminus{\mathbb{R}}$, the resolvent $R_{z}(L)=(L-z)^{-1}$
of its unique selfadjoint extension (which we continue to denote by the letter
$L$) is of the form

$R_{z}(L)\phi(x)=\frac{1}{W(u_{a},u_{b})}\left(u_{b}(x)\int_{a}^{x}u_{a}(y)\phi(y)\,dy+u_{a}(x)\int_{x}^{b}u_{b}(y)\phi(y)\,dy\right).$
(A.1)

Here $u_{a}$ and $u_{b}$ are the non-vanishing solutions of the equation
$(L-z)u=0$, uniquely determined up to a factor by the conditions $u_{a}\in
L^{2}(a,c)$ and $u_{b}\in L^{2}(c,b)$ for some $c\in(a,b)$, and $W$ denotes
the Wronskian

$W(u_{a},u_{b})=p(x)(u_{a}^{\prime}(x)u_{b}(x)-u_{a}(x)u_{b}^{\prime}(x)),$

which is a constant for $x\in(a,b)$.

A more general formulation (and a proof) that includes the LC case can be
found, e.g., in ?.161616See ? for a proof that $C^{\infty}_{0}(a,b)$ is a core
if the coefficients are smooth. In the following, we write (A.1) briefly in
the form

$R_{z}(L)\phi(x)=\int_{a}^{b}G_{z}(x,y)\phi(y)\,dy.$

If the imaginary part of the thus defined Green’s kernel $G_{z}(x,y)$ has
finite boundary values as $z$ approaches the real line from above, there is a
simple formula for the spectral projection associated with $L$ that often
applies if the spectrum of $L$ is absolutely continuous.

######  Theorem 4.

(i) Assume that there exits, as $\epsilon\downarrow 0$, the limit

$\pi^{-1}{\operator@font Im\,}G_{\lambda+i\epsilon}(x,y)\to K_{\lambda}(x,y),$

locally uniform in $x,y\in(a,b)$ for each $\lambda\in{\mathbb{R}}$ except for
some isolated points $\lambda$ for which the limit is replaced by

$\epsilon\,{\operator@font Im\,}G_{\lambda+i\epsilon}(x,y)\to 0.$

Then the spectrum is absolutely continuous, $\sigma(L)=\sigma_{\text{ac}}(L)$,
and, for a Borel set $\Delta$,

$\langle{\mathbb{1}}_{\Delta}(L)\phi,\psi\rangle=\int_{\Delta}\langle
K_{\lambda}\,\phi,\psi\rangle\,d\lambda\qquad(\phi,\psi\in
C^{\infty}_{0}(a,b)).$ (A.2)

(ii) Assume further, for some $(a^{\prime},b^{\prime})\subset(a,b)$, that

$\int_{a^{\prime}}^{b^{\prime}}\int_{a^{\prime}}^{b^{\prime}}\left(\int_{\Delta}|K_{\lambda}(x,y)|\,d\lambda\right)^{2}\,dx\,dy\,<\,\infty.$

Then ${\mathbb{1}}_{\Delta}(L){\mathbb{1}}_{(a^{\prime},b^{\prime})}$ is a
Hilbert-Schmidt operator on $L^{2}(a,b)$ with kernel

${\mathbb{1}}_{(a^{\prime},b^{\prime})}(y)\int_{\Delta}K_{\lambda}(x,y)\,d\lambda.$

If $\int_{\Delta}K_{\lambda}(x,y)\,d\lambda$ is a continuous function of
$x,y\in(a^{\prime},b^{\prime})$,
${\mathbb{1}}_{\Delta}(L){\mathbb{1}}_{(a^{\prime},b^{\prime})}$ is a trace
class operator with trace

${\operator@font
tr}\,{\mathbb{1}}_{\Delta}(L){\mathbb{1}}_{(a^{\prime},b^{\prime})}=\int_{a^{\prime}}^{b^{\prime}}\int_{\Delta}K_{\lambda}(x,x)\,d\lambda\,dx.$

###### Proof.

With $E$ denoting the spectral resolution of the selfadjoint operator $L$, we
observe that, for a given $\phi\in C^{\infty}_{0}(a,b)$, the Borel–Stieltjes
transform of the positive measure $\mu_{\phi}(\lambda)=\langle
E(\lambda)\phi,\phi\rangle$ can be simply expressed in terms of the resolvent
as follows (see ?, §32.1):

$\int_{-\infty}^{\infty}\frac{d\mu_{\phi}(\lambda)}{\lambda-z}=\langle
R_{z}(L)\phi,\phi\rangle.$

If we take $z=\lambda+i\epsilon$ and let $\epsilon\downarrow 0$, we obtain by
the locally uniform convergence of the integral kernel of $R_{z}$ that there
exits either the limit

$\pi^{-1}{\operator@font Im\,}\langle
R_{\lambda+i\epsilon}(L)\phi,\phi\rangle\to\langle
K_{\lambda}\phi,\phi\rangle$

or, at isolated points $\lambda$,

$\epsilon\,{\operator@font Im\,}\langle
R_{\lambda+i\epsilon}(L)\phi,\phi\rangle\to 0.$

By a theorem of de la Vallée-Poussin (see ?, Thm. 11.6(ii/iii)), the singular
part of $\mu_{\phi}$ vanishes, $\mu_{\phi,\text{sing}}=0$; by Plemelj’s
reconstruction the absolutely continuous part satisfies (see ?, Thm. 11.6(iv))

$d\mu_{\phi,\text{ac}}(\lambda)=\langle
K_{\lambda}\phi,\phi\rangle\,d\lambda.$

Since $C^{\infty}_{0}(a,b)$ is dense in $L^{2}(a,b)$, approximation shows that
$E_{\text{sing}}=0$, that is, $\sigma(L)=\sigma_{\text{ac}}(L)$. Since
$\langle{\mathbb{1}}_{\Delta}(L)\phi,\phi\rangle=\int_{\Delta}d\mu_{\phi}(\lambda)$,
we thus get, by the symmetry of the bilinear expressions, the representation
(A.2), which finishes the proof of (i). The Hilbert–Schmidt part of part (ii)
follows using the Cauchy–Schwarz inequality and Fubini’s theorem and yet
another density argument; the trace class part follows from ? since
${\mathbb{1}}_{(a^{\prime},b^{\prime})}{\mathbb{1}}_{\Delta}(L){\mathbb{1}}_{(a^{\prime},b^{\prime})}$
is a selfadjoint, positive-semidefinite operator.∎∎

We apply this theorem to the spectral projections used in the proof of Theorem
1. The first two examples could have been dealt with by Fourier techniques (?,
§3.3); applying, however, the same method in all the examples renders the
approach more systematic.

######  Example 1 (Dyson kernel).

Consider $Lu=-u^{\prime\prime}$ on $(-\infty,\infty)$. Since $u\equiv 1$ is a
solution of $Lu=0$, both endpoints are LP; for a given ${\operator@font
Im\,}z>0$ the solutions $u_{a}$ ($u_{b}$) of $(L-z)u=0$ being $L^{2}$ at
$-\infty$ ($\infty$) are spanned by

$u_{a}(x)=e^{-ix\sqrt{z}},\qquad u_{b}(x)=e^{ix\sqrt{z}}.$

Thus, Theorem 3 applies: $L$ is essentially selfadjoint on
$C^{\infty}_{0}(-\infty,\infty)$, the resolvent of its unique selfadjoint
extension is represented, for ${\operator@font Im\,}z>0$, by the Green’s
kernel

$G_{z}(x,y)=\frac{i}{2\sqrt{z}}\begin{cases}e^{i(x-y)\sqrt{z}}&x>y\\\\[2.84526pt]
e^{-i(x-y)\sqrt{z}}&\text{otherwise}.\end{cases}$

For $\lambda>0$ there is the limit

$\pi^{-1}{\operator@font
Im\,}G_{\lambda+i0}(x,y)=K_{\lambda}(x,y)=\frac{\cos((x-y)\sqrt{\lambda})}{2\pi\sqrt{\lambda}},$

for $\lambda<0$ the limit is zero; both limits are locally uniform in
$x,y\in{\mathbb{R}}$. For $\lambda=0$ there would be divergence, but we
obviously have

$\epsilon\,{\operator@font Im\,}G_{i\epsilon}(x,y)\to
0\qquad(\epsilon\downarrow 0),$

locally uniform in $x,y\in{\mathbb{R}}$. Hence, Theorem 4 applies:
$\sigma(L)=\sigma_{\text{ac}}(L)=[0,\infty)$ and (A.2) holds for each Borel
set $\Delta\subset{\mathbb{R}}$. Given a bounded interval $(a,b)$, we may
estimate for the specific choice $\Delta=(-\infty,\pi^{2})$ that

$\int_{a}^{b}\int_{a}^{b}\left(\int_{-\infty}^{\pi^{2}}|K_{\lambda}(x,y)|\,d\lambda\right)^{2}\,dx\,dy\\\\[5.69054pt]
=\int_{a}^{b}\int_{a}^{b}\left(\int_{0}^{\pi^{2}}\left|\frac{\cos((x-y)\sqrt{\lambda})}{2\pi\sqrt{\lambda}}\right|\,d\lambda\right)^{2}\,dx\,dy\leqslant\left(\int_{a}^{b}\int_{0}^{\pi^{2}}\frac{d\lambda}{2\pi\sqrt{\lambda}}\right)^{2}=(b-a)^{2}.$

Therefore, Theorem 4 yields that
${\mathbb{1}}_{(-\infty,\pi^{2})}(L){\mathbb{1}}_{(a,b)}$ is Hilbert–Schmidt
with the _Dyson kernel_

$\int_{-\infty}^{\pi^{2}}K_{\lambda}(x,y)\,d\lambda=\int_{0}^{\pi^{2}}\frac{\cos((x-y)\sqrt{\lambda})}{2\pi\sqrt{\lambda}}\,d\lambda=\frac{\sin(\pi(x-y))}{\pi(x-y)},$

restricted to $x,y\in(a,b)$. Here, the last equality is simply obtained from

$(x-y)\int_{0}^{\pi^{2}}\frac{\cos((x-y)\sqrt{\lambda})}{2\sqrt{\lambda}}\,d\lambda=\int_{0}^{\pi^{2}}\frac{d}{d\lambda}\sin((x-y)\sqrt{\lambda})\,d\lambda=\sin(\pi(x-y)).$

Since the resulting kernel is continuous for $x,y\in(a,b)$, Theorem 4 gives
that ${\mathbb{1}}_{(-\infty,\pi^{2})}(L){\mathbb{1}}_{(a,b)}$ is a trace
class operator with trace

${\operator@font
tr}\,{\mathbb{1}}_{(-\infty,\pi^{2})}(L){\mathbb{1}}_{(a,b)}=b-a.$

To summarize, we have thus obtained the following lemma.

######  Lemma A.1.

The operator $Lu=-u^{\prime\prime}$ is essentially selfadjoint on
$C^{\infty}_{0}(-\infty,\infty)$. The spectrum of its unique selfadjoint
extension is

$\sigma(L)=\sigma_{\rm ac}(L)=[0,\infty).$

Given $(a,b)$ bounded,
${\mathbb{1}}_{(-\infty,\pi^{2})}(L){\mathbb{1}}_{(a,b)}$ is trace class with
trace $b-a$ and kernel

$K_{\text{\rm
Dyson}}(x,y)=\int_{0}^{\pi^{2}}\frac{\cos((x-y)\sqrt{\lambda})}{2\pi\sqrt{\lambda}}\,d\lambda=\frac{\sin(\pi(x-y))}{\pi(x-y)}.$
(A.3)

######  Example 2 (Airy kernel).

Consider the differential operator $Lu=-u^{\prime\prime}+xu$ on
$(-\infty,\infty)$. Since the specific solution $u(x)={\operator@font Bi}(x)$
of $Lu=0$ is not locally $L^{2}$ at each of the endpoints, both endpoints are
LP. For a given ${\operator@font Im\,}z>0$ the solutions $u_{a}$ ($u_{b}$) of
$(L-z)u=0$ being $L^{2}$ at $-\infty$ ($\infty$) are spanned by (see ?, Eq.
(10.4.59–64))

$u_{a}(x)={\operator@font Ai}(x-z)-i\,{\operator@font Bi}(x-z),\qquad
u_{b}(x)={\operator@font Ai}(x-z).$

Thus, Theorem 3 applies: $L$ is essentially selfadjoint on
$C^{\infty}_{0}(-\infty,\infty)$, the resolvent of its unique selfadjoint
extension is represented, for ${\operator@font Im\,}z>0$, by the Green’s
kernel

$G_{z}(x,y)=i\pi\begin{cases}{\operator@font Ai}(x-z)\left({\operator@font
Ai}(y-z)-i\,{\operator@font Bi}(y-z)\right)&x>y\\\\[2.84526pt] {\operator@font
Ai}(y-z)\left({\operator@font Ai}(x-z)-i\,{\operator@font
Bi}(x-z)\right)&\text{otherwise}.\end{cases}$

For $\lambda\in{\mathbb{R}}$ there is thus the limit

$\pi^{-1}{\operator@font
Im\,}G_{\lambda+i0}(x,y)=K_{\lambda}(x,y)={\operator@font
Ai}(x-\lambda){\operator@font Ai}(y-\lambda),$

locally uniform in $x,y\in{\mathbb{R}}$. Hence, Theorem 4 applies:
$\sigma(L)=\sigma_{\text{ac}}(L)={\mathbb{R}}$ and (A.2) holds for each Borel
set $\Delta\subset{\mathbb{R}}$. Given $s>-\infty$, we may estimate for the
specific choice $\Delta=(-\infty,0)$ that

$\left(\int_{s}^{\infty}\int_{s}^{\infty}\left(\int_{\Delta}|K_{\lambda}(x,y)|\,d\lambda\right)^{2}\,dx\,dy\right)^{1/2}\leqslant\int_{s}^{\infty}\int_{0}^{\infty}{\operator@font
Ai}(x+\lambda)^{2}\,d\lambda\,dx=\tau(s)$

with

$\tau(s)=\frac{1}{3}\left(2s^{2}\text{Ai}(s)^{2}-2s\text{Ai}^{\prime}(s)^{2}-\text{Ai}(s)\text{Ai}^{\prime}(s)\right).$

Therefore, Theorem 4 yields that
${\mathbb{1}}_{(-\infty,0)}(L){\mathbb{1}}_{(s,\infty)}$ is Hilbert–Schmidt
with the _Airy kernel_

$\int_{-\infty}^{0}K_{\lambda}(x,y)\,d\lambda=\int_{0}^{\infty}{\operator@font
Ai}(x+\lambda){\operator@font Ai}(y+\lambda)\,d\lambda=\frac{{\operator@font
Ai}(x){\operator@font Ai}^{\prime}(y)-{\operator@font
Ai}^{\prime}(x){\operator@font Ai}(y)}{x-y},$

restricted to $x,y\in(s,\infty)$. Here, the last equality is obtained from a
Christoffel–Darboux type of argument: First, we use the underlying
differential equation,

$x\,{\operator@font Ai}(x+\lambda)={\operator@font
Ai}^{\prime\prime}(x+\lambda)-\lambda\,{\operator@font Ai}(x+\lambda),$

and partial integration to obtain

$x\int_{0}^{\infty}{\operator@font Ai}(x+\lambda){\operator@font
Ai}(y+\lambda)\,d\lambda\\\\[5.69054pt] =\int_{0}^{\infty}{\operator@font
Ai}^{\prime\prime}(x+\lambda){\operator@font
Ai}(y+\lambda)\,d\lambda-\int_{0}^{\infty}\lambda\,{\operator@font
Ai}(x+\lambda){\operator@font Ai}(y+\lambda)\,d\lambda\\\\[5.69054pt]
=-{\operator@font Ai}^{\prime}(x){\operator@font
Ai}(y)-\int_{0}^{\infty}{\operator@font Ai}^{\prime}(x+\lambda){\operator@font
Ai}^{\prime}(y+\lambda)\,d\lambda-\int_{0}^{\infty}\lambda\,{\operator@font
Ai}(x+\lambda){\operator@font Ai}(y+\lambda)\,d\lambda.$

Next, we exchange the roles of $x$ and $y$ and substract to get the assertion.
Since the resulting kernel is continuous, Theorem 4 gives that
${\mathbb{1}}_{(-\infty,0)}(L){\mathbb{1}}_{(s,\infty)}$ is a trace class
operator with trace

${\operator@font
tr}\,{\mathbb{1}}_{(-\infty,0)}(L){\mathbb{1}}_{(s,\infty)}=\tau(s)\to\infty\qquad(s\to-\infty).$

To summarize, we have thus obtained the following lemma.

######  Lemma A.2.

The differential operator $Lu=-u^{\prime\prime}+xu$ is essentially selfadjoint
on $C^{\infty}_{0}(-\infty,\infty)$. The spectrum of its unique selfadjoint
extension is

$\sigma(L)=\sigma_{\rm ac}(L)=(-\infty,\infty).$

Given $s>-\infty$, the operator
${\mathbb{1}}_{(-\infty,0)}(L){\mathbb{1}}_{(s,\infty)}$ is trace class with
kernel

$K_{\text{\rm Airy}}(x,y)=\int_{0}^{\infty}{\operator@font
Ai}(x+\lambda){\operator@font Ai}(y+\lambda)\,d\lambda=\frac{{\operator@font
Ai}(x){\operator@font Ai}^{\prime}(y)-{\operator@font
Ai}^{\prime}(x){\operator@font Ai}(y)}{x-y}.$ (A.4)

######  Example 3 (Bessel kernel).

Given $\alpha>0$, take $Lu=-4(xu^{\prime})^{\prime}+\alpha^{2}x^{-1}u$ on
$(0,\infty)$. Since a fundamental system of solutions of $Lu=0$ is given by
$u(x)=x^{\pm\alpha/2}$, the endpoint $x=0$ is LP for $\alpha\geqslant 1$ and
LC otherwise; the endpoint $x=\infty$ is LP in both cases. Fixing the LP case
at $x=0$, we restrict ourselves to the case $\alpha\geqslant 1$.

For a given ${\operator@font Im\,}z>0$ the solutions $u_{a}$ ($u_{b}$) of
$(L-z)u=0$ being $L^{2}$ at $0$ ($\infty$) are spanned by (see ?, Eq.
(9.1.7–9) and (9.2.5–6))

$u_{a}(x)=J_{\alpha}(\sqrt{xz}),\qquad
u_{b}(x)=J_{\alpha}(\sqrt{xz})+i\,Y_{\alpha}(\sqrt{xz}).$

Thus, Theorem 3 applies: $L$ is essentially selfadjoint on
$C^{\infty}_{0}(0,\infty)$, the resolvent of its unique selfadjoint extension
is represented, for ${\operator@font Im\,}z>0$, by the Green’s kernel

$G_{z}(x,y)=\frac{i\pi}{4}\begin{cases}J_{\alpha}(\sqrt{xz})\left(J_{\alpha}(\sqrt{\smash[b]{yz}})+i\,Y_{\alpha}(\sqrt{\smash[b]{yz}})\right)&x>y\\\\[2.84526pt]
J_{\alpha}(\sqrt{\smash[b]{yz}})\left(J_{\alpha}(\sqrt{xz})+i\,Y_{\alpha}(\sqrt{xz})\right)&\text{otherwise}.\end{cases}$

For $\lambda>0$ there is the limit

$\pi^{-1}{\operator@font
Im\,}G_{\lambda+i0}(x,y)=K_{\lambda}(x,y)=\frac{1}{4}J_{\alpha}(\sqrt{x\lambda})J_{\alpha}(\sqrt{\smash[b]{y\lambda}}),$

for $\lambda\leqslant 0$ the limit is zero; both limits are locally uniform in
$x,y\in{\mathbb{R}}$. Hence, Theorem 4 applies:
$\sigma(L)=\sigma_{\text{ac}}(L)=[0,\infty)$ and (A.2) holds for each Borel
set $\Delta\subset{\mathbb{R}}$. Given $0\leqslant s<\infty$, we may estimate
for the specific choice $\Delta=(-\infty,1)$ that

$\left(\int_{0}^{s}\int_{0}^{s}\left(\int_{\Delta}|K_{\lambda}(x,y)|\,d\lambda\right)^{2}\,dx\,dy\right)^{1/2}\leqslant\frac{1}{4}\int_{0}^{s}\int_{0}^{1}J_{\alpha}(\sqrt{x\lambda})^{2}\,d\lambda\,dx=\tau_{\alpha}(s).$

Therefore, Theorem 4 yields that
${\mathbb{1}}_{(-\infty,1)}(L){\mathbb{1}}_{(0,s)}$ is Hilbert–Schmidt with
the _Bessel kernel_

$\int_{-\infty}^{1}K_{\lambda}(x,y)\,d\lambda=\frac{1}{4}\int_{0}^{1}J_{\alpha}(\sqrt{x\lambda})J_{\alpha}(\sqrt{y\lambda})\,d\lambda\\\\[5.69054pt]
=\frac{J_{\alpha}(\sqrt{x})\sqrt{\smash[b]{y}}\,J_{\alpha}^{\prime}(\sqrt{\smash[b]{y}})-\sqrt{x}\,J_{\alpha}^{\prime}(\sqrt{x})J_{\alpha}(\sqrt{\smash[b]{y}})}{2(x-y)},$

restricted to $x,y\in(0,s)$. Here, the last equality is obtained from a
Christoffel–Darboux type of argument: First, we use the underlying
differential equation,

$x\,J_{\alpha}(\sqrt{x\lambda})=-4\,\frac{d}{d\lambda}\left(\lambda\frac{d}{d\lambda}\,J_{\alpha}(\sqrt{x\lambda})\right)+\alpha^{2}\lambda^{-1}J_{\alpha}(\sqrt{x\lambda}),$

and partial integration to obtain

$\frac{x}{4}\int_{0}^{1}J_{\alpha}(\sqrt{x\lambda})J_{\alpha}(\sqrt{y\lambda})\,d\lambda\\\\[5.69054pt]
=-\int_{0}^{1}\frac{d}{d\lambda}\left(\lambda\frac{d}{d\lambda}\,J_{\alpha}(\sqrt{x\lambda})\right)J_{\alpha}(\sqrt{y\lambda})\,d\lambda+\frac{\alpha^{2}}{4}\int_{0}^{1}\lambda^{-1}J_{\alpha}(\sqrt{x\lambda})J_{\alpha}(\sqrt{y\lambda})\,d\lambda\\\\[5.69054pt]
=-\frac{1}{2}\sqrt{x}J^{\prime}_{\alpha}(\sqrt{x})J_{\alpha}(\sqrt{\smash[b]{y}})\\\\[5.69054pt]
+\int_{0}^{1}\lambda\left(\frac{d}{d\lambda}\,J_{\alpha}(\sqrt{x\lambda})\right)\left(\frac{d}{d\lambda}J_{\alpha}(\sqrt{y\lambda})\right)\,d\lambda+\frac{\alpha^{2}}{4}\int_{0}^{1}\lambda^{-1}J_{\alpha}(\sqrt{x\lambda})J_{\alpha}(\sqrt{y\lambda})\,d\lambda.$

Next, we exchange the roles of $x$ and $y$ and substract to get the assertion.
Since the resulting kernel is continuous, Theorem 4 gives that
${\mathbb{1}}_{(-\infty,1)}(L){\mathbb{1}}_{(0,s)}$ is a trace class operator
with trace

${\operator@font
tr}\,{\mathbb{1}}_{(-\infty,1)}(L){\mathbb{1}}_{(0,s)}=\tau_{\alpha}(s)\to\infty\qquad(s\to\infty).$

To summarize, we have thus obtained the following lemma.

######  Lemma A.3.

Given $\alpha\geqslant 1$, the differential operator
$Lu=-4(xu^{\prime})^{\prime}+\alpha^{2}x^{-1}u$ is essentially selfadjoint on
$C^{\infty}_{0}(0,\infty)$. The spectrum of its unique selfadjoint extension
is

$\sigma(L)=\sigma_{\rm ac}(L)=[0,\infty).$

Given $0\leqslant s<\infty$, the operator
${\mathbb{1}}_{(-\infty,1)}(L){\mathbb{1}}_{(0,s)}$ is trace class with kernel

$K_{\text{\rm
Bessel}}^{(\alpha)}(x,y)=\frac{1}{4}\int_{0}^{1}J_{\alpha}(\sqrt{x\lambda})J_{\alpha}(\sqrt{y\lambda})\,d\lambda\\\\[5.69054pt]
=\frac{J_{\alpha}(\sqrt{x})\sqrt{\smash[b]{y}}\,J_{\alpha}^{\prime}(\sqrt{\smash[b]{y}})-\sqrt{x}\,J_{\alpha}^{\prime}(\sqrt{x})J_{\alpha}(\sqrt{\smash[b]{y}})}{2(x-y)}.$
(A.5)

######  Remark A.1.

Lemma A.3 extends to $0\leqslant\alpha<1$ if we choose the particular
selfadjoint realization of $L$ that is defined by the boundary condition
(1.6), cf. ?.

## References

  * [1]
  * [2] [] Abramowitz, M. and Stegun, I. A.: 1964, Handbook of mathematical functions with formulas, graphs, and mathematical tables, National Bureau of Standards, Washington.
  * [3]
  * [4] [] Anderson, G. W., Guionnet, A. and Zeitouni, O.: 2010, An Introduction to Random Matrices, Cambridge University Press.
  * [5]
  * [6] [] Andrews, G. E., Askey, R. and Roy, R.: 1999, Special Functions, Cambridge University Press, Cambridge.
  * [7]
  * [8] [] Borodin, A. and Olshanski, G.: 2007, Asymptotics of Plancherel-type random partitions, J. Algebra 313, 40–60.
  * [9]
  * [10] [] Collins, B.: 2005, Product of random projections, Jacobi ensembles and universality problems arising from free probability, Probab. Theory Related Fields 133, 315–344.
  * [11]
  * [12] [] Deift, P.: 1999, Orthogonal polynomials and random matrices: a Riemann-Hilbert approach, AMS, Providence.
  * [13]
  * [14] [] Erdélyi, A., Magnus, W., Oberhettinger, F. and Tricomi, F. G.: 1953, Higher transcendental functions. Vol. II, McGraw-Hill, New York.
  * [15]
  * [16] [] Gohberg, I., Goldberg, S. and Krupnik, N.: 2000, Traces and determinants of linear operators, Birkhäuser Verlag, Basel.
  * [17]
  * [18] [] Hutson, V., Pym, J. S. and Cloud, M. J.: 2005, Applications of functional analysis and operator theory, second edn, Elsevier B. V., Amsterdam.
  * [19]
  * [20] [] Johnstone, I. M.: 2001, On the distribution of the largest eigenvalue in principal components analysis, Ann. Statist. 29, 295–327.
  * [21]
  * [22] [] Johnstone, I. M.: 2008, Multivariate analysis and Jacobi ensembles: largest eigenvalue, Tracy-Widom limits and rates of convergence, Ann. Statist. 36, 2638–2716.
  * [23]
  * [24] [] Kuijlaars, A. B. J.: 2011, Universality, in G. Akemann, J. Baik and P. D. Francesco (eds), Oxford Handbook of Random Matrix Theory, Oxford University Press.  arXiv:1103.5922.
  * [25]
  * [26] [] Lax, P. D.: 2002, Functional analysis, Wiley-Interscience, New York.
  * [27]
  * [28] [] Simon, B.: 2005, Trace ideals and their applications, second edn, American Mathematical Society, Providence, RI.
  * [29]
  * [30] [] Stolz, G. and Weidmann, J.: 1993, Approximation of isolated eigenvalues of ordinary differential operators, J. Reine Angew. Math. 445, 31–44.
  * [31]
  * [32] [] Tao, T.: 2012, Topics in random matrix theory, American Mathematical Society.
  * [33]
  * [34] [] Wachter, K. W.: 1980, The limiting empirical measure of multiple discriminant ratios, Ann. Statist. 8, 937–957.
  * [35]
  * [36] [] Weidmann, J.: 1980, Linear operators in Hilbert spaces, Springer-Verlag, New York.
  * [37]
  * [38] [] Weidmann, J.: 1987, Spectral theory of ordinary differential operators, Springer-Verlag, Berlin.
  * [39]