A Riemann–Hilbert Approach to Asymptotic Analysis of Toeplitz+Hankel
Determinants

A Riemann–Hilbert Approach to Asymptotic Analysis
of Toeplitz+Hankel Determinants

Roozbeh GHARAKHLOO † and Alexander ITS ‡§

R. Gharakhloo and A. Its

† Department of Mathematics, Colorado State University,
† 1874 Campus Delivery, Fort Collins, CO 80523-1874, USA
<EMAIL_ADDRESS>https://www.math.colostate.edu/~gharakhl

‡ Department of Mathematical Sciences, Indiana University-Purdue University
Indianapolis,
‡ 402 N. Blackford St., Indianapolis, IN 46202, USA<EMAIL_ADDRESS>
§ St. Petersburg State University, Universitetskaya emb. 7/9, 199034, St.
Petersburg, Russia

Received November 03, 2019, in final form August 26, 2020; Published online
October 06, 2020

In this paper we will formulate $4\times 4$ Riemann–Hilbert problems for
Toeplitz+Hankel determinants and the associated system of orthogonal
polynomials, when the Hankel symbol is supported on the unit circle and also
when it is supported on an interval $[a,b]$, $0<a<b<1$. The distinguishing
feature of this work is that in the formulation of the Riemann–Hilbert problem
no specific relationship is assumed between the Toeplitz and Hankel symbols.
We will develop nonlinear steepest descent methods for analysing these
problems in the case where the symbols are smooth (i.e., in the absence of
Fisher–Hartwig singularities) and admit an analytic continuation in a
neighborhood of the unit circle (if the symbol’s support is the unit circle).
We will finally introduce a model problem and will present its solution
requiring certain conditions on the ratio of Hankel and Toeplitz symbols. This
in turn will allow us to find the asymptotics of the norms $h_{n}$ of the
corresponding orthogonal polynomials and, in fact, the large $n$ asymptotics
of the polynomials themselves. We will explain how this solvable case is
related to the recent operator-theoretic approach in [Basor E., Ehrhardt T.,
in Large Truncated Toeplitz Matrices, Toeplitz Operators, and Related Topics,
Oper. Theory Adv. Appl., Vol. 259, Birkhäuser/Springer, Cham, 2017, 125–154,
arXiv:1603.00506] to Toeplitz+Hankel determinants. At the end we will discuss
the prospects of future work and outline several technical, as well as
conceptual, issues which we are going to address next within the $4\times 4$
Riemann–Hilbert framework introduced in this paper.

Toeplitz+Hankel determinants; Riemann–Hilbert problem; asymptotic analysis

15B05; 30E15; 35Q15

##  1 Introduction and preliminaries

The $n\times n$ Toeplitz and Hankel matrices associated respectively to the
symbols $\phi$ and $w$, supported on the unit circle $\mathbb{T}$ are
respectively defined as

$\displaystyle T_{n}[\phi;r]:=\\{\phi_{j-k+r}\\},\qquad
j,k=0,\dots,n-1,\qquad\phi_{k}=\int_{{\mathbb{T}}}z^{-k}\phi(z)\frac{{\rm
d}z}{2\pi{\rm i}z},$

and

$\displaystyle H_{n}[w;s]:=\\{w_{j+k+s}\\},\qquad j,k=0,\dots,n-1,\qquad
w_{k}=\int_{{\mathbb{T}}}z^{-k}w(z)\frac{{\rm d}z}{2\pi{\rm i}z},$ (1.1)

for fixed offset values $r,s\in{\mathbb{Z}}$. If the Hankel symbol $w$ is
supported on a subset $I$ of the real line, then $w_{k}$ in (1.1) are instead
given by

$\displaystyle w_{k}=\int_{I}x^{k}w(x)\,{\rm d}x.$ (1.2)

The Toeplitz and Hankel determinants characterize important objects
particularly in random matrix theory, statistical mechanics, theory of
orthogonal polynomials, theory of Fredhom determinants, etc. For more on the
history of the development of the theory of Toeplitz and Hankel determinants
and their numerous applications we refer the reader to the review articles
[15] and [21]. We also refer to the monographs [8] and [9] as the main sources
for general facts concerning the theory of Toeplitz matrices and operators.
The asymptotic results concerning the Hankel determinants and their
applications – both recent and classical, are featured in the papers [10, 11,
15, 19, 20] and in the references therein.

There has been a growing interest in the asymptotics of Toeplitz+Hankel
determinants in recent years. A Toeplitz+Hankel matrix is naturally the sum
$T_{n}[\phi;r]+H_{n}[w;s]$, and thus it has a determinant of the form

$\displaystyle
D_{n}(\phi,w;r,s):=\det\begin{pmatrix}\phi_{r}+w_{s}&\phi_{r-1}+w_{s+1}&\cdots&\phi_{r-n+1}+w_{s+n-1}\\\
\phi_{r+1}+w_{s+1}&\phi_{r}+w_{s+2}&\cdots&\phi_{r-n+2}+w_{s+n}\\\
\vdots&\vdots&\ddots&\vdots\\\
\phi_{r+n-1}+w_{s+n-1}&\phi_{r+n-2}+w_{s+n}&\cdots&\phi_{r}+w_{s+2n-2}\end{pmatrix}\\!,$
(1.3)

$r,s\in{\mathbb{Z}}$, where, naturally, $\phi$ and $w$ respectively denote the
Toeplitz and Hankel symbols. Although there are no results in the literature
for the Toeplitz+Hankel determinants where $w$ is supported on the line, the
case where $w$ is supported on the unit circle has been considered under
specific assumptions. E. Basor and T. Ehrhardt have studied different aspects
of these determinants in a series of papers [2, 3, 4, 5, 6] via operator-
theoretic tools over the last 20 years or so. In [13], the Riemann–Hilbert
technique which has already been proven very effective to study the
asymptotics of Toeplitz and Hankel determinants was extended for the first
time to the determinants of Toeplitz+Hankel matrices generated by the same
symbol $w=\phi$, where the Hankel weight is supported on ${\mathbb{T}}$. In
that work the symbol was assumed to be of Fisher–Hartwig type and it was
further required that the symbol be even, i.e., $w=\tilde{w}$.111Notation.
Throughout the paper we will frequently use the notation $\tilde{f}(z)$, to
denote $f\big{(}z^{-1}\big{)}$. In [2], by employing the relevant results in
[13], the authors managed for the first time to find the asymptotics of
Toeplitz+Hankel determinants for certain non-coinciding symbols. Indeed, they
considered

$\displaystyle\phi(z)=c(z)\phi_{0}(z),\qquad\mbox{and}\qquad
w(z)=c(z)d(z)w_{0}(z),$ (1.4)

where the functions $c$ and $d$ are assumed to be smooth and nonvanishing on
the unit circle with zero winding number. Neither $c$ nor $d$ are assumed to
be even functions but it is further required that $d$ satisfies the conditions
$d\tilde{d}=1$ (on the unit circle) and $d(\pm 1)=1$. Furthermore, $\phi_{0}$
is assumed to be an even function of FH type and $w_{0}$ is related to
$\phi_{0}$ in one of the following four ways: a) $w_{0}(z)=\pm\phi_{0}(z)$, b)
$w_{0}(z)=z\phi_{0}(z)$ and c) $w_{0}(z)=\displaystyle-z^{-1}\phi_{0}(z)$.

Since the Riemann–Hilbert analysis carried out in [13] does not allow for
different symbols, the primary goal of this paper is to develop a
Riemann–Hilbert framework for asymptotic analysis of Toeplitz+Hankel
determinants $D_{n}(\phi,w;r,s)$ where $\phi$ and $w$ are not a priori
related, at least at the level of formulation of the problem. Indeed,
asymptotics of Toeplitz+Hankel determinants with different symbols are
interesting for several reasons that prompts this research project. For
example, the type (1.4) of Toeplitz+Hankel determinants has appeared in the
very recent work [12] in connection with the analysis of Ising model on the
$45^{\circ}$ rotated half-plane, or the so-called zig-zag half-plane.

Perhaps, our most important motivation behind studying Toeplitz+Hankel
determinants is to study the large $n$ asymptotics of the eigenvalues of the
Hankel matrix $H_{n}[w]$ associated to the symbol $w$. Specifically, we want
to extend the recent results [14] concerning the spectral asymptotics of the
Toeplitz matrices222 The large $n$ behavior of the individual eigenvalues of
Toeplitz matrices has been also addressed in a number of works – see [7] and
references therein. to the Hankel case. The key feature which allows an
effective asymptotic spectral analysis of Toeplitz matrices and, in
particular, the use of the Riemann–Hilbert method, is that the characteristic
polynomial of a Toeplitz matrix is again a Toeplitz determinant with a symbol
of general Fisher–Hartwig type (i.e., no conditions on the
$\beta$-parameters). The asymptotics of such Toeplitz determinants is given by
the Basor–Tracy formula (first conjectured by E. Basor and C. Tracy and then
proved in [13]). However, in the case of Hankel matrices, and this is the crux
of the matter, their characteristic polynomials are not Hankel determinants.
Indeed, the characteristic polynomial $\det(H_{n}[w]-\lambda I)$ of the Hankel
matrix $H_{n}[w]$ is a particular Toeplitz+Hankel determinant, with
$\phi(z)\equiv-\lambda$. Clearly in the case of characteristic polynomial of a
Hankel determinant, there is no relationship between $\phi$ and $w$, so to
study the asymptotics of this determinant, one can not refer to the works [13]
or [2] mentioned above. Here again we are directed to a methodological issue
which has to be addressed at a fundamental level by formulation of a suitable
Riemann–Hilbert problem.

In this paper, we are proposing a version of the Riemann–Hilbert formalism for
the asymptotic analysis of Toeplitz+Hankel determinants based on a certain
$4\times 4$ Riemann–Hilbert problem. When the Hankel symbol is supported on
the unit circle, we introduce the following system of monic orthogonal
polynomials $\\{\mathcal{P}_{n}(z)\\}$, $\deg\mathcal{P}_{n}(z)=n$, associated
to $D_{n}(\phi,w;r,s)$:

$\displaystyle\int_{{\mathbb{T}}}\mathcal{P}_{n}(z)z^{-k-r}\phi(z)\frac{{\rm
d}z}{2\pi{\rm
i}z}+\int_{{\mathbb{T}}}\mathcal{P}_{n}(z)z^{k+s}\tilde{w}(z)\frac{{\rm
d}z}{2\pi{\rm i}z}=h_{n}\delta_{n,k},\qquad k=0,1,\dots,n.$

We also show that for $r=s=1$, if the symbols are analytic in a neighborhood
of the unit circle, one can proceed with a $4\times 4$ analogue of the
Deift–Zhou non-linear steepest descent method and arrive at a $4\times 4$
model Riemann–Hilbert problem on the unit circle which does not contain the
parameter $n$. It is significant to note that one arrives at the same model
Riemann–Hilbert in the fundamentally different case where $w$ is supported on
the interval $[a,b]$, with $0<a<b<1$. In this situation we consider the
following system of monic orthogonal polynomials $\\{P_{n}(z)\\}$, $\deg
P_{n}(z)=n$, associated to $D_{n}(\phi,w;r,s)$:

$\displaystyle\int_{{\mathbb{T}}}P_{n}(z)z^{-k-r}\phi(z)\frac{{\rm
d}z}{2\pi{\rm i}z}+\int^{b}_{a}P_{n}(x)x^{k+s}w(x)\,{\rm
d}x=h_{n}\delta_{n,k},\qquad k=0,1,\dots,n.$

In this case, we can proceed with the Riemann–Hilbert analysis with $r=1$ and
an arbitrary value for $s\in{\mathbb{Z}}$.

We have been able to solve the model problem for the class of symbols (1.4)
considered in [2], in the absence of Fisher–Hartwig singularities. It is
important to discuss the relevancy of the two conditions assumed to be
satisfied by the function $d$ in [2], in our Riemann–Hilbert framework (see
(1.4) and below). Unlike the condition $d\big{(}{\rm e}^{{\rm
i}\theta}\big{)}d\big{(}{\rm e}^{-{\rm i}\theta}\big{)}=1$ which is,
remarkably, a simplifying condition for the factorization of the model
Riemann–Hilbert problem, it should be noticed that the condition $d(\pm 1)=1$
is not required in the entirety of our Riemann–Hilbert approach. Solving the
model problem allows us to find the asymptotics for the norm $h_{n}$ of the
associated orthogonal polynomials. We provide the details of this calculation
for the case where $w$ is supported on the unit circle.

In what follows we formulate our main asymptotic result. To do so we need to
introduce some notations. Throughout the paper, we will occasionally refer to
a symbol $f$ as a Szegő-type symbol, if a) it is smooth and nonzero on the
unit circle, b) has no winding number, and c) admits an analytic continuation
in a neighborhood of the unit circle. Also for a given function $f$, and an
oriented contour $\Gamma$, we write $f_{+}(z)$ (resp. $f_{-}(z)$) to denote
the limiting value of $f(\zeta)$, as $\zeta$ approaches $z\in\Gamma$ from the
left (resp. right) hand side of the oriented contour $\Gamma$ with respect to
its orientation. We also note that the expressions like $\tilde{f}_{\pm}(z)$,
should be understood as the boundary values of the function $\tilde{f}$ at $z$
and should not be confused with the boundary value of the function $f$ at the
point $\frac{1}{z}$, in other words, the operation of $z\mapsto\frac{1}{z}$
precedes that of taking the boundary values. Given the Szegő-type symbols
$\phi(z)$ and $w(z)=d(z)\phi(z)$, we define

$\displaystyle\alpha(z)=\exp\biggl{[}\frac{1}{2\pi{\rm
i}}\int_{{\mathbb{T}}}\frac{\ln(\phi(\tau))}{\tau-z}\,{\rm
d}\tau\biggr{]},\qquad\beta(z)=\exp\biggl{[}\frac{1}{2\pi{\rm
i}}\int_{{\mathbb{T}}}\frac{\ln(d(\tau))}{\tau-z}\,{\rm d}\tau\biggr{]},$
$\displaystyle\mathcal{C}_{\rho}(z)=-\frac{1}{2\pi{\rm
i}}\int_{{\mathbb{T}}}\frac{1}{\beta_{-}(\tau)\beta_{+}(\tau)\tilde{\alpha}_{-}(\tau)\alpha_{+}(\tau)(\tau-z)}\,{\rm
d}\tau,$ $\displaystyle
g_{23}(z)=-\frac{\alpha(0)\tilde{d}(z)\beta(z)}{\tilde{\alpha}(z)},\qquad
g_{43}(z)=-\alpha^{2}(0)\beta(z)\biggl{(}\frac{\alpha(z)}{\tilde{\phi}(z)}+\frac{\tilde{d}(z)C_{\rho}(z)}{\tilde{\alpha}(z)}\biggr{)},$
$\displaystyle R_{1,23}(z;n)=\frac{1}{2\pi{\rm
i}}\int_{\Gamma^{\prime}_{i}}\frac{\mu^{n}g_{23}(\mu)}{\mu-z}\,{\rm
d}\mu,\qquad R_{1,43}(z;n)=\frac{1}{2\pi{\rm
i}}\int_{\Gamma^{\prime}_{i}}\frac{\mu^{n}g_{43}(\mu)}{\mu-z}\,{\rm d}\mu,$
(1.5)

and finally

$\displaystyle\mathcal{E}(n)=\frac{2}{\alpha(0)}R_{1,43}(0;n)-C_{\rho}(0)R_{1,23}(0;n).$
(1.6)

In (1.5), the contour $\Gamma^{\prime}_{i}$ is a circle, oriented counter-
clockwise, with radius $r^{\prime}<1$ so that the functions $\phi$ and $d$ are
analytic in the annulus $\\{z\colon r^{\prime}\leq|z|<1\\}$.

######  Theorem 1.1.

Suppose that $\phi({\rm e}^{i\theta})$ is smooth and nonzero on the unit
circle with zero winding number, which admits an analytic continuation in a
neighborhood of the unit circle. Let $w=d\phi$, where $d$ satisfies all the
properties of $\phi$ in addition to $d\big{(}{\rm e}^{{\rm
i}\theta}\big{)}d\big{(}{\rm e}^{-{\rm i}\theta}\big{)}=1$, for all
$\theta\in[0,2\pi)$. Let also

$\displaystyle U_{0}:=\left\\{z\colon r_{i}<|z|<r_{o}\colon
0<r_{i}<1<r_{o}\right\\},$ (1.7)

be the neighborhood of the unit circle where both functions, $\phi(z)$ and
$d(z)$ are analytic. Denote

$\displaystyle r_{0}:=\max\big{\\{}r_{i},r^{-1}_{o}\big{\\}},$ (1.8)

and suppose now that there exists such $C>0$ that for sufficiently large $n$,

$\displaystyle|\mathcal{E}(n)|\geq Cr^{n}$ (1.9)

for some $r\in[r_{0},1)$, where $\mathcal{E}(n)$ is the functional of the
weights $\phi$ and $w$ defined in (1.6). Then, for sufficiently large $n$ the
determinant $D_{n}(\phi,w;1,1)\neq 0$ and the asymptotics of

$\displaystyle h_{n-1}\equiv\frac{D_{n}(\phi,w;1,1)}{D_{n-1}(\phi,w;1,1)},$

is given by

$\displaystyle
h_{n-1}=-\alpha(0)\frac{\mathcal{E}(n)}{\mathcal{E}(n-1)}\big{(}1+{\cal
O}{\big{(}{\rm e}^{-c_{1}n}\big{)}}\big{)},\qquad n\to\infty,$ (1.10)

where $c_{1}=-\log\big{(}\frac{r^{2}_{1}}{r}\big{)}>0$, and $r_{1}$ is any
number satisfying the conditions: $r<r_{1}<1$ and $r^{2}_{1}<r$.

######  Remark 1.2.

As a slight generalization of Theorem 1.1, one could replace the condition
(1.9) by

$\displaystyle|\mathcal{E}(n)|\geq Cr^{n}\varkappa(n)$ (1.11)

for some $r\in[r_{0},1)$, where $\varkappa(n)>0$ for all sufficiently large
$n$ and is such that ${\rm e}^{-\varepsilon n}=o(\varkappa(n))$ for all
$\varepsilon>0$ as $n\to\infty$. So as a result, (1.10) should be replaced by

$\displaystyle
h_{n-1}=-\alpha(0)\frac{\mathcal{E}(n)}{\mathcal{E}(n-1)}\biggl{(}1+{\cal
O}{\biggl{(}\frac{{\rm e}^{-c_{1}n}}{\varkappa(n)}\biggr{)}}\biggr{)},\qquad
n\to\infty,$

where the constant $c_{1}$ above is the same as the one in (1.10).

######  Remark 1.3.

The analyticity of $\phi$ in the neighborhood of the unit circle is a
technical condition. It can be lifted and replaced by certain smoothness
conditions using the approximation type arguments similar to the ones used in
[13, Section 6.2]. However, because of the condition (1.9), the corresponding
analysis becomes more subtle. We shall address this issue together with
several other technical points in the forthcoming publications.

###  1.1 Outline

In Section 2 we will analyze the case where Hankel symbol $w$ is supported on
the unit circle. We will propose a $2\times 2$ Riemann–Hilbert problem with a
shift for the associated orthogonal polynomials. For an effective
Riemann–Hilbert analysis, we will then propose a $2\times 4$ Riemann–Hilbert
problem whose jump conditions could be written in the usual form of matrix
multiplications. We will then formulate a $4\times 4$ Riemann–Hilbert problem
which is the suitable framework for our analysis. In the formulation of the
$4\times 4$ Riemann–Hilbert problem, for technical reasons, we will restrict
to the particular offset values $r=s=1$. Following the natural steps of
steepest descent analysis we will arrive at a model problem in Section 2.6,
which we will refer to as the model Riemann–Hilbert problem for the pair
$(\phi,w)$.

Section 3 is devoted to analysis of the case where the Hankel symbol is
supported on the interval $[a,b]$, with $0<a<b<1$. Similar to Section 2, we
will propose a $2\times 2$ Riemann–Hilbert problem with a shift for the
associated system of orthogonal polynomials and for the same reasons mentioned
above we pass through a $2\times 4$ to arrive at a suitable $4\times 4$
Riemann–Hilbert problem. In this case, our methods allow for considering an
arbitrary offset for the Hankel part, more precisely, we can pursue the
steepest descent analysis for $r=1$ and an arbitrary $s\in{\mathbb{Z}}$. This
steepest descent analysis leads us to a model problem, which is the same as
the model problem of Section 2, except that it is for the pair
$(\phi,-\tilde{u})$, where

$\displaystyle u(z)=z\int^{b}_{a}\frac{x^{s-1}w(x)}{x-z}\,{\rm d}x.$

Although there are similarities between the steepest descent analysis of
Section 2 and Section 3, we feel obliged to lay out a thorough exposition to
illustrate the remarkable fact that the same model RH problem emerges in both
cases.

In Section 4, we present the factorization of the model RH problem for the
pair $(\phi,d\phi)$, where the functions $\phi$ and $d$ are of Szegő-type. The
function $d$ further satisfies $d\tilde{d}=1$ on the unit circle. We will then
use this solution to construct the solutions to the global parametrix and the
small-norm Riemann–Hilbert problems, which finally enables us to find the
asymptotics of the norms $h_{n}$ of the associated monic orthogonal
polynomials. Although we do not formulate our main asymptotic result for the
symbol pairs discussed in Section 3, in Remark 4.2 we justify that the
analysis of Section 4 is completely relevant for the symbol pairs of Section 3
as well.

Finally, in Section 5 we summarize the still open technical and conceptual
questions which we are going to address in our future work.

##  2 Toeplitz+Hankel determinants: Hankel weight
supported on ${\mathbb{T}}$

In this section we assume that $w$ is supported on the unit circle and that
both symbols $\phi$ and $w$, admit analytic continuations to a neighborhood of
the unit circle. A key observation is that the determinant (1.3) is related to
the system of monic polynomials $\\{\mathcal{P}_{n}(z)\\}$,
$\deg\mathcal{P}_{n}(z)=n$, determined by the orthogonality relations

$\displaystyle\int_{{\mathbb{T}}}\mathcal{P}_{n}(z)z^{-k-r}\phi(z)\frac{{\rm
d}z}{2\pi{\rm
i}z}+\int_{{\mathbb{T}}}\mathcal{P}_{n}(z)z^{k+s}\tilde{w}(z)\frac{{\rm
d}z}{2\pi{\rm i}z}=h_{n}\delta_{n,k},\qquad k=0,1,\dots,n.$ (2.1)

These polynomials exist and are unique if the Toeplitz+Hankel determinants
(1.3) are non-zero. This can be seen as follows. Expectedly, if $D_{n}\equiv
D_{n}(\phi,w;r,s)\neq 0$, the polynomials $\mathcal{P}_{n}$ can be written as
the following determinants

$\displaystyle\mathcal{P}_{n}(z):=\frac{1}{D_{n}}\det\begin{pmatrix}\phi_{r}+w_{s}&\phi_{r-1}+w_{s+1}&\cdots&\phi_{r-n}+w_{s+n}\\\
\phi_{r+1}+w_{s+1}&\phi_{r}+w_{s+2}&\cdots&\phi_{r-n+1}+w_{s+n+1}\\\
\vdots&\vdots&\ddots&\vdots\\\
\phi_{r+n-1}+w_{s+n-1}&\phi_{r+n-2}+w_{s+n}&\cdots&\phi_{r-1}+w_{s+2n-1}\\\
1&z&\cdots&z^{n}\end{pmatrix}\\!.$ (2.2)

Indeed, for the polynomials defined by (2.2) we have that

$\displaystyle\int_{{\mathbb{T}}}\mathcal{P}_{n}(z)z^{k+s}\tilde{w}(z)\frac{{\rm
d}z}{2\pi{\rm i}z}$
$\displaystyle\qquad=\frac{1}{D_{n}}\det\begin{pmatrix}\phi_{r}+w_{s}&\phi_{r-1}+w_{s+1}&\cdots&\phi_{r-n}+w_{s+n}\\\
\phi_{r+1}+w_{s+1}&\phi_{r}+w_{s+2}&\cdots&\phi_{r-n+1}+w_{s+n+1}\\\
\vdots&\vdots&\ddots&\vdots\\\
\phi_{r+n-1}+w_{s+n-1}&\phi_{r+n-2}+w_{s+n}&\cdots&\phi_{r-1}+w_{s+2n-1}\\\
w_{k+s}&w_{k+s+1}&\cdots&w_{k+s+n}\end{pmatrix}\\!,$

and

$\displaystyle\int_{{\mathbb{T}}}\mathcal{P}_{n}(z)z^{-k-r}\phi(z)\frac{{\rm
d}z}{2\pi{\rm i}z}$
$\displaystyle\qquad=\frac{1}{D_{n}}\det\begin{pmatrix}\phi_{r}+w_{s}&\phi_{r-1}+w_{s+1}&\cdots&\phi_{r-n}+w_{s+n}\\\
\phi_{r+1}+w_{s+1}&\phi_{r}+w_{s+2}&\cdots&\phi_{r-n+1}+w_{s+n+1}\\\
\vdots&\vdots&\ddots&\vdots\\\
\phi_{r+n-1}+w_{s+n-1}&\phi_{r+n-2}+w_{s+n}&\cdots&\phi_{r-1}+w_{s+2n-1}\\\
\phi_{k+r}&\phi_{k+r-1}&\cdots&\phi_{k+r-n}\end{pmatrix}\\!,$

hence

$\displaystyle\int_{{\mathbb{T}}}\mathcal{P}_{n}(z)z^{-k-r}\phi(z)\frac{{\rm
d}z}{2\pi{\rm
i}z}\\!+\\!\int_{{\mathbb{T}}}\mathcal{P}_{n}(z)z^{k+s}\tilde{w}(z)\frac{{\rm
d}z}{2\pi{\rm i}z}$
$\displaystyle=\frac{1}{D_{n}}\det\begin{pmatrix}\phi_{r}+w_{s}&\phi_{r-1}+w_{s+1}&\cdots&\phi_{r-n}+w_{s+n}\\\
\phi_{r+1}+w_{s+1}&\phi_{r}+w_{s+2}&\cdots&\phi_{r-n+1}+w_{s+n+1}\\\
\vdots&\vdots&\ddots&\vdots\\\
\phi_{r+n-1}+w_{s+n-1}&\phi_{r+n-2}+w_{s+n}&\cdots&\phi_{r-1}+w_{s+2n-1}\\\
\phi_{k+r}+w_{k+s}&\phi_{k+r-1}+w_{k+s+1}&\cdots&\phi_{k+r-n}+w_{k+s+n}\end{pmatrix}\\!=\frac{D_{n+1}}{D_{n}}\delta_{n,k}.$

The uniqueness of the polynomial
$\mathcal{P}_{n}(z)=z^{n}+a_{n-1}z^{n-1}+\cdots+a_{0}$ satisfying (2.1),
simply follows from the fact that one has the following linear system for the
coefficients $a_{j},0\leq j\leq n-1$:

$\displaystyle(T_{n}[\phi;r]+H_{n}[w;s])\begin{pmatrix}a_{0}\\\ a_{1}\\\
\vdots\\\ a_{n-1}\end{pmatrix}=\begin{pmatrix}-\phi_{-n+r}-w_{n+s}\\\
-\phi_{1-n+r}-w_{1+n+s}\\\ \vdots\\\ -\phi_{-1+r}-w_{2n-1+s}\end{pmatrix}\\!.$
(2.3)

So if $D_{n}\neq 0$, the coefficients $a_{j}$ and hence $\mathcal{P}_{n}$, can
be uniquely determined by inverting the Toeplitz+Hankel matrix in (2.3). So
the polynomials defined by (2.2) are the unique polynomials satisfying (2.1),
and

$\displaystyle h_{n}=\frac{D_{n+1}(\phi,w;r,s)}{D_{n}(\phi,w;r,s)}.$ (2.4)

We consider the function $\mathcal{Y}$ defined as

$\displaystyle\mathcal{Y}(z;n)=\begin{pmatrix}\mathcal{P}_{n}(z)&\displaystyle\int_{{\mathbb{T}}}\frac{\xi^{s}\tilde{w}(\xi)\mathcal{P}_{n}(\xi)+\xi^{r}\tilde{\phi}(\xi)\tilde{\mathcal{P}}_{n}(\xi)}{\xi-z}\frac{{\rm
d}\xi}{2\pi{\rm i}\xi}\vspace{1ex}\\\
-\displaystyle\frac{1}{h_{n-1}}\mathcal{P}_{n-1}(z)&-\displaystyle\frac{1}{h_{n-1}}\int_{{\mathbb{T}}}\frac{\xi^{s}\tilde{w}(\xi)\mathcal{P}_{n-1}(\xi)+\xi^{r}\tilde{\phi}(\xi)\tilde{\mathcal{P}}_{n-1}(\xi)}{\xi-z}\frac{{\rm
d}\xi}{2\pi{\rm i}\xi}\end{pmatrix}\\!,$ (2.5)

assuming that $D_{n},D_{n-1}\neq 0$, which ensures that a)
$h_{n-1}=D_{n}/D_{n-1}$ is well-defined and nonzero and b) that
$\mathcal{P}_{n}$ and $\mathcal{P}_{n-1}$ exist and are unique. Now consider
the following Riemann–Hilbert problem for finding the $2\times 2$ matrix
$\mathcal{Y}$ satisfying:

  * •

RH-$\mathcal{Y}$1 $\mathcal{Y}$ is holomorphic in
${\mathbb{C}}\setminus{\mathbb{T}}.$

  * •

RH-$\mathcal{Y}$2 For $z\in{\mathbb{T}}$ we have

$\displaystyle\mathcal{Y}_{+}^{(1)}(z;n)=\mathcal{Y}^{(1)}_{-}(z;n),\qquad
z\in{\mathbb{T}},$ (2.6)

and

$\displaystyle\mathcal{Y}_{+}^{(2)}(z;n)=\mathcal{Y}^{(2)}_{-}(z;n)\\!+\\!z^{-1+s}\tilde{w}(z)\mathcal{Y}^{(1)}_{-}(z;n)\\!+\\!z^{-1+r}\tilde{\phi}(z)\mathcal{Y}^{(1)}_{-}\big{(}z^{-1};n\big{)},\quad
z\in{\mathbb{T}},$ (2.7)

where ${\mathbb{T}}$ is positively oriented in the counter-clockwise
direction.

  * •

RH-$\mathcal{Y}$3 As $z\to\infty$, $\mathcal{Y}$ satisfies

$\displaystyle\mathcal{Y}(z;n)=\big{(}I+\mathcal{O}\big{(}z^{-1}\big{)}\big{)}z^{n\sigma_{3}}=\begin{pmatrix}z^{n}+\mathcal{O}\big{(}z^{n-1}\big{)}&\mathcal{O}\big{(}z^{-n-1}\big{)}\vspace{1ex}\\\
\mathcal{O}\big{(}z^{n-1}\big{)}&z^{-n}+\mathcal{O}\big{(}z^{-n-1}\big{)}\end{pmatrix}\\!,$
(2.8)

where $\mathcal{Y}^{(1)}$ and $\mathcal{Y}^{(2)}$ are the first and second
columns of $\mathcal{Y}$ respectively and

$\displaystyle\sigma_{3}=\begin{pmatrix}1&0\\\ 0&-1\end{pmatrix}$

is the third Pauli matrix. Notice that the jump conditions for the
$\mathcal{Y}$-RHP can not be written in the usual form of matrix
multiplications. Expectedly, this feature of the $\mathcal{Y}$-RHP (which will
be addressed in more detail in Section 2.1) significantly affects the
progression of the Riemann–Hilbert analysis as it necessitates studying larger
size Riemann–Hilbert problems.

The next theorem establishes the association of the function (2.5) with the
$\mathcal{Y}$-RHP.

######  Theorem 2.1.

The following statements are true.

  1. 1.

Suppose that $D_{n}$, $D_{n-1}\neq 0$. Then, the Riemann–Hilbert problem _RH-
$\mathcal{Y}$1_ through _RH- $\mathcal{Y}$3_ is uniquely solvable and its
solution $\mathcal{Y}$ is defined by (2.5). Moreover,

$\displaystyle h_{n-1}=-\lim_{z\to\infty}z^{n-1}/\mathcal{Y}_{21}(z;n).$ (2.9)

  2. 2.

Suppose that the Riemann–Hilbert problem _RH- $\mathcal{Y}$1_ through _RH-
$\mathcal{Y}$3_ has a unique solution. Then $D_{n}\neq 0$,
$\operatorname{rank}(T_{n-1}[\phi;r]+H_{n-1}[w;s])\geq n-2$, and
$\mathcal{P}_{n}(z)=\mathcal{Y}_{11}(z;n)$.

  3. 3.

Suppose that the Riemann–Hilbert problem _RH- $\mathcal{Y}$1_ through _RH-
$\mathcal{Y}$3_ has a unique solution. Suppose also that

$\displaystyle\lim_{z\to\infty}\mathcal{Y}_{21}(z;n)z^{-n+1}\neq 0.$

Then, as before, $D_{n}\neq 0$, $\mathcal{P}_{n}(z)=\mathcal{Y}_{11}(z;n)$,
and, in addition,

$\displaystyle D_{n-1}\neq 0,\qquad
h_{n-1}=-\lim_{z\to\infty}\mathcal{Y}^{-1}_{21}(z;n)z^{n-1},\qquad\mathcal{P}_{n-1}(z)=-h_{n-1}\mathcal{Y}_{21}(z;n).$

###### Proof.

Assume that $D_{n},D_{n-1}\neq 0$, and thus the function $\mathcal{Y}$ is
uniquely defined by (2.5), being identified with the unique orthogonal
polynomials satisfying the orthogonality conditions (2.1). We now show that
$\mathcal{Y}$ given by (2.5) is a unique solution of RH-$\mathcal{Y}$1 through
RH-$\mathcal{Y}$3. It is clear that RH-$\mathcal{Y}$1 is satisfied due to
general properties of Cauchy integrals. From (2.6) we see that
$\mathcal{Y}_{11}$ and $\mathcal{Y}_{21}$ are entire functions, and from (2.8)
we know that $\mathcal{Y}_{11}$ has to be a monic polynomial of degree $n$ and
$\mathcal{Y}_{21}$ has to be a polynomial of degree $n-1$ or less,

$\displaystyle\mathcal{Y}_{11}(z;n)=z^{n}+\sum_{k=0}^{n-1}\alpha_{k}z^{k},\qquad\mathcal{Y}_{21}(z;n)=\sum_{k=0}^{n-1}\beta_{k}z^{k}.$

From (2.7) and what we just mentioned about $\mathcal{Y}_{11}$ we would have

$\displaystyle\mathcal{Y}_{12,+}(z;n)-\mathcal{Y}_{12,-}(z;n)=z^{-1+s}\tilde{w}(z)\mathcal{Y}_{11}(z;n)+z^{-1+r}\tilde{\phi}(z)\widetilde{\mathcal{Y}}_{11}(z;n).$

So by Plemelj–Sokhotskii formula we have

$\displaystyle\mathcal{Y}_{12}(z;n)=\frac{1}{2\pi{\rm
i}}\int_{{\mathbb{T}}}\frac{\xi^{-1+s}\tilde{w}(\xi)\mathcal{Y}_{11}(\xi;n)+\xi^{-1+r}\tilde{\phi}(\xi)\widetilde{\mathcal{Y}}_{11}(\xi;n)}{\xi-z}\,{\rm
d}\xi.$

Using the identity

$\displaystyle\frac{1}{\xi-z}=-\sum_{k=0}^{n}\frac{\xi^{k}}{z^{k+1}}+\frac{\xi^{n+1}}{(\xi-z)z^{n+1}},$

we get

$\displaystyle\mathcal{Y}_{12}(z;n)=-\sum_{k=0}^{n}\frac{1}{z^{k+1}}\int_{{\mathbb{T}}}\big{[}\xi^{-1+s}\tilde{w}(\xi)\mathcal{Y}_{11}(\xi;n)\xi^{k}+\xi^{-1+r}\tilde{\phi}(\xi)\widetilde{\mathcal{Y}}_{11}(\xi;n)\xi^{k}\big{]}\frac{{\rm
d}\xi}{2\pi{\rm i}}$
$\displaystyle\phantom{\mathcal{Y}_{12}(z;n)={}}+\frac{1}{z^{n+1}}\int_{{\mathbb{T}}}\frac{\xi^{n+1}}{(\xi-z)}\big{[}\xi^{-1+s}\tilde{w}(\xi)\mathcal{Y}_{11}(\xi;n)+\xi^{-1+r}\tilde{\phi}(\xi)\widetilde{\mathcal{Y}}_{11}(\xi;n)\big{]}\frac{{\rm
d}\xi}{2\pi{\rm i}}.$

Note that since $\mathcal{Y}_{12}(z;n)=O(z^{-n-1})$, we must have:

$\displaystyle\int_{{\mathbb{T}}}\tilde{w}(\xi)\mathcal{Y}_{11}(\xi;n)\xi^{k+s}\frac{{\rm
d}\xi}{2\pi{\rm
i}\xi}+\int_{{\mathbb{T}}}\tilde{\phi}(\xi)\widetilde{\mathcal{Y}}_{11}(\xi;n)\xi^{k+r}\frac{{\rm
d}\xi}{2\pi{\rm i}\xi}=0,\qquad 0\leq k\leq n-1.$

In the second integral we make the change of variable
$\xi\mapsto\tau:=\xi^{-1}$ and as a result we will arrive at

$\displaystyle\int_{{\mathbb{T}}}\mathcal{Y}_{11}(\xi;n)\xi^{k+s}\tilde{w}(\xi)\frac{{\rm
d}\xi}{2\pi{\rm
i}\xi}+\int_{{\mathbb{T}}}\mathcal{Y}_{11}(\tau;n)\tau^{-k-r}\phi(\tau)\frac{{\rm
d}\tau}{2\pi{\rm i}\tau}=0,\qquad 0\leq k\leq n-1.$

Since $\mathcal{Y}_{11}$ satisfies the orthogonality relations (2.1) we
necessarily have

$\displaystyle\mathcal{Y}_{11}(z;n)=\mathcal{P}_{n}(z).$

In a similar fashion one can show that

$\displaystyle\mathcal{Y}_{22}(z;n)=\frac{1}{2\pi{\rm
i}}\int_{{\mathbb{T}}}\frac{\xi^{-1+s}\tilde{w}(\xi)\mathcal{Y}_{21}(\xi;n)+\xi^{-1+r}\tilde{\phi}(\xi)\widetilde{\mathcal{Y}}_{21}(\xi;n)}{\xi-z}{\rm
d}\xi.$

The asymptotic condition,
$\mathcal{Y}_{22}(z;n)=z^{-n}+O\big{(}z^{-n-1}\big{)}$, would then yield the
orthogonality relations,

$\displaystyle\int_{{\mathbb{T}}}\mathcal{Y}_{21}(\xi;n)\xi^{k+s}\tilde{w}(\xi)\frac{{\rm
d}\xi}{2\pi{\rm
i}\xi}+\int_{{\mathbb{T}}}\mathcal{Y}_{21}(\tau;n)\tau^{-k-r}\phi(\tau)\frac{{\rm
d}\tau}{2\pi{\rm i}\tau}=-\delta_{k,n-1},\qquad 0\leq k\leq n-1.$

These relations in turn are equivalent to the following linear system for the
coefficients $\beta_{j}$,

$\displaystyle(T_{n}[\phi;r]+H_{n}[w;s])\begin{pmatrix}\beta_{0}\\\
\beta_{1}\\\ \vdots\\\ \beta_{n-1}\end{pmatrix}=\begin{pmatrix}0\\\ 0\\\
\vdots\\\ -1\end{pmatrix}\\!.$

From this it follows that $\beta_{n-1}$ is necessarily not zero. Indeed, using
Cramer’s rule, we arrive at the formula,

$\displaystyle\beta_{n-1}=-\frac{D_{n-1}}{D_{n}}\neq 0.$

This would also mean that $\beta_{n-1}=-\frac{1}{h_{n-1}}$ and

$\displaystyle\mathcal{Y}_{21}(z;n)=-\frac{1}{h_{n-1}}\mathcal{P}_{n-1}(z).$
(2.10)

This finishes the argument why (2.5) uniquely satisfies both RH-$\mathcal{Y}$2
and RH-$\mathcal{Y}$3 as well. Also, equation (2.10) implies equation (2.9).

To prove the second statement, now assume that the Riemann–Hilbert problem
RH-$\mathcal{Y}$1 through RH-$\mathcal{Y}$3 has a unique solution. Because of
similar considerations mentioned above, it can be written as

$\displaystyle\mathcal{Y}(z;n)=\begin{pmatrix}\mathcal{Q}_{n}(z)&\displaystyle\int_{{\mathbb{T}}}\frac{\xi^{s}\tilde{w}(\xi)\mathcal{Q}_{n}(\xi)+\xi^{r}\tilde{\phi}(\xi)\tilde{\mathcal{Q}}_{n}(\xi)}{\xi-z}\frac{{\rm
d}\xi}{2\pi{\rm i}\xi}\\\\[4.30554pt]
\mathcal{R}_{n-1}(z)&\displaystyle\int_{{\mathbb{T}}}\frac{\xi^{s}\tilde{w}(\xi)\mathcal{R}_{n-1}(\xi)+\xi^{r}\tilde{\phi}(\xi)\tilde{\mathcal{R}}_{n-1}(\xi)}{\xi-z}\frac{{\rm
d}\xi}{2\pi{\rm i}\xi}\end{pmatrix}\\!,$ (2.11)

where $\mathcal{Q}_{n}(z)$ and $\mathcal{R}_{m}(z)$ are polynomials,

$\displaystyle\mathcal{Q}_{n}(z)=z^{n}+\sum_{k=0}^{n-1}\alpha_{k}z^{k},\qquad\mathcal{R}_{n-1}(z)=\sum_{k=0}^{n-1}\beta_{k}z^{k},$

satisfying the orthogonality conditions,

$\displaystyle\int_{{\mathbb{T}}}\mathcal{Q}_{n}(z)z^{-k-r}\phi(z)\frac{{\rm
d}z}{2\pi{\rm
i}z}+\int_{{\mathbb{T}}}\mathcal{Q}_{n}(z)z^{k+s}\tilde{w}(z)\frac{{\rm
d}z}{2\pi{\rm i}z}=0,\qquad k=0,1,\dots,n-1,$ (2.12)

and

$\displaystyle\int_{{\mathbb{T}}}\mathcal{R}_{n-1}(z)z^{-k-r}\phi(z)\frac{{\rm
d}z}{2\pi{\rm
i}z}+\int_{{\mathbb{T}}}\mathcal{R}_{n-1}(z)z^{k+s}\tilde{w}(z)\frac{{\rm
d}z}{2\pi{\rm i}z}=-\delta_{k,n-1},$ (2.13) $\displaystyle k=0,1,\dots,n-1.$

Note that the above equations can be written as the following linear systems
on the coefficients $\alpha_{j}$ and $\beta_{j}$:

$\displaystyle(T_{n}[\phi;r]+H_{n}[w;s])\begin{pmatrix}\alpha_{0}\\\
\alpha_{1}\\\ \vdots\\\
\alpha_{n-1}\end{pmatrix}=\begin{pmatrix}-\phi_{-n+r}-w_{n+s}\\\
-\phi_{1-n+r}-w_{1+n+s}\\\ \vdots\\\ -\phi_{-1+r}-w_{2n-1+s}\end{pmatrix}\\!,$
(2.14)

and

$\displaystyle(T_{n}[\phi;r]+H_{n}[w;s])\begin{pmatrix}\beta_{0}\\\
\beta_{1}\\\ \vdots\\\ \beta_{n-1}\end{pmatrix}=\begin{pmatrix}0\\\ 0\\\
\vdots\\\ -1\end{pmatrix}\\!.$ (2.15)

Note that since we have assumed that the solution of $\mathcal{Y}$-RHP exists,
both linear systems (2.14) and (2.15) must have solutions; and since the
solution of $\mathcal{Y}$-RHP is unique, it follows that the determinant of
the systems (2.14) and (2.15), which is nothing but $D_{n}$, is nonzero. Thus,
we certainly have that

$\displaystyle\mathcal{Q}_{n}(z)=\mathcal{P}_{n}(z).$

It is now obvious that $\mbox{rank}\,(T_{n-1}[\phi;r]+H_{n-1}[w;s])\geq n-2$.
Because if $D_{n-1}=0$ and $D_{n}\neq 0$, one can easily show that
$\mbox{rank}\,(T_{n-1}[\phi;r]+H_{n-1}[w;s])=n-2$. Although the proof of the
second statement is complete, it is worthwhile to say more about
$\mathcal{Y}_{21}$ when $D_{n-1}=0$, because then the objects
$\mathcal{P}_{n-1}(z)$ and $h_{n-1}$ do not exist. Indeed the homogenous
system,

$\displaystyle(T_{n-1}[\phi;r]+H_{n-1}[w;s])\begin{pmatrix}c_{0}\\\ c_{1}\\\
\vdots\\\ c_{n-2}\end{pmatrix}=0,$ (2.16)

has infinitely many nontrivial solutions. Let us take one of them and define
the following polynomial of degree $n-2$

$\displaystyle\hat{\mathcal{R}}_{n-2}(z)=\sum_{j=0}^{n-2}c_{j}z^{j},$

which automatically satisfies

$\displaystyle\int_{{\mathbb{T}}}\hat{\mathcal{R}}_{n-2}(z)z^{-k-r}\phi(z)\frac{{\rm
d}z}{2\pi{\rm
i}z}+\int_{{\mathbb{T}}}\hat{\mathcal{R}}_{n-2}(z)z^{k+s}\tilde{w}(z)\frac{{\rm
d}z}{2\pi{\rm i}z}=0,\qquad k=0,1,\dots,n-2.$

Put

$\displaystyle\delta_{n-1}:=\int_{{\mathbb{T}}}\hat{\mathcal{R}}_{n-2}(z)z^{-n+1-r}\phi(z)\frac{{\rm
d}z}{2\pi{\rm
i}z}+\int_{{\mathbb{T}}}\hat{\mathcal{R}}_{n-2}(z)z^{n-1+s}\tilde{w}(z)\frac{{\rm
d}z}{2\pi{\rm i}z}.$

Note that $\delta_{n-1}\neq 0$. Otherwise, we could redefine
$\mathcal{R}_{n-1}(z)$ as

$\displaystyle\mathcal{R}_{n-1}(z)\to\mathcal{R}_{n-1}(z)+\hat{\mathcal{R}}_{n-2}(z),$

which, on the contrary, means that the solution of the $\mathcal{Y}$-RHP is
not unique. Hence, we can find a unique normalization of the nontrivial
solution of (2.16), to assure that

$\displaystyle\delta_{n-1}=-1.$

This, together with the uniqueness of the solution of the $\mathcal{Y}$-RHP
implies that $\mathcal{R}_{n-1}(z)=\hat{\mathcal{R}}_{n-2}(z)$.

To prove the third statement, we notice that, due to the additional condition
at $z=\infty$, the unique solution of the Riemann–Hilbert problem
RH-$\mathcal{Y}$1 through RH-$\mathcal{Y}$3 can be written in the same form
(2.11) with polynomial $\mathcal{R}_{n-1}(z)$ replaced by

$\displaystyle\mathcal{R}_{n-1}(z)=q_{n}\mathcal{Q}_{n-1}(z),\qquad q\neq 0,$

where both $\mathcal{Q}_{n}(z)$ and $\mathcal{Q}_{n-1}(z)$ are monic
polynomials,

$\displaystyle\mathcal{Q}_{n}(z)=z^{n}+\sum_{k=0}^{n-1}\alpha_{k}z^{k},\qquad\mathcal{Q}_{n-1}(z)=z^{n-1}+\sum_{k=0}^{n-2}\beta_{k}z^{k},$

satisfying the orthogonality conditions,

$\displaystyle\int_{{\mathbb{T}}}\mathcal{Q}_{n}(z)z^{-k-r}\phi(z)\frac{{\rm
d}z}{2\pi{\rm
i}z}+\int_{{\mathbb{T}}}\mathcal{Q}_{n}(z)z^{k+s}\tilde{w}(z)\frac{{\rm
d}z}{2\pi{\rm i}z}=0,\qquad k=0,1,\dots,n-1,$

and

$\displaystyle
q_{n}\Biggl{(}\int_{{\mathbb{T}}}\mathcal{Q}_{n-1}(z)z^{-k-r}\phi(z)\frac{dz}{2\pi{\rm
i}z}+\int_{{\mathbb{T}}}\mathcal{Q}_{n-1}(z)z^{k+s}\tilde{w}(z)\frac{dz}{2\pi{\rm
i}z}\Biggr{)}=-\delta_{k,n-1},$ (2.17) $\displaystyle k=0,1,\dots,n-1.$

As before with equations (2.12) and (2.13), the above equations can be written
as the following linear systems on the coefficients $\alpha_{j}$ and
$\beta_{j}$:

$\displaystyle(T_{n}[\phi;r]+H_{n}[w;s])\begin{pmatrix}\alpha_{0}\\\
\alpha_{1}\\\ \vdots\\\
\alpha_{n-1}\end{pmatrix}=\begin{pmatrix}-\phi_{-n+r}-w_{n+s}\\\
-\phi_{1-n+r}-w_{1+n+s}\\\ \vdots\\\ -\phi_{-1+r}-w_{2n-1+s}\end{pmatrix}\\!,$

and

$\displaystyle(T_{n-1}[\phi;r]+H_{n-1}[w;s])\begin{pmatrix}\beta_{0}\\\
\beta_{1}\\\ \vdots\\\
\beta_{n-2}\end{pmatrix}=\begin{pmatrix}-\phi_{-n+1+r}-w_{n-1+s}\\\
-\phi_{1-n+1+r}-w_{1+n-1+s}\\\ \vdots\\\
-\phi_{-1+r}-w_{2n-3+s}\end{pmatrix}\\!.$

We now argue, that both $D_{n}$ and $D_{n-1}$ are nonzero and thus
$\mathcal{Q}_{n}=\mathcal{P}_{n}$ and $\mathcal{Q}_{n-1}=\mathcal{P}_{n-1}$
are given by (2.2). Otherwise, we shall have more than one solution for one or
both of the above linear systems, which means that we could find distinct
polynomials $\mathcal{Q}_{n}(z)$ (if $D_{n}=0$), or distinct polynomials
$\mathcal{Q}_{n-1}(z)$ (if $D_{n-1}=0$), or both (if $D_{n},D_{n-1}=0$).
Either way, using these distinct polynomials, we could construct distinct
solutions of the $\mathcal{Y}$-RHP which contradicts our assumption. This
finishes the proof of the theorem. Note that from (2.17) we necessarily have
$q_{n}\neq 0$, and since $\mathcal{Q}_{n-1}=\mathcal{P}_{n-1}$ we conclude
that $h_{n-1}=-1/q_{n}\neq 0,$ which could also be seen from
$h_{n-1}=D_{n}/D_{n-1}$. ∎

######  Corollary 2.2.

Suppose that the $\mathcal{Y}$-RH problem has a unique solution for $n$ and
$n-1$. Then

$\displaystyle D_{n}\neq 0,\qquad D_{n-1}\neq 0,\qquad\mbox{and}\qquad
h_{n-1}\neq 0,$

where $h_{n-1}$ can be reconstructed form the RHP data as

$\displaystyle h_{n-1}=-\lim_{z\to\infty}z^{n-1}/\mathcal{Y}_{21}(z;n).$
(2.18)

Equation (2.18), as in the pure Toeplitz or Hankel cases, in conjunction with
(2.4) reduces the asymptotic analysis of the Toeplitz+Hankel determinants to
the asymptotic analysis of the Riemann–Hilbert problem for $\mathcal{Y}$.

###  2.1 The associated $\boldsymbol{2\times 4}$ and $\boldsymbol{4\times 4}$
Riemann–Hilbert problems

In the rest of this section we will develop a $4\times 4$ analogue of the
Deift/Zhou non-linear steepest descent method for the Toeplitz+Hankel
determinants (1.3). The $\mathcal{Y}$-RHP is a particular case of the matrix
Riemann–Hilbert problem with a shift, or the matrix analytical boundary
problem of the Carleman type. Indeed, the matrix form of the equations
(2.6)–(2.7) reads as follows

$\displaystyle\mathcal{Y}_{+}(z;n)=\mathcal{Y}_{-}(z;n)G_{1}(z)+\mathcal{Y}_{-}(\kappa(z);n)G_{2}(z),$

where

$\displaystyle G_{1}(z)=\begin{pmatrix}1&z^{s-1}\tilde{w}(z)\\\
0&1\end{pmatrix}\\!,\qquad G_{2}(z)=\begin{pmatrix}0&z^{r-1}\tilde{\phi}(z)\\\
0&0\end{pmatrix}\\!,$

and the “shift” $\kappa$ is the mapping

$\displaystyle\kappa(z)=\frac{1}{z}.$

The presence of the shift makes it impossible to directly apply the usual
$2\times 2$ version of the Deift–Zhou nonlinear steepest descent method to the
$\mathcal{Y}$-RHP. However, the mapping $\kappa$ satisfies the Carleman
condition, $\kappa(\kappa(z))=z$, and hence we can translate the $2\times 2$
$\mathcal{Y}$-RHP to the usual matrix form by doubling the relevant matrix
sizes. More precisely, we first propose the associated $2\times 4$ and then
the associated $4\times 4$ Riemann–Hilbert problems. Although more
complicated, the analysis of the proposed $4\times 4$ Riemann–Hilbert problem
follows in the same spirit as the lower dimensional RHPs until we get to the
model Riemann–Hilbert problem for Toeplitz+Hankel determinants introduced in
Section 2.6.

Let us define the $2\times 4$ matrix $\overset{\circ}{\mathcal{X}}$ out of the
columns of $\mathcal{Y}$ as follows

$\displaystyle\overset{\circ}{\mathcal{X}}(z;n):=\bigl{(}\mathcal{Y}^{(1)}(z;n),\widetilde{\mathcal{Y}}^{(1)}(z;n),\mathcal{Y}^{(2)}(z;n),\widetilde{\mathcal{Y}}^{(2)}(z;n)\bigr{)}.$
(2.19)

From (2.6), (2.7) and (2.8) we obtain the following Riemann–Hilbert problem
for $\overset{\circ}{\mathcal{X}}$:

  * •

RH-$\overset{\circ}{\mathcal{X}}$1 $\overset{\circ}{\mathcal{X}}$ is
holomorphic in ${\mathbb{C}}\setminus\left({\mathbb{T}}\cup\\{0\\}\right)$.

  * •

RH-$\overset{\circ}{\mathcal{X}}$2 For $z\in{\mathbb{T}}$,
$\overset{\circ}{\mathcal{X}}$ satisfies

$\displaystyle\overset{\circ}{\mathcal{X}}_{+}(z;n)=\overset{\circ}{\mathcal{X}}_{-}(z;n)\begin{pmatrix}1&0&z^{s-1}\tilde{w}(z)&-z^{-r+1}\phi(z)\\\
0&1&z^{r-1}\tilde{\phi}(z)&-z^{-s+1}w(z)\\\ 0&0&1&0\\\
0&0&0&1\end{pmatrix}\\!.$ (2.20)

  * •

RH-$\overset{\circ}{\mathcal{X}}$3 As $z\to\infty$ we have

$\displaystyle\overset{\circ}{\mathcal{X}}(z;n)=\begin{pmatrix}1+\mathcal{O}\big{(}z^{-1}\big{)}&C_{1}(n)+\mathcal{O}\big{(}z^{-1}\big{)}&\mathcal{O}\big{(}z^{-1}\big{)}&C_{3}(n)+\mathcal{O}\big{(}z^{-1}\big{)}\\\
\mathcal{O}\big{(}z^{-1}\big{)}&C_{2}(n)+\mathcal{O}\big{(}z^{-1}\big{)}&1+\mathcal{O}\big{(}z^{-1}\big{)}&C_{4}(n)+\mathcal{O}\big{(}z^{-1}\big{)}\end{pmatrix}$
$\displaystyle\phantom{\overset{\circ}{\mathcal{X}}(z;n)=}{}\times\begin{pmatrix}z^{n}&0&0&0\\\
0&1&0&0\\\ 0&0&z^{-n}&0\\\ 0&0&0&1\end{pmatrix}\\!.$ (2.21)

  * •

RH-$\overset{\circ}{\mathcal{X}}$4 As $z\to 0$ we have

$\displaystyle\overset{\circ}{\mathcal{X}}(z;n)=\begin{pmatrix}C_{1}(n)+\mathcal{O}(z)&1+\mathcal{O}(z)&C_{3}(n)+\mathcal{O}(z)&\mathcal{O}(z)\\\
C_{2}(n)+\mathcal{O}(z)&\mathcal{O}(z)&C_{4}(n)+\mathcal{O}(z)&1+\mathcal{O}(z)\end{pmatrix}$
$\displaystyle\phantom{\overset{\circ}{\mathcal{X}}(z;n)=}{}\times\begin{pmatrix}1&0&0&0\\\
0&z^{-n}&0&0\\\ 0&0&1&0\\\ 0&0&0&z^{n}\end{pmatrix}\\!,$ (2.22)

where

$\displaystyle C_{1}(n)=\mathcal{Y}_{11}(0;n),\qquad
C_{3}(n)=\mathcal{Y}_{12}(0;n),\qquad C_{2}(n)=\mathcal{Y}_{21}(0;n),\qquad
C_{4}(n)=\mathcal{Y}_{22}(0;n).$

######  Remark 2.3.

It is well-known that for construction of the global parametrix in the
Riemann–Hilbert analysis, one has to construct the so-called Szegő functions
of the following type:

$\displaystyle\mathcal{S}_{f}(z):=\exp\Bigg{[}\frac{1}{2\pi{\rm
i}}\int_{{\mathbb{T}}}\frac{\ln(f(\tau))}{\tau-z}{\rm d}\tau\Bigg{]},$

where one assumes that $f$ is non-zero and continuous on the unit circle with
zero winding number. The natural progression of the Riemann–Hilbert analysis
with general offset values $r,s\in{\mathbb{Z}}$, leads us to constructing
Szegő functions for $f_{1}(z)=z^{1-r}\phi(z)$ and $f_{2}(z)=z^{1-s}w(z)$ (or,
equivalently, for $f_{1}$ and $f_{2}/f_{1}$). Although there are ways to “peel
off” the winding generating parts of $f_{1}$ and $f_{2}$ (see Section 5.2).
The first natural case to consider is when $f_{1}$ and $f_{2}$ have zero
winding numbers. To that end, in this work we shall only consider the case
$r=s=1$ and the symbols $\phi$ and $w$ which are of Szegő type and analytic in
a neighborhood of the unit circle. We also discuss prospects of extension of
our method to symbols with Fisher–Hartwig singularities in Section 5.4. Also
it is worth mentioning that our method in the present work trivially extends
to the case where we consider general, but fixed, $r,s\in{\mathbb{Z}}$ with
symbols $\phi$ and $w$ respectively with winding numbers $r-1$ and $s-1$ (so
that the overall winding number of $f_{1}$ and $f_{2}$ are zero, which is the
main condition to be considered). It should be also mentioned that in the pure
Toeplitz case there is another, and historically the first, way to put the
analysis of Toeplitz determinants in the framework of the Riemann–Hilbert
scheme. This approach was suggested by Jinho Baik, Percy Deift, and Kurt
Johansson in [1]. The Riemann–Hilbert problem that appear there yields to the
Szegő function for $f(z)$ coinciding with the original Toeplitz symbol
$\phi(z)$. Therefore, the Riemann–Hilbert method of [1] is applied directly to
the symbols with zero winding numbers. The crux of the matter is that the
construction of [1] can not be extended to the mixed, T+H, situation. The
$\mathcal{Y}$-RHP we are introducing in this paper seems to be the only way to
put the mixed problem into the Riemann–Hilbert setting; hence the
inevitability of the choice $r=s=1$, or equivalently, starting with symbols
that have nonzero winding numbers.

In view of this remark, in the rest of the paper we assume that $r=s=1$. In a
natural way we now consider the following $4\times 4$ Riemann–Hilbert problem
which we will refer to as the $\mathcal{X}$-RHP.

  * •

RH-$\mathcal{X}$1 $\mathcal{X}$ is holomorphic in
${\mathbb{C}}\setminus\left({\mathbb{T}}\cup\\{0\\}\right)$.

  * •

RH-$\mathcal{X}$2 For $z\in{\mathbb{T}}$, $\mathcal{X}$ satisfies

$\displaystyle\mathcal{X}_{+}(z;n)=\mathcal{X}_{-}(z;n)\begin{pmatrix}1&0&\tilde{w}(z)&-\phi(z)\\\
0&1&\tilde{\phi}(z)&-w(z)\\\ 0&0&1&0\\\ 0&0&0&1\end{pmatrix}\\!.$ (2.23)

  * •

RH-$\mathcal{X}$3 As $z\to\infty$ we have

$\displaystyle\mathcal{X}(z;n)=\big{(}\displaystyle
I+\mathcal{O}\big{(}z^{-1}\big{)}\big{)}\begin{pmatrix}z^{n}&0&0&0\\\
0&1&0&0\\\ 0&0&z^{-n}&0\\\ 0&0&0&1\end{pmatrix}\\!.$ (2.24)

  * •

RH-$\mathcal{X}$4 As $z\to 0$ we have

$\displaystyle\mathcal{X}(z;n)=P(n)(I+\mathcal{O}(z))\begin{pmatrix}1&0&0&0\\\
0&z^{-n}&0&0\\\ 0&0&1&0\\\ 0&0&0&z^{n}\end{pmatrix}\\!.$ (2.25)

######  Remark 2.4.

The uniqueness of the solution of $\mathcal{X}$-RHP is established using the
standard Liouville theorem-based arguments. We also note that the matrix
factor $P(n)$ in (2.25) is not a priori prescribed.

######  Remark 2.5.

It is easy to see that the solution $\mathcal{X}$ of the $\mathcal{X}$-RHP
satisfies the symmetry relation,

$\displaystyle WP^{-1}(n)\mathcal{X}\big{(}z^{-1};n\big{)}W=\mathcal{X}(z;n),$
(2.26)

where

$\displaystyle W=\begin{pmatrix}0&1&0&0\\\ 1&0&0&0\\\ 0&0&0&1\\\
0&0&1&0\end{pmatrix}\\!.$ (2.27)

Equation (2.26) in turn yields the following symmetry equation for $P(n)$,

$\displaystyle P(n)=WP^{-1}(n)W,$ (2.28)

or (taking into account that $W^{-1}=W$),

$\displaystyle\bigl{(}WP(n)\bigr{)}^{2}=\bigl{(}P(n)W\bigr{)}^{2}=I_{4}.$

######  Remark 2.6.

The matrix $P(n)W-I_{4}$ has rank 2. Here is the proof of this statement which
is motivated by some of the referee’s remarks.333These valuable remarks also
led us to formulate and prove Lemmas 2.7 and 2.8, and their analogues in
Section 3.

If we take the limit $z\to 1$, with $|z|>1$ in (2.26) it follows that

$\displaystyle WP^{-1}(n)\mathcal{X}_{+}(1;n)W=\mathcal{X}_{-}(1;n),$

or

$\displaystyle\mathcal{X}_{+}(1;n)W\mathcal{X}^{-1}_{-}(1;n)=P(n)W.$

Using (2.23), the last equation reads,

$\displaystyle\mathcal{X}_{-}(1;n)G_{\mathcal{X}}(1)W\mathcal{X}^{-1}_{-}(1;n)=P(n)W,$

where $G_{\mathcal{X}}(z)$ is the jump matrix in (2.23). Hence the matrices
$P(n)W$ and $G_{\mathcal{X}}(1)W$ are similar, and thus the claim is true if
the matrix $G_{\mathcal{X}}(1)W$ has rank $2$. We have,

$\displaystyle G_{\mathcal{X}}(1)W-I_{4}=\begin{pmatrix}-1&1&-\phi&w\\\
1&-1&-w&\phi\\\ 0&0&-1&1\\\ 0&0&1&-1\end{pmatrix}\\!,$

where $\phi\equiv\phi(1)=\tilde{\phi}(1)$, and $w\equiv w(1)=\tilde{w}(1)$.
The row echelon form of this matrix is

$\displaystyle\begin{pmatrix}-1&1&-\phi&w\\\ 0&0&-1&1\\\ 0&0&0&0\\\
0&0&0&0\end{pmatrix}\\!,$

and the statement about the rank of matrix $G_{\mathcal{X}}(1)W-I_{4}$ (and
hence the rank of matrix $P(n)W-I_{4}$) follows.

###  2.2 Relation of the $\boldsymbol{2\times 4}$ and the $\boldsymbol{4\times
4}$ Riemann–Hilbert problems

Put

$\displaystyle\mathfrak{R}(z;n):=\overset{\circ}{\mathcal{X}}(z;n)\mathcal{X}^{-1}(z;n).$
(2.29)

From (2.20) and (2.23) it is clear that $\mathfrak{R}$ has no jumps. From
(2.22) and (2.25) we can obtain the behavior of $\mathfrak{R}$ near zero:

$\displaystyle\mathfrak{R}(z;n)=\begin{pmatrix}C_{1}(n)+\mathcal{O}(z)&1+\mathcal{O}(z)&C_{3}(n)+\mathcal{O}(z)&\mathcal{O}(z)\\\
C_{2}(n)+\mathcal{O}(z)&\mathcal{O}(z)&C_{4}(n)+\mathcal{O}(z)&1+\mathcal{O}(z)\end{pmatrix}P^{-1}(n).$

Therefore $\mathfrak{R}$ is an entire function. Also note that from (2.21) and
(2.24) we have

$\displaystyle\mathfrak{R}(z;n)=\begin{pmatrix}1+\mathcal{O}\big{(}z^{-1}\big{)}&C_{1}(n)+\mathcal{O}\big{(}z^{-1}\big{)}&\mathcal{O}\big{(}z^{-1}\big{)}&C_{3}(n)+\mathcal{O}\big{(}z^{-1}\big{)}\\\
\mathcal{O}\big{(}z^{-1}\big{)}&C_{2}(n)+\mathcal{O}\big{(}z^{-1}\big{)}&1+\mathcal{O}\big{(}z^{-1}\big{)}&C_{4}(n)+\mathcal{O}\big{(}z^{-1}\big{)}\end{pmatrix}\\!,\qquad
z\to\infty.$

Therefore by Liouville’s theorem we conclude that

$\displaystyle\mathfrak{R}(z;n)=\begin{pmatrix}1&C_{1}(n)&0&C_{3}(n)\\\
0&C_{2}(n)&1&C_{4}(n)\end{pmatrix}\\!.$ (2.30)

And therefore we have

$\displaystyle\begin{pmatrix}1&C_{1}(n)&0&C_{3}(n)\\\
0&C_{2}(n)&1&C_{4}(n)\end{pmatrix}=\begin{pmatrix}C_{1}(n)&1&C_{3}(n)&0\\\
C_{2}(n)&0&C_{4}(n)&1\end{pmatrix}P^{-1}(n).$ (2.31)

We argue that this system, under certain generic assumptions, is a well-
defined linear system on $C_{j}(n)$ which is uniquely solvable. To see this,
let us first double the system, that is consider instead of (2.31), the system

$\displaystyle\begin{pmatrix}1&C_{1}(n)&0&C_{3}(n)\\\
0&C_{2}(n)&1&C_{4}(n)\end{pmatrix}=\begin{pmatrix}C^{\prime}_{1}(n)&1&C^{\prime}_{3}(n)&0\\\
C^{\prime}_{2}(n)&0&C^{\prime}_{4}(n)&1\end{pmatrix}P^{-1}(n).$ (2.32)

This is an $8\times 8$ system of linear equations for eight unknowns –
$C_{j}(n)$ and $C^{\prime}_{j}(n)$. By a straightforward and, in fact, rather
simple calculations one finds that the determinant of this $8\times 8$ system
is

$\displaystyle\big{(}P_{22}(n)P_{44}(n)-P_{42}(n)P_{24}(n)\big{)}^{2},$

where $P_{jk}(n)$, $j,k=1,...,4$ denote the entries of matrix $P(n)$. Hence,
assuming that

$\displaystyle P_{22}(n)P_{44}(n)-P_{42}(n)P_{24}(n)\neq 0,$ (2.33)

we would have the unique solvability of system (2.32). That is, there is only
one 8-vector,

$\displaystyle\vec{C}(n):=\big{(}C_{1}(n),C_{2}(n),C_{3}(n),C_{4}(n),C^{\prime}_{1}(n),C^{\prime}_{2}(n),C^{\prime}_{3}(n),C^{\prime}_{4}(n)\big{)},$
(2.34)

which solves (2.32). At the same time, the symmetry relation (2.28) would
imply that

$\displaystyle\begin{pmatrix}1&C_{1}(n)&0&C_{3}(n)\\\
0&C_{2}(n)&1&C_{4}(n)\end{pmatrix}W=\begin{pmatrix}C^{\prime}_{1}(n)&1&C^{\prime}_{3}(n)&0\\\
C^{\prime}_{2}(n)&0&C^{\prime}_{4}(n)&1\end{pmatrix}WP(n),$

or

$\displaystyle\begin{pmatrix}1&C^{\prime}_{1}(n)&0&C^{\prime}_{3}(n)\\\
0&C^{\prime}_{2}(n)&1&C^{\prime}_{4}(n)\end{pmatrix}=\begin{pmatrix}C_{1}(n)&1&C_{3}(n)&0\\\
C_{2}(n)&0&C_{4}(n)&1\end{pmatrix}P^{-1}(n).$

In other words, together with (2.34), the system (2.32) will be also solved by
the vector

$\displaystyle\vec{C}^{\prime}(n):=\big{(}C^{\prime}_{1}(n),C^{\prime}_{2}(n),C^{\prime}_{3}(n),C^{\prime}_{4}(n),C_{1}(n),C_{2}(n),C_{3}(n),C_{4}(n)\big{)}.$

Because of the uniqueness, we conclude that the vectors $\vec{C}^{\prime}(n)$
and $\vec{C}(n)$ must coincide and hence we must have,

$\displaystyle C_{j}(n)=C^{\prime}_{j}(n),\qquad j=1,2,3,4.$

Therefore we have the unique solvability of the original system (2.31) under
the generic condition (2.33).

Indeed, we can make the solvability condition of the system (2.31) more
flexible. This is shown in the next lemma.

######  Lemma 2.7.

Assume that at least one of the following six inequalities is true,

$\displaystyle P_{22}(n)P_{44}(n)-P_{42}(n)P_{24}(n)\neq 0,$ (2.35)
$\displaystyle(1-P_{21}(n))P_{42}(n)+P_{22}(n)P_{41}(n)\neq 0,$ (2.36)
$\displaystyle(1-P_{43}(n))P_{22}(n)+P_{23}(n)P_{42}(n)\neq 0,$ (2.37)
$\displaystyle(1-P_{21}(n))P_{44}(n)+P_{41}(n)P_{24}(n)\neq 0,$ (2.38)
$\displaystyle(1-P_{21}(n))(P_{43}(n)-1)+P_{41}(n)P_{23}(n)\neq 0,$ (2.39)
$\displaystyle(1-P_{43}(n))P_{24}(n)+P_{23}(n)P_{44}(n)\neq 0.$ (2.40)

Then, the system (2.31) is a well-defined linear system on $C_{j}(n)$ which is
uniquely solvable.

###### Proof.

We first notice that system (2.31) can be rewritten as

$\displaystyle\begin{pmatrix}1&C_{1}(n)&0&C_{3}(n)\\\
0&C_{2}(n)&1&C_{4}(n)\end{pmatrix}P(n)W=\begin{pmatrix}C_{1}(n)&1&C_{3}(n)&0\\\
C_{2}(n)&0&C_{4}(n)&1\end{pmatrix}W=\begin{pmatrix}1&C_{1}(n)&0&C_{3}(n)\\\
0&C_{2}(n)&1&C_{4}(n)\end{pmatrix}\\!,$

or

$\displaystyle\begin{pmatrix}1&C_{1}(n)&0&C_{3}(n)\\\
0&C_{2}(n)&1&C_{4}(n)\end{pmatrix}(P(n)W-I_{4})=0.$ (2.41)

Notice that

$\displaystyle
P(n)W-I_{4}=\begin{pmatrix}P_{12}(n)-1&P_{11}(n)&P_{14}(n)&P_{13}(n)\\\
P_{22}(n)&P_{21}(n)-1&P_{24}(n)&P_{23}(n)\\\
P_{32}(n)&P_{31}(n)&P_{34}(n)-1&P_{33}(n)\\\
P_{42}(n)&P_{41}(n)&P_{44}(n)&P_{43}(n)-1\\\ \end{pmatrix}\\!.$

Therefore, the conditions of this lemma mean that the second and the forth
rows of $P(n)W-I_{4}$ are linearly independent. This, in conjunction with
Remark 2.6, implies that the other two rows are linear combinations of the
second and the fourth rows. In other words, there exists a unique collection
of four numbers, $C_{1}(n)$, $C_{2}(n)$, $C_{3}(n)$, $C_{4}(n)$ such that

$\displaystyle\bigl{(}P_{12}(n)-1,P_{11}(n),P_{14}(n),P_{13}(n)\bigr{)}=-C_{1}(n)\bigl{(}P_{22}(n),P_{21}(n)-1,P_{24}(n),P_{23}(n)\bigr{)}$
$\displaystyle\qquad{}-C_{3}(n)\bigl{(}P_{42}(n),P_{41}(n),P_{44}(n),P_{43}(n)-1\bigr{)},$
(2.42)

and

$\displaystyle\bigl{(}P_{32}(n),P_{31}(n),P_{34}(n)-1,P_{33}(n)\bigr{)}=-C_{2}(n)\bigl{(}P_{22}(n),P_{21}(n)-1,P_{24}(n),P_{23}(n)\bigr{)}$
$\displaystyle\qquad{}-C_{4}(n)\bigl{(}P_{42}(n),P_{41}(n),P_{44}(n),P_{43}(n)-1\bigr{)}.$
(2.43)

Vector equations (2.42), (2.43) are just the rows of matrix equation (2.41). ∎

Investigation of the possibility of linear independence of other rows of the
matrix $P(n)W-I_{4}$, leads us to the following observation about the
uniqueness of a solution to (2.31), if one exists.

######  Lemma 2.8.

If the system (2.31) has a solution, it has to be unique.

###### Proof.

For simplicity of notation let us drop the dependence on $n$ in $C_{j}(n)$ and
$P(n)$, and respectively write $C_{j}$ and $P$ instead. Denote the rows of the
matrix $PW-I_{4}$ by $\mathcal{R}_{j}$, $1\leq j\leq 4$, and thus the system
of equations (2.41) can be written as the following vector equations

$\displaystyle\mathcal{R}_{1}+C_{1}\mathcal{R}_{2}+C_{3}\mathcal{R}_{4}=0,$
(2.44)

and

$\displaystyle\mathcal{R}_{3}+C_{2}\mathcal{R}_{2}+C_{4}\mathcal{R}_{4}=0.$
(2.45)

Since $PW-I_{4}$ is of rank $2$, necessarily, at least one of the six pairs of
vectors has to be linearly independent. We consider each possibility
separately.

First, let us assume that $\mathcal{R}_{1}$ and $\mathcal{R}_{3}$ are linearly
independent. Note that $C_{1}C_{4}-C_{2}C_{3}\neq 0$, because otherwise,
multiplying (2.44) by $-C_{2}$ and (2.45) by $C_{1}$ and adding the results
yields that $\mathcal{R}_{1}$ and $\mathcal{R}_{3}$ are linearly dependent to
the contrary. There exist unique constants $a_{1}$, $a_{2}$, $b_{1}$, and
$b_{2}$ such that

$\displaystyle\mathcal{R}_{2}=a_{1}\mathcal{R}_{1}+a_{2}\mathcal{R}_{3},\qquad\mbox{and}\qquad\mathcal{R}_{4}=b_{1}\mathcal{R}_{1}+b_{2}\mathcal{R}_{3}.$
(2.46)

Also observe that, $a_{1}b_{2}-a_{2}b_{1}=0$ contradicts our assumption that
$\mathcal{R}_{1}$ and $\mathcal{R}_{3}$ are linearly independent. Indeed,
multiplying the first member and second member of (2.46) respectively by
$b_{2}$ and $-a_{2}$ and adding the results yields that $\mathcal{R}_{2}$ and
$\mathcal{R}_{4}$ are linearly dependent, and consequently the equations
(2.44) and (2.45) imply that $\mathcal{R}_{1}$ and $\mathcal{R}_{3}$ are
linearly dependent, a contradiction.

Solving the system (2.44)–(2.45) for $\mathcal{R}_{2}$ and $\mathcal{R}_{4}$
and comparing with (2.46) yields

$\displaystyle a_{1}=-\frac{C_{4}}{C_{1}C_{4}-C_{2}C_{3}},\qquad$
$\displaystyle a_{2}=\frac{C_{3}}{C_{1}C_{4}-C_{2}C_{3}},$ $\displaystyle
b_{1}=\frac{C_{2}}{C_{1}C_{4}-C_{2}C_{3}},\qquad$ $\displaystyle
b_{2}=-\frac{C_{1}}{C_{1}C_{4}-C_{2}C_{3}}.$

Using these equalities we find
$a_{1}b_{2}-a_{2}b_{1}=1/(C_{1}C_{4}-C_{2}C_{3})$, and since
$a_{1}b_{2}-a_{2}b_{1}\neq 0$, $C_{1}C_{4}-C_{2}C_{3}$ is uniquely expressed
in terms of $a_{1}b_{2}-a_{2}b_{1}$ and hence we can write the unique solution
as

$\displaystyle C_{1}=-\frac{b_{2}}{a_{1}b_{2}-a_{2}b_{1}},\qquad$
$\displaystyle C_{2}=\frac{b_{1}}{a_{1}b_{2}-a_{2}b_{1}},$ $\displaystyle
C_{3}=\frac{a_{2}}{a_{1}b_{2}-a_{2}b_{1}},\qquad$ $\displaystyle
C_{4}=-\frac{a_{1}}{a_{1}b_{2}-a_{2}b_{1}}.$

Next, assume that $\mathcal{R}_{1}$ and $\mathcal{R}_{2}$ are linearly
independent and thus from (2.44) we necessarily have

$\displaystyle C_{3}\neq 0.$

So there exist unique constants $a_{1}$, $a_{2}$, $b_{1}$, and $b_{2}$ such
that

$\displaystyle\mathcal{R}_{3}=a_{1}\mathcal{R}_{1}+a_{2}\mathcal{R}_{2},\qquad\mbox{and}\qquad\mathcal{R}_{4}=b_{1}\mathcal{R}_{1}+b_{2}\mathcal{R}_{2}.$
(2.47)

Note that $b_{1}=0$ contradicts our assumption that $\mathcal{R}_{1}$ and
$\mathcal{R}_{2}$ are linearly independent, because then
$\mathcal{R}_{4}=b_{2}\mathcal{R}_{2}$ and substitution into (2.44) yields
$\mathcal{R}_{1}=-(C_{1}+b_{2}C_{3})\mathcal{R}_{2}$, contrary to our
assumption. Solving the system (2.44)–(2.45) for $\mathcal{R}_{3}$ and
$\mathcal{R}_{4}$ and comparing with (2.47) yields the equalities:

$\displaystyle a_{1}=\frac{C_{4}}{C_{3}},\qquad$ $\displaystyle
a_{2}=\frac{C_{4}C_{1}-C_{2}C_{3}}{C_{3}},$ $\displaystyle
b_{1}=-\frac{1}{C_{3}},\qquad$ $\displaystyle b_{2}=-\frac{C_{1}}{C_{3}}.$

These relationships yield the unique solution for the system (2.44)–(2.45),
because $C_{3}$ is uniquely determined from $b_{1}$ (since $b_{1}\neq 0$),
then $C_{1}$ is uniquely determined from $C_{3}$ and $b_{2}$, and
simultaneously $C_{4}$ is uniquely determined from $C_{3}$ and $a_{1}$, and
finally $C_{2}$ is uniquely determined from $C_{3}$, $C_{1}$, $C_{4}$ and
$a_{2}$. Thus, we can write the unique solution as

$\displaystyle C_{1}=\frac{b_{2}}{b_{1}},\qquad$ $\displaystyle
C_{2}=\frac{a_{1}b_{2}-b_{1}a_{2}}{b_{1}},$ $\displaystyle
C_{3}=-\frac{1}{b_{1}},\qquad$ $\displaystyle C_{4}=-\frac{a_{1}}{b_{1}}.$

Among the four remaining cases,the argument for each of the following three:

  * •

$\mathcal{R}_{1}$ and $\mathcal{R}_{4}$ are linearly independent,

  * •

$\mathcal{R}_{2}$ and $\mathcal{R}_{3}$ are linearly independent, or

  * •

$\mathcal{R}_{3}$ and $\mathcal{R}_{4}$ are linearly independent,

is similar to the one presented above for linear independence of
$\mathcal{R}_{1}$ and $\mathcal{R}_{2}$ and thus we do not provide the details
here. Finally, for the case where $\mathcal{R}_{2}$ and $\mathcal{R}_{4}$ are
linearly independent, we refer to Lemma 2.7 where we prove the stronger
assertion that the system (2.31) is uniquely solvable. ∎

######  Lemma 2.9.

Suppose that the solution of the $\mathcal{X}$-RHP exists. Then, if at least
one of the conditions (2.35) through (2.40) holds, one can uniquely
reconstruct the solution of the $\mathcal{Y}$-RHP.

###### Proof.

If the solution of the $\mathcal{X}$-RHP exists, then the expression for
$P(n)$ can be found from

$\displaystyle P(n)=\mathcal{X}(z;n)\left.\begin{pmatrix}1&0&0&0\\\
0&z^{n}&0&0\\\ 0&0&1&0\\\ 0&0&0&z^{-n}\end{pmatrix}\right|_{z=0},$ (2.48)

and due to our assumption, the constants $C_{j}(n)$ can be uniquely found
according to Lemma 2.7. Then, according to (2.29) and (2.30) we find the
solution to the $\overset{\circ}{\mathcal{X}}$-RHP as

$\displaystyle\overset{\circ}{\mathcal{X}}(z;n)=\begin{pmatrix}1&C_{1}(n)&0&C_{3}(n)\\\
0&C_{2}(n)&1&C_{4}(n)\end{pmatrix}\mathcal{X}(z;n).$ (2.49)

Because, if it exists, the solution of the $\mathcal{X}$-RHP is unique (recall
Remark 2.4), we note that (2.49) is the unique solution of the
$\overset{\circ}{\mathcal{X}}$-RHP. Indeed, suppose that
$\overset{\circ}{\mathcal{X}}^{\prime}$ is another solution of the same
$\overset{\circ}{\mathcal{X}}$-RHP. Put

$\displaystyle\mathfrak{R}^{\prime}(z;n):=\overset{\circ}{\mathcal{X}^{\prime}}(z;n)\mathcal{X}^{-1}(z;n).$

Then, as with $\mathfrak{R}(z;n)$ before, we will arrive to the conclusion
that

$\displaystyle\mathfrak{R}^{\prime}(z;n)\equiv\begin{pmatrix}1&C^{\prime}_{1}(n)&0&C^{\prime}_{3}(n)\\\
0&C^{\prime}_{2}(n)&1&C^{\prime}_{4}(n)\end{pmatrix}\\!,$

where the constants $C^{\prime}_{j}(n)$ satisfy the system (2.31). Since the
solution of this system is unique according to Lemma 2.8, we conclude that
$\mathfrak{R}^{\prime}(z;n)=\mathfrak{R}(z;n)$ and, hence,

$\displaystyle\overset{\circ}{\mathcal{X}}^{\prime}(z;n)=\overset{\circ}{\mathcal{X}}(z;n).$

Now, observe that the symmetry relation (2.26) implies

$\displaystyle\overset{\circ}{\mathcal{X}}\big{(}z^{-1};n\big{)}=\overset{\circ}{\mathcal{X}}(z;n)W.$
(2.50)

Indeed, we have

$\displaystyle\overset{\circ}{\mathcal{X}}\big{(}z^{-1};n\big{)}=\mathfrak{R}(n)\mathcal{X}(z^{-1};n)=\mathfrak{R}(n)P(n)W\mathcal{X}(z;n)W.$

But, by (2.41), $\mathfrak{R}(n)P(n)W=\mathfrak{R}(n)$, therefore

$\displaystyle\overset{\circ}{\mathcal{X}}\big{(}z^{-1};n\big{)}=\mathfrak{R}(n)\mathcal{X}(z;n)W=\overset{\circ}{\mathcal{X}}(z;n)W.$

Now, by $\overset{\circ}{\mathcal{X}^{(j)}}$, $j=1,2,3,4$ denote the columns
of the $2\times 4$ matrix $\overset{\circ}{\mathcal{X}}$. Equation (2.50)
means that

$\displaystyle\overset{\circ}{\mathcal{X}}^{(2)}(z;n)=\overset{\circ}{\mathcal{X}}^{(1)}\big{(}z^{-1};n\big{)},\qquad\mbox{and}\qquad\overset{\circ}{\mathcal{X}}^{(4)}(z;n)=\overset{\circ}{\mathcal{X}}^{(3)}\big{(}z^{-1};n\big{)}.$

In other words, the matrix valued function $\overset{\circ}{\mathcal{X}}$ can
be written in the form (2.19), i.e.,

$\displaystyle\overset{\circ}{\mathcal{X}}(z;n)=\big{(}\mathcal{Y}^{(1)}(z;n),\widetilde{\mathcal{Y}}^{(1)}(z;n),\mathcal{Y}^{(2)}(z;n),\widetilde{\mathcal{Y}}^{(2)}(z;n)\big{)},$

with

$\displaystyle\mathcal{Y}^{(1)}(z;n):=\overset{\circ}{\mathcal{X}}^{(1)}(z;n),\qquad\mbox{and}\qquad\mathcal{Y}^{(2)}(z;n):=\overset{\circ}{\mathcal{X}}^{(3)}(z;n).$

Furthermore, the $2\times 2$ matrix valued function

$\displaystyle\mathcal{Y}(z;n):=\begin{pmatrix}\mathcal{Y}^{(1)}(z;n),\mathcal{Y}^{(2)}(z;n)\end{pmatrix}\\!,$

will be a solution of the $2\times 2$ $\mathcal{Y}$-RHP. From the unique
solvability of the $\overset{\circ}{\mathcal{X}}$-RHP, it follows that this
solution of the $\mathcal{Y}$-RHP is unique. Because otherwise, if
$\mathcal{Y}^{\prime}$ is another solution of the $\mathcal{Y}$-RHP, via
(2.19) we can construct another solution of the
$\overset{\circ}{\mathcal{X}}$-RHP on the contrary. ∎

The following corollary is the direct consequence of Corollary 2.2 and Lemma
2.9.

######  Corollary 2.10.

Suppose that the solution of the $\mathcal{X}$-RHP exists for $n$ and $n-1$,
then if at least one of the conditions (2.35) through (2.40) holds also for
$n$ and $n-1$, then we have

$\displaystyle D_{n}\neq 0,\qquad D_{n-1}\neq 0,\qquad\mbox{and}\qquad
h_{n-1}\neq 0.$

Moreover,

$\displaystyle h_{n-1}=-\lim_{z\to\infty}z^{n-1}/\mathcal{Y}_{21}(z;n).$
(2.51)

###  2.3 The primary opening of the lenses

Let us consider the contour $\Gamma:=\Gamma_{i}\cup{\mathbb{T}}\cup\Gamma_{o}$
shown in Fig. 1. Define the function $\mathcal{Z}$ as

$\displaystyle\mathcal{Z}(z;n):=\mathcal{X}(z;n)\begin{cases}J^{-1}_{\mathcal{X},i}(z),&z\in\Omega_{1},\vspace{1mm}\\\
J_{\mathcal{X},o}(z),&z\in\Omega_{2},\vspace{1mm}\\\
I,&z\in\Omega_{0}\cup\Omega_{\infty},\end{cases}$

where $J_{\mathcal{X},i}$ and $J_{\mathcal{X},o}$ are defined in the following
factorization for the jump matrix of the $\mathcal{X}$-RHP, which we denote by
$J_{\mathcal{X}}$:

$\displaystyle J_{\mathcal{X}}(z):=\begin{pmatrix}1&0&\tilde{w}(z)&-\phi(z)\\\
0&1&\tilde{\phi}(z)&-w(z)\\\ 0&0&1&0\\\
0&0&0&1\end{pmatrix}=\begin{pmatrix}1&0&0&0\\\ 0&1&0&-w(z)\\\ 0&0&1&0\\\
0&0&0&1\end{pmatrix}\begin{pmatrix}1&0&0&-\phi(z)\\\ 0&1&\tilde{\phi}(z)&0\\\
0&0&1&0\\\ 0&0&0&1\end{pmatrix}$
$\displaystyle\phantom{J_{\mathcal{X}}(z):=}\times\begin{pmatrix}1&0&\tilde{w}(z)&0\\\
0&1&0&0\\\ 0&0&1&0\\\ 0&0&0&1\end{pmatrix}\equiv
J_{\mathcal{X},o}(z)J_{\mathcal{X},{\mathbb{T}}}(z)J_{\mathcal{X},i}(z).$
(2.52)

We remind that the symbol $w$, and hence $\tilde{w}$, are analytic in the
neighborhood $U_{0}$ (cf. (1.7)) of ${\mathbb{T}}$ which is supposed to
include the domains $\Omega_{1}$ and $\Omega_{2}$.

$0$$\Omega_{1}$$\Omega_{2}$$\Gamma_{o}$${\mathbb{T}}$$\Gamma_{i}$$\Omega_{0}$$\Omega_{\infty}$
Figure 1: The jump contour $\Gamma$ for the $\mathcal{Z}$, $T$ and the global
parametrix Riemann–Hilbert problems.

The function $\mathcal{Z}$ satisfies the following Riemann–Hilbert problem:

  * •

RH-$\mathcal{Z}$1 $\mathcal{Z}$ is holomorphic in
${\mathbb{C}}\setminus\left(\Gamma\cup\\{0\\}\right)$.

  * •

RH-$\mathcal{Z}$2
$\mathcal{Z}_{+}(z;n)=\mathcal{Z}_{-}(z;n)J_{\mathcal{Z}}(z)$, where

$\displaystyle
J_{\mathcal{Z}}(z)=\begin{cases}J_{\mathcal{X},{\mathbb{T}}}(z),&z\in{\mathbb{T}},\\\
J_{\mathcal{X},i}(z),&z\in\Gamma_{i},\\\
J_{\mathcal{X},o}(z),&z\in\Gamma_{o}.\end{cases}$

  * •

RH-$\mathcal{Z}$3 As $z\to\infty$ we have

$\displaystyle\mathcal{Z}(z;n)=\big{(}\displaystyle
I+\mathcal{O}\big{(}z^{-1}\big{)}\big{)}\begin{pmatrix}z^{n}&0&0&0\\\
0&1&0&0\\\ 0&0&z^{-n}&0\\\ 0&0&0&1\end{pmatrix}\\!.$

  * •

RH-$\mathcal{Z}$4 As $z\to 0$ we have

$\displaystyle\mathcal{Z}(z;n)=P(n)(I+\mathcal{O}(z))\begin{pmatrix}1&0&0&0\\\
0&z^{-n}&0&0\\\ 0&0&1&0\\\ 0&0&0&z^{n}\end{pmatrix}\\!.$

######  Remark 2.11.

The term “opening of the lenses” is usually used to describe situations where
the jump matrix on the added contours is exponentially close to the identity
matrix for large values of the parameter $n$. The passage
${\mathbb{T}}\mapsto\Gamma$, corresponding to the RH transformation
$\mathcal{X}\mapsto\mathcal{Z}$, is clearly not of this type. However, our
secondary opening of the lenses (the passage $\Gamma\mapsto\Gamma_{S}$ which
corresponds to the RH transformation $T\mapsto S$) in Section 2.5 is an
example of a usual opening of the lenses.

######  Remark 2.12.

The primary opening of the lenses is essential for the progression of the RH
analysis in the following sections. This is due to a technical reason that
will be elaborated at the end of next section. Since the structure of jump
matrices is different in Section 3, we do not have an analogous step when the
Hankel symbol is supported on $[a,b]$, $0<a<b<1$.

###  2.4 Normalization of behaviours at $\mathbf{0}$ and $\boldsymbol{\infty}$

Following the natural steps of Riemann–Hilbert analysis, we will normalize the
behavior of $\mathcal{Z}$ at $0$ and $\infty$; to this end let us define

$\displaystyle
T(z;n):=\mathcal{Z}(z;n)\begin{cases}\begin{pmatrix}z^{-n}&0&0&0\\\ 0&1&0&0\\\
0&0&z^{n}&0\\\ 0&0&0&1\end{pmatrix}\\!,&|z|>1,\vspace{1mm}\\\
\begin{pmatrix}1&0&0&0\\\ 0&z^{n}&0&0\\\ 0&0&1&0\\\
0&0&0&z^{-n}\end{pmatrix}\\!,&|z|<1.\end{cases}$ (2.53)

It is very important to note that in order to have a suitable Riemann–Hilbert
analysis, the normalization of behaviors at $0$ and $\infty$ can only be
carried out only after the undressing $\mathcal{X}\mapsto\mathcal{Z}$; this is
due to technical reasons that will be further commented about at the end of
this section. We have the following RHP for $T$:

  * •

RH-$T$1 $T$ is holomorphic in
${\mathbb{C}}\setminus\left({\mathbb{T}}\cup\Gamma_{i}\cup\Gamma_{o}\right)$.

  * •

RH-$T$2 $T_{+}(z;n)=T_{-}(z;n)J_{T}(z;n)$, where

$\displaystyle J_{T}(z;n)=\begin{cases}\widehat{J}(z;n),&z\in{\mathbb{T}},\\\
J_{\mathcal{X},i}(z),&z\in\Gamma_{i},\\\
J_{\mathcal{X},o}(z),&z\in\Gamma_{o},\end{cases}\quad\mbox{where}\quad\widehat{J}(z;n)=\begin{pmatrix}z^{n}&0&0&-\phi(z)\\\
0&z^{n}&\tilde{\phi}(z)&0\\\ 0&0&z^{-n}&0\\\
0&0&0&z^{-n}\end{pmatrix}\\!,\\!\\!\\!$ (2.54)

and the matrices $J_{\mathcal{X},i}$ and $J_{\mathcal{X},o}$ are defined by
(2.52).

  * •

RH-$T$3 As $z\to\infty$, we have $T(z;n)=\big{(}\displaystyle
I+\mathcal{O}\big{(}z^{-1}\big{)}\big{)}$.

We observe that for $z\in{\mathbb{T}}$, $J_{T}$ can be factorized as follows

$\displaystyle\widehat{J}(z;n)=\begin{pmatrix}I_{2}&0_{2}\\\
z^{-n}\Phi^{-1}(z)&I_{2}\end{pmatrix}\begin{pmatrix}0_{2}&\Phi(z)\\\
-\Phi^{-1}(z)&0_{2}\end{pmatrix}\begin{pmatrix}I_{2}&0_{2}\\\
z^{n}\Phi^{-1}(z)&I_{2}\end{pmatrix}$
$\displaystyle\phantom{\widehat{J}(z;n)}\equiv
J_{T,o}(z;n)\overset{\circ}{J}(z)J_{T,i}(z;n),$ (2.55)

where $0_{2}$ and $I_{2}$ are respectively $2\times 2$ zero and identity
matrices and

$\displaystyle\Phi(z)=\begin{pmatrix}0&-\phi(z)\\\
\tilde{\phi}(z)&0\end{pmatrix}\\!.$

Note that $J_{T,i}$ is exponentially close to the identity matrix for $z$
inside of the unit circle and $J_{T,o}$ is exponentially close to the identity
matrix for $z$ outside of the unit circle.

Now we are in a position to address Remark 2.12 in the previous section.
Indeed, if one normalizes the behaviors at $0$ and $\infty$ without the
undressing transformation $\mathcal{X}\mapsto\mathcal{Z}$; i.e., by directly
defining the function $\mathcal{T}$ as

$\displaystyle\mathcal{T}(z;n):=\mathcal{X}(z;n)\begin{cases}\begin{pmatrix}z^{-n}&0&0&0\\\
0&1&0&0\\\ 0&0&z^{n}&0\\\ 0&0&0&1\end{pmatrix}\\!,&|z|>1,\vspace{1mm}\\\
\begin{pmatrix}1&0&0&0\\\ 0&z^{n}&0&0\\\ 0&0&1&0\\\
0&0&0&z^{-n}\end{pmatrix}\\!,&|z|<1.\end{cases}$

Then the jump matrix $J_{\mathcal{T}}:=\mathcal{T}^{-1}_{-}\mathcal{T}_{+}$ on
the unit circle would be

$\displaystyle
J_{\mathcal{T}}(z;n)=\begin{pmatrix}z^{n}&0&z^{n}\tilde{w}(z)&-\phi(z)\\\
0&z^{n}&\tilde{\phi}(z)&-z^{-n}w(z)\\\ 0&0&z^{-n}&0\\\
0&0&0&z^{-n}\end{pmatrix}\\!,$

for which finding a factorization like (2.55) remains a challenge, mainly due
to presence of the large parameter $n$ in the $13$ and $24$ elements of
$J_{\mathcal{T}}$. This fact justifies the necessity of the undressing step
$\mathcal{X}\mapsto\mathcal{Z}$. Indeed, due to the specific matrix structure
of the jump matrices $J_{\mathcal{X},i}$ and $J_{\mathcal{X},o}$ they do not
change under the transformation (2.53).

###  2.5 The secondary opening of the lenses

The next Riemann–Hilbert transformation $T\mapsto S$, provides us with a
problem with jump conditions on five contours where three jump matrices do not
depend on $n$ and the other two converge exponentially fast to the identity
matrix as $n\to\infty$. Let us define the function $S$, suggested by (2.55),
as

$\displaystyle
S(z;n):=T(z;n)\times\begin{cases}J^{-1}_{T,i}(z;n),&z\in\Omega^{\prime}_{1},\\\
J_{T,o}(z;n),&z\in\Omega^{\prime}_{2},\\\
I,&z\in\Omega^{\prime\prime}_{1}\cup\Omega^{\prime\prime}_{2}\cup\Omega_{0}\cup\Omega_{\infty},\end{cases}$

where the regions $\Omega^{\prime}_{1}$, $\Omega^{\prime}_{2}$,
$\Omega^{\prime\prime}_{1}$ and $\Omega^{\prime\prime}_{2}$ are shown in Fig.
2. We have the following Riemann–Hilbert problem for $S$

  * •

RH-$S$1 $S$ is holomorphic in
${\mathbb{C}}\setminus\left({\mathbb{T}}\cup\Gamma_{i}\cup\Gamma_{o}\cup\Gamma^{\prime}_{i}\cup\Gamma^{\prime}_{o}\right)$.

  * •

RH-$S$2 $S_{+}(z;n)=S_{-}(z;n)J_{S}(z;n)$, where

$\displaystyle
J_{S}(z;n)=\begin{cases}\overset{\circ}{J}(z),&z\in{\mathbb{T}},\\\
J_{T,i}(z;n),&z\in\Gamma^{\prime}_{i},\\\
J_{T,o}(z;n),&z\in\Gamma^{\prime}_{o},\\\
J_{\mathcal{X},i}(z),&z\in\Gamma_{i},\\\
J_{\mathcal{X},o}(z),&z\in\Gamma_{o}.\end{cases}$

  * •

RH-$S$3 As $z\to\infty$, we have $S(z;n)=\displaystyle
I+\mathcal{O}\big{(}z^{-1}\big{)}$.

$0$$\Omega^{\prime}_{1}$$\Omega^{\prime\prime}_{1}$$\Omega^{\prime\prime}_{2}$$\Omega^{\prime}_{2}$$\Gamma_{o}$$\Gamma^{\prime}_{o}$${\mathbb{T}}$$\Gamma_{i}$$\Gamma^{\prime}_{i}$$\Omega_{0}$$\Omega_{\infty}$
Figure 2: The jump contour $\Gamma_{S}$ of the $S$-RHP.

In the usual way, we will first try to solve this Riemann–Hilbert problem by
disregarding the jump matrices which depend on $n$, this solution is denoted
by $\overset{\circ}{S}$ and will be referred to as the global parametrix. Once
we construct the global parametrix, we will consider the small-norm
Riemann–Hilbert problem for the ratio $R:=S(\overset{\circ}{S})^{-1}$ and
discuss its solvability in the forthcoming sections.

###  2.6 The global parametrix and the model Riemann–Hilbert problem
for the pair $\boldsymbol{(\phi,w)}$

The $S$-RHP reduces to the following Riemann–Hilbert problem for the global
parametrix $\overset{\circ}{S}$, when we ignore the jump matrices which are
exponentially close to the identity matrix:

  * •

RH-$\overset{\circ}{S}$1 $\overset{\circ}{S}$ is holomorphic in
${\mathbb{C}}\setminus({\mathbb{T}}\cup\Gamma_{i}\cup\Gamma_{o})$.

  * •

RH-$\overset{\circ}{S}$2
$\overset{\circ}{S}_{+}(z)=\overset{\circ}{S}_{-}(z)J_{\overset{\circ}{S}}(z)$,
where

$\displaystyle
J_{\overset{\circ}{S}}(z)=\begin{cases}\overset{\circ}{J}(z),&z\in{\mathbb{T}},\\\
J_{\mathcal{X},i}(z),&z\in\Gamma_{i},\\\
J_{\mathcal{X},o}(z),&z\in\Gamma_{o}.\\\ \end{cases}$

  * •

RH-$\overset{\circ}{S}$3 As $z\to\infty$, we have
$\overset{\circ}{S}(z)=\displaystyle I+\mathcal{O}\big{(}z^{-1}\big{)}$.

And we finally dress the $\overset{\circ}{S}$-RHP to obtain a model problem
for the global parametrix having jumps only on the unit circle. We define the
function $\Lambda$ as

$\displaystyle\Lambda(z):=\overset{\circ}{S}(z)\times\begin{cases}J_{\mathcal{X},i}(z),&z\in\Omega_{1},\\\
J^{-1}_{\mathcal{X},o}(z),&z\in\Omega_{2},\\\
I,&z\in\Omega_{0}\cup\Omega_{\infty}.\end{cases}$ (2.56)

Now we arrive at the following Riemann–Hilbert problem for $\Lambda$ that from
now on we will refer to as the model Riemann–Hilbert problem for the pair
$(\phi,w)$:

  * •

RH-$\Lambda$1 $\Lambda$ is holomorphic in ${\mathbb{C}}\setminus{\mathbb{T}}$.

  * •

RH-$\Lambda$2 $\Lambda_{+}(z)=\Lambda_{-}(z)J_{\Lambda}(z)$, for
$z\in{\mathbb{T}}$, where

$\displaystyle J_{\Lambda}(z)=\begin{pmatrix}0&0&0&-\phi(z)\\\
\displaystyle-\frac{w(z)}{\phi(z)}&0&\displaystyle\tilde{\phi}(z)-\frac{w(z)\tilde{w}(z)}{\phi(z)}&0\\\
0&\displaystyle-\frac{1}{\tilde{\phi}(z)}&0&0\\\
\displaystyle\frac{1}{\phi(z)}&0&\displaystyle\frac{\tilde{w}(z)}{\phi(z)}&0\end{pmatrix}\\!.$

  * •

RH-$\Lambda$3 As $z\to\infty$, we have $\Lambda(z)=\displaystyle
I+\mathcal{O}\big{(}z^{-1}\big{)}$.

The conditions on $w$ and $\phi$ which ensure the solvability of this model
problem are not completely known and categorized at this point. We also want
to stress that the appearance of the $4\times 4$ model $\Lambda$-problem in
the asymptotic analysis of the original $\mathcal{X}$-RHP is the crucial
difference of the Toeplitz+Hankel case we consider in this work comparing to
the pure Toeplitz or pure Hankel or Toeplitz+Hankel with the same symbols
cases. Indeed, even if the pair $(\phi,w)$ is such that the $\Lambda$-RHP is
solvable it does not mean that it is explicitly solvable. Hence, one should
not expect the closed form of the asymptotic answer in the case of the generic
pair $(\phi,w)$.444In this respect the Toeplitz+Hankel determinants with
generic symbols are similar to the block Toeplitz determinants, where the
explicit answers can be obtained only in two cases: (a) the Fourier expansion
of the corresponding matrix symbol is one side truncated or (b) one can
produce an explicit Wiener–Hopf factorization of the symbol. However, in
Section 4 we will present a detailed analysis of this model problem for a
specific family of pairs $(\phi,w)$ within the broader class of Toeplitz and
Hankel weights considered by E. Basor and T. Ehrhardt in [2] for which the
model $\Lambda$-problem is explicitly solvable.

Remarkably, we arrive at the same model $\Lambda$-Riemann–Hilbert problem, if
we start with a Hankel weight supported on the interval $[a,b]$, $0<a<b<1$.
This will be shown in the next section.

##  3 Toeplitz+Hankel determinants: Hankel weight supported
on the interval $\boldsymbol{[a,b]}$, $\boldsymbol{0<a<b<1}$

In this section we consider the determinant (1.3) where $w_{k}$ exist and are
given by (1.2), with $I=[a,b]$, $0<a<b<1$. We further assume that $w$ does not
have Fisher–Hartwig singularities (see [10, 11, 19], or [20] for instances of
Fisher–Hartwig singularities on the real line). Let us again assume that the
symbol $\phi$ is of Szegő-type. The Riemann–Hilbert approach outlined in this
section can be naturally extended to the three other cases: i) $-1<a<b<0$, ii)
$-\infty<a<b<-1$, and iii) $1<a<b<\infty$. We consider the system of
orthogonal polynomials $\\{P_{n}(z)\\}$, $\deg P_{n}(z)=n$, satisfying the
following orthogonality conditions

$\displaystyle\int^{b}_{a}P_{n}(x)x^{k+s}w(x)\,{\rm
d}x+\int_{{\mathbb{T}}}P_{n}(z)z^{-k-r}\phi(z)\frac{{\rm d}z}{2\pi{\rm
i}z}=h_{n}\delta_{n,k},\qquad k=0,1,\dots,n.$ (3.1)

One can write a determinantal formula for $P_{n}$ like in (2.2), which yields

$\displaystyle h_{n}=\frac{D_{n+1}(\phi,w;r,s)}{D_{n}(\phi,w;r,s)}.$

By similar considerations as those mentioned in Section 2, the orthogonal
polynomials $P_{n}$ exist and are unique if $D_{n}\neq 0$. Now, assuming that
$D_{n}$, $D_{n-1}\neq 0$, we consider the function $Y$ defined as

$\displaystyle
Y(z;n)\\!=\\!\begin{pmatrix}P_{n}(z)&\displaystyle\int^{b}_{a}\frac{P_{n}(x)x^{s}w(x)}{x-z}{\rm
d}x+\int_{{\mathbb{T}}}\frac{\tilde{\phi}(\xi)\xi^{r}\tilde{P}_{n}(\xi)}{\xi-z}\frac{{\rm
d}\xi}{2\pi{\rm i}\xi}\\\\[8.61108pt]
-\displaystyle\frac{P_{n-1}(z)}{h_{n-1}}&-\displaystyle\frac{1}{h_{n-1}}\Bigg{\\{}\int^{b}_{a}\frac{P_{n-1}(x)x^{s}w(x)}{x-z}{\rm
d}x+\\!\\!\int_{{\mathbb{T}}}\\!\frac{\tilde{\phi}(\xi)\xi^{r}\tilde{P}_{n-1}(\xi)}{\xi-z}\frac{{\rm
d}\xi}{2\pi{\rm i}\xi}\Bigg{\\}}\end{pmatrix}\\!,$ (3.2)

built from the orthogonal polynomials $P_{n}$ satisfying (3.1). Consider the
following Riemann–Hilbert problem for finding the $2\times 2$ matrix $Y$
satisfying

  * •

RH-Y1 $Y$ is holomorphic in ${\mathbb{C}}\setminus({\mathbb{T}}\cup[a,b])$.

  * •

RH-Y2 For $z\in{\mathbb{T}}$ we have

$\displaystyle Y_{+}^{(1)}(z;n)=Y^{(1)}_{-}(z;n),$

and

$\displaystyle
Y_{+}^{(2)}(z;n)=Y^{(2)}_{-}(z;n)+z^{r-1}\tilde{\phi}(z)Y^{(1)}_{-}\big{(}z^{-1};n\big{)}.$

  * •

RH-Y3 For $x\in(a,b)$ we have

$\displaystyle Y_{+}^{(1)}(x;n)=Y^{(1)}_{-}(x;n),$

and

$\displaystyle Y_{+}^{(2)}(x;n)=Y^{(2)}_{-}(x;n)+2\pi{\rm
i}x^{s}w(x)Y^{(1)}_{-}(x;n).$

  * •

RH-Y4 As $z\to\infty$

$\displaystyle
Y(z;n)=\bigg{(}I+O\bigg{(}\frac{1}{z}\bigg{)}\bigg{)}z^{n\sigma_{3}}=\begin{pmatrix}z^{n}+\mathcal{O}\big{(}z^{n-1}\big{)}&\mathcal{O}\big{(}z^{-n-1}\big{)}\\\
\mathcal{O}\big{(}z^{n-1}\big{)}&z^{-n}+\mathcal{O}\big{(}z^{-n-1}\big{)}\end{pmatrix}\\!,$

where $Y^{(1)}$ and $Y^{(2)}$ are the first and second columns of $Y$,
respectively. We have the analogue of Theorem 2.1 here as well.

######  Theorem 3.1.

The following statements are true.

  1. $\rm 1.$

Suppose that $D_{n},D_{n-1}\neq 0$. Then, the Riemann–Hilbert problem _RH-
$Y$1_ through _RH- $Y$4_ is uniquely solvable and its solution $\mathcal{Y}$
is defined by (3.2). Moreover,

$\displaystyle h_{n-1}=-\lim_{z\to\infty}z^{n-1}/Y_{21}(z;n).$

  2. $\rm 2.$

Suppose that the Riemann–Hilbert problem _RH- $Y$1_ through _RH- $Y$4_ has a
unique solution. Then $D_{n}\neq 0$,
$\operatorname{rank}(T_{n-1}[\phi;r]+H_{n-1}[w;s])\geq n-2$, and
$P_{n}(z)=Y_{11}(z;n)$.

  3. $\rm 3.$

Suppose that the Riemann–Hilbert problem _RH- $Y$1_ through _RH- $Y$4_ has a
unique solution. Suppose also that

$\displaystyle\lim_{z\to\infty}Y_{21}(z;n)z^{-n+1}\neq 0.$

Then, as before, $D_{n}\neq 0$, $P_{n}(z)=Y_{11}(z;n)$, and, in addition,

$\displaystyle D_{n-1}\neq 0,\qquad
h_{n-1}=-\lim_{z\to\infty}\mathcal{Y}^{-1}_{21}(z;n)z^{n-1},\qquad
P_{n-1}(z)=-h_{n-1}Y_{21}(z;n).$

We omit the proof here as it is similar to the proof of Theorem 2.1.

######  Corollary 3.2.

Suppose that the $Y$-RH problem has a unique solution for $n$ and $n-1$. Then

$\displaystyle D_{n}\neq 0,\qquad D_{n-1}\neq 0,\qquad\mbox{and}\qquad
h_{n-1}\neq 0.$

Moreover,

$\displaystyle h_{n-1}=-\lim_{z\to\infty}z^{n-1}/Y_{21}(z;n).$

###  3.1 The associated $\boldsymbol{2\times 4}$ and $\boldsymbol{4\times 4}$
Riemann–Hilbert problems

The formulation of the $2\times 4$ and $4\times 4$ Riemann–Hilbert problems
are very similar to those of Section 2.1, however there are minor differences
that convinces us to practice clarity in our exposition. Let us consider the
following $2\times 4$ matrix function, constructed from the columns of $Y$
given by (3.2):

$0$$a$$b$$b^{-1}$$a^{-1}$${\mathbb{T}}$ Figure 3: The jump contour $\Sigma$.

$\displaystyle\overset{\circ}{X}(z;n):=\big{(}Y^{(1)}(z;n),\widetilde{Y}^{(1)}(z;n),Y^{(2)}(z;n),\widetilde{Y}^{(2)}(z;n)\big{)}.$
(3.3)

Let us define $\Sigma:={\mathbb{T}}\cup[a,b]\cup\big{[}b^{-1},a^{-1}\big{]}$,
and $\Sigma^{\prime}:=\Sigma\setminus\big{\\{}a,b,b^{-1},a^{-1}\big{\\}}$.
$\overset{\circ}{X}(z;n)$ satisfies the following Riemann–Hilbert problem

  * •

RH-$\overset{\circ}{X}$1 $\overset{\circ}{X}$ is analytic in
${\mathbb{C}}\setminus\left(\Sigma\cup\\{0\\}\right)$,

  * •

RH-$\overset{\circ}{X}$2 For $z\in\Sigma^{\prime}$, we have
$\overset{\circ}{X}_{+}(z;n)=\overset{\circ}{X}_{-}(z;n)J_{\overset{\circ}{X}}(z)$,
where

$\displaystyle
J_{\overset{\circ}{X}}(z)=\begin{cases}\begin{pmatrix}1&0&0&-z^{-r+1}\phi(z)\\\
0&1&z^{r-1}\tilde{\phi}(z)&0\\\ 0&0&1&0\\\
0&0&0&1\end{pmatrix}\\!,&z\in{\mathbb{T}},\vspace{1mm}\\\
\begin{pmatrix}1&0&2\pi{\rm i}x^{s}w(x)&0\\\ 0&1&0&0\\\ 0&0&1&0\\\
0&0&0&1\end{pmatrix}\\!,&z\equiv x\in(a,b),\vspace{1mm}\\\
\begin{pmatrix}1&0&0&0\\\ 0&1&0&-2\pi{\rm i}x^{-s}\tilde{w}(x)\\\ 0&0&1&0\\\
0&0&0&1\end{pmatrix}\\!,&z\equiv x\in\big{(}b^{-1},a^{-1}\big{)}.\end{cases}$

  * •

RH-$\overset{\circ}{X}$3 As $z\to\infty$

$\displaystyle\overset{\circ}{X}(z;n)=\begin{pmatrix}1+O\big{(}z^{-1}\big{)}&E_{1}(n)+O\big{(}z^{-1}\big{)}&O\big{(}z^{-1}\big{)}&E_{3}(n)+O\big{(}z^{-1}\big{)}\vspace{1mm}\\\
O\big{(}z^{-1}\big{)}&E_{2}(n)+O\big{(}z^{-1}\big{)}&1+O\big{(}z^{-1}\big{)}&E_{4}(n)+O\big{(}z^{-1}\big{)}\end{pmatrix}$
$\displaystyle\phantom{\overset{\circ}{X}(z;n)=}\times\begin{pmatrix}z^{n}&0&0&0\\\
0&1&0&0\\\ 0&0&z^{-n}&0\\\ 0&0&0&1\end{pmatrix}\\!.$

  * •

RH-$\overset{\circ}{X}$4 As $z\to 0$

$\displaystyle\overset{\circ}{X}(z;n)\\!=\\!\begin{pmatrix}E_{1}(n)+O(z)&1+O(z)&E_{3}(n)+O(z)&O(z)\\\
E_{2}(n)+O(z)&O(z)&E_{4}(n)+O(z)&1+O(z)\end{pmatrix}\begin{pmatrix}1&0&0&0\\\
0&z^{-n}&0&0\\\ 0&0&1&0\\\ 0&0&0&z^{n}\end{pmatrix}\\!,$

where

$\displaystyle E_{1}(n)=Y_{11}(0;n),\qquad$ $\displaystyle
E_{3}(n)=Y_{12}(0;n),$ $\displaystyle E_{2}(n)=Y_{21}(0;n),\qquad$
$\displaystyle E_{4}(n)=Y_{22}(0;n).$

It is straightforward to check that $\overset{\circ}{X}$ given by (3.3) and
(3.2) satisfies the Riemann–Hilbert problem RH-$\overset{\circ}{X}$1 through
RH-$\overset{\circ}{X}$4.

One of the differences between the case when the Hankel symbol is supported on
the unit circle versus the case when it is supported on the interval $[a,b]$,
is discussed in the following remark about the values of offsets that can be
handled without much difficulty in each case.

######  Remark 3.3.

Let $u$ be defined by

$\displaystyle u(z):=z\int^{b}_{a}\frac{t^{s-1}w(t)}{t-z}{\rm d}t.$ (3.4)

When the Hankel symbol is supported on the interval, the natural progression
of the Riemann–Hilbert analysis with general offset values
$r,s\in{\mathbb{Z}}$, finally requires us to construct Szegő functions for the
functions $f_{1}(z)=z^{r-1}\phi(z)$ and $f_{2}(z)=\tilde{u}(z)$ (Compare with
Remark 2.3). Note that the function $u$ given by (3.4) (and $\tilde{u}$) has
no winding number for all $s\in{\mathbb{Z}}$. Therefore, in this work it seems
natural for us to focus on the determinants of the type $D_{n}(\phi,w;1,s)$,
in view of the points mentioned above and in Remark 2.3.

Similar to our approach in Section 2.1, we introduce the following
Riemann–Hilbert problem of finding the $4\times 4$ matrix function $X$
satisfying:

  * •

RH-X1 $X$ is analytic in ${\mathbb{C}}\setminus(\Sigma\cup\\{0\\})$.

  * •

RH-X2 For $z\in\Sigma^{\prime}$, we have $X_{+}(z;n)=X_{-}(z;n)J_{X}(z)$,
where

$\displaystyle J_{X}(z)=\begin{cases}\begin{pmatrix}1&0&0&-\phi(z)\\\
0&1&\tilde{\phi}(z)&0\\\ 0&0&1&0\\\
0&0&0&1\end{pmatrix}\\!,&z\in{\mathbb{T}},\vspace{1mm}\\\
\begin{pmatrix}1&0&2\pi{\rm i}x^{s}w(x)&0\\\ 0&1&0&0\\\ 0&0&1&0\\\
0&0&0&1\end{pmatrix}\\!,&z\equiv x\in(a,b),\vspace{1mm}\\\
\begin{pmatrix}1&0&0&0\\\ 0&1&0&-2\pi{\rm i}x^{-s}\tilde{w}(x)\\\ 0&0&1&0\\\
0&0&0&1\end{pmatrix}\\!,&z\equiv x\in\big{(}b^{-1},a^{-1}\big{)},\end{cases}$
(3.5)

that is, $J_{X}$ is exactly equal to $J_{\overset{\circ}{X}}$ when $r=1$.

  * •

RH-X3 As $z\to\infty$

$\displaystyle X(z;n)=\big{(}I+{\cal
O}{\big{(}z^{-1}\big{)}}\big{)}\begin{pmatrix}z^{n}&0&0&0\\\ 0&1&0&0\\\
0&0&z^{-n}&0\\\ 0&0&0&1\end{pmatrix}\\!.$

  * •

RH-X4 As $z\to 0$

$\displaystyle X(z;n)=Q(n)(I+{\cal O}{(z)})\begin{pmatrix}1&0&0&0\\\
0&z^{-n}&0&0\\\ 0&0&1&0\\\ 0&0&0&z^{n}\end{pmatrix}\\!,$ (3.6)

where we emphasize that the matrix factor $Q(n)$ in (3.6) is not a priori
prescribed. Now we are going to briefly mention some facts about this
Riemann–Hilbert problem which are similar to those of the $\mathcal{X}$-RHP.
Using the usual Liouville theorem-based arguments one can easily show that the
solution of $X$-RHP is unique, if it exists. Also, without much difficulty one
can show that the function $WP^{-1}(n)X\big{(}z^{-1};n\big{)}W$ is also a
solution of the $X$-RHP, and thus due to the uniqueness of the solution, we
get the symmetry relation

$\displaystyle WP^{-1}(n)\mathcal{X}\big{(}z^{-1};n\big{)}W=\mathcal{X}(z;n),$
(3.7)

where $W$ is given by (2.27). Equation (3.7) yields the following symmetry
equation for $Q(n)$,

$\displaystyle Q(n)=WQ^{-1}(n)W,$

or,

$\displaystyle(WQ(n))^{2}=(Q(n)W)^{2}=I_{4}.$

Exact similar argument used in Remark 2.6 proves that the matrix $Q(n)W-I_{4}$
has rank $2$. Here we also have the relationship between $\overset{\circ}{X}$
and $X$ given by

$\displaystyle\overset{\circ}{X}(z;n)=\begin{pmatrix}1&E_{1}(n)&0&E_{3}(n)\\\
0&E_{2}(n)&1&E_{4}(n)\end{pmatrix}X(z;n),$

and moreover,

$\displaystyle\begin{pmatrix}1&E_{1}(n)&0&E_{3}(n)\\\
0&E_{2}(n)&1&E_{4}(n)\end{pmatrix}=\begin{pmatrix}E_{1}(n)&1&E_{3}(n)&0\\\
E_{2}(n)&0&E_{4}(n)&1\end{pmatrix}Q^{-1}(n).$ (3.8)

Since this is exactly the system (2.31), where $C_{j}(n)$ and $P(n)$ are
respectively replaced by $E_{j}(n)$ and $Q(n)$, $1\leq j\leq 4$, and because
$P(n)$ and $Q(n)$ enjoy the same symmetry and rank properties, we readily have
the following statements, whose counterparts are already proven in Section 2.

######  Lemma 3.4.

Let $Q_{jk}(n)$, $1\leq j,k\leq 4$, be the entries of the matrix $Q(n)$.
Assume that at least one of the following six inequalities is true,

$\displaystyle Q_{22}(n)Q_{44}(n)-Q_{42}(n)Q_{24}(n)\neq 0,$ (3.9)
$\displaystyle(1-Q_{21}(n))Q_{42}(n)+Q_{22}(n)Q_{41}(n)\neq 0,$ (3.10)
$\displaystyle(1-Q_{43}(n))Q_{22}(n)+Q_{23}(n)Q_{42}(n)\neq 0,$ (3.11)
$\displaystyle(1-Q_{21}(n))Q_{44}(n)+Q_{41}(n)Q_{24}(n)\neq 0,$ (3.12)
$\displaystyle(1-Q_{21}(n))(Q_{43}(n)-1)+Q_{41}(n)Q_{23}(n)\neq 0,$ (3.13)
$\displaystyle(1-Q_{43}(n))Q_{24}(n)+Q_{23}(n)Q_{44}(n)\neq 0.$ (3.14)

Then, the system (3.8) is a well-defined linear system on $E_{j}(n)$ which is
uniquely solvable.

######  Lemma 3.5.

If the system (3.8) has a solution, it has to be unique.

######  Lemma 3.6.

Suppose that the solution of the $X$-RHP exists. Then, if at least one of the
conditions (3.9) through (3.14) holds, one can uniquely reconstruct the
solution of the $Y$-RHP.

######  Corollary 3.7.

Suppose that the solution of the $X$-RHP exists for $n$ and $n-1$, then if at
least one of the conditions (3.9) through (3.14) also holds for $n$ and $n-1$,
then we have

$\displaystyle D_{n}\neq 0,\qquad D_{n-1}\neq 0,\qquad\mbox{and}\qquad
h_{n-1}\neq 0,$

where $h_{n-1}$ can be reconstructed form the RHP data as

$\displaystyle h_{n-1}=-\lim_{z\to\infty}z^{n-1}/Y_{21}(z;n).$

###  3.2 Normalization of behaviors at $\boldsymbol{0}$ and
$\boldsymbol{\infty}$

Unlike the situation in Section 2 where we had to make the transformation
$\mathcal{X}\mapsto\mathcal{Z}$ before normalization of behaviors at zero and
infinity, when the Hankel symbol is supported on the interval $[a,b]$ we can
immediately normalize the asymptotic behaviors at $0$ and infinity, due to the
desired structure of jump matrices. Indeed, it is natural to define

$\displaystyle T(z;n):=X(z;n)\begin{cases}\begin{pmatrix}z^{-n}&0&0&0\\\
0&1&0&0\\\ 0&0&z^{n}&0\\\ 0&0&0&1\end{pmatrix}\\!,&|z|>1,\vspace{1mm}\\\
\begin{pmatrix}1&0&0&0\\\ 0&z^{n}&0&0\\\ 0&0&1&0\\\
0&0&0&z^{-n}\end{pmatrix}\\!,&|z|<1.\end{cases}$ (3.15)

The function $T$ satisfies the following Riemann–Hilbert problem:

  * •

RH-$T$1 $T$ is holomorphic in ${\mathbb{C}}\setminus\Sigma$.

  * •

RH-$T$2 For $z\in\Sigma^{\prime}$, we have $T_{+}(z;n)=T_{-}(z;n)J_{T}(z;n)$,
where

$\displaystyle J_{T}(z;n)=\begin{cases}\widehat{J}(z;n),&z\in{\mathbb{T}},\\\
J_{X}(z),&z\in(a,b)\cup\big{(}b^{-1},a^{-1}\big{)}.\end{cases}$

We recall that $\widehat{J}$ is given by (2.54) and the matrices $J_{X}$ for
$z\in(a,b)$ and $z\in\big{(}b^{-1},a^{-1}\big{)}$ are given by (3.5).

  * •

RH-$T$3 As $z\to\infty$, we have $T(z;n)=\big{(}\displaystyle
I+\mathcal{O}\big{(}z^{-1}\big{)}\big{)}$.

We bring the reader’s attention to the fact that the transformation (3.15)
does not change the jump matrices $J_{X}$.

###  3.3 Opening of the lenses

Using (2.55), we open the lenses off the unit circle as shown in the Fig. 4
and we define

$\displaystyle
S(z;n):=T(z;n)\times\begin{cases}J^{-1}_{T,i}(z;n),&z\in\Omega_{1},\\\
J_{T,o}(z;n),&z\in\Omega_{2},\\\
I,&z\in{\mathbb{C}}\setminus\left(\overline{\Omega_{1}}\cup\overline{\Omega_{2}}\cup[a,b]\cup\big{[}b^{-1},a^{-1}\big{]}\right),\end{cases}$

where $J_{T,i}$ and $J_{T,o}$ are defined in (2.55). Let
$\Sigma_{S}\equiv\Sigma\cup\Sigma_{o}\cup\Sigma_{i}$ and
$\Sigma^{\prime}_{S}\equiv\Sigma^{\prime}\cup\Sigma_{o}\cup\Sigma_{i}$ (see
Fig. 4). It is straightforward to check that $S$ satisfies the following
Riemann–Hilbert problem

$0$$a$$b$$b^{-1}$$a^{-1}$${\mathbb{T}}$$\Sigma_{o}$$\Sigma_{i}$$\Omega_{0}$$\Omega_{1}$$\Omega_{2}$$\Omega_{\infty}$
Figure 4: The jump contour
$\Sigma_{S}\equiv\Sigma\cup\Sigma_{o}\cup\Sigma_{i}$ of the $S$-RHP.

  * •

RH-$S$1 $S$ is holomorphic in ${\mathbb{C}}\setminus\Sigma_{S}$,

  * •

RH-$S$2 For $z\in\Sigma^{\prime}_{S}$ we have
$S_{+}(z;n)=S_{-}(z;n)J_{S}(z;n)$, where

$\displaystyle
J_{S}(z;n)=\begin{cases}\overset{\circ}{J}(z),&z\in{\mathbb{T}},\\\
J_{T,i}(z;n),&z\in\Sigma_{i},\\\ J_{T,o}(z;n),&z\in\Sigma_{o},\\\
J_{X}(z),&z\in(a,b)\cup\big{(}b^{-1},a^{-1}\big{)},\\\ \end{cases}$

where these matrices are defined in (2.55) and (3.5).

  * •

RH-$S$3 As $z\to\infty$, we have $S(z;n)=\displaystyle
I+\mathcal{O}\big{(}z^{-1}\big{)}$.

###  3.4 The global parametrix and a model Riemann–Hilbert problem

Let us consider the following Riemann–Hilbert for $\overset{\circ}{S}$, or the
global parametrix, which is expected to be a good approximation to $S$ for
large parameter $n$. This RHP is simply obtained from the $S$-RHP by ignoring
the jumps on $\Sigma_{i}$ and $\Sigma_{o}$:

  * •

RH-$\overset{\circ}{S}$1 $\overset{\circ}{S}$ is holomorphic in
${\mathbb{C}}\setminus\Sigma$ (see Fig. 3).

  * •

RH-$\overset{\circ}{S}$2 For $z\in\Sigma^{\prime}$, we have
$\overset{\circ}{S}_{+}(z)=\overset{\circ}{S}_{-}(z)J_{\overset{\circ}{S}}(z)$,
where

$\displaystyle
J_{\overset{\circ}{S}}(z)=\begin{cases}\overset{\circ}{J}(z),&z\in{\mathbb{T}},\\\
J_{X}(z),&z\in(a,b)\cup\big{(}b^{-1},a^{-1}\big{)}.\\\ \end{cases}$

  * •

RH-$\overset{\circ}{S}$3 As $z\to\infty$, we have
$\overset{\circ}{S}(z)=\displaystyle I+\mathcal{O}\big{(}z^{-1}\big{)}$.

Let us recall the function $u$ defined in Remark 3.3:

$\displaystyle u(z)=z\int^{b}_{a}\frac{t^{s-1}w(t)}{t-z}\,{\rm d}t.$

The Plemelj–Sokhotskii formula implies that

$\displaystyle u_{+}(x)-u_{-}(x)=2\pi{\rm i}x^{s}w(x),\qquad x\in(a,b),$
$\displaystyle\tilde{u}_{+}(x)-\tilde{u}_{-}(x)=-2\pi{\rm
i}x^{-s}\tilde{w}(x),\qquad x\in\big{(}b^{-1},a^{-1}\big{)}.$

Put

$\displaystyle\Theta(z):=\overset{\circ}{S}(z)\begin{cases}\begin{pmatrix}1&0&-u(z)&0\\\
0&1&0&0\\\ 0&0&1&0\\\ 0&0&0&1\end{pmatrix}\\!,&|z|<1,\vspace{1mm}\\\
\begin{pmatrix}1&0&0&0\\\ 0&1&0&-\tilde{u}(z)\\\ 0&0&1&0\\\
0&0&0&1\end{pmatrix}\\!,&|z|>1.\end{cases}$

It can be checked that $\Theta$ does not have jumps on the intervals $(a,b)$
and $\big{(}b^{-1},a^{-1}\big{)}$. We have arrived at the following model
Riemann–Hilbert problem on the unit circle:

  * •

RH-$\Theta$1 $\Theta$ is holomorphic in ${\mathbb{C}}\setminus{\mathbb{T}}$.

  * •

RH-$\Theta$2 $\Theta_{+}(z)=\Theta_{-}(z)J_{\Theta}(z)$, for
$z\in{\mathbb{T}}$, where

$\displaystyle J_{\Theta}(z)=\begin{pmatrix}0&0&0&-\phi(z)\\\
\displaystyle\frac{\tilde{u}(z)}{\phi(z)}&0&\displaystyle\tilde{\phi}(z)-\frac{u(z)\tilde{u}(z)}{\phi(z)}&0\\\
0&\displaystyle-\frac{1}{\tilde{\phi}(z)}&0&0\\\
\displaystyle\frac{1}{\phi(z)}&0&\displaystyle-\frac{u(z)}{\phi(z)}&0\end{pmatrix}\\!.$

  * •

RH-$\Theta$3 As $z\to\infty$, we have $\Theta(z)=\displaystyle
I+\mathcal{O}\big{(}z^{-1}\big{)}$,

where, in RH-$\Theta$3 we have used the fact that $\tilde{u}(\infty)=u(0)=0$.

######  Remark 3.8.

Recalling Section 2.6, we note that this is exactly the model Riemann–Hilbert
problem for the pair $(\phi,-\tilde{u})$. Hence, it can be concluded that the
study of the Toeplitz+Hankel determinants both when the Hankel symbol is
supported on the unit circle and also when it is supported on the interval
$[a,b]$, reduces to the study of the model Riemann–Hilbert problem
RH-$\Lambda$1 through RH-$\Lambda$3.

For a specific class of symbols $\phi$ and $w$, we will present the solution
to the model Riemann–Hilbert problem RH-$\Lambda$1 through RH-$\Lambda$3 for
the pair $(\phi,w)$ in the next section.

##  4 Analysis of the model problem and a solvable pair

As mentioned before, it is an ambitious task to classify all the pairs
$(\phi,w)$ for which the $\Lambda$-model Riemann–Hilbert problem is solvable.
However, it is reasonable to start our analysis with the class of symbols
(1.4) considered in [2]. Since in our work the symbols are not assumed to be
of the Fisher–Hartwig type (which needs a more delicate treatment, see Section
5.4), we should still expect that the model Riemann–Hilbert problem be
solvable for the class of symbols (1.4) when there is no Fisher–Hartwig
singularity($a_{0}(z)=b_{0}(z)\equiv 1$). Indeed this is the case as will be
elaborated in this section. As commented in the beginning of Section 2.1,
asymptotics of $D_{n}(\phi,d\phi;r,s)$, for general $r$ and $s$ requires a
more delicate approach (see Section 5.2) and we do not discuss the details
here. So let us consider $D_{n}(\phi,d\phi;1,1)$, where $d$ is of Szegő-type
and further satisfies the condition $d(z)\tilde{d}(z)=1$ on the unit circle.
For instance, a class of functions satisfying these conditions is given by

$\displaystyle d(z)=\prod^{m}_{j=1}d_{j}(z),\qquad
d_{j}(z)=\pm\bigg{(}\frac{z-b_{j}}{z-a_{j}}\bigg{)}^{\alpha_{j}}\bigg{(}\frac{a_{j}z-1}{b_{j}z-1}\bigg{)}^{\alpha_{j}},$
(4.1)

where $\alpha_{j}\in{\mathbb{C}}$, all factors are defined by their principal
branch, and

$\displaystyle 0<a_{1}<b_{1}<a_{2}<b_{2}<\cdots<a_{m}<b_{m}<1.$

Note that a similar construction can be found for
$-1<b_{m}<a_{m}<\cdots<b_{1}<a_{1}<0$, and thus a larger class of functions
can be found from multiplying functions of the first class with those of the
second class. Although we have a class of functions satisfying the required
properties, a complete categorization of functions satisfying the four
required properties for $d$ is yet to be found. We emphasize that the
conditions $d(\pm 1)=1$ required in [2] do not play a role in the
Riemann–Hilbert analysis. Indeed, for $d$ as defined in (4.1) one can check
that $d(\pm 1)=(-1)^{\epsilon_{0}}$, where $\epsilon_{0}$ is the number of the
$d_{j}$-factors in whose definition the sign “$-$” is taken. So in this sense
we are considering functions $d$ which are slightly more general than those
considered in [2]. At the same time, we have our technical assumption of
analyticity of the symbols in a neighborhood of the unit circle which is not
needed in the analysis of [2].

Note that the condition $d\tilde{d}=1$ on the unit circle renders the
23-element of the jump matrix $J_{\Lambda}$ zero; indeed

$\displaystyle
J_{\Lambda,23}(z)=\displaystyle\tilde{\phi}(z)-\frac{w(z)\tilde{w}(z)}{\phi(z)}=\tilde{\phi}(z)\big{(}1-d(z)\tilde{d}(z)\big{)}=0.$

Hence, for the particular choices made above, the jump matrix $G_{\Lambda}$
reduces to

$\displaystyle J_{\Lambda}(z)=\begin{pmatrix}0&0&0&-\phi(z)\\\
\displaystyle-d(z)&0&0&0\\\ 0&\displaystyle-\frac{1}{\tilde{\phi}(z)}&0&0\\\
\displaystyle\frac{1}{\phi(z)}&0&\displaystyle\frac{\tilde{w}(z)}{\phi(z)}&0\end{pmatrix}\\!.$
(4.2)

In order to factorize $J_{\Lambda}$, let us first consider the following Szegő
functions

$\displaystyle\alpha(z)=\exp\bigg{[}\frac{1}{2\pi{\rm
i}}\int_{{\mathbb{T}}}\frac{\ln(\phi(\tau))}{\tau-z}\,{\rm
d}\tau\bigg{]},\qquad\beta(z)=\exp\bigg{[}\frac{1}{2\pi{\rm
i}}\int_{{\mathbb{T}}}\frac{\ln(d(\tau))}{\tau-z}\,{\rm d}\tau\bigg{]}.$ (4.3)

By Plemelj–Sokhotskii formula $\alpha$, $\beta$, $\tilde{\alpha}$ and
$\tilde{\beta}$ satisfy the following jump conditions on the unit circle:

$\displaystyle\begin{split}&\alpha_{+}(z)=\alpha_{-}(z)\phi(z),\qquad\beta_{+}(z)=\beta_{-}(z)d(z),\\\
&\tilde{\alpha}_{-}(z)=\tilde{\alpha}_{+}(z)\tilde{\phi}(z),\qquad\tilde{\beta}_{-}(z)=\tilde{\beta}_{+}(z)\tilde{d}(z).\end{split}$
(4.4)

It turns out that knowing the value of $\beta(0)$ is crucial for finding an
asymptotic expression for $h_{n}$ (see Section 4.2) and the condition
$d\tilde{d}=1$ on the unit circle allows us to evaluate $\beta(0)$ easily.
Indeed

$\displaystyle\int_{{\mathbb{T}}}\ln(d(\tau))\,\frac{{\rm
d}\tau}{\tau}=\int_{{\mathbb{T}}}\ln\big{(}\tilde{d}(\tau)\big{)}\,\frac{{\rm
d}\tau}{\tau}=\int_{{\mathbb{T}}}\ln\big{(}d^{-1}(\tau)\big{)}\,\frac{{\rm
d}\tau}{\tau}=-\int_{{\mathbb{T}}}\ln(d(\tau))\,\frac{{\rm d}\tau}{\tau}.$

Thus

$\displaystyle\int_{{\mathbb{T}}}\ln(d(\tau))\frac{{\rm
d}\tau}{\tau}=0,\qquad\mbox{and therefore,}\qquad\beta(0)=1.$ (4.5)

Next, we show that $\beta=\tilde{\beta}$. Note that

$\displaystyle\begin{split}\tilde{\beta}(z)&=\exp\bigg{[}\frac{1}{2\pi{\rm
i}}\int_{{\mathbb{T}}}\frac{\ln(d(\tau))}{\tau-z^{-1}}\,{\rm
d}\tau\bigg{]}=\exp\bigg{[}-\frac{z}{2\pi{\rm
i}}\int_{{\mathbb{T}}}\frac{\ln(d(\tau))}{\tau^{-1}-z}\,\frac{{\rm
d}\tau}{\tau}\bigg{]}\\\ &=\exp\bigg{[}-\frac{z}{2\pi{\rm
i}}\int_{{\mathbb{T}}}\frac{\ln\big{(}\tilde{d}(\tau)\big{)}}{\tau-z}\frac{{\rm
d}\tau}{\tau}\bigg{]}=\exp\bigg{[}\frac{z}{2\pi{\rm
i}}\int_{{\mathbb{T}}}\frac{\ln(d(\tau))}{\tau-z}\,\frac{{\rm
d}\tau}{\tau}\bigg{]},\end{split}$

where we have again used the fact that $d\tilde{d}\equiv 1$ on the unit
circle. Using

$\displaystyle\frac{1}{(\tau-z)\tau}=\frac{z^{-1}}{\tau-z}-\frac{z^{-1}}{\tau},$

we can write the last expression for $\tilde{\beta}$ as

$\displaystyle\tilde{\beta}(z)=\exp\bigg{[}\frac{1}{2\pi{\rm
i}}\int_{{\mathbb{T}}}\frac{\ln(d(\tau))}{\tau-z}\,{\rm
d}\tau-\frac{1}{2\pi{\rm i}}\int_{{\mathbb{T}}}\frac{\ln(d(\tau))}{\tau}\,{\rm
d}\tau\bigg{]}=\frac{\beta(z)}{\beta(0)}=\beta(z),$

by (4.5). To show that $\beta=\tilde{\beta}$ one could also argue that they
both solve the same scalar RHP which has a unique solution. We also note that
$\alpha(z),\beta(z)=1+\mathcal{O}(z^{-1})$, and
$\tilde{\alpha}(z)=\alpha(0)(1+\mathcal{O}(z^{-1}))$ as $z\to\infty$. Now we
can write the solution of the $\Lambda$-RHP (in the case $d\tilde{d}\equiv 1$
on ${\mathbb{T}}$) as

$\displaystyle\Lambda(z)=\Lambda^{-1}_{\infty}\\!\begin{pmatrix}1&0&0&0\\\
\mathcal{C}_{\rho}(z)&1&0&0\\\ 0&0&1&0\\\
0&0&0&1\end{pmatrix}\displaystyle\\!\times\\!\begin{cases}\begin{pmatrix}\displaystyle-\beta(z)&0&0&0\\\
0&0&\displaystyle\frac{1}{\tilde{\alpha}(z)\beta(z)\alpha(z)}&0\\\
0&\displaystyle-\tilde{\alpha}(z)&0&0\\\
0&0&0&\displaystyle-\alpha(z)\end{pmatrix}\\!,&|z|<1,\\\
\begin{pmatrix}0&\beta(z)&0&0\\\
0&0&0&\displaystyle\frac{1}{\beta(z)\tilde{\alpha}(z)\alpha(z)}\\\
0&0&\tilde{\alpha}(z)&0\\\ \alpha(z)&0&0&0\end{pmatrix}\\!,&|z|>1,\end{cases}$
(4.6)

where $\mathcal{C}_{f}(z)$ is the Cauchy-transform of $f(z)$:

$\displaystyle\mathcal{C}_{f}(z)=\frac{1}{2\pi{\rm
i}}\int_{{\mathbb{T}}}\frac{f(\tau)}{\tau-z}\,{\rm d}\tau,$

and

$\displaystyle\Lambda^{-1}_{\infty}=\begin{pmatrix}0&0&0&1\\\ 1&0&0&0\\\
0&0&\displaystyle\frac{1}{\alpha(0)}&0\\\
0&\alpha(0)&0&0\end{pmatrix}\\!,\qquad\rho(z)=\displaystyle-\frac{1}{\beta_{-}(z)\beta_{+}(z)\tilde{\alpha}_{-}(z)\alpha_{+}(z)}.$
(4.7)

Using (4.4), the Plemelj–Sokhotskii formula and general properties of the
Cauchy integral, it can be checked that $\Lambda$ given by (4.6) satisfies the
$\Lambda$-RHP.

###  4.1 The small-norm Riemann–Hilbert problem associated to
$\boldsymbol{D_{n}(\phi,d\phi,1,1)}$

Let us consider

$\displaystyle R(z;n):=S(z;n)\overset{\circ}{S}(z)^{-1}.$

This function clearly has no jumps on $\Gamma_{i}$, $\Gamma_{o}$ and
${\mathbb{T}}$, since $S$ and $\overset{\circ}{S}$ have the same jumps on
these contours. Thus, $R$ satisfies the following small-norm Riemann–Hilbert
problem

  * •

RH-$R$1 $R$ is holomorphic in ${\mathbb{C}}\setminus\Gamma_{R}$.

  * •

RH-$R$2 $R_{+}(z;n)=R_{-}(z;n)J_{R}(z;n)$, for $z\in\Gamma_{R}$.

  * •

RH-$R$3 As $z\to\infty$, $R(z;n)=I+\mathcal{O}\big{(}z^{-1}\big{)}$,

where $\Gamma_{R}:=\Gamma_{i}^{\prime}\cup\Gamma_{o}^{\prime}$, and $J_{R}$ is
given by

$\displaystyle
J_{R}(z;n)=\overset{\circ}{S}(z)J_{S}(z;n)\overset{\circ}{S}(z)^{-1}=\begin{cases}\overset{\circ}{S}(z)J_{T,i}(z;n)\overset{\circ}{S}(z)^{-1},&z\in\Gamma_{i}^{\prime},\\\
\overset{\circ}{S}(z)J_{T,o}(z;n)\overset{\circ}{S}(z)^{-1},&z\in\Gamma_{o}^{\prime}.\end{cases}$

Using (4.6), (4.7), (2.56), (2.55) and (2.52) we find

$\displaystyle
J_{R}(z;n)-I=\begin{cases}z^{n}\begin{pmatrix}0&g_{12}(z)&0&g_{14}(z)\\\
0&0&g_{23}(z)&0\\\ 0&0&0&0\\\
0&0&g_{43}(z)&0\end{pmatrix}\\!,&z\in\Gamma_{i}^{\prime},\\\
z^{-n}\begin{pmatrix}0&0&0&0\\\ g_{21}(z)&0&0&0\\\ 0&g_{32}(z)&0&g_{34}(z)\\\
g_{41}(z)&0&0&0\end{pmatrix}\\!,&z\in\Gamma_{o}^{\prime},\end{cases}$ (4.8)

where

$\displaystyle
g_{12}(z)=-\frac{\alpha(z)}{\phi(z)\beta(z)}-\frac{\tilde{w}(z)C_{\rho}(z)}{\phi(z)\beta(z)\tilde{\alpha}(z)},\qquad
g_{14}(z)=\frac{\tilde{w}(z)}{\phi(z)\beta(z)\tilde{\alpha}(z)\alpha(0)},$
$\displaystyle
g_{23}(z)=-\frac{\alpha(0)\tilde{w}(z)\beta(z)}{\tilde{\phi}(z)\tilde{\alpha}(z)},\qquad
g_{43}(z)=-\alpha^{2}(0)\bigg{(}\frac{\alpha(z)\beta(z)}{\tilde{\phi}(z)}+\frac{\beta(z)\tilde{w}(z)C_{\rho}(z)}{\tilde{\alpha}(z)\tilde{\phi}(z)}\bigg{)},$
$\displaystyle g_{21}(z)=\frac{w(z)\beta(z)}{\phi(z)\alpha(z)},\qquad
g_{32}(z)=-\frac{1}{\alpha(0)\tilde{\phi}(z)}\bigg{(}\frac{\tilde{\alpha}(z)}{\beta(z)}-w(z)\tilde{\alpha}^{2}(z)\beta(z)\alpha(z)C_{\rho}(z)\bigg{)},$
$\displaystyle
g_{34}(z)=\frac{w(z)\tilde{\alpha}^{2}(z)\beta(z)\alpha(z)}{\tilde{\phi}(z)\alpha^{2}(0)},\qquad
g_{41}(z)=-\frac{\alpha(0)}{\phi(z)}\bigg{(}\frac{1}{\tilde{\alpha}(z)\beta(z)\alpha^{2}(z)}-\frac{w(z)\beta(z)C_{\rho}(z)}{\alpha(z)}\bigg{)}.$

From (4.8) it follows that the jump matrix $J_{R}$ satisfies on $\Gamma_{R}$
the small-norm estimate,

$\displaystyle||J_{R}-I||_{L_{2}\cap L_{\infty}}\leq C{\rm e}^{-cn},$ (4.9)

for some positive $C$ and $c=-\log r_{1}$, where $r_{1}$ is any number
satisfying the condition $r_{0}<r_{1}<1$ (see (1.7) and (1.8) for the
definition and meaning of the number $r_{0}$). Therefore, by standard theory
of small-norm Riemann–Hilbert problems [16, 17], there exists $n_{*}$ such
that for all $n>n_{*}$ the $R$-RH problem is solvable and

$\displaystyle R(z)=I+R_{1}(z)+R_{2}(z)+R_{3}(z)+\cdots,\qquad
z\in{\mathbb{C}}\setminus\Gamma_{R},\qquad n\geq n_{*},$

where each $R_{k}$ is of order $O\big{(}{\rm e}^{-kcn}\big{)}$ and they can be
found recursively from

$\displaystyle R_{k}(z)=\frac{1}{2\pi{\rm
i}}\int_{\Gamma_{R}}\frac{[R_{k-1}(\mu)]_{-}(J_{R}(\mu)-I)}{\mu-z}{\rm
d}\mu,\qquad z\in{\mathbb{C}}\setminus\Gamma_{R},\qquad k\geq 1.$ (4.10)

Note that this recurrence also means that

$\displaystyle R_{k+1}(n)=o(R_{k}(n)),\qquad n\to\infty.$

In particular,

$\displaystyle R_{1}(z;n)=\frac{1}{2\pi{\rm
i}}\int_{\Gamma_{R}}\frac{J_{R}(\mu;n)-I}{\mu-z}\,{\rm d}\mu$
$\displaystyle\phantom{R_{1}(z;n)}=\begin{pmatrix}0&R_{1,12}(z;n)&0&R_{1,14}(z;n)\\\
R_{1,21}(z;n)&0&R_{1,23}(z;n)&0\\\ 0&R_{1,32}(z;n)&0&R_{1,34}(z;n)\\\
R_{1,41}(z;n)&0&R_{1,43}(z;n)&0\end{pmatrix}\\!,$ (4.11)

where

$\displaystyle\begin{split}&R_{1,jk}(z;n)=\frac{1}{2\pi{\rm
i}}\int_{\Gamma^{\prime}_{i}}\frac{\mu^{n}g_{jk}(\mu)}{\mu-z}\,{\rm
d}\mu,\qquad jk=12,14,23,43,\\\ &R_{1,jk}(z;n)=\frac{1}{2\pi{\rm
i}}\int_{\Gamma^{\prime}_{o}}\frac{\mu^{-n}g_{jk}(\mu)}{\mu-z}\,{\rm
d}\mu,\qquad jk=21,32,34,41.\end{split}$ (4.12)

Also from (4.10) we can write an expression for $R_{2}$:

$\displaystyle R_{2}(z;n)=\frac{1}{2\pi{\rm
i}}\int_{\Gamma_{R}}\frac{[R_{1}(\mu;n)]_{-}(J_{R}(\mu;n)-I)}{\mu-z}\,{\rm
d}\mu$
$\displaystyle\phantom{R_{2}(z;n)}=\begin{pmatrix}R_{2,11}(z;n)&0&R_{2,13}(z;n)&0\\\
0&R_{2,22}(z;n)&0&R_{2,24}(z;n)\\\ R_{2,31}(z;n)&0&R_{2,33}(z;n)&0\\\
0&R_{2,42}(z;n)&0&R_{2,44}(z;n)\end{pmatrix}\\!,$

where

$\displaystyle
R_{2,kj}(z;n)=\begin{cases}\displaystyle\sum_{\ell\in\\{2,4\\}}\frac{1}{2\pi{\rm
i}}\int_{\Gamma^{\prime}_{o}}\frac{\mu^{-n}[R_{1,k\ell}(\mu;n)]_{-}g_{\ell
j}(\mu)}{\mu-z}\,{\rm d}\mu,\qquad j=1,\ k=1,3,\\\
\displaystyle\sum_{\ell\in\\{2,4\\}}\frac{1}{2\pi{\rm
i}}\int_{\Gamma^{\prime}_{i}}\frac{\mu^{n}[R_{1,k\ell}(\mu;n)]_{-}g_{\ell
j}(\mu)}{\mu-z}\,{\rm d}\mu,\qquad j=3,\ k=1,3,\\\
\displaystyle\frac{1}{2\pi{\rm
i}}\int_{\Gamma^{\prime}_{i}}\frac{\mu^{n}[R_{1,k1}(\mu;n)]_{-}g_{1j}(\mu)}{\mu-z}\,{\rm
d}\mu\\\ \phantom{\frac{1}{2\pi{\rm
i}}\int_{\Gamma^{\prime}_{i}}}\displaystyle+\frac{1}{2\pi{\rm
i}}\int_{\Gamma^{\prime}_{o}}\frac{\mu^{-n}[R_{1,k3}(\mu;n)]_{-}g_{3j}(\mu)}{\mu-z}\,{\rm
d}\mu,\qquad k,j=2,4.\end{cases}$

Moreover, using (4.10) and a straightforward calculation one can justify that
the matrix structure (i.e., the location of zero and nonzero elements) of
$R_{2k+1}$ and $R_{2k}$, $k\geq 1$, are similar to that of $R_{1}$ and
$R_{2}$, respectively. It is also straightforward to show that

$\displaystyle R_{k,ij}(z;n)=\frac{{\cal O}{\big{(}{\rm
e}^{-kcn}\big{)}}}{|z|+1},\qquad n\to\infty,\qquad k\geq 1,$

uniformly for $z\in{\mathbb{C}}\setminus\Gamma_{R}$, and the positive constant
$c$ is the same as in (4.9).

###  4.2 Asymptotics of $\boldsymbol{h_{n}}$

The analysis of the previous section shows that the $\mathcal{X}$-RH problem
has unique solution for all $n>n_{*}$. We now proceed to the reconstruction of
the corresponding functions $\overset{\circ}{\mathcal{X}}(z;n)$,
$\mathcal{Y}(z;n)$, and to the asymptotics of $h_{n}$. To this end we need the
asymptotic information about the matrix $P(n)$ which is needed to determine
constants $C_{j}$ that participate in equation (2.49).

Tracing back the Riemann–Hilbert transformations, we find that for
$z\in\Omega_{0}$ we have

$\mathcal{X}(z;n)=R(z;n)\Lambda(z)\begin{pmatrix}1&0&0&0\\\ 0&z^{-n}&0&0\\\
0&0&1&0\\\ 0&0&0&z^{n}\end{pmatrix}\\!.$ (4.13)

From (2.48) and (4.13) we conclude that

$P(n)=R(0;n)\Lambda(0).$ (4.14)

From (4.11) we arrive at the estimate

$\displaystyle R(0;n)=I+R_{1}(0;n)+O\big{(}{\rm e}^{-2cn}\big{)}$
$\displaystyle\phantom{R(0;n)}=\begin{pmatrix}1&R_{1,12}(0;n)&0&R_{1,14}(0;n)\\\
R_{1,21}(0;n)&1&R_{1,23}(0;n)&0\\\ 0&R_{1,32}(0;n)&1&R_{1,34}(0;n)\\\
R_{1,41}(0;n)&0&R_{1,43}(0;n)&1\end{pmatrix}+O\big{(}{\rm e}^{-2cn}\big{)}.$
(4.15)

Simultaneously, from (4.6) and (4.7) we have

$\displaystyle\Lambda(0)=\begin{pmatrix}0&0&0&-\alpha(0)\\\ -1&0&0&0\\\
0&-\displaystyle\frac{1}{\alpha(0)}&0&0\\\
-C_{\rho}(0)\alpha(0)&0&1&0\end{pmatrix}\\!.$ (4.16)

Equations (4.14), (4.15) and (4.16) yield the following asymptotic formula for
$P(n)$,

$\displaystyle
P(n)\\!=\\!\\!\begin{pmatrix}-C_{\rho}(0)\alpha(0)R_{1,14}(0;\\!n)\\!-\\!R_{1,12}(0;\\!n)&0&R_{1,14}(0;\\!n)&-\alpha(0)\\\\[4.30554pt]
-1&-\displaystyle\frac{R_{1,23}(0;\\!n)}{\alpha(0)}&0&-\alpha(0)R_{1,21}(0;\\!n)\\\\[8.61108pt]
-C_{\rho}(0)\alpha(0)R_{1,34}(0;\\!n)\\!-\\!R_{1,32}(0;\\!n)&-\displaystyle\frac{1}{\alpha(0)}&R_{1,34}(0;\\!n)&0\\\\[8.61108pt]
-C_{\rho}(0)\alpha(0)&-\displaystyle\frac{R_{1,43}(0;\\!n)}{\alpha(0)}&1&-\alpha(0)R_{1,41}(0;\\!n)\end{pmatrix}$
$\displaystyle\phantom{P(n)}+{\cal O}{\big{(}{\rm e}^{-2cn}\big{)}},$ (4.17)

as $n\to\infty$.

It is time now for the conditions of Lemma 2.7. We are not going to study each
and every condition of this lemma, rather as a case study we consider in
particular the condition (2.36):

$\displaystyle(1-P_{21}(n))P_{42}(n)+P_{22}(n)P_{41}(n)\neq 0.$ (4.18)

From (4.17) we have that

$\displaystyle(1-P_{21}(n))P_{42}(n)+P_{22}(n)P_{41}(n)=-\mathcal{E}(n)+O\big{(}{\rm
e}^{-2cn}\big{)},\qquad n\to\infty,$ (4.19)

where (cf. (1.6))

$\displaystyle\mathcal{E}(n):=\frac{2}{\alpha(0)}R_{1,43}(0;n)-C_{\rho}(0)R_{1,23}(0;n).$
(4.20)

This is when we arrive at condition (1.9) of Theorem 1.1. Indeed, let us
suppose that there exist such $C>0$ and $n_{0}\geq n_{*}$ that

$\displaystyle|\mathcal{E}(n)|\geq Cr^{n},\qquad\mbox{for some}\quad r\colon
r_{0}\leq r<1,\qquad\mbox{and}\qquad n>n_{0},$

and choose $r_{1}$ in the definition of the constant $c$ so that $r<r_{1}<1$
and $r^{2}_{1}<r$. Then, estimate (4.19) can be rewritten as

$\displaystyle(1-P_{21}(n))P_{42}(n)+P_{22}(n)P_{41}(n)=-\mathcal{E}(n)\big{(}1+O\big{(}{\rm
e}^{-c_{1}n}\big{)}\big{)},\qquad n\to\infty,$ (4.21)

where $c_{1}=-\log\big{(}\frac{r^{2}_{1}}{r}\big{)}>0$. The last estimate in
turn means that there exists such $n_{1}\geq n_{0}$ that for all $n>n_{1}+1$
we should have that condition (4.18) holds for $n$ and $n-1$ and hence by
Lemma 2.9 and Corollary (2.10), we can uniquely reconstruct the solution of
the $\mathcal{Y}$-RHP, having already the unique solution of the
$\mathcal{X}$-RHP, and, moreover, we could use equation (cf. (2.51)),

$\displaystyle-\frac{1}{h_{n-1}}=\lim_{z\to
0}z^{n-1}\mathcal{Y}_{21}\big{(}z^{-1};n\big{)}.$ (4.22)

for evaluation of the large $n$ behavior of $h_{n}$. Let us denote

$\displaystyle\mathcal{A}(z;n):=P^{-1}(n)\mathcal{X}(z;n)\begin{pmatrix}1&0&0&0\\\
0&z^{n}&0&0\\\ 0&0&1&0\\\ 0&0&0&z^{-n}\end{pmatrix}\\!,$ (4.23)

and also let us define the matrix $\mathcal{B}(n)$ in the following expansion
for $\mathcal{A}(z;n)$, which is equivalent to RH-$\mathcal{X}$4:

$\displaystyle\mathcal{A}(z;n)=I+\mathcal{B}(n)z+{\cal
O}{\big{(}z^{2}\big{)}},\qquad z\to 0.$ (4.24)

Therefore by (2.29), (2.30), (2.31) and (4.23) we can write

$\displaystyle\overset{\circ}{\mathcal{X}}(z,n)=\begin{pmatrix}C_{1}(n)&1&C_{3}(n)&0\\\
C_{2}(n)&0&C_{4}(n)&1\end{pmatrix}\mathcal{A}(z;n)\begin{pmatrix}1&0&0&0\\\
0&z^{-n}&0&0\\\ 0&0&1&0\\\ 0&0&0&z^{n}\end{pmatrix}\\!.$ (4.25)

Using (2.19) and (4.25) we can write

$\displaystyle\mathcal{Y}_{21}\big{(}z^{-1};n\big{)}=\overset{\circ}{\mathcal{X}}_{22}(z;n)=C_{2}(n)\mathcal{A}_{12}(z;n)z^{-n}\\!+C_{4}(n)\mathcal{A}_{32}(z;n)z^{-n}\\!+\mathcal{A}_{42}(z;n)z^{-n}.\\!$
(4.26)

From (4.24) we have

$\displaystyle
z^{-n}\mathcal{A}(z;n)=z^{-n}I+z^{-n+1}\mathcal{B}(n)+\mathcal{O}\big{(}z^{-n+2}\big{)},\qquad
z\to 0.$

Therefore, as $z\to 0$

$\displaystyle
z^{-n}\mathcal{A}_{ij}(z;n)=\begin{cases}z^{-n+1}\mathcal{B}_{ij}(n)+\mathcal{O}\big{(}z^{-n+2}\big{)},&i\neq
j,\\\
z^{-n}+z^{-n+1}\mathcal{B}_{ii}(n)+\mathcal{O}\big{(}z^{-n+2}\big{)},&i=j.\end{cases}$
(4.27)

Therefore by (4.22), (4.26) and (4.27) we have

$\displaystyle-\frac{1}{h_{n-1}}=C_{2}(n)\mathcal{B}_{12}(n)+C_{4}(n)\mathcal{B}_{32}(n)+\mathcal{B}_{42}(n).$
(4.28)

Let us denote the coefficients in the expansions of $R(z;n)$ and $\Lambda(z)$,
as $z\to 0$, by

$\displaystyle R(z;n)=R(0;n)+R^{(1)}(n)z+R^{(2)}(n)z^{2}+{\cal
O}{\big{(}z^{3}\big{)}},$
$\displaystyle\Lambda(z)=\Lambda(0)+\Lambda^{(1)}z+\Lambda^{(2)}z^{2}+{\cal
O}{\big{(}z^{3}\big{)}}.$

From (4.24), (4.13), and (4.14) we have

$\displaystyle\mathcal{B}(n)=\Lambda^{-1}(0)R^{-1}(0;n)R^{(1)}(n)\Lambda(0)+\Lambda^{-1}(0)\Lambda^{(1)}.$
(4.29)

Note that

$\displaystyle R^{(1)}(n)=\frac{1}{2\pi{\rm
i}}\int_{\Gamma_{R}}\\!(J_{R}(\mu;n)-I)\frac{{\rm
d}\mu}{\mu^{2}}+\mathcal{O}\big{(}{\rm
e}^{-2cn}\big{)},\qquad\\!\\!R^{-1}(0;n)=I-R_{1}(0;n)+{\cal O}{\big{(}{\rm
e}^{-2cn}\big{)}},$

as $n\to\infty$. More precisely, we have

$\displaystyle
R^{(1)}(n)=\begin{pmatrix}0&R^{(1)}_{12}(n)&0&R^{(1)}_{14}(n)\\\
R^{(1)}_{21}(n)&0&R^{(1)}_{23}(n)&0\\\ 0&R^{(1)}_{32}(n)&0&R^{(1)}_{34}(n)\\\
R^{(1)}_{41}(n)&0&R^{(1)}_{43}(n)&0\end{pmatrix}+\mathcal{O}\big{(}{\rm
e}^{-2cn}\big{)},$ (4.30)

as $n\to\infty$, where

$\displaystyle\begin{split}&R^{(1)}_{jk}(n)=\frac{1}{2\pi{\rm
i}}\int_{\Gamma^{\prime}_{i}}\mu^{n-2}g_{jk}(\mu)\,{\rm d}\mu,\qquad\ \
jk=12,14,23,43,\\\ &R^{(1)}_{jk}(n)=\frac{1}{2\pi{\rm
i}}\int_{\Gamma^{\prime}_{o}}\mu^{-n-2}g_{jk}(\mu)\,{\rm d}\mu,\qquad
jk=21,32,34,41,\end{split}$ (4.31)

and

$\displaystyle
R^{-1}(0;n)\\!=\\!\begin{pmatrix}1&-R_{1,12}(0;n)&0&-R_{1,14}(0;n)\\\
-R_{1,21}(0;n)&1&-R_{1,23}(0;n)&0\\\ 0&-R_{1,32}(0;n)&1&-R_{1,34}(0;n)\\\
-R_{1,41}(0;n)&0&-R_{1,43}(0;n)&1\end{pmatrix}\\!+{\cal O}{\big{(}{\rm
e}^{-2cn}\big{)}},$ (4.32) $\displaystyle\qquad n\to\infty.$

Regarding the coefficients of $\Lambda(z)$, we will actually only need the
zero-term, $\Lambda(0)$, which has already been presented in (4.16).

From (4.29), (4.30), (4.32) and (4.16) we find that

$\displaystyle\mathcal{B}_{12}(n)=\frac{R^{(1)}_{23}(n)}{\alpha(0)}+\mathcal{O}\big{(}{\rm
e}^{-2cn}\big{)},\qquad\mathcal{B}_{32}(n)=C_{\rho}(0)R^{(1)}_{23}(n)-\frac{R^{(1)}_{43}(n)}{\alpha(0)}+\mathcal{O}\big{(}{\rm
e}^{-2cn}\big{)},$
$\displaystyle\mathcal{B}_{42}(n)=-\frac{1}{\alpha^{2}(0)}\big{(}R_{1,12}(0;n)R^{(1)}_{23}(n)+R_{1,14}(0;n)R^{(1)}_{43}(n)\big{)}+\mathcal{O}\big{(}{\rm
e}^{-3cn}\big{)}.$ (4.33)

Note that $\mathcal{B}_{12}(n)$, $\mathcal{B}_{32}(n)$ are of order ${\cal
O}{\big{(}{\rm e}^{-cn}\big{)}}$, while $\mathcal{B}_{42}(n)$ is of order
${\cal O}{\big{(}{\rm e}^{-2cn}\big{)}}$.

Revisiting (2.31) we have

$\displaystyle\begin{pmatrix}1&C_{1}(n)&0&C_{3}(n)\\\
0&C_{2}(n)&1&C_{4}(n)\end{pmatrix}P(n)=\begin{pmatrix}C_{1}(n)&1&C_{3}(n)&0\\\
C_{2}(n)&0&C_{4}(n)&1\end{pmatrix}\\!.$

In particular, in view of (4.18), we can write the following two equations for
the constants $C_{2}$ and $C_{4}$

$\displaystyle C_{2}(n)P_{21}(n)+P_{31}(n)+C_{4}(n)P_{41}(n)=C_{2}(n),$
$\displaystyle C_{2}(n)P_{22}(n)+P_{32}(n)+C_{4}(n)P_{42}(n)=0.$

Solving for $C_{2}$ and $C_{4}$ we find

$\displaystyle
C_{2}(n)=\frac{P_{42}(n)P_{31}(n)-P_{41}(n)P_{32}(n)}{(1-P_{21}(n))P_{42}(n)+P_{41}(n)P_{22}(n)},$
$\displaystyle
C_{4}(n)=-\frac{P_{22}(n)P_{31}(n)+[1-P_{21}(n)]P_{32}(n)}{(1-P_{21}(n))P_{42}(n)+P_{41}(n)P_{22}(n)}.$

From (4.17) and (4.21) we have

$\displaystyle C_{2}(n)=\frac{C_{\rho}(0)}{\mathcal{E}(n)}\big{(}1+{\cal
O}{\big{(}{\rm e}^{-c_{1}n}\big{)}}\big{)},$ (4.34)

and

$\displaystyle C_{4}(n)=-\frac{2}{\alpha(0)\mathcal{E}(n)}\big{(}1+{\cal
O}{\big{(}{\rm e}^{-c_{1}n}\big{)}}\big{)}.$ (4.35)

Combining (4.28), (4.33), (4.34) and (4.35) we obtain

$\displaystyle
h_{n-1}=-\alpha(0)\frac{\mathcal{E}(n)}{\frac{2}{\alpha(0)}R^{(1)}_{43}(n)-C_{\rho}(0)R^{(1)}_{23}(n)}\big{(}1+{\cal
O}{\big{(}{\rm e}^{-c_{1}n}\big{)}}\big{)},\qquad n\to\infty.$ (4.36)

Note that from (4.12) and (4.31) we have

$\displaystyle R_{1,jk}(0;n)=R^{(1)}_{jk}(n+1),\qquad\mbox{for}\quad
jk=12,14,23,43,$ $\displaystyle
R_{1,jk}(0;n)=R^{(1)}_{jk}(n-1),\qquad\mbox{for}\quad jk=21,32,34,41.$

This allows us to rewrite (4.36) as

$\displaystyle
h_{n-1}=-\alpha(0)\frac{\mathcal{E}(n)}{\mathcal{E}(n-1)}\big{(}1+{\cal
O}{\big{(}{\rm e}^{-c_{1}n}\big{)}}\big{)},\qquad n\to\infty.$ (4.37)

This concludes the proof of Theorem 1.1.

Let $\mathfrak{C}$ denote the class of symbol pairs $(\phi,d\phi)$, where
$\phi$ and $d$ satisfy the properties mentioned in Theorem 1.1, for which the
corresponding matrix $P(n)$ satisfies the condition (4.18) for sufficiently
large $n$. Now, consider the subclass $\mathfrak{C}_{0}\subset\mathfrak{C}$,
which includes symbol pairs $(\phi,d\phi)$ for which $\mathcal{E}(n)$
satisfies inequality (1.9), or rather its generalization (1.11) for
sufficiently large $n$ (see also Section 5.5). For the purposes of this paper,
it is worthwhile to prove that the subclass $\mathfrak{C}_{0}$ is not empty by
providing an explicit example. To this end, let us consider the symbol pairs
$(\phi,d\phi)$, where

$\displaystyle\phi(z)=\left(\frac{z-b}{z-a}\right)^{\alpha},$ (4.38)

and

$\displaystyle
d(z)=\left(\frac{z-b_{1}}{z-a_{1}}\right)^{\alpha_{1}}\left(\frac{a_{1}z-1}{b_{1}z-1}\right)^{\alpha_{1}}.$
(4.39)

We assume that $0<a<b<a_{1}<b_{1}<1,$ and $|\Re\alpha_{1}|<1.$ Then, for
$\mathcal{E}(n)$ defined in (4.20), by standard Watson lemma type asymptotic
analysis of integrals we will arrive at

$\displaystyle\mathcal{E}(n)=\kappa
b_{1}^{n-\alpha_{1}}n^{\alpha_{1}-1}\bigg{(}1+\mathcal{O}\bigg{(}\frac{1}{n}\bigg{)}\bigg{)},\qquad
n\to\infty,$ (4.40)

where

$\displaystyle\kappa=-\frac{{\rm i}}{\pi}{\rm e}^{-{\rm
i}\pi\alpha_{1}}\Gamma(1-\alpha_{1})(b_{1}-a_{1})^{\alpha_{1}}\bigg{(}\frac{1-bb_{1}}{1-ab_{1}}\bigg{)}^{\alpha}\bigg{(}\frac{a_{1}}{b_{1}}\bigg{)}^{-\alpha_{1}}$
$\displaystyle\phantom{\kappa=}\times\bigg{[}{\rm e}^{-{\rm
i}\pi\alpha_{1}}\frac{{\rm
i}}{\pi}\int_{1/b_{1}}^{1/a_{1}}\bigg{(}\frac{z-b_{1}}{z-a_{1}}\bigg{)}^{\alpha_{1}}\bigg{(}\frac{1/a_{1}-z}{z-1/b_{1}}\bigg{)}^{-\alpha_{1}}\frac{z+b_{1}}{z-b_{1}}\frac{dz}{z}-1\bigg{]},$
(4.41)

and

$\displaystyle\arg\bigg{(}\frac{z-b_{1}}{z-a_{1}}\bigg{)}=\arg\bigg{(}\frac{1/a_{1}-z}{z-1/b_{1}}\bigg{)}=0,\qquad
z\in[1/b_{1},1/a_{1}].$

We argue, that for generic values of $a$, $b$, $a_{1}$, $b_{1}$, $\alpha$ and
$\alpha_{1}$ the number $\kappa\neq 0$. Indeed, let us take, for instance,
$\alpha_{1}$ real, i.e.,

$\displaystyle-1<\alpha_{1}<1.$

Then, the integrand in (4.41) is strictly positive and hence, except for
$\alpha_{1}=\pm 1/2$, the first term in the brackets of (4.41) has nonzero
imaginary part and thus can not cancel the second term. It is only needed now
to notice that in the case under consideration $r_{0}=b_{1}$ and hence
inequality $\kappa\neq 0$ implies that $\mathcal{E}(n)$ corresponding to the
weights (4.38) and (4.39) satisfies the condition (1.11) indicated in Remark
1.2 with $\varkappa(n)=n^{\alpha_{1}-1}$.

We conclude the discussion of this example by indicating the asymptotic
behavior of the corresponding norm parameter $h_{n-1}$ and the determinant
$D_{n}$. To this end we notice that, in fact, the estimate (4.40) can be
extended to the full asymptotic series,

$\displaystyle\mathcal{E}(n)\sim\kappa
b_{1}^{n-\alpha_{1}}n^{\alpha_{1}-1}\Bigg{(}1+\sum_{k=1}^{\infty}d_{k}n^{-k}\Bigg{)},\qquad
n\to\infty.$

Therefore, from (4.37) we should have that

$\displaystyle
h_{n-1}=-b_{1}\bigg{(}\frac{n}{n-1}\bigg{)}^{\alpha_{1}-1}\bigg{(}1+O\bigg{(}\frac{1}{n^{2}}\bigg{)}\bigg{)},\qquad
n\to\infty,$

and hence

$\displaystyle
D_{n}=C(-b_{1})^{n}n^{\alpha_{1}-1}\bigg{(}1+O\bigg{(}\frac{1}{n}\bigg{)}\bigg{)},\qquad
n\to\infty,$

for some constant $C$ which is yet to be determined. It is interesting that
this formula indicates an oscillatory behavior of the determinant $D_{n}$ for
large $n$. This fact is confirmed numerically.

######  Remark 4.1.

From (2.5), (2.19), (2.29), and (2.30) we have the following representation
for the orthogonal polynomials $\mathcal{P}_{n}(z)$ in terms of the solution
$\mathcal{X}(z;n)$ of the $\mathcal{X}$-RHP,

$\displaystyle\mathcal{P}_{n}(z)=\mathcal{X}_{11}(z;n)+C_{1}(n)\mathcal{X}_{21}(z;n)+C_{3}(n)\mathcal{X}_{41}(z;n).$

The asymptotic results concerning the function $\mathcal{X}(z;n)$ obtained in
this section can be translated to the large $n$ asymptotic formulae for the
polynomials $\mathcal{P}_{n}(z)$. Indeed, skipping the rather tedious though
straightforward calculations, we arrive at the following asymptotics for
$\mathcal{P}_{n}(z)$ on the unit circle:

$\displaystyle\mathcal{P}_{n}(z)=\frac{\beta_{+}(z)(2\alpha(0)C_{\rho,+}(z)-C_{\rho}(0)\alpha(0))}{\mathcal{E}(n)}$
$\displaystyle\phantom{\mathcal{P}_{n}(z)=}{}+\alpha_{-}(z)z^{n}\bigg{(}1+\frac{C_{\rho}(0)\alpha(0)R_{1,21}(z;n)-2R_{1,41}(z;n)}{\mathcal{E}(n)}\bigg{)}+\mathcal{O}\big{(}{\rm
e}^{-c_{1}n}\big{)},$ (4.42)

as $n\to\infty$. While in the interior and exterior of the unit circle we have
the following asymptotic formulae for $\mathcal{P}_{n}(z)$:

$\displaystyle\mathcal{P}_{n}(z)=\frac{\beta(z)(2\alpha(0)C_{\rho}(z)-C_{\rho}(0)\alpha(0))}{\mathcal{E}(n)}+\mathcal{O}\big{(}{\rm
e}^{-c_{1}n}\big{)},\qquad|z|<1,$
$\displaystyle\mathcal{P}_{n}(z)=\alpha(z)z^{n}\bigg{(}1+\frac{C_{\rho}(0)\alpha(0)R_{1,21}(z;n)-2R_{1,41}(z;n)}{\mathcal{E}(n)}+\mathcal{O}\big{(}{\rm
e}^{-2c_{1}n}\big{)}\bigg{)},\qquad|z|>1,$

as $n\to\infty$ (compare with (4.42)).

######  Remark 4.2.

In a similar fashion, we could have obtained the large $n$ asymptotic
expression for the norm $h_{n}$ of orthogonal polynomials (3.1). In that case,
the methods of this section would work analogously if the function

$\displaystyle
d:=-\phi^{-1}(z)\tilde{u}(z)\equiv\phi^{-1}(z)\int_{a}^{b}\frac{t^{s-1}w(t)}{1-tz}\,\mathrm{d}t,$

satisfies $d\tilde{d}\equiv 1$ on the unit circle, or

$\displaystyle\phi(z)\tilde{\phi}(z)=\int_{a}^{b}\frac{t^{s-1}w(t)}{1-tz}\mathrm{d}t\int_{a}^{b}\frac{t^{s-1}w(t)}{1-tz^{-1}}\,\mathrm{d}t,\qquad
z\in{\mathbb{T}}.$ (4.43)

##  5 Remaining open questions

We consider this work as a starting point of a long term research project.
There are many challenging technical as well as conceptual open questions
related to the Riemann–Hilbert formalism we are suggesting in this paper. Here
we highlight some of them that we consider the most pressing.

###  5.1 Derivation of the relevant Christoffel–Darboux formulae
and differential identities

Our main objective in this paper has been to develop a $4\times 4$ steepest
descent analysis for certain Toeplitz+Hankel determinants and we have achieved
that. However, to obtain the asymptotics of $D_{n}(\phi,d\phi,1,1)$ one has to
derive suitable differential identities. We propose that the differential
identity has to be with respect to the parameters $\alpha_{i}$ in the function
$d$ given by (4.1). Thus, one has to perform $m$ integrations in the
parameters $\alpha_{i}$, $1\leq i\leq m$. Note that for
$\alpha_{1}=\alpha_{2}=\cdots=\alpha_{m}=0$, we have $d\equiv 1$ and hence
$\phi=w$. Hence the starting point of the integration in $\alpha_{1}$ could be
taken from the results of [2].555Although the authors in [2] do not
particularly study the asymptotics of $D_{n}(\phi,\phi,1,1)$, this asymptotics
is achievable by their methods. Integration of the differential identity in
$\alpha_{1}$ will provide us with an asymptotic expression for
$D_{n}(\phi,d_{1}\phi,1,1)$, which also serves as the starting point of
integration in $\alpha_{2}$. Thus we can find asymptotics of
$D_{n}(\phi,d_{1}d_{2}\phi,1,1)$ which also serves as the starting point of
integration in $\alpha_{3}$, and so on. Repeating this procedure will finally
lead us to the asymptotics of $D_{n}(\phi,d\phi,1,1)$.

In order to derive the differential identities mentioned above, one has to
find recurrence relations and prove a Christoffel–Darboux formula for the
polynomials (2.1) and follow a path similar to that introduced by I. Krasovsky
in [20]. Note that the recurrence relations can be found by analyzing the
function $\mathcal{M}(z;n):=\mathcal{X}(z;n+1)\mathcal{X}^{-1}(z;n)$, which is
holomorphic in ${\mathbb{C}}\setminus\\{0\\}$ and can be globally determined
by its singular parts at zero and infinity.

###  5.2 Extension of the analysis to general offset values
$\boldsymbol{r,s\neq 1}$

If we lift the restriction $r=s=1$ then in the jump matrix of the
$\mathcal{X}$-RHP the functions $\phi(z)$ and $w(z)$ should be replaced by
$z^{1-r}\phi(z)$ and $z^{1-s}w(z)$, respectively. This, in turn would raise a
serious question about solvabilty of the $\Lambda$-RHP. Indeed, for instance,
we would not be able to define the function $\alpha$ in (4.3) and hence to
factorize the jump matrix (4.2). The way out of this difficulty could be the
use of the relation between the determinants $D_{n}(\phi,w;r,s)$ with
different values of $r$, $s$ or both. Such relations are well known in the
pure Toeplitz case (for example see [13, Lemma 2.4]). However, for general
Toeplitz+Hankel determinants they are yet to be found. Another way to approach
the problem could be to develop the so-called Bäklund–Schlesinger
transformations of the $\mathcal{X}$-RHP itself. That is, the transformations
of the form,

$\displaystyle\mathcal{X}(z)\mapsto R(z)\mathcal{X}(z),$

where $R(z)$ is a properly chosen rational function for which the above
transformation results in the desired shifting of the parameters $r$ and $s$.
Also, one can try to allow the matrix $P(n)$ in the setting of the
$\mathcal{X}$-RHP to depend on $z$.

It is worth to point out that the problem with the extension of our scheme to
the general values of $r$ and $s$ is not actually a problem of the setting of
the relevant Riemann–Hilbert problem. Indeed, it is rather the question of the
correct way to approach to its asymptotic solution. Let us demonstrate this
point by considering the pure Toeplitz case.

Assume that $D_{n}(\phi,0;r,0)\neq 0$ for all $n$, so that
$\mathcal{P}_{n}(z)$ and, correspondingly, the solution $\mathcal{Y}(z;n)$ of
the $\mathcal{Y}$-RHP exist for all $n$. Put,

$\displaystyle\mathcal{X}(z;n):=P^{-1}_{\infty}(n)\begin{pmatrix}\mathcal{Y}_{11}(z;n)&0&0&\widetilde{\mathcal{Y}}_{12}(z;n)\\\
0&\widetilde{\mathcal{Y}}_{11}(z;n)&\mathcal{Y}_{12}(z;n)&0\\\
0&\widetilde{\mathcal{Y}}_{21}(z;n)&\mathcal{Y}_{22}(z;n)&0\\\
\mathcal{Y}_{21}(z;n)&0&0&\widetilde{\mathcal{Y}}_{22}(z;n)\end{pmatrix}\\!,$

where the normalizing matrix $P_{\infty}(n)$ is given by

$\displaystyle P_{\infty}(n)=\begin{pmatrix}1&0&0&\mathcal{Y}_{12}(0;n)\\\
0&\mathcal{Y}_{11}(0;n)&0&0\\\ 0&\mathcal{Y}_{21}(0;n)&1&0\\\
0&0&0&\mathcal{Y}_{22}(0;n)\end{pmatrix}\\!,$

which is invertible for generic $\phi$. It is straightforward to check that
the so defined $4\times 4$ matrix-valued function $\mathcal{X}$ solves the
$\mathcal{X}$-Riemann–Hilbert problem, RH-$\mathcal{X}$1–RH-$\mathcal{X}$4
with $w\equiv 0$, and $\phi(z)$ replaced by $z^{1-r}\phi(z)$. Take now $r=0$.
That is, let us consider the standard orthogonal polynomials on the circle
with the weight $\phi(z)$ having zero winding number. Then, from the standard
$2\times 2$ Riemann–Hilbert formalism [1], we know everything about the
asymptotic behavior of the corresponding orthogonal polynomials
$\mathcal{P}_{n}(z)$ and hence we know asymptotic solution of the
$\mathcal{X}$-Riemann–Hilbert problem corresponding to $D_{n}(\phi,0;0,0)$.
And, we know this in spite of the fact that the approach developed in the body
of this work can not be satisfactorily used for the case $r=s=0$. We believe
that this observation might entail a useful hint on how to modify our
Riemann–Hilbert approach to deal with general values of $r$ and $s$.

###  5.3 Extension of the Riemann–Hilbert analysis of Section 3
for more general choices of $\boldsymbol{I}$

We recall that our Riemann–Hilbert analysis of Section 3, with minimal
modifications, naturally extends to the following three cases as well: i)
$-1<a<b<0$, ii) $-\infty<a<b<-1$, and iii) $1<a<b<\infty$. A natural first
step in generalizing the results of Section 3 beyond the above cases is
considering the case where $I$ has $0$ as an end point. This would slightly
affect the analysis as one has to take into account the behavior of $w$ at $0$
in the set up of the $2\times 4$ and the subsequent Riemann–Hilbert problems.

The other interesting case to be studied is when $I$ intersects the unit
circle. Clearly, in this case one has to perform local analysis in a
neighborhood of the intersection point(s) of $I$ and the unit circle. Although
these local constructions are reminiscent of what one does near the
Fisher–Hartwig singularities or the endpoints of the support of the symbol,
here even if the possible intersection points $\pm 1$, are regular points for
non-FH symbols $\phi$ and $w$, one has to still perform local analysis due to
collision of the supports of $\phi$ and $w$.

Another possible generalization would be to consider $I$ to be the union of
two symmetric intervals with respect to the unit circle, i.e.,
$I=[a,b]\cup\big{[}b^{-1},a^{-1}\big{]}$. This generalization should be
accessible by slight modification of our approach explained in Section 3.
However, generalization to the case where $I$ is a union of two non-
symmetrical intervals with respect to the unit circle needs a more special
treatment.

###  5.4 Extension to Fisher–Hartwig symbols

One can study the large-$n$ asymptotics of determinant $D_{n}(\phi,d\phi,1,1)$
(and with increasing effort $D_{n}(\phi,d\phi,r,s)$ for fixed
$r,s\in{\mathbb{Z}}$) assuming that $\phi$ possesses Fisher–Hartwig
singularities $\\{z_{i}\\}^{m}_{i=1}$ on the unit circle. It is in fact in