On a ${\mathbb{C}}^{2}$-valued integral index transform
and bilateral hypergeometric series

Yury A.Neretin111Supported by the grant FWF, P31591.

We discuss the spectral decomposition of the hypergeometric differential
operators on the line $\mathop{\mathrm{Re}}\nolimits z=1/2$, such operators
arise in the problem of decomposition of tensor products of unitary
representations of the universal covering of the group
$\mathrm{SL}(2,{\mathbb{R}})$. Our main purpose is a search of natural bases
in generalized eigenspaces and variants of the inversion formula.

##  1\. Introduction

1.1. The hypergeometric opertor $\mathcal{D}$ on the line. It is well-known
that classical hypergeometric systems of orthogonal polynomials are
eigenfunctions of certain differential or difference operators (see, e.g.,
[13], [9], Sect. 10.8-10.13, 10.21-22, [2], Sect. 6.10, Ex.6.29-6.37). On the
other hand many classical integral transforms, as the Hankel transform, the
Kontorovich–Lebedev transform, the ${}_{2}F_{1}$-Wimp transform, the Jacobi
transform (synonyms: the Olevskiĭ transform, the generalized Mehler–Fock
transform), etc., can be obtained as spectral decompositions of certain
differential or difference operators with continuous spectra (see collections
of examples with differential operators in [28], Chapter 4, [7], Sect.
XIII.8).

We consider the following differential operator

(1.1)
$\mathcal{D}:=\frac{d}{dx}\Bigl{(}\frac{1}{4}+x^{2}\Bigr{)}\frac{d}{dx}+\frac{(\alpha+i\beta)^{2}}{4(1/2+ix)}+\frac{(\alpha-i\beta)^{2}}{4(1/2-ix)}+\frac{1}{4}$

in $L^{2}({\mathbb{R}})$. The parameters $\alpha$, $\beta$ are real. Clearly,
the replacing $(\alpha,\beta)$ by $(-\alpha,-\beta)$ does not change the
operator, so we can assume $\alpha\geqslant 0$.

As an algebraic expression $\mathcal{D}$ is a hypergeometric differential
operator, spectral expansions of similar operators produce the Jacobi
polynomials (see, e.g.,[13], Sect.9.8) and the ’Jacobi integral transform’,
see [29], [28], Sect.4.16, [7], Sect. XIII.8, Theorem on p.1526, [14], [15],
[30]). Once more counterpart of $\mathcal{D}$ was considered in [17].

The operator $\mathcal{D}$ has a continuous spectrum on the half-line
$\lambda\leqslant 0$ with multiplicity two and a finite number of discrete
points in the domain $\lambda>0$. The explicit spectral decomposition of
$\mathcal{D}$ was obtained in [20]. Since the spectrum has multiplicity, there
arise a question about possible choices of natural bases in spaces of
solutions of the equation $\mathcal{D}f=\lambda f$ for $\lambda\leqslant 0$.

Remark. The operator $\mathcal{D}$ appears in a natural way in the problem of
decomposition of tensor products of unitary representations of the group
$\mathrm{SL}(2,{\mathbb{R}})$ and its universal covering group, see [20].
Tensor products of unitary representation of $\mathrm{SL}(2,{\mathbb{R}})$
were topics of many papers, in particular, [25], [16], [12], [20], [11].
However, appearance of a multiplicity make the topic non-flexible for a
further development, the same obstacles in more serious forms arise for
numerous spectral problems of non-commutative harmonic analysis with
multiplicities. An informal purpose of the present paper is a search of an
approach to such problems. $\boxtimes$

1.2. Notation. Denote

$\Gamma\left[\begin{matrix}a_{1},\dots,a_{n}\\\
b_{1},\dots,b_{m}\end{matrix}\right]:=\frac{\Gamma(a_{1})\dots\Gamma(a_{n})}{\Gamma(b_{1})\dots\Gamma(b_{m})}.$

By

$\vphantom{F}{}_{p}F_{q}\left[\begin{matrix}a_{1},\dots,a_{p}\\\
b_{1},\dots,b_{q}\end{matrix};z\right]:=\sum_{n=0}^{\infty}\frac{(a_{1})_{n}\dots(a_{p})_{n}}{(b_{1})_{n}\dots(b_{q})_{n}\,n!}\,z^{n}$

we denote generalized hypergeometric functions, here

$(a)_{n}:=\frac{\Gamma(a+n)}{\Gamma(a)}:=\begin{cases}a(a+1)\dots(a+n-1)&\qquad\text{if
$n\geqslant 0$;}\\\ \frac{1}{(a-1)\dots(a-n)}&\qquad\text{if $n\leqslant
0$,}\end{cases}$

is the Pochhammer symbol. By

$\vphantom{F}{}_{p}H_{p}\left[\begin{matrix}a_{1},\dots,a_{p}\\\
b_{1},\dots,b_{p}\end{matrix};z\right]:=\sum_{n=-\infty}^{\infty}\frac{(a_{1})_{n}\dots(a_{p})_{n}}{(b_{1})_{n}\dots(b_{p})_{n}}\,z^{n},\qquad\text{where
$|z|=1$.}$

we denote bilateral hypergeometric series, see, e.g., [27], Chapter 6. If
$b_{p}=1$, then $\sum_{n<0}$ vanishes and $(b_{p})_{n}=(1)_{n}=n!$, so we get
a hypergeometric function $\vphantom{F}{}_{p}F_{p-1}[\dots]$. We prefer
another normalization of bilateral series

$\vphantom{F}{}^{\phantom{\star}}_{p}H_{p}^{\star}\left[\begin{matrix}a_{1},\dots,a_{p}\\\
b_{1},\dots,b_{p}\end{matrix};z\right]:=\frac{1}{\Gamma[1-a_{1},\dots,1-a_{p},\,\,b_{1},\dots,b_{p}]}\,\,\vphantom{F}_{p}H_{p}\left[\begin{matrix}a_{1},\dots,a_{p}\\\
b_{1},\dots,b_{p}\end{matrix};z\right]=\\\
=\sum_{n=-\infty}^{\infty}\frac{((-1)^{p}z)^{n}}{\Gamma[1-a_{1}-n,\dots,1-a_{p}-n,\,\,b_{1}+n,\dots,b_{p}+n]}.$

All summands of the latter sum are well-defined (singularities are removable).
The series absolutely converges on the circle $|z|=1$ if
$\mathop{\mathrm{Re}}\nolimits(\sum a_{j}-\sum b_{j})<-1$. If
$\mathop{\mathrm{Re}}\nolimits(\sum a_{j}-\sum b_{j})<0$, then the series
conditionally converges for $|z|=1$, $z\neq 1$. The sum has a
continuation222In any case, coefficients of the series
$\vphantom{F}{}^{\phantom{\star}}_{p}H_{p}^{\star}[\dots]$ have polynomial
growth, therefore the series always converge in the sense of distributions on
the circle $|z|=1$. to a function real analytic in $z=e^{i\varphi}$ and
meromorphic in $a_{1}$,…, $a_{p}$, $b_{1}$, …, $b_{p}$.

The Dougall formula (see [8], formula (1.4.1), or [27], formula (6.1.2.1))
gives

(1.2)
$\,\vphantom{H}{}^{\phantom{\star}}_{2}H_{2}^{\star}\left[\begin{matrix}a_{1},a_{2}\\\
b_{1},b_{2}\end{matrix};1\right]=\frac{\Gamma[b_{1}+b_{2}-a_{1}-a_{2}-1]}{\Gamma[b_{1}-a_{1},\,b_{1}-a_{2},\,b_{2}-a_{1},\,b_{2}-a_{2}]}.$

1.3. The integral transform and the inversion formula. For any
$\sigma\in{\mathbb{C}}$, $t\in{\mathbb{C}}$ we define a function
$\Phi(\sigma,t;x)=\Phi_{\alpha,\beta}(\sigma,t;x)$ on ${\mathbb{R}}$ by

(1.3)
$\Phi(\sigma,t;x):=\left(\tfrac{1}{2}+ix\right)^{t}\left(\tfrac{1}{2}-ix\right)^{-1/2-t-\sigma}\times\\\
\times\,\vphantom{H}^{\phantom{\star}}_{2}H_{2}^{\star}\left[\begin{matrix}\frac{1-\alpha+i\beta}{2}+\sigma+t,\frac{1+\alpha-i\beta}{2}+\sigma+t\\\
1-\frac{\alpha+i\beta}{2}+t,1+\frac{\alpha+i\beta}{2}+t\end{matrix};-\frac{\frac{1}{2}+ix}{\frac{1}{2}-ix}\right],$

where branches of the power functions in (1.3) are defined by the condition

$\left(\tfrac{1}{2}\pm ix\right)^{\tau}\Bigr{|}_{x=0}=e^{\tau\ln(1/2)}.$

We have

$\mathcal{D}\,\Phi(\sigma,t;x)=\sigma^{2}\Phi(\sigma,t;x),$

so for each $\sigma$ we have a family of functions depending on a complex
parameter $t$ in a two-dimensional space of solutions. For $\sigma\in
i{\mathbb{R}}$ we have

$\Phi(\sigma,t;x)=O(|x|^{-1/2})\qquad\text{as $x\to\infty$},$

in this case $\Phi(\sigma,t;x)$ are generalized eigenfunctions of
$\mathcal{D}$ (see [4], Sect. 2.2).

For $f\in L^{2}({\mathbb{R}})$ we define a function

$J_{\alpha,\beta}f(\sigma;t):=\int_{-\infty}^{\infty}f(x)\,\overline{\Phi_{\alpha,\beta}(\sigma,t;x)}\,dx:=L^{2}\text{-}\\!\lim_{A\to\infty}\int_{-A}^{A}f(x)\,\overline{\Phi_{\alpha,\beta}(\sigma,t;x)}\,dx$

depending on $\sigma=i\nu\in i{\mathbb{R}}$, $t\in{\mathbb{C}}$. The
$L^{2}$-$\lim$ is a limit of the family of functions
$\varphi_{A}:=\int_{-A}^{A}(\dots)dx$ in the sense of $L^{2}({\mathbb{R}})$.

If $f\in L^{2}({\mathbb{R}})$ is compactly supported, then
$J_{\alpha,\beta}f(\sigma;t)$ is well-defined for all $\sigma$,
$t\in{\mathbb{C}}^{2}$, so we get a function on ${\mathbb{C}}^{2}$
holomorphic333We have
$\Phi(\sigma,t;x)=O(x^{-1/2+|\mathop{\mathrm{Re}}\nolimits\sigma|}\ln|x|)$ as
$x\to\pm\infty$, see (2.20), (2.13)–(2.14). The logarithm arise since formulas
(2.13)–(2.14) are valid if $\sigma\neq 0$, 1, 2, $\dots$. For integer $\sigma$
we come to logarithmic solutions of the hypergeometric differential equation,
see [8], Subsect. 2.3.1. in $\overline{\sigma}$, $\overline{t}$.

We started with a function of one variable $x$ and get a function
$J_{\alpha,\beta}f(\sigma,t)$ of two variables $\sigma\in i{\mathbb{R}}$,
$t\in{\mathbb{C}}$. These data are overfilled, for a reconstruction of $f$ it
is sufficient to know values of $J_{\alpha,\beta}f(\sigma,t)$ for two values
of $t$ for each $\sigma$.

######  Theorem 1.1.

Let $0\leqslant\alpha\leqslant 1/2$. Consider two measurable maps $\sigma\to
t(\sigma)$, $\sigma\mapsto s(\sigma)$ defined for $\sigma=i\nu$, where
$\nu\geqslant 0$. Assume that $s-t\notin{\mathbb{Z}}$ a.s. Then

a) For $f_{1}\in L^{2}({\mathbb{R}})$ we have

(1.4)
$f(x)=\frac{1}{2\pi}\int_{0}^{\infty}\begin{pmatrix}\Phi(i\nu,t(i\nu);x)&\Phi(i\nu,s(i\nu);x)\end{pmatrix}R\begin{pmatrix}J_{\alpha,\beta}f(i\nu,t(i\nu))\\\
J_{\alpha,\beta}f(i\nu,s(i\nu))\end{pmatrix}\,d\nu,$

where the matrix spectral density $R$ is

(1.5) $R_{t(i\nu),\,s(i\nu)}:=\\\
=\frac{\pi^{4}}{2\cosh\pi(\beta+i\sigma)\,\cosh\pi(\beta-i\sigma)\,\cos\pi(\alpha-\sigma)\,\cos\pi(\alpha+\sigma)\,\Gamma[2\sigma,-2\sigma]}\times\\\
\times\frac{1}{\sin\pi(s-t)\,\sin\pi(\overline{s}-\overline{t})}\begin{pmatrix}\cos\pi(\sigma+s-\overline{s})&-\cos\pi(\sigma+t-\overline{s})\\\
-\cos\pi(\sigma+s-\overline{t})&\cos\pi(\sigma+t-\overline{t})\end{pmatrix}.$

We understand the integral in (1.4) as a $L^{2}$-limit as $B\to\infty$ of
integrals444See the general statement about self-adjoint differential
operators in [7], Theorem XIII.5.1 and Corollary XIII.5.2.
$\int_{0}^{B}(\dots)d\nu$.

b) For $f_{1}$, $f_{2}\in L^{2}({\mathbb{R}})$ we have the Plancherel formula

(1.6) $\int_{-\infty}^{\infty}f_{1}(x)\overline{f_{2}(x)}\,dx=\\\
=\frac{1}{2\pi}\int\limits_{0}^{\infty}\begin{pmatrix}J_{\alpha,\beta}f_{1}(i\nu,t(i\nu))&J_{\alpha,\beta}f_{1}(i\nu,s(i\nu))\end{pmatrix}R_{t(i\nu),\,s(i\nu)}\begin{pmatrix}J_{\alpha,\beta}f_{2}(i\nu,t(i\nu))\\\
J_{\alpha,\beta}f_{2}(i\nu,s(i\nu))\end{pmatrix}\,d\nu.$

Theorem is proved in Section 2.

Remark. It is more-or-less obvious that the matrix $R$ admits an explicit
expression in the terms of $\Gamma$-functions. But a multiplicative structure
of the matrix elements is a result of a long calculation and looks as a happy-
end, see, e.g., transformation (2.25) below. $\boxtimes$

Remark. Let $t=\pm\frac{\alpha+i\beta}{2}$. In this case
$\Phi=\Phi\bigl{(}\sigma,\pm\tfrac{\alpha+i\beta}{2};x\bigr{)}$ are
hypergeometric functions up to simple functional factors, for this case our
statement is formulated separately in Proposition 2.2.

1.4. The case $\alpha>1/2$ and Romanovski polynomials. This subsection
contains nothing new comparatively [20], however it is important for
understanding of our topic. For $\alpha>1/2$ the operator $\mathcal{D}$ has
also a finite family of $L^{2}$-eigenfunctions

$\Theta^{k}_{\alpha,\beta}(x):=\\\
=\left(\tfrac{1}{2}+ix\right)^{-(\alpha+i\beta)/2}\left(\tfrac{1}{2}-ix\right)^{-(\alpha-i\beta)/2}\,{\vphantom{F}}_{2}F_{1}\left[\begin{matrix}-k,k-2\alpha+1\\\
1-\alpha-i\beta\end{matrix};\frac{1}{2}+ix\right]=\\\
=\left(\tfrac{1}{2}+ix\right)^{-(\alpha+i\beta)/2}\left(\tfrac{1}{2}-ix\right)^{(3\alpha+i\beta)/2-1-k}\times\qquad\qquad\qquad\qquad\qquad\,\,\,\,\\\
\times\,{\vphantom{F}}_{2}F_{1}\left[\begin{matrix}k-2\alpha+1,1-\alpha-i\beta+k\\\
1-\alpha-i\beta\end{matrix};-\frac{\tfrac{1}{2}+ix}{\tfrac{1}{2}-ix}\right]=\\\
=\Gamma\bigl{[}2\alpha-k,\alpha+i\beta-k,1-\alpha-i\beta\bigr{]}\,\Phi\bigl{(}-k+\alpha-1/2;-(\alpha+i\beta)/2;x\bigr{)},\qquad$

where $k$ ranges in integers satisfying the condition

(1.7) $0\leqslant k<\alpha-1/2$

(so for $\alpha\leqslant 1/2$ such functions are absent). The functions

$R_{\alpha,\beta}^{k}(x):=\,{\vphantom{F}}_{2}F_{1}\left[\begin{matrix}-k,k-2\alpha+1\\\
1-\alpha-i\beta\end{matrix};\frac{1}{2}+ix\right]$

are the Romanovski polynomials [26] (we use a nonstandard normalization), they
are orthogonal on the line with respect to the weight

$w(x)=(1/2+ix)^{-(\alpha+i\beta)}(1/2-ix)^{-(\alpha-i\beta)}.$

This weight decreases at infinity as a power, for this reason we have only a
finite family of orthogonal polynomials. The $L^{2}$-norms are given by (see
[3])

(1.8)
$\|\Theta_{\alpha,\beta}^{k}\|^{2}_{L^{2}({\mathbb{R}})}=\frac{1}{2\pi}\int_{-\infty}^{\infty}\frac{R_{\alpha,\beta}^{k}(x)\,\overline{R_{\alpha,\beta}^{k}(x)}\,dx}{(1/2+ix)^{\alpha+i\beta}(1/2-ix)^{\alpha-i\beta}}=\\\
=\frac{k!\,\Gamma(2\alpha-k)}{(2n-2\alpha+1)\Gamma(\alpha+i\beta)\Gamma(\alpha-i\beta)}.$

For $\alpha>1/2$ in the right-hand side of the inversion formula (1.4) there
arise additional terms

$+\sum_{k}\|\Theta_{\alpha,\beta}^{k}\|_{L^{2}({\mathbb{R}})}^{-2}\,\langle
f,\Theta_{\alpha,\beta}^{k}\rangle\,\,\Theta_{\alpha,\beta}^{k}(x),$

where the summation is taken over $k$ satisfying (1.7).

1.5. Difference operators. Next, we find the image of the operator of
multiplication by $x$ under the transformation $J_{\alpha,\beta}$.

######  Theorem 1.2.

Let $f(x)$ be a compactly supported integrable function on ${\mathbb{R}}$. Let
the operator $J_{\alpha,\beta}$ send $f(x)$ to $F(\sigma,t)$. Then
$J_{\alpha,\beta}$ sends $ix\,f(x)$ to the function

(1.9) $\mathcal{Z}F(\sigma,t)=\\\
=\frac{(1/2+\alpha-\overline{\sigma})(1/2-\alpha-\overline{\sigma})(1/2+i\beta-\overline{\sigma})(1/2-i\beta-\overline{\sigma})}{(-2\overline{\sigma})(1-2\overline{\sigma})}F(\sigma-1,t)+\\\
+\frac{2i\alpha\beta}{(-1+2\overline{\sigma})(1+2\overline{\sigma})}F(\sigma,t)+\frac{1}{2\overline{\sigma}(1+2\overline{\sigma})}F(\sigma+1,t).$

Remark. Recall that in the inversion formula and in the Plancherel formula the
integrations are taken over the imaginary axis $\sigma\in i{\mathbb{R}}$. So
$\mathcal{Z}$ is a difference operator in the direction transversal to the
contour of integration. Similar facts take place for other classical index
integral transform as the Jacobi transform (see [18], Theorem 2.1), the
Kontorovich–Lebedev transform (see [22], Theorem 3.2, Proposition 3.3), the
Wimp transform (see [22], Theorem 4.2). Moreover, Cherednik showed that the
multi-dimensional Harish-Chandra transform sends a certain algebra of
operators of multiplications to an algebra of difference operators (see [5],
[6]). $\boxtimes$

1.6. The further structure of the paper. Proof of Theorem 1.1 is contained in
Section 2, this section contains also two other variants of the inversion
formula, see Proposition 2.2, Subsect. 2. Theorem 1.2 is proved in Section 3.
The last Section 4 contains evaluations of the transform of some functions.

##  2\. Proof of the inversion formula

2.1. A reduction of $\mathcal{D}$ to a Schrödinger operator. We consider a
unitary operator $S:L^{2}({\mathbb{R}})\to L^{2}({\mathbb{R}})$ given by the
formula

(2.1) $Sf(y):=f\bigl{(}\tfrac{1}{2}\sinh y\bigr{)}\bigl{(}\tfrac{1}{2}\cosh
y\bigr{)}^{1/2}.$

It send the operator $\mathcal{D}$ to the operator

$\mathcal{H}:=\frac{d^{2}}{dy^{2}}-q(y),\qquad\text{where
$q(y):=-\frac{-1+4\alpha^{2}-4\beta^{2}+8\alpha\beta\sinh y}{\cosh^{2}y}$}$

(cf. [28], Sect. 4.16). We get a Schrödinger operator with a rapidly decaying
potential $q(y)$, and we can apply general statements about such operators,
see, e.g., [4], Sect. II.6. The operator $\mathcal{H}$ defined on the space
$C_{0}^{\infty}({\mathbb{R}})$ of smooth compactly supported functions is
essentially self-adjoint in $L^{2}({\mathbb{R}})$, see [4], Theorem II.1.1
(therefore $\mathcal{D}$ also is essentially self-adjoint). The space $L^{2}$
splits as a direct sum $L^{2}({\mathbb{R}}):=V^{\mathrm{disc}}\oplus
V^{\mathrm{cont}}$ of two $\mathcal{H}$-invariant subspaces corresponding to
discrete and continuous spectrum. The subspace $V^{\mathrm{disc}}$ is finite-
dimensional, eigenfunctions are $L^{2}$-solutions of the equation
$\mathcal{H}f_{s}=s^{2}f$, $s>0$ and they have asymptotics of the form

$\displaystyle f_{s}(y)$
$\displaystyle=C_{1}e^{-sy}\bigl{(}1+o(1)\bigr{)},\quad$ as $y\to+\infty$.
$\displaystyle=C_{2}e^{sy}\bigl{(}1+o(1)\bigr{)},\quad$ as $y\to-\infty$.

Next, let $\sigma=i\nu\in i{\mathbb{R}}$. Consider the two-dimensional space
$V_{\sigma}$ consisting of solutions of the equation
$\mathcal{H}f=\sigma^{2}f$, they have asymptotics of the form

$\displaystyle f(y)$ $\displaystyle=a\,e^{-i\nu
y}\bigl{(}1+o(1)\bigr{)}+b\,e^{i\nu y}\bigl{(}1+o(1)\bigr{)},\quad$
$\displaystyle\text{as $y\to+\infty$};$ $\displaystyle=c\,e^{-i\nu
y}\bigl{(}1+o(1)\bigr{)}+d\,e^{i\nu y}\bigl{(}1+o(1)\bigr{)},\quad$ as
$y\to-\infty$,

so these functions are not in $L^{2}$. We define an inner product in
$V_{\sigma}$ by

(2.2) $\langle
f_{1},f_{2}\rangle_{V_{\sigma}}:=\tfrac{1}{2}(a_{1}\overline{a}_{2}+b_{1}\overline{b}_{2}+c_{1}\overline{c}_{2}+d_{1}\overline{d}_{2}).$

Next, we define two special canonical solutions of $\mathcal{H}f=\sigma^{2}f$,
they have asymptotics of the form

(2.3) $\displaystyle\theta_{1}(\nu;y)$ $\displaystyle=e^{i\nu
y}\bigl{(}1+o(1)\bigr{)}+A(\nu)e^{-i\nu y}\bigl{(}1+o(1)\bigr{)}\quad$ as
$y\to-\infty$; $\displaystyle=B(\nu)e^{i\nu y}\bigl{(}1+o(1)\bigr{)}\quad$ as
$y\to+\infty$,

and

(2.4) $\displaystyle\theta_{2}(\nu;y)$ $\displaystyle=\qquad\qquad\qquad\quad
D(\nu)e^{-i\nu y}\bigl{(}1+o(1)\bigr{)}\quad$ as $y\to-\infty$;
$\displaystyle=C(\nu)e^{i\nu y}\bigl{(}1+o(1)\bigr{)}+e^{-i\nu
y}\bigl{(}1+o(1)\bigr{)}\quad$ as $y\to+\infty$,

it can be shown that the scattering matrix $\begin{pmatrix}A(\nu)&B(\nu)\\\
D(\nu)&C(\nu)\end{pmatrix}$ is unitary and symmetric (see [4], Sect. II.6,
[10], Sect.36). For this reason, $\theta_{1}(\nu;y)$, $\theta_{2}(\nu;y)$ form
an orthogonal basis in $V_{\sigma}$ with respect to the inner product (2.2).

Next, consider two operators,

$I:L^{2}({\mathbb{R}})\to L^{2}({\mathbb{R}}^{+})\oplus
L^{2}({\mathbb{R}}^{+}),\qquad J:L^{2}({\mathbb{R}}^{+})\oplus
L^{2}({\mathbb{R}}^{+})\to L^{2}({\mathbb{R}})$

given by

$I:f(y)\mapsto\Bigl{(}\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}f(y)\,\overline{\theta_{1}(\nu,y)}\,dy,\quad\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}f(y)\,\overline{\theta_{2}(\nu,y)}\,dy\Bigr{)}$

and

$J:\bigl{(}\varphi_{1}(\nu),\varphi_{2}(\nu)\bigr{)}\mapsto\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}\varphi_{1}(\nu)\,\theta_{1}(\nu,y)\,d\nu+\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}\varphi_{2}(\nu)\,\theta_{2}(\nu,y)\,d\nu.$

Then $\ker I=V^{\mathrm{disc}}$, $\operatorname{im}J=V^{\mathrm{cont}}$. The
operator $I$ is a unitary operator $V^{\mathrm{cont}}\to
L^{2}({\mathbb{R}})\oplus L^{2}({\mathbb{R}})$ and $J$ is the inverse operator
$L^{2}({\mathbb{R}})\oplus L^{2}({\mathbb{R}})\to V^{\mathrm{cont}}$. See [4],
Theorem 6.2.

We will use the statement in the following form. Let us choose (in a
measurable way) a basis $\Psi_{1}(\nu;y)$, $\Psi_{2}(\nu;y)$ in each
$V_{\sigma}$. Consider the corresponding Gram matrix

(2.5)
$\Delta(\nu):=\begin{pmatrix}\bigl{\langle}\Psi_{1}(\nu;y),\Psi_{1}(\nu;y)\bigr{\rangle}&\bigl{\langle}\Psi_{1}(\nu;y),\Psi_{2}(\nu;y)\bigr{\rangle}\\\
\bigl{\langle}\Psi_{2}(\nu;y),\Psi_{1}(\nu;y)\bigr{\rangle}&\bigl{\langle}\Psi_{2}(\nu;y),\Psi_{2}(\nu;y)\bigr{\rangle}\vphantom{\Bigr{|}}\end{pmatrix}.$

Denote

(2.6) $\Xi(\nu):=\Delta(\nu)^{-1}.$

Consider the space $C_{0}^{\infty}({\mathbb{R}}_{+})\oplus
C_{0}^{\infty}({\mathbb{R}}_{+})$ equipped with the inner product

(2.7)
$\Bigl{\langle}(h_{1},h_{2}),(h_{1}^{\prime},h_{2}^{\prime})\Bigr{\rangle}:=\frac{1}{2\pi}\int_{0}^{\infty}\begin{pmatrix}h_{1}(\nu)&h_{2}(\nu)\end{pmatrix}\Xi(\nu)\begin{pmatrix}\overline{h}_{1}^{\,\prime}(\nu)\\\
\overline{h}_{2}^{\,\prime}(\nu)\end{pmatrix}\,d\nu,$

denote by $\mathcal{L}[\Xi]$ the completion of
$C_{0}^{\infty}({\mathbb{R}}_{+})\oplus C_{0}^{\infty}({\mathbb{R}}_{+})$ with
respect to this inner product. Then the operator

$\mathcal{I}:f\mapsto\Bigl{(}\bigl{\langle}f(x),\Psi_{1}(\nu,x)\bigr{\rangle}_{L^{2}({\mathbb{R}})},\bigl{\langle}f(x),\Psi_{2}(\nu,x)\bigr{\rangle}_{L^{2}({\mathbb{R}})}\Bigr{)}$

is a unitary operator from $V^{\mathrm{cont}}$ to $\mathcal{L}[\Xi]$.

2.2. A reduction of $\mathcal{D}$ to a hypergeometric differential operator.
We set

$r(x):=\left(\tfrac{1}{2}+ix\right)^{(\alpha+i\beta)/2}\left(\tfrac{1}{2}+ix\right)^{(\alpha-i\beta)/2}$

and pass to the differential operator

$\mathcal{B}f(x):=r(x)^{-1}\mathcal{D}\bigl{(}r(x)f(x)\bigr{)}.$

Next, we pass to a complex variable

$z=\tfrac{1}{2}+ix$

and come to a new operator

$\mathcal{A}:=-z(1-z)\frac{d^{2}}{dz^{2}}-\bigl{(}1+\alpha+i\beta-z(2+2\alpha)\bigr{)}\frac{d}{dz}+\left(\alpha+\tfrac{1}{2}\right)^{2}.$

The equation for eigenfunctions $\mathcal{A}\varphi=\sigma^{2}\varphi$ becomes
a special case of the hypergeometric differential equation

$\Bigl{[}z(1-z)\frac{d^{2}}{dz^{2}}+(c-z(a+b+1))\frac{d^{2}}{dz^{2}}-ab\Bigr{]}\varphi(z)=0$

with

(2.8) $c=1+\alpha+i\beta,\qquad a=\frac{1}{2}+\alpha+\sigma,\qquad
b=\frac{1}{2}+\alpha-\sigma.$

We write two Kummer solutions [8], (2.9.3), (2.9.20) of the hypergeometric
equation

$(1-z)^{-a}\,{\vphantom{F}}_{2}F_{1}\left[\begin{matrix}a,c-b\\\
c\end{matrix};\frac{z}{z-1}\right],\quad
z^{1-c}(1-z)^{c-a-1}\,{\vphantom{F}}_{2}F_{1}\left[\begin{matrix}a+1-c,1-b\\\
2-c\end{matrix};\frac{z}{z-1}\right].$

Substituting (2.8), $z=1/2+ix$ and multiplying by $r(x)$ we get two following
solutions of the equation $\mathcal{D}\psi=\sigma^{2}\psi$:

(2.9)
$\Psi_{1}(\sigma;x)=\left(\tfrac{1}{2}+ix\right)^{(\alpha+i\beta)/2}\left(\tfrac{1}{2}-ix\right)^{-(\alpha+i\beta)/2-1/2-\sigma}\times\\\
\times\,{\vphantom{F}}_{2}F_{1}\left[\begin{matrix}\tfrac{1}{2}+\alpha+\sigma,\tfrac{1}{2}+i\beta+\sigma\\\
1+\alpha+i\beta\end{matrix};-\frac{1/2+ix}{1/2-ix}\right];$ (2.10)
$\Psi_{2}(\sigma;x)=\left(\tfrac{1}{2}+ix\right)^{-(\alpha+i\beta)/2}\left(\tfrac{1}{2}-ix\right)^{(\alpha+i\beta)/2-1/2-\sigma}\times\\\
\times\,{\vphantom{F}}_{2}F_{1}\left[\begin{matrix}\tfrac{1}{2}-\alpha+\sigma,\tfrac{1}{2}-i\beta+\sigma\\\
1-\alpha-i\beta\end{matrix};-\frac{1/2+ix}{1/2-ix}\right].$

These solutions are obtained one from another by a substitution
$(\alpha,\beta)\leadsto(-\alpha,-\beta)$, this substitution does not change
the operator $\mathcal{D}$.

Remark. In this place we must assume $(\alpha,\beta)\neq(0,0)$. Otherwise
$\Psi_{1}$, $\Psi_{2}$ coincide, and we come to the logarithmic case of the
hypergeometric differential equation, see [8], Sect. 2.3, [2], end of Section
2.3. $\boxtimes$

We need asymptotics of these functions as $x\to\pm\infty$. In this case the
argument of hypergeometric function tends to 1, we apply formulas [8],
(2.10.1), (2.10.5),

$\,{\vphantom{F}}_{2}F_{1}\left[\begin{matrix}a,b\\\
c\end{matrix};u\right]=\Gamma\begin{bmatrix}c,c-a-b\\\
c-a,c-b\end{bmatrix}\,{\vphantom{F}}_{2}F_{1}\left[\begin{matrix}a,b\\\
a+b-c+1\end{matrix};1-u\right]+\\\ +\Gamma\begin{bmatrix}c,a+b-c\\\
a,b\end{bmatrix}(1-u)^{c-a-b}\,{\vphantom{F}}_{2}F_{1}\left[\begin{matrix}c-a,c-b\\\
c-a-b+1\end{matrix};1-u\right].$

We have

$(1-u)\Bigr{|}_{u=-(1/2+ix)/(1/2-ix)}=\bigl{(}\tfrac{1}{2}-ix\bigr{)}^{-1}.$

Denote

(2.11) $\displaystyle A(\alpha,\beta,\sigma)$
$\displaystyle:=\Gamma\begin{bmatrix}1+\alpha+i\beta,\,-2\sigma\\\
1/2+\alpha-\sigma,\,1/2+i\beta-\sigma\end{bmatrix};$ (2.12)
$\displaystyle\gamma(\alpha,\beta,\sigma)$
$\displaystyle:=\exp\Bigl{\\{}\frac{\pi}{2}(i\alpha-\beta+i\sigma)\Bigr{\\}}.$

Then

(2.13) $\Psi_{1}(\sigma;x)=e^{\pi
i/4}\gamma(\alpha,\beta,\sigma)\,A(\alpha,\beta,\sigma)\,x^{-1/2-\sigma}\bigl{(}1+o(1)\bigr{)}+\\\
+e^{\pi
i/4}\gamma(\alpha,\beta,-\sigma)\,A(\alpha,\beta,-\sigma)\,x^{-1/2+\sigma}\bigl{(}1+o(1)\bigr{)},\quad\text{as
$x\to+\infty$};$ (2.14) $\phantom{\Psi_{1}(\sigma;x)}=e^{-\pi
i/4}\gamma(-\alpha,-\beta,-\sigma)\,A(\alpha,\beta,\sigma)\,(-x)^{-1/2-\sigma}\bigl{(}1+o(1)\bigr{)}+\\\
+e^{-\pi
i/4}\gamma(-\alpha,-\beta,\sigma)\,A(\alpha,\beta,-\sigma)\,(-x)^{-1/2+\sigma}\bigl{(}1+o(1)\bigr{)},\quad\text{as
$x\to-\infty$}.$

For $\Psi_{2}(\sigma;x)$ we have a similar expression with $(\alpha,\beta)$
replaced by $(-\alpha,-\beta)$. The formulas hold for any
$\sigma\in{\mathbb{C}}$. For $\sigma\in i{\mathbb{R}}$ both summands of the
asymptotic have the same order (and $\Psi_{1,2}(\sigma;x)$ are almost
$L^{2}$-functions), for $\sigma\notin i{\mathbb{R}}$ one summand dominates
another.

To adapt the general reasoning from Subsect. 2 we must apply the unitary
operator (2.1), $x^{-1/2\pm\sigma}$ transform as

(2.15) $\displaystyle x^{-1/2\pm\sigma}$
$\displaystyle\leadsto\bigl{(}\tfrac{1}{2}\sinh
y\bigr{)}^{-1/2\pm\sigma}\bigl{(}\tfrac{1}{2}\cosh y\bigr{)}^{1/2}\sim
2^{\mp\sigma}e^{\pm\sigma y}\quad\text{as $y\to+\infty$};$ (2.16)
$\displaystyle(-x)^{-1/2\pm\sigma}$
$\displaystyle\leadsto\bigl{(}-\tfrac{1}{2}\sinh
y\bigr{)}^{-1/2\pm\sigma}\bigl{(}\tfrac{1}{2}\cosh y\bigr{)}^{1/2}\sim
2^{\mp\sigma}e^{\mp\sigma y}\quad\text{as $y\to-\infty$}.$

2.3. The Gram matrix for the hypergeometric eigenfunctions. Let $\sigma\in
i{\mathbb{R}}$. Our next purpose is to evaluate the matrices $\Delta$ and
$\Xi=\Delta^{-1}$ (see (2.5)) for the eigenfunctions $\Psi_{1}$, $\Psi_{2}$
given by (2.9), (2.10).

######  Lemma 2.1.

a) The matrix elements of the Gram matrix $\Delta$ for eigenfunctions
$\Psi_{1}$, $\Psi_{2}$ are

$\displaystyle\Delta_{11}$
$\displaystyle=\frac{2}{\pi}\cosh\pi(\beta-i\sigma)\,\cosh\pi(\beta+i\sigma)\,\Gamma\begin{bmatrix}1+\alpha+i\beta,\,1+\alpha-i\beta,\,2\sigma,-2\sigma\\\
1/2+\alpha-\sigma,\,1/2+\alpha+\sigma\end{bmatrix};$
$\displaystyle\Delta_{12}$
$\displaystyle=\frac{2}{\pi}\cos\pi(\alpha-\sigma)\,\cos\pi(\alpha+\sigma)\,\Gamma\begin{bmatrix}1-\alpha+i\beta,\,1+\alpha+i\beta,\,2\sigma,\,-2\sigma\\\
1/2+i\beta-\sigma,\,1/2+i\beta+\sigma\end{bmatrix};$
$\displaystyle\Delta_{21}$
$\displaystyle=\frac{2}{\pi}\cos\pi(\alpha-\sigma)\,\cos\pi(\alpha+\sigma)\,\Gamma\begin{bmatrix}1-\alpha-i\beta,\,1+\alpha-i\beta,\,2\sigma,\,-2\sigma\\\
1/2-i\beta-\sigma,\,1/2-i\beta+\sigma\end{bmatrix};$
$\displaystyle\Delta_{22}$
$\displaystyle=\frac{2}{\pi}\cosh\pi(\beta-i\sigma)\,\cosh\pi(\beta+i\sigma)\,\Gamma\begin{bmatrix}1-\alpha-i\beta,\,1-\alpha+i\beta,\,2\sigma,\,-2\sigma\\\
1/2-\alpha-\sigma,\,1/2-\alpha+\sigma\end{bmatrix}.$

b) The determinant of $\Delta$ is

$\det\Delta=\frac{4}{\pi^{2}}\cos\pi(\alpha-\sigma)\,\cos\pi(\alpha+\sigma)\,\cosh\pi(\beta-i\sigma)\,\cosh\pi(\beta+i\sigma)\times\\\
\times(\alpha^{2}+\beta^{2})\,\Gamma[2\sigma,-2\sigma]^{2}.$

Proof. a) We present a calculation of $\Delta_{11}$,

(2.17)
$\Delta_{11}=\frac{1}{2}\bigl{(}|\gamma(\alpha,\beta,\sigma)|^{2}+|\gamma(-\alpha,-\beta,-\sigma)|^{2}\bigr{)}\,\,|A(\alpha,\beta,\sigma)|^{2}+\\\
+\frac{1}{2}\bigl{(}|\gamma(\alpha,\beta,-\sigma)|^{2}+|\gamma(-\alpha,-\beta,+\sigma)|^{2}\bigr{)}\,\,|A(\alpha,\beta,-\sigma)|^{2}.$

Let us evaluate the first summand. We have

$\tfrac{1}{2}\bigl{(}|\gamma(\alpha,\beta,\sigma)|^{2}+|\gamma(-\alpha,-\beta,-\sigma)|^{2}\bigr{)}=\tfrac{1}{2}(e^{\pi(-\beta+i\sigma)}+e^{-\pi(-\beta+i\sigma)})=\cosh\pi(\beta-i\sigma)$

and

$|A(\alpha,\beta,\sigma)|^{2}=\Gamma\begin{bmatrix}1+\alpha+i\beta,\,-2\sigma\\\
1/2+\alpha-\sigma,\,1/2+i\beta-\sigma\end{bmatrix}\Gamma\begin{bmatrix}1+\alpha-i\beta,\,2\sigma\\\
1/2+\alpha+\sigma,\,1/2-i\beta+\sigma\end{bmatrix}=\\\
=\frac{1}{\pi}\cosh\pi(\beta+i\sigma)\,\Gamma\begin{bmatrix}1+\alpha+i\beta,\,1+\alpha-i\beta,\,-2\sigma,\,2\sigma\\\
1/2+\alpha-\sigma,\,1/2+\alpha+\sigma\end{bmatrix}.$

We see that the first summand in (2.17) is even in $\sigma$. Therefore it is
equal to the second summand, and we come to the final expression.

Evaluations of other matrix elements are similar.

b) Evaluating $\det\Delta=\Delta_{11}\Delta_{22}-\Delta_{12}\Delta_{21}$ we
meet the following subexpressions:

$\displaystyle\frac{1}{\Gamma[\tfrac{1}{2}+\alpha+\sigma,\,\tfrac{1}{2}+\alpha-\sigma,\,\tfrac{1}{2}-\alpha+\sigma,\,\tfrac{1}{2}-\alpha-\sigma]}=\frac{1}{\pi^{2}}\cos\pi(\alpha-\sigma)\,\cos\pi(\alpha+\sigma);$
$\displaystyle\frac{1}{\Gamma[\tfrac{1}{2}+i\beta+\sigma,\,\tfrac{1}{2}+i\beta-\sigma,\,\tfrac{1}{2}-i\beta+\sigma,\,\tfrac{1}{2}-i\beta-\sigma]}=$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad=\frac{1}{\pi^{2}}\cosh\pi(\beta-i\sigma)\,\cosh\pi(\beta+i\sigma);$
$\displaystyle\Gamma[1+\alpha+i\beta,\,1+\alpha-i\beta,\,1-\alpha+i\beta,\,1-\alpha-i\beta]=\frac{\pi^{2}(\alpha+i\beta)(\alpha-i\beta)}{\sin\pi(\alpha+i\beta)\,\sin\pi(\alpha-i\beta)}.$

Applying these transformations we get the following expression for
$\det\Delta$:

$\\!\\!\\!\\!\\!\\!\frac{4}{\pi^{2}}\frac{(\alpha^{2}+\beta^{2})\cos\pi(\alpha-\sigma)\cos\pi(\alpha+\sigma)\cosh\pi(\beta-i\sigma)\cosh\pi(\beta+i\sigma)\Gamma[2\sigma,-2\sigma]^{2}}{\sin\pi(\alpha+i\beta)\,\sin\pi(\alpha-i\beta)}\times\\\
\times\Bigl{\\{}\cosh\pi(\beta-i\sigma)\cosh\pi(\beta+i\sigma)-\cos\pi(\alpha-\sigma)\cos\pi(\alpha+\sigma)\Bigr{\\}}.$

Simplifying the expression in the curly brackets we get

$\Bigl{\\{}\dots\Bigr{\\}}=\sin\pi(\alpha+i\beta)\,\sin\pi(\alpha-i\beta)$

and we come to the final expression. $\square$

Now we write the matrix $\Delta^{-1}$ in a straightforward way and get the
following statement, see [20]:

######  Proposition 2.2.

Let $(\alpha,\beta)\neq(0,0)$. Then for the eigenfunctions $\Psi_{1}(x)$,
$\Psi_{2}(x)$ given by (2.9)–(2.10) the spectral matrix $\Xi$ in (2.7) is
given by555We write each matrix element as a two-line formula, this allows to
obtain a readable expression.

$\frac{1}{2\pi\Gamma[2\sigma,-2\sigma]}\begin{pmatrix}\begin{matrix}\Gamma[\frac{1}{2}+\alpha+\sigma,\frac{1}{2}+\alpha-\sigma]\times\,\,\\\
\,\,\,\,\,\,\times\Gamma[-\alpha-i\beta,-\alpha+i\beta]\end{matrix}_{\vphantom{\Bigl{|}}}&\begin{matrix}\Gamma[\frac{1}{2}-i\beta+\sigma,\frac{1}{2}-i\beta-\sigma]\times\,\\\
\,\,\,\,\,\,\times\Gamma[-\alpha+i\beta,\alpha+i\beta]\end{matrix}\\\
\begin{matrix}\Gamma[\frac{1}{2}+i\beta+\sigma,\frac{1}{2}+i\beta-\sigma]\times\,\\\
\,\,\,\,\,\,\times\Gamma[-\alpha-i\beta,\alpha-i\beta]\end{matrix}^{\vphantom{A^{A}}}&\begin{matrix}\Gamma[\frac{1}{2}-\alpha+\sigma,\frac{1}{2}-\alpha-\sigma]\times\,\,\\\
\,\,\,\,\,\,\times\Gamma[\alpha-i\beta,\alpha+i\beta]\end{matrix}\end{pmatrix}.$

2.4. Bilateral hypergeometric functions
$\,\vphantom{H}{}^{\phantom{\star}}_{2}H_{2}^{\star}$. It is easy to see that
functions $\,\vphantom{H}{}_{2}H_{2}\left[\begin{matrix}a_{1},a_{2}\\\
b_{1},b_{2}\end{matrix};z\right]$ satisfy the differential equation

$\Bigl{\\{}z\Bigl{(}z\frac{d}{dz}+a_{1}\Bigr{)}\Bigl{(}z\frac{d}{dz}+a_{2}\Bigr{)}-\Bigl{(}z\frac{d}{dz}+b_{1}-1\Bigr{)}\Bigl{(}z\frac{d}{dz}+b_{2}-1\Bigr{)}\Bigr{\\}}\,F(z)=0$

(cf. [27], formula (2.1.2.1)). Functions
$\,\vphantom{H}{}^{\phantom{\star}}_{2}H_{2}^{\star}$ differ from
$\,\vphantom{H}{}_{2}H_{2}$ by constant factors. Moreover for any
$t\in{\mathbb{C}}$ the function

$J_{t}(z):=(-z)^{t}\,\vphantom{H}{}^{\phantom{\star}}_{2}H_{2}^{\star}\left[\begin{matrix}a_{1}+t,a_{2}+t\\\
b_{1}+t,b_{2}+t\end{matrix};z\right]$

satisfy the same differential equation (we assume that
$(-z)^{t}\bigr{|}_{z=-1}=1$). Therefore any 3 functions $J_{t_{1}}(z)$,
$J_{t_{2}}(z)$, $J_{t_{3}}(z)$ are linear dependent, i.e.,

(2.18) $C_{1}J_{t_{1}}(z)+C_{2}J_{t_{2}}(z)+C_{3}J_{t_{3}}(z)=0$

for some $C_{1}$, $C_{2}$, $C_{3}$. In fact, see [1],

(2.19)
$\sin\pi(t_{2}-t_{3})\,J_{t_{1}}(z)+\sin\pi(t_{3}-t_{1})\,J_{t_{2}}(z)+\sin\pi(t_{1}-t_{2})\,J_{t_{3}}(z)=0.$

Remark. These coefficients $C_{1}$, $C_{2}$, $C_{3}$ of the linear dependence
can be derived in the following way. The Dougall formula (1.2) provides us an
explicit value for any
$\,\vphantom{H}{}^{\phantom{\star}}_{2}H_{2}^{\star}(z)$ at $z=1$. We
substitute $z=e^{0_{+}i}$ and $z=e^{2\pi_{-}i}$ and get two equations for the
coefficients. $\boxtimes$

Setting

$t_{1}=0,\qquad t_{2}=1-b_{1},\quad t_{3}=1-b_{2}$

to (2.19) we get an expression of an arbitrary function
$\,\vphantom{H}{}^{\phantom{\star}}_{2}H_{2}^{\star}(z)$ in terms of Gauss
hypergeometric functions.

In particular, we get an expression for the functions

$\Phi(\sigma,t;x):=\left(\tfrac{1}{2}+ix\right)^{t}\left(\tfrac{1}{2}-ix\right)^{-1/2-t-\sigma}\times\\\
\times\,\vphantom{H}^{\phantom{\star}}_{2}H_{2}^{\star}\left[\begin{matrix}\frac{1-\alpha+i\beta}{2}+\sigma+t,\frac{1+\alpha-i\beta}{2}+\sigma+t\\\
1-\frac{\alpha+i\beta}{2}+t,1+\frac{\alpha+i\beta}{2}+t\end{matrix};-\frac{\frac{1}{2}+ix}{\frac{1}{2}-ix}\right],$

defined in Subsect. 1. Namely,

(2.20) $\Phi(\sigma,t;x)\sin\pi(\alpha+i\beta)=\\\
=C_{1}(\sigma)\Psi_{1}(\sigma;x)\sin\pi\bigl{(}t+\tfrac{\alpha+i\beta}{2}\bigr{)}+C_{2}(\sigma)\Psi_{2}(\sigma;x)\sin\pi\bigl{(}-t+\tfrac{\alpha+i\beta}{2}\bigr{)},$

where

(2.21) $\displaystyle C_{1}(\sigma)$
$\displaystyle=\frac{1}{\Gamma[\frac{1}{2}-i\beta-\sigma,\frac{1}{2}-\alpha-\sigma,1+\alpha+i\beta]}:=C(\alpha,\beta,\sigma);$
(2.22) $\displaystyle C_{2}(\sigma)$
$\displaystyle=\frac{1}{\Gamma[\frac{1}{2}+i\beta-\sigma,\frac{1}{2}+\alpha-\sigma,1-\alpha-i\beta]}=C(-\alpha,-\beta,\sigma).$

######  Lemma 2.3.

(2.23)
$\bigl{\langle}\Phi(\sigma,t;x),\Phi(\sigma,s;x)\bigr{\rangle}_{V_{\sigma}}=M\cdot\cos\pi(\sigma+t-\overline{s}),$

where

$M=\cosh\pi(\beta+i\sigma)\,\cosh\pi(\beta-i\sigma)\,\cos\pi(\alpha+\sigma)\,\cos\pi(\alpha-\sigma)\,\Gamma[2\sigma,-2\sigma].$

Proof. Let $\Delta$ be the Gram matrix of the eigenfunctions $\Psi_{1}$,
$\Psi_{2}$, see Lemma 2.1. Then

$\begin{pmatrix}C_{1}\Delta_{11}\overline{C}_{1}&C_{1}\Delta_{12}\overline{C}_{2}\\\
C_{2}\Delta_{21}\overline{C}_{1}&C_{2}\Delta_{22}\overline{C}_{2}\end{pmatrix}=\frac{2}{\pi^{4}}M\cdot
S,$

where

$S=\begin{pmatrix}\cosh\pi(\beta-i\sigma)&\cos\pi(\alpha+\sigma)\\\
\cos\pi(\alpha-\sigma)&\cosh\pi(\beta+i\sigma)\end{pmatrix}.$

Let us verify the identity for the first matrix element:

$C_{1}\Delta_{11}\overline{C}_{1}=\frac{2}{\pi}\cosh\pi(\beta-i\sigma)\,\cosh\pi(\beta+i\sigma)\,\Gamma\begin{bmatrix}1+\alpha+i\beta,\,1+\alpha-i\beta,\,2\sigma,-2\sigma\\\
1/2+\alpha-\sigma,\,1/2+\alpha+\sigma\end{bmatrix}\times\\\
\times\frac{1}{\Gamma[\frac{1}{2}-i\beta-\sigma,\frac{1}{2}-\alpha-\sigma,1+\alpha+i\beta]}\cdot\frac{1}{\Gamma[\frac{1}{2}+i\beta+\sigma,\frac{1}{2}-\alpha+\sigma,1+\alpha-i\beta]}.$

The product of three $\Gamma$-factors is

$\frac{1}{\pi^{3}}\cos\pi(\alpha+\sigma)\,\cos\pi(\alpha-\sigma)\,\Gamma[-2\sigma,2\sigma]\,\cosh\pi(\beta-i\sigma),$

and we come to the desired expression.

Now we are ready to evaluate

(2.24)
$\bigl{\langle}\Phi(\sigma,t;x),\Phi(\sigma,s;x)\bigr{\rangle}_{V_{\sigma}}=\frac{2}{\pi^{4}}\frac{M}{\sin\pi(\alpha+i\beta)\,\sin\pi(\alpha-i\beta)}\times\\\
\times\left\\{\begin{pmatrix}\sin\pi(\frac{\alpha+i\beta}{2}+t)&\sin\pi(\frac{\alpha+i\beta}{2}-t)\end{pmatrix}\,S\,\begin{pmatrix}\sin\pi(\frac{\alpha-i\beta}{2}+\overline{s})\\\
\sin\pi(\frac{\alpha-i\beta}{2}-\overline{s})\end{pmatrix}\right\\}.$

The expression in the curly bracket is

(2.25)
$\Bigl{\\{}\dotsc\Bigr{\\}}=\sin\pi(\tfrac{\alpha+i\beta}{2}+t)\,\cosh\pi(\beta-i\sigma)\,\sin\pi(\tfrac{\alpha-i\beta}{2}+\overline{s})+\\\
+\sin\pi(\tfrac{\alpha+i\beta}{2}+t)\,\cos\pi(\alpha+\sigma)\,\sin\pi(\tfrac{\alpha-i\beta}{2}-\overline{s})+\\\
+\sin\pi(\tfrac{\alpha+i\beta}{2}-t)\,\cos\pi(\alpha-\sigma)\,\sin\pi(\tfrac{\alpha-i\beta}{2}+\overline{s})+\\\
+\sin\pi(\tfrac{\alpha+i\beta}{2}-t)\,\cosh\pi(\beta+i\sigma)\,\sin\pi(\tfrac{\alpha-i\beta}{2}-\overline{s})=\\\
=\cos\pi(\sigma+t-\overline{s})\,\sin\pi(\alpha+i\beta)\,\sin\pi(\alpha-i\beta),$

this implies the statement of the lemma. The last identity is not obvious, but
when written, it admits a straightforward verification. $\square$

Proof of Theorem 1.1. Thus, the Gram matrix of $\Phi(\sigma,t;x)$ and
$\Phi(\sigma,s;x)$ is

$\frac{2}{\pi^{4}}\cdot
M\begin{pmatrix}\cos\pi(\sigma+t-\overline{t})&\cos\pi(\sigma+t-\overline{s})\\\
\cos\pi(\sigma+s-\overline{t})&\cos\pi(\sigma+s-\overline{s})\end{pmatrix}.$

The inverse matrix is

$\frac{\pi^{4}}{2}\frac{1}{M}\cdot\frac{1}{\sin\pi(s-t)\,\sin\pi(\overline{s}-\overline{t})}\begin{pmatrix}\cos\pi(\sigma+s-\overline{s})&-\cos\pi(\sigma+t-\overline{s})\\\
-\cos\pi(\sigma+s-\overline{t})&\cos\pi(\sigma+t-\overline{t})\end{pmatrix}$

and we come to the formula (1.6). $\square$

2.5. A generalized orthogonal system. For completeness we present formulas for
the eigenfunctions $\theta_{1}$, $\theta_{2}$, see (2.3)–(2.4). Set

$\theta_{1}(\sigma;x):=\\\ =-\frac{e^{\frac{3\pi
i}{4}}}{2\pi}\,\Bigl{(}\mu(\alpha,\beta,\sigma)M(\alpha,\beta,\sigma)\Psi_{1}(\sigma;x)+\mu(-\alpha,-\beta,\sigma)M(-\alpha,-\beta,\sigma)\Psi_{2}(\sigma;x)\Bigr{)};$
$\theta_{2}(\sigma;x):=\\\ =\frac{e^{\frac{\pi
i}{4}}}{2\pi}\,\Bigl{(}\mu(-\alpha,-\beta,-\sigma)M(\alpha,\beta,\sigma)\Psi_{1}(\sigma;x)+\mu(\alpha,\beta,-\sigma)M(-\alpha,-\beta,\sigma)\Psi_{2}(\sigma;x)\Bigr{)},$

where

$\displaystyle\mu(\alpha,\beta,\sigma)$
$\displaystyle:=e^{\frac{\pi}{2}(-i\alpha+\beta+i\sigma)};$ $\displaystyle
M(\alpha,\beta,\sigma)$
$\displaystyle:=\Gamma\left[\begin{matrix}-\alpha-i\beta,\frac{1}{2}+\alpha-\sigma,1/2+i\beta-\sigma\\\
-2\sigma\end{matrix}\right].$

Then $\theta_{1}$, $\theta_{2}$ have the following asymptotics at infinity
(see (2.15)–(2.16)):

$\displaystyle\theta_{1}(\sigma;x)$
$\displaystyle=(-x)^{-\frac{1}{2}-\sigma}(1+O(x^{-1}))+A(\sigma)(-x)^{-\frac{1}{2}+\sigma}(1+O(x^{-1}))\,\,$
as $x\to-\infty$;
$\displaystyle=B(\sigma)x^{-\frac{1}{2}+\sigma}(1+O(x^{-1}))\,\,$ as
$x\to+\infty$,

and

$\displaystyle\theta_{2}(\sigma;y)$ $\displaystyle=\qquad\qquad\qquad\quad
D(\sigma)(-x)^{-\frac{1}{2}+\sigma}(1+O(x^{-1}))\,\,$ as $x\to-\infty$;
$\displaystyle=C(\sigma)x^{-\frac{1}{2}+\sigma}(1+O(x^{-1}))+x^{-\frac{1}{2}-\sigma}(1+O(x^{-1}))\,\,$
as $x\to+\infty$,

where the elements of the scattering matrix are given by

$\displaystyle A$
$\displaystyle:=\frac{1}{2\pi^{2}}\bigl{(}e^{-\pi\beta}\cos\pi(\alpha-\sigma)+e^{\pi\beta}\cos\pi(\alpha+\sigma)\bigr{)}\Gamma\left[\begin{matrix}2\sigma\\\
-2\sigma\end{matrix}\right]\cdot\mathcal{G};$ $\displaystyle B$
$\displaystyle=D:=\frac{1}{2\pi\Gamma[1-2\sigma,-2\sigma]}\cdot\mathcal{G};$
$\displaystyle C$
$\displaystyle:=\frac{1}{2\pi^{2}}\bigl{(}e^{\pi\beta}\cos\pi(\alpha-\sigma)+e^{-\pi\beta}\cos\pi(\alpha+\sigma)\bigr{)}\Gamma\left[\begin{matrix}2\sigma\\\
-2\sigma\end{matrix}\right]\cdot\mathcal{G},$

and

$\mathcal{G}:=\Gamma\bigl{[}\tfrac{1}{2}-\alpha-\sigma,\,\tfrac{1}{2}+\alpha-\sigma,\,\tfrac{1}{2}-i\beta-\sigma,\,\tfrac{1}{2}+i\beta-\sigma\bigr{]}.$

For such functions $\theta_{1}$, $\theta_{2}$ the matrix $\Xi$ in (2.7) is
$\begin{pmatrix}1&0\\\ 0&1\end{pmatrix}$, but we pay for this by longer and
less flexible expressions for eigenfunctions.

2.6. The case $\alpha=0$, $\beta=0$. For this case the calculations of this
section are not valid, but we can easily apply continuity arguments. Our final
formula (1.5) follows from (2.23). To extend the latter formula to our case,
it is sufficient to show, that coefficients at high terms of asymptotics
$|x|^{-1/2\pm\sigma}$ of $\Phi_{\alpha,\beta}(\sigma,t;x)$ at infinities are
continuous at the point $(\alpha,\beta)=(0,0)$ for $\sigma=i\nu$, where
$\nu>0$. These coefficients can be easily written explicitly with formulas
(2.20), (2.13)-(2.14). For instance, the coefficient in front of
$x^{-1/2-\sigma}$ as $x\to+\infty$ is

(2.26)
$\frac{1}{\sin\pi(\alpha+i\beta)/2}\Bigl{\\{}\gamma(\alpha,\beta,\sigma)\,A(\alpha,\beta,\sigma)\,C(\alpha,\beta,\sigma)\sin\pi\bigl{(}t+\tfrac{\alpha+i\beta}{2}\bigr{)}-\\\
-\gamma(-\alpha,-\beta,\sigma)\,A(-\alpha,-\beta,\sigma)\,C(-\alpha,-\beta,\sigma)\sin\pi\bigl{(}t-\tfrac{\alpha+i\beta}{2}\bigr{)}\Bigr{\\}},$

where $\gamma(\dots)$, $A(\dots)$, $C(\dots)$ are defined by formulas
(2.11)-(2.12), (2.21). Substitute $\alpha=-i\beta$ to the bracket
$\Bigl{\\{}\dots\Bigr{\\}}$. It is easy to see that

$\gamma(\alpha,\beta,\sigma)\Bigr{|}_{\alpha=-i\beta}=\gamma(-\alpha,-\beta,\sigma)\Bigr{|}_{\alpha=-i\beta},$

similar identities take place also for $A(\dots)$, $C(\dots)$. Therefore for
$\alpha=-i\beta$ the expression $\Bigl{\\{}\dots\Bigr{\\}}$ is zero. So the
singularity on the surface $\alpha+i\beta=0$ in (2.26) is removable and the
whole expression is continuous.

##  3\. The difference operator

The topic of this section is the proof of Theorem 1.2. In fact, we must show
that the kernel $F=\Phi(\sigma,t;x)$ satisfies the equation

(3.1) $-ix\,F(\sigma,t;x)=\\\
=\frac{(1/2+\alpha-\sigma)(1/2-\alpha-\sigma)(1/2+i\beta-\sigma)(1/2-i\beta-\sigma)}{(-2\sigma)(1-2\sigma)}F(\sigma-1,t;x)-\\\
-\frac{2i\alpha\beta}{(-1+2\sigma)(1+2\sigma)}F(\sigma,t;x)+\frac{1}{2\sigma(1+2\sigma)}F(\sigma+1,t;x)$

(the variable $t$ is absent in the coefficients).

We use the expression (2.20) for $\Phi$, it is sufficient to show that two
terms $C_{1}\Psi_{1}(x)$, $C_{2}\Psi_{2}(x)$ satisfy the same difference
equation. We write $\Psi_{1}(x)$ as

$\Psi_{1}(\sigma;x)=\left(\tfrac{1}{2}+ix\right)^{(\alpha+i\beta)/2}\left(\tfrac{1}{2}-ix\right)^{(\alpha-i\beta)/2}\,{\vphantom{F}}_{2}F_{1}\left[\begin{matrix}\frac{1}{2}+\alpha+\sigma,\frac{1}{2}+\alpha-\sigma\\\
1+\alpha+i\beta\end{matrix};\frac{1}{2}+ix\right],$

see [8], formulas (2.9.1), (2.9.3). The expression for $\Psi_{2}(\sigma;x)$ is
obtained by replacing $(\alpha,\beta)\leadsto(-\alpha,-\beta)$

By [18], formula (2.3), the Gauss hypergeometric function satisfies the
following contiguous relation

$-y\,{\vphantom{F}}_{2}F_{1}\left[\begin{matrix}p,q\\\
r\end{matrix};y\right]=\frac{q(r-p)}{(q-p)(1+q-p)}\,{\vphantom{F}}_{2}F_{1}\left[\begin{matrix}p-1,q+1\\\
r\end{matrix};y\right]-\\\
-\Bigl{(}\frac{q(r-p)}{(q-p)(1+q-p)}+\frac{p(r-q)}{(p-q)(1+p-q)}\Bigr{)}\,\,{\vphantom{F}}_{2}F_{1}\left[\begin{matrix}p,q\\\
r\end{matrix};y\right]+\\\
+\frac{p(r-q)}{(p-q)(1+p-q)}\,{\vphantom{F}}_{2}F_{1}\left[\begin{matrix}p+1,q-1\\\
r\end{matrix};y\right].$

Therefore $G(\sigma;x)=\Psi_{1}(\sigma;x)$ satisfies the difference equation

(3.2)
$-\left(\tfrac{1}{2}+ix\right)G(\sigma,x)=\frac{(\tfrac{1}{2}+\alpha-\sigma)(\tfrac{1}{2}+i\beta-\sigma)}{-2\sigma(1-2\sigma)}G(\sigma-1,x)-\\\
-\left\\{\frac{(\tfrac{1}{2}+\alpha-\sigma)(\tfrac{1}{2}+i\beta-\sigma)}{-2\sigma(1-2\sigma)}+\frac{(\tfrac{1}{2}+\alpha+\sigma)(\tfrac{1}{2}+i\beta+\sigma)}{2\sigma(1+2\sigma)}\right\\}G(\sigma,x)+\\\
+\frac{(\tfrac{1}{2}+\alpha+\sigma)(\tfrac{1}{2}+i\beta+\sigma)}{2\sigma(1+2\sigma)}F(\sigma+1,x)$

The expression in the curly brackets can be transformed as

$\Bigl{\\{}\dots\Bigr{\\}}=\frac{1}{2}+\frac{2i\alpha\beta}{(-1+2\sigma)(1+2\sigma)}.$

If $G(\sigma,x)$ satisfies the difference equation (3.2), then
$F(\sigma;x)=C_{1}(\sigma)G(\sigma;x)$ satisfies equation (3.1). So
$C_{1}(\sigma)\Psi_{1}(\sigma;x)$ satisfies (3.1). Since expression (3.1) is
invariant with respect to the transformation
$(\alpha,\beta)\leadsto(-\alpha,-\beta)$, the summand
$C_{2}(\sigma)\Psi_{2}(\sigma;x)$ also satisfies the difference equation.

##  4\. Some evaluations

There are many explicit evaluations for the Jacobi transform, this allows to
use it as a tool for obtaining non-trivial properties of special functions,
see [15], [19], [21] (see also, [23] for the complex analog of the Jacobi
transform). It is interesting to find a collection of evaluations of
$J_{\alpha,\beta}f$ for some functions $f$. This section contains few
examples.

The transform $J_{\alpha,\beta}$ sends the function

$\left(\tfrac{1}{2}+ix\right)^{-p}\left(\tfrac{1}{2}-ix\right)^{-q}$

to the function

(4.1)
$2\pi\Gamma\left(p+q+\overline{\sigma}-\tfrac{1}{2}\right)\,\,\vphantom{H}^{\phantom{\star}}_{3}H_{3}^{\star}\left[\begin{matrix}\frac{1-\alpha-i\beta}{2}+\overline{\sigma}+\overline{t},\,\,\frac{1+\alpha+i\beta}{2}+\overline{\sigma}+\overline{t},\,\,1-q-\overline{t}\\\
1-\frac{\alpha-i\beta}{2}+\overline{t},\,\,1+\frac{\alpha-i\beta}{2}+\overline{t},\,\,p+\frac{1}{2}+\overline{\sigma}+\overline{t}\end{matrix};1\right].$

To verify this, we must evaluate the integral

$\int_{-\infty}^{\infty}\left(\tfrac{1}{2}+ix\right)^{-p}\left(\tfrac{1}{2}-ix\right)^{-q}\times\\\
\times\overline{\left(\tfrac{1}{2}+ix\right)^{t}\left(\tfrac{1}{2}-ix\right)^{-1/2-t-\overline{\sigma}}\\!\\!\,\vphantom{H}{}^{\phantom{\star}}_{2}H_{2}^{\star}\left[\begin{matrix}\frac{1-\alpha+i\beta}{2}+\overline{\sigma}+\overline{t},\,\,\frac{1+\alpha-i\beta}{2}+\overline{\sigma}+t\\\
1-\frac{\alpha+i\beta}{2}+\overline{t},\,\,1+\frac{\alpha+i\beta}{2}+\overline{t}\end{matrix};-\frac{\frac{1}{2}+ix}{\frac{1}{2}-ix}\right]}\,dx.$

We expand $\,\vphantom{H}{}^{\phantom{\star}}_{2}H_{2}^{\star}$ into a series.
Integrating term-wise with the formula

$\int_{-\infty}^{\infty}\frac{dx}{\left(\frac{1}{2}+ix\right)^{\mu}\left(\frac{1}{2}-ix\right)^{\nu}}=\frac{2\pi\,\Gamma(\mu+\nu-1)}{\Gamma(\mu)\,\Gamma(\nu)},$

we come to (4.1).

Remark. The functions (4.1) are bilateral version of Hahn functions, which
were considered in [12]. $\boxtimes$

Next, for cases $q=\pm\frac{\alpha-i\beta}{2}$ and for
$p=\pm\frac{\alpha+i\beta}{2}$ the expression (4.1) can be simplified. For
instance, the transformation $J_{\alpha,\beta}$ sends a function

$\left(\tfrac{1}{2}+ix\right)^{-p}\left(\tfrac{1}{2}-ix\right)^{\alpha/2-i\beta/2}$

to

$\frac{-2\sin\pi(\frac{\alpha-i\beta}{2}+\overline{t})}{\Gamma(p+\frac{\alpha+i\beta}{2})\,\Gamma(p-\frac{\alpha+i\beta}{2})}\cdot\frac{\Gamma(p+\frac{-\alpha+i\beta-1}{2}+\overline{\sigma})\,\Gamma(p+\frac{-\alpha+i\beta-1}{2}+\overline{\sigma})}{\Gamma(\frac{1}{2}-\alpha-\overline{\sigma})\,\Gamma(\frac{1}{2}+i\beta-\overline{\sigma})}.$

To establish this statement, we substitute $q=-\alpha/2+i\beta/2$ to (4.1).
Then we get a function of the form

$\,\vphantom{H}{}^{\phantom{\star}}_{3}H_{3}^{\star}\left[\begin{matrix}a_{1},a_{2},c\\\
b_{1},b_{2},c\end{matrix};1\right]=\frac{\sin\pi
c}{\pi}\,\vphantom{H}^{\phantom{\star}}_{2}H_{2}^{\star}\left[\begin{matrix}a_{1},a_{2}\\\
b_{1},b_{2}\end{matrix};1\right]$

and apply the Dougall formula (1.2).

In a similar way we set $p=\alpha/2+i\beta/2$ and observe that our transform
sends

$\left(\tfrac{1}{2}+ix\right)^{-(\alpha+i\beta)/2}\left(\tfrac{1}{2}-ix\right)^{-q}$

to

$\frac{2\cos\pi(\frac{\alpha+i\beta}{2}+\overline{\sigma}+\overline{t})}{\Gamma(q+\frac{\alpha-i\beta}{2})\,\Gamma(q-\frac{\alpha-i\beta}{2})}\cdot\frac{\Gamma(q+\frac{\alpha+i\beta-1}{2}+\overline{\sigma})\Gamma(q+\frac{\alpha+i\beta-1}{2}-\overline{\sigma})}{\Gamma(\frac{1}{2}+\alpha-\overline{\sigma})\Gamma(\frac{1}{2}+i\beta-\overline{\sigma})}.$

Two remaining cases are similar.

## References

  * [1] Agarwal R. P. General transformations of bilateral cognate trigonometrical series of ordinary hypergeometric type. Canad. J. Math. 5 (1953), 544-553.
  * [2] Andrews G. E., Askey R., Roy R. Special functions. Cambridge University Press, Cambridge, 1999.
  * [3] Askey R., An integral of Ramanujan and orthogonal polynomials, J. Indian Math. Soc. 51 (1987), 27-36.
  * [4] Berezin F.A., Shubin M. A. The Schrödinger equation. Kluwer, Dordrecht, 1991.
  * [5] Cherednik I., Inverse Harish-Chandra transform and difference operators. Internat. Math. Res. Notices 1997 (1997) 733–750.
  * [6] van Diejen J. F., Emsiz E. Difference equation for the Heckman–Opdam hypergeometric function and its confluent Whittaker limit. Adv. Math. 285 (2015), 1225-1240.
  * [7] Dunford N., Schwartz J. T. Linear operators. Part II: Spectral theory. Self adjoint operators in Hilbert space. John Wiley & Sons, New York-London 1963.
  * [8] Erdélyi A., Magnus W., Oberhettinger F., Tricomi F. G. Higher transcendental functions. Vol. I, Based, in part, on notes left by Harry Bateman. McGraw-Hill, New York-Toronto-London, 1953.
  * [9] Erdélyi A., Magnus W., Oberhettinger F., Tricomi F. G. Higher transcendental functions. Vol. II, Based, in part, on notes left by Harry Bateman. McGraw-Hill, New York-Toronto-London, 1953.
  * [10] Faddeev L. D., Yakubovskiĭ O. A. Lectures on quantum mechanics for mathematics students. Amer. Math. Soc., Providence, RI, 2009.
  * [11] Groenevelt W. Wilson function transforms related to Racah coefficients. Acta Appl. Math. 91 (2006), no. 2, 133-191.
  * [12] Groenevelt W., Koelink E., Rosengren H. Continuous Hahn functions as Clebsch-Gordan coefficients, in Theory and applications of special functions, 221-284, Dev. Math., 13, Springer, New York, 2005.
  * [13] Koekoek R., Swarttouw R. F. The Askey-scheme of hypergeometric orthogonal polynomials and its $q$-analogue. Delft University of Technology, Faculty of Information Technology and Systems, Department of Technical Mathematics and Informatics, Report no. 98-17, 1998. Preprint https://arXiv.org/pdf/math/9602214.pdf
  * [14] Koornwinder T. H., Jacobi functions and analysis on noncompact symmetric spaces, in Special functions: Group theoretical aspects and applications, Reidel, Dordrecht 1984, pp. 1-85.
  * [15] Koornwinder T. H., Special orthogonal polynomial systems mapping to each other by the Fourier-Jacobi transform, Lecture Notes in Math., vol. 1171, Springer-Verlag, Berlin 1985, pp. 174-183.
  * [16] Molchanov V. F., Tensor products of unitary representations of the three-dimensional Lorentz group, Math. USSR-Izv., 15:1 (1980), 113-143.
  * [17] Molchanov V. F., Neretin Yu. A. A pair of commuting hypergeometric operators on the complex plane and bispectrality. J. of Spectral Theory (2021), DOI: 10.4171/JST/349.
  * [18] Neretin Yu. A., Index hypergeometric transform and imitation of analysis of Berezin kernels on hyperbolic spaces, Sb. Math., 192:3 (2001), 403-432.
  * [19] Neretin Yu. A., Beta-integrals and finite orthogonal systems of Wilson polynomials, Sb. Math., 193:7 (2002), 1071-1089.
  * [20] Neretin Yu. A., Some Continuous Analogs of the Expansion in Jacobi Polynomials and Vector-Valued Orthogonal Bases, Funct. Anal. Appl., 39:2 (2005), 106–119.
  * [21] Neretin Yu. A., Index hypergeometric integral transform. Addendum to Russian translation of Andrews G.E., Askey R., and Roy R., Special Functions, MCCME, 2013, 607–624; English version: Preprint, arXiv:1208.3342.
  * [22] Neretin, Yu. A. Difference Sturm-Liouville problems in the imaginary direction. J. Spectr. Theory 3 (2013), no. 3, 237–269.
  * [23] Neretin, Yury A. An analog of the Dougall formula and of the de Branges-Wilson integral. Ramanujan J. 54 (2021), no. 1, 93-106.
  * [24] Olevskiĭ M. N. On the representation of an arbitrary function in the form of an integral with a kernel containing a hypergeometric function. (Russian) Doklady Akad. Nauk SSSR (N.S.) 69, (1949), 11–14.
  * [25] Pukanszky L. On the Kronecker products of irreducible unitary representations of the $2\times 2$ real unimodular group. Trans. Amer. Math. Soc., 100 (1961), 116-152.
  * [26] Romanovski V., Sur quelques classes nouvelles de polynómes orthogonaux, C. R. Acad. Sci., Paris, 188 (1929), 1023-1025.
  * [27] Slater L. J. Generalized hypergeometric functions. Cambridge University Press, Cambridge, 1966.
  * [28] Titchmarsh E. C., Eigenfunction expansions with second-order differential operators, Clarendon Press, Oxford, 1946.
  * [29] Weyl H., Über gew onliche lineare Differentialgleichungen mit singulären Stellen und ihre Eigenfunktionen (2 Note), Nachr. Konig. Gess. Wiss. Göttingen. Math.-Phys. (1910), 442-467; in H. Weyl, Gesammelte Abhandlungen, vol. 1, Springer-Verlag, Berlin 1968, pp. 222-247.
  * [30] Yakubovich, S. B. Index transforms. World Scientific, River Edge, NJ, 1996.

Math. Dept., University of Vienna& Pauli Institute
&Institute for Theoretical and Experimental Physics (Moscow);
&MechMath Dept., Moscow State University;
&Institute for Information Transmission Problems;
URL: http://mat.univie.ac.at/$\sim$neretin/