# BCS-BEC crossover and quantum phase transition in an ultracold Fermi gas
under spin-orbit coupling

Fan Wu  Key Laboratory of Quantum Information, University of Science and
Technology of China, CAS, Hefei, Anhui, 230026, People’s Republic of China
Synergetic Innovation Center of Quantum Information and Quantum Physics,
University of Science and Technology of China, Hefei, Anhui 230026, People’s
Republic of China  Ren Zhang  Department of Physics, Renmin University of
China, Beijing 100872, People’s Republic of China  Tian-Shu Deng  Key
Laboratory of Quantum Information, University of Science and Technology of
China, CAS, Hefei, Anhui, 230026, People’s Republic of China  Synergetic
Innovation Center of Quantum Information and Quantum Physics, University of
Science and Technology of China, Hefei, Anhui 230026, People’s Republic of
China  Wei Zhang<EMAIL_ADDRESS>Department of Physics, Renmin University
of China, Beijing 100872, People’s Republic of China  Beijing Key Laboratory
of Opto-electronic Functional Materials and Micro-nano Devices, Beijing
100872, People’s Republic of China  Wei Yi<EMAIL_ADDRESS>Key Laboratory of
Quantum Information, University of Science and Technology of China, CAS,
Hefei, Anhui, 230026, People’s Republic of China  Synergetic Innovation Center
of Quantum Information and Quantum Physics, University of Science and
Technology of China, Hefei, Anhui 230026, People’s Republic of China  Guang-
Can Guo  Key Laboratory of Quantum Information, University of Science and
Technology of China, CAS, Hefei, Anhui, 230026, People’s Republic of China
Synergetic Innovation Center of Quantum Information and Quantum Physics,
University of Science and Technology of China, Hefei, Anhui 230026, People’s
Republic of China

###### Abstract

In this work, we study the BCS-BEC crossover and quantum phase transition in a
Fermi gas under Rashba spin-orbit coupling close to a Feshbach resonance. By
adopting a two-channel model, we take into account of the closed channel
molecules, and show that combined with spin-orbit coupling, a finite
background scattering in the open channel can lead to two branches of solution
for both the two-body and the many-body ground states. The branching of the
two-body bound state solution originates from the avoided crossing between
bound states in the open and the closed channels, respectively. For the many-
body states, we identify a quantum phase transition in the upper branch
regardless of the sign of the background scattering length, which is in clear
contrast to the case without spin-orbit coupling. For systems with negative
background scattering length in particular, we show that the bound state in
the open channel, and hence the quantum phase transition in the upper branch,
are induced by spin-orbit coupling. We then characterize the critical detuning
of the quantum phase transition for both positive and negative background
scattering lengths, and demonstrate the optimal parameters for the critical
point to be probed experimentally.

###### pacs:

67.85.Lm, 03.75.Ss, 05.30.Fk

##  I Introduction

Synthetic spin-orbit coupling (SOC), a recent addition to the toolbox
available for quantum simulation in ultracold atomic gases, can give rise to
interesting two-body and many-body properties by modifying the single-particle
dispersion spectrum of the underlying system gauge2exp ; fermisocexp1 ;
fermisocexp2 ; fermisocfeshnist ; fermisocexpfeshbach . In ultracold Fermi
gases, it has been shown that the implementation of synthetic SOC can lead to
an unconventional superfluid with non-trivial topological features, or a
superfluid with non-zero center-of-mass (CoM) momentum, or a combination of
both, depending on the spatial dimensions of the gas, and on the form of
synthetic SOC implemented soc3 ; TFZheng ; TFWei ; Ren_W ; chenggang ;
huhuitwo-c ; WF_A ; zhang ; sato ; chuanwei ; soc4 ; soc6 ; iskin ; thermo ;
2d2 ; 2d1 ; melo ; wmliu ; helianyi ; wy2d ; xiaosen ; wypolaron ;
iskinnistsoc ; puhantwobody ; shenoy ; wyfflo ; melosupp2 ; puhan3dsoc ; XF ;
xiongjun ; hufflo ; chuanweifflo ; bdg1 ; bdg2 ; HUHUI_2 ; zhangpeng ;
zhangpeng2 ; shenoy2 . Most of these studies have assumed the system to be
close to a Feshbach resonance, so that the interaction is tunable via an
external magnetic field. However, to characterize the Feshbach resonance, most
of the previous studies on spin-orbit coupled Fermi systems have adopted a
single-channel model TFZheng ; TFWei ; WF_A ; zhang ; sato ; chuanwei ; soc4 ;
soc6 ; iskin ; thermo ; 2d2 ; 2d1 ; melo ; wmliu ; helianyi ; wy2d ; xiaosen ;
wypolaron ; iskinnistsoc ; puhantwobody ; wyfflo ; melosupp2 ; puhan3dsoc ; XF
; xiongjun ; hufflo ; chuanweifflo ; bdg1 ; bdg2 ; HUHUI_2 ; zhangpeng ;
zhangpeng2 ; shenoy2 . While on a phenomenological level, a two-channel model
is more appropriate, where the Feshbach resonance is described as a multi-
channel resonant scattering process when the bound state in a closed channel
is tuned close to the continuum threshold of an open channel FBR . The two-
channel model reduces to a single-channel model only when the population of
the closed channel molecule becomes negligible, which is not always the case,
particularly under SOC huhuitwo-c ; Ren_W .

Recently, there have been several studies using two-channel models for the
characterization of spin-orbit coupled Fermi gases near a Feshbach resonance
huhuitwo-c ; chenggang ; Ren_W ; shenoy . A particularly interesting finding
is that the SOC can induce a new branch of two-body bound state. While it has
been reported before that in the absence of SOC, two branches of bound state
can be found near a Feshbach resonance for a Fermi gas with positive
background scattering length, the extra bound state under negative background
scattering length is purely induced by SOC. The existence of this new two-body
bound state should leave signatures on the many-body level. Indeed, for a
Fermi gas without SOC, a quantum phase transition exists for a positive
background scattering length, which is intimately connected with the
corresponding two-body bound state WY_D . We expect that similar phase
transitions may appear in a spin-orbit coupled Fermi gas when the proper two-
channel resonant scattering process is considered.

In this work, we study an ultracold Fermi gas under Rashba SOC close to a
Feshbach resonance using a two-channel model. We first confirm the two-body
calculations in Refs. shenoy ; chenggang , and study the branching of the two-
body bound state in the presence of a finite background scattering length. For
a positive background scattering length, the open channel supports a bound
state even in the absence of SOC, and the two branches of two-body bound state
originate from the avoided level crossing between the bound states in the open
and the closed channels. For a negative background scattering length, an SOC-
induced bound state emerges in the open channel for any finite SOC. The SOC-
induced bound state then couples with the bound state in the closed channel,
also leading to two branches of bound state.

With these understandings, we characterize many-body properties of the system
using a Bardeen-Cooper-Schrieffer (BCS) mean field approach. As expected, we
find two-branches of many-body solutions, the upper and the lower branch, for
any finite background scattering length. With a positive background scattering
length, we find that the lower branch is always bosonic with negative chemical
potential, essentially a condensate of tightly bound molecules. A quantum
phase transition exists in the upper branch, across which the ground state of
the Fermi gas changes from a superfluid state to a normal state. The position
of the phase transition can be controlled by tuning the SOC strength. We also
notice that by tuning the interaction or the SOC strength, a Bardeen-Cooper-
Schrieffer to Bose-Einstein condensate (BCS-BEC) crossover occurs in the upper
branch. With a negative background scattering length, the upper branch emerges
from the scattering threshold on the low-field-side of the Feshbach resonance
via a quantum phase transition for any finite SOC.

We discuss in detail the many-body properties of different branches under
various parameters, and characterize the critical detuning for the onset of
the quantum phase transition. We show that the results in this work, the
quantum phase transitions in particular, should best be observed in narrow
Feshbach resonances under appropriate SOC. While experimentally, only an equal
mixture of Rashba and Dresselhaus SOC has been realized in cold atomic gases
gauge2exp ; fermisocexp1 ; fermisocexp2 , there have been various proposals
for realizing the Rashba-type SOC xiongjun ; rashbagen1 ; rashbagen2 ;
rashbagen3 . With the recent experimental implementation of Feshbach resonance
in spin-orbit coupled degenerate Fermi gases fermisocfeshnist ;
fermisocexpfeshbach , we expect that the SOC-induced quantum phase transition
reported here can be experimentally probed in the future.

The paper is organized as the following: in Sec. II, we introduce the two-
channel model for a Fermi gas under Rashba SOC and close to a Feshbach
resonance. In Sec. III, we study the two-body bound state solutions under a
finite background scattering length. In Sec. IV, we discuss in detail the
many-body ground state of the two-channel model using the standard BCS mean-
field theory. For a finite background scattering length and a finite SOC,
there are typically two branches of ground state, where a quantum phase
transition can be identified in the upper branch. We then characterize the
critical point of the quantum phase transition in Sec. V, and finally
summarize in Sec. VI.

##  II Two-channel model

We consider a three-dimensional two-component Fermi gas close to a Feshbach
resonance under Rashba SOC. This system can be described by a two-channel
model

$\displaystyle H=H_{0}+H_{\rm SOC}+H_{\rm bf}+H_{\rm int},$ (1)

where the terms take the following forms

$\displaystyle H_{0}=$
$\displaystyle\sum_{\mathbf{k},\sigma=\uparrow,\downarrow}\epsilon_{\mathbf{k}}a^{{\dagger}}_{\mathbf{k}\sigma}a_{\mathbf{k}\sigma}+\sum_{\mathbf{q}}({\gamma}+\frac{\epsilon_{\mathbf{q}}}{2})b^{{\dagger}}_{\mathbf{q}}b_{\mathbf{q}},$
$\displaystyle H_{\rm SOC}=$
$\displaystyle\sum_{\mathbf{k}}{\alpha}[(k_{x}-ik_{y})a^{{\dagger}}_{\mathbf{k},\uparrow}a_{\mathbf{k},\downarrow}+h.c.],$
$\displaystyle H_{\rm bf}=$
$\displaystyle\frac{{g}}{\sqrt{V}}\sum_{\mathbf{k},\mathbf{q}}(a^{{\dagger}}_{\mathbf{k}+\mathbf{q}\uparrow}a^{{\dagger}}_{-\mathbf{k}+\mathbf{q}\downarrow}b_{\mathbf{q}}+h.c.),$
$\displaystyle H_{\rm int}=$
$\displaystyle\frac{{U}}{V}\sum_{\mathbf{k},\mathbf{k}^{\prime},\mathbf{q}}a^{{\dagger}}_{\mathbf{k}+\mathbf{q}\uparrow}a^{{\dagger}}_{-\mathbf{k}+\mathbf{q}\downarrow}a_{-\mathbf{k}^{\prime}+\mathbf{q}\downarrow}a_{\mathbf{k}^{\prime}+\mathbf{q}\uparrow}.$

Here, $a_{\mathbf{k},\sigma}$($a^{{\dagger}}_{\mathbf{k},\sigma}$) is the
annihilation (creation) operator for atoms with pseudo-spin $\sigma$ and
momentum ${\bf k}$, $\epsilon_{\mathbf{k}}=\hbar^{2}k^{2}/2m$ is the single
fermion dispersion, $b_{\mathbf{q}}$($b^{{\dagger}}_{\mathbf{q}}$) is the
annihilation (creation) operator for the closed channel molecules, ${\alpha}$
is the Rashba SOC strength, $V$ is the quantization volume, and $h.c.$ stands
for Hermitian conjugate. The bare atom-molecule coupling rate ${g}$, the bare
background interaction rate ${U}$, and the bare detuning ${\gamma}$ are
connected with the physical ones $\\{{g}_{p},{U}_{p},{\gamma}_{p}\\}$ through
the standard renormalization relations:
${U}=\Gamma{U}_{p},{g}=\Gamma{g}_{p},{\gamma}={\gamma}_{p}-\Gamma{g}^{2}_{p}/U_{c}$,
where
$\Gamma=(1+{U}_{p}/U_{c})^{-1},U^{-1}_{c}=-\sum_{\mathbf{k}}1/2\epsilon_{\mathbf{k}}$,
and ${U}_{p}=4\pi\hbar^{2}a_{\rm bg}/m,{g}^{2}_{p}=4\pi\hbar^{2}a_{\rm
bg}W\mu_{\rm co}/m$, and ${\gamma}_{p}=\mu_{\rm co}(B-B_{0})$ crossoverreview
. Here, $a_{\rm bg}$ is the background scattering length in the open channel,
$W$ is the Feshbach resonance width, $\mu_{\rm co}$ is the magnetic moment
difference between the closed and open channels, and $B-B_{0}$ is the magnetic
field detuning with $B_{0}$ the Feshbach resonance point.

To be consistent with the experimental parameters, we adopt the unit of energy
as $E_{0}$, and define the unit of the momentum $k_{0}$ and the unit of
density $n_{0}$ as

$k_{0}=\sqrt{\frac{2mE_{0}}{\hbar^{2}}},\quad\quad
n_{0}=\frac{k^{3}_{0}}{3\pi^{2}}.$ (2)

We then obtain a dimensionless version of the parameters in the Hamiltonian,
which will be used in the following discussion.

##  III Two-body bound states

In this section, we investigate the two-body bound-state solution under the
Hamiltonian Eq. (1). Due to the presence of SOC, the relative motion of the
fermions is dependent on the CoM motion. As a result, the bound-state energy
also acquires dependence on the CoM momentum. For the lowest energy case of
zero CoM momentum, the two-body bound state wave function can be written as

$\displaystyle|\Psi\rangle$ $\displaystyle=$ $\displaystyle\bigg{\\{}\beta
b_{0}^{\dagger}+\sum_{{\bf k}}{}^{\prime}\Big{[}\eta^{\uparrow\downarrow}({\bf
k})a_{{\bf k},\uparrow}^{\dagger}a_{-{\bf
k},\downarrow}^{\dagger}+\eta^{\downarrow\uparrow}({\bf k})a_{{\bf
k},\downarrow}^{\dagger}a_{-{\bf k},\uparrow}^{\dagger}$ (3)
$\displaystyle+\eta^{\uparrow\uparrow}({\bf k})a_{{\bf
k},\uparrow}^{\dagger}a_{-{\bf
k},\uparrow}^{\dagger}+\eta^{\downarrow\downarrow}({\bf k})a_{{\bf
k},\downarrow}^{\dagger}a_{-{\bf
k},\downarrow}^{\dagger}\Big{]}\bigg{\\}}|0\rangle,$

where $\beta$ and $\eta^{\sigma\sigma^{\prime}}$ denote the closed channel and
open channel coefficients, respectively, and the summation over momentum space
$\sum_{\bf k}^{\prime}$ runs over half of the momentum space with $k_{y}>0$.

By solving the Shrödinger’s equation $H|\Psi\rangle=E|\Psi\rangle$ and
matching coefficients, we obtain the equation for the two-body binding energy
$E$

$\displaystyle\left[{U}_{p}-\frac{{g}_{p}^{2}}{{\gamma}_{p}-E}\right]^{-1}={\cal
S}({\alpha},E),$ (4)

where ${\cal S}({\alpha},E)$ is defined as,

$\displaystyle{\cal
S}({\alpha},E)\equiv\frac{3\pi}{8\sqrt{2}}\left[\sqrt{-E}-\frac{\alpha}{\sqrt{2}}{\rm
arctanh}\left(\frac{{\alpha}}{\sqrt{-2E}}\right)\right].$ (5)

Figure 1: (Color online) Two-body binding energy as a function of detuning for
(a-b) positive background scattering length, and (c-d) negative background
scattering length. The results for a two-channel model (solid, red) are
compared with those from a single-channel model (dashed, blue). Using the unit
system defined within the text, the dimensionless atom-molecule coupling
constant $g_{p}=7$ for all panels. Other parameters are (a)
$U_{p}=0.17,\alpha=0$; (b) $U_{p}=0.17,\alpha=5$; (c) $U_{p}=-0.17,\alpha=0$;
(d) $U_{p}=-0.17,\alpha=5$.

In Fig. 1, we plot typical results for the two-body binding energy
$E_{b}\equiv E-E_{\rm th}$ for both cases of positive and negative background
scattering lengths, where $E_{\rm th}=-\alpha^{2}/2$ is the threshold. In the
absence of SOC [see Fig. 1(a) and (c)], there are two branches of bound state
solution for positive background scattering length, while there is only one
branch for negative background scattering length. This is consistent with the
calculations of Ref. WY_D . For finite SOC, an additional branch of bound
state emerges for the case with negative background scattering length [see
Fig. 1(d)], which is consistent with the results of Ref. shenoy ; chenggang .
As a comparison, results obtained from a single-channel model with
corresponding parameters are also shown.

The position of the bound state threshold in the upper branch can be
determined analytically for both cases of positive and negative background
scattering length, leading to

$\displaystyle\gamma_{c}=-\frac{\alpha^{2}}{2}+\frac{g_{p}^{2}}{U_{p}}.$ (6)

It is clear that the positions of the bound state threshold in both cases are
pushed towards the BEC-side of the Feshbach resonance as the SOC strength
increases. This is the direct result of decreasing threshold energy with
increasing SOC.

In the large-detuning limit, the binding energy in either branch
asymptotically approaches a common value $E_{\inf}$, which is determined by
the following equation,

$\displaystyle{U}_{p}=\frac{16}{3\pi}\frac{1}{\sqrt{-2E_{\inf}}-{\alpha}{\rm
arctanh}\left({\alpha}/\sqrt{-2E_{\inf}}\right)}.$ (7)

One can easily read from this result that $E_{\rm inf}$ becomes more negative
with increasing SOC strength.

Figure 2: (Color online) The superfluid order parameter $\Delta$, the shifted
chemical potential $\mu_{b}\equiv\mu+\alpha^{2}/4$, and the molecular fraction
$n_{b}$ associated with (a-c) the upper branch solution, and (d-f) the lower
branch solution for the case of a positive background scattering length. For
all panels, dimensionless parameters are chose as
$U_{p}=0.17,g_{p}=7,\alpha=1$ (solid, red), $U_{p}=0.14,g_{p}=5,\alpha=1$
(dashed, black), and $U_{p}=0.1,g_{p}=2.5,\alpha=1$ (dash-dotted, blue).

##  IV Manby-body pairing states

In this section, we characterize the many-body properties of the system at
zero temperature. Following the standard BCS mean-field theory, the effective
Hamiltonian can be written in a matrix form in the pseudo-spin basis
$\\{a_{\mathbf{k},\uparrow},a_{-\mathbf{k},\uparrow}^{{\dagger}},a_{\mathbf{k},\downarrow}a_{-\mathbf{k},\downarrow}^{{\dagger}}\\}^{T}$:

$\displaystyle H_{\text{eff}}-{\mu}N$ $\displaystyle=$
$\displaystyle\frac{1}{2}\sum_{\mathbf{k}}\left[\begin{array}[]{cccc}\lambda_{\mathbf{k}}&{\Delta}&0&\kappa_{\mathbf{k}}^{-}\\\
{\Delta}&-\lambda_{\mathbf{k}}&\kappa_{\mathbf{k}}^{-}&0\\\
0&\kappa_{\mathbf{k}}^{+}&-\lambda_{\mathbf{k}}&-{\Delta}\\\
\kappa_{\mathbf{k}}^{+}&0&-{\Delta}&\lambda_{\mathbf{k}}\end{array}\right]$
(13)
$\displaystyle+\sum_{\mathbf{k}}\left(\epsilon_{\mathbf{k}}+{U}f\right)+({\gamma}-2{\mu})|\psi_{m}|^{2}-{U}(|p|^{2}+f^{2}).$

Here,
$\lambda_{\mathbf{k}}=\epsilon_{\mathbf{k}}-{\mu}+Uf,\kappa_{\mathbf{k}}^{\pm}={\alpha}(k_{x}\pm
ik_{y})$, the order parameter ${\Delta}={g}\psi_{m}+{U}p$, and the mean-field
parameters are defined as:

$\displaystyle\psi_{m}=$ $\displaystyle<b_{0}>$ $\displaystyle p=$
$\displaystyle\sum_{\mathbf{k}}<a_{-\mathbf{k},\downarrow}a_{\mathbf{k},\uparrow}>$
$\displaystyle f=$
$\displaystyle\sum_{\mathbf{k}}<a^{{\dagger}}_{\mathbf{k},\uparrow}a_{\mathbf{k},\uparrow}>=\sum_{\mathbf{k}}<a^{{\dagger}}_{\mathbf{k},\downarrow}a_{\mathbf{k},\downarrow}>.$
(14)

Notice that the dimensionless total particle number $N=1$ in the unit system
defined in Sec. II.

By diagonalizing the effective Hamiltonian Eq. (13) and by imposing the
renormalization condition with physical parameters, we obtain the expression
for the ground state thermodynamic potential at zero temperature

$\displaystyle\Omega=$
$\displaystyle\sum_{\mathbf{k}}\left[\epsilon_{\mathbf{k}}-\frac{1}{2}(E_{\mathbf{k},+}+E_{\mathbf{k},-})+{U}f\right]$
$\displaystyle+({\gamma}-2{\mu})|\psi_{m}|^{2}-{U}(|p|^{2}+f^{2}),$ (15)

where the quasi-particle dispersion
$E_{\mathbf{k},\pm}=\sqrt{A^{2}_{\mathbf{k},\pm}+{\Delta}^{2}}$, and
$A_{\mathbf{k},\pm}=\epsilon_{\mathbf{k}}-{\mu}+{U}f\pm{\alpha}k_{\perp}$ with
$k_{\perp}=\sqrt{k_{x}^{2}+k_{z}^{2}}$ is defined to simplify notation.

From the extrema conditions $\partial\Omega/\partial f=0$,
$\partial\Omega/\partial p=0$, $\partial\Omega/\partial\psi_{m}=0$, and the
number equation $N=-\partial\Omega/\partial{\mu}$, we have a set of self-
consistent equations huhuitwo-c :

$\displaystyle\psi_{m}=$
$\displaystyle-\frac{{g}_{p}p}{({\gamma}_{p}-2{\mu})},$ $\displaystyle 2f=$
$\displaystyle\sum_{\mathbf{k}}\left(1-\frac{A_{+}}{2E_{+}}-\frac{A_{-}}{2E_{-}}\right),$
$\displaystyle 1=$
$\displaystyle\left({U}_{p}-\frac{{g}^{2}_{p}}{{\gamma}_{p}-2{\mu}}\right)\sum_{\mathbf{k}}\left(\frac{1}{2\epsilon_{\mathbf{k}}}-\frac{1}{4E_{\mathbf{k},+}}-\frac{1}{4E_{\mathbf{k},-}}\right),$
$\displaystyle 1=$ $\displaystyle
2f+2|{\Delta}|^{2}\left[{g}_{p}-\frac{({\gamma}_{p}-2{\mu}){U}_{p}}{{g}_{p}}\right]^{-2},$
(16)

from which the ground state parameters can be determined. Note that for the
parameter regime discussed in this work, we find the influence on the chemical
potential induced by the Hartree term $f$ remains negligible. Thus, the quasi-
particle dispersion can be approximated as

$E_{\mathbf{k},\pm}\approx\sqrt{(\epsilon_{\mathbf{k}}-{\mu}\pm{\alpha}k_{\perp})^{2}+{\Delta}^{2}}.$
(17)

Note that the self-consistent Eqs. (16) can be reduced to the more familiar
forms of the gap and number equations under SOC in a single-channel model by
setting ${g}_{p}=0$ wy2d . We also define the closed channel fraction as

$n_{b}=2|{\Delta}|^{2}\left[({g}_{p}-\frac{({\gamma}_{p}-2{\mu}){U}_{p}}{{g}_{p}})^{2}\right]^{-1},$
(18)

which will be used to describe the properties of the underlying system.

Figure 3: (Color online) The superfluid order parameter $\Delta$, the shifted
chemical potential $\mu_{b}\equiv\mu+\alpha^{2}/4$, and the molecular fraction
$n_{b}$ associated with (a-c) the upper branch solution, and (d-f) the lower
branch solution for the case of a negative background scattering length.
Dimensionless parameters used in this figure are
$U_{p}=-0.17,g_{p}=7,\alpha=5$ (solid, red), $U_{p}=-0.14,g_{p}=5,\alpha=5$
(dashed, black), and $U_{p}=-0.1,g_{p}=2.5,\alpha=5$ (dash-dotted, blue).

###  IV.1 Positive background scattering length

For the case of a positive background scattering length $a_{\rm bg}>0$, there
exists a weakly-bound state in the open channel away from the Fesbach
resonance. As the magnetic field is tuned close to the resonance point, the
coupling between the bound states in the open and the closed channels gives
rise to the two branches of many-body solution in the absence of SOC. Previous
studies have shown the existence of a quantum phase transition in the upper
branch where the many-body ground state changes from a superfluid state to a
normal state WY_D . The qualitative picture remains valid in the presence of
SOC, where quantitative modifications can be induced by the SOC. In Fig. 2, we
map out various mean-field quantities as functions of the detuning for several
scattering parameters.

In Fig. 2(a-c) we show the results of order parameter, the chemical potential,
and the molecular fraction as functions of detuning for the upper branch,
which corresponds to the weakly-bound state in the two-body case. Here, the
chemical potential is plotted after subtracting the single-particle threshold
with SOC, i.e., $\mu_{b}\equiv\mu+\alpha^{2}/4$. An interesting feature here
is the existence of a quantum phase transition, whose location can be
identified as the detuning where the order parameter approaches zero [see Fig.
2 (a)]. As the resonance width narrows, the location of the phase transition
point moves towards the BEC-side of the Feshbach resonance. Importantly, with
appropriate resonance width and SOC strength, the location of the quantum
phase transition point may cross the Feshbach resonance and reach the BEC
side, as we will show in Sec. V. We also note that the shifted chemical
potential $\mu_{b}$ in the upper branch crosses zero on the BEC side of the
resonance, demonstrating the existence of a BCS-BEC crossover [see the inset
of Fig. 2(b)]. We show in Fig. 2(d-f) the properties of the lower branch,
which corresponds to the deeply-bound state in the two-body case. It is
apparent that the shifted chemical potential stays negative with an order
parameter approaching finite values in both the weak and strong coupling
limit. Physically, the solution in the lower branch corresponds to a
condensate of tightly bound molecules, which become Rashbons in the large SOC
limit soc3 .

###  IV.2 Negative scattering length

We now turn to the case with negative background scattering length $a_{\rm
bg}<0$. In the absence of SOC, there is only one branch of many-body solution,
which features a BCS-BEC crossover as the interaction strength is tuned WY_D .
When SOC is turned on, however, this picture is drastically modified. Similar
to the two-body case, a new branch (upper branch) of many-body solution
emerges. Interestingly, a quantum phase transition can also be identified in
this upper branch.

In Fig. 3 (a-c) we show the results of the order parameter, the shifted
chemical potential, and the molecular fraction as functions of detuning for
the upper branch. Here, the location of the quantum phase transition is pushed
towards the BEC-limit with increasing SOC strength or Feshbach resonance
width, and the shifted chemical potential $\mu_{b}$ remains positive for
arbitrary detuning. In Fig. 3 (d-f), we show the same quantities for the lower
branch. Notice that there is no quantum phase transition in this branch, as
the order parameter is always finite. For small SOC, the shifted chemical
potential is positive in the BCS-limit, indicating the existence of a Fermi
surface. Hence, the system in the lower branch undergoes a BCS-BEC crossover
as the interaction becomes stronger or as the SOC strength increases.

##  V Quantum Phase Transition

In previous sections, we see that the quantum phase transition in the upper
branch is intimately connected with the SOC. For positive background
scattering length, SOC can modify the location of the phase transition point,
while for negative background scattering length, SOC can induce a new quantum
phase transition. In this section, we discuss in detail the dependence of the
phase transition point on various parameters.

The condition for the onset of this quantum phase transition can in fact be
obtained analytically by examining the gap and the number equations (16). At
the critical detuning where the quantum phase transition occurs, we have
$\Delta=0$. For the upper branch, regardless of the sign of the background
scattering length, this is only possible when the right-hand-side of the gap
equation also tends to infinity as the quantum critical point is approached.
Therefore, at the critical point, the denominator of the left-hand-side of the
gap equation must vanish, leading to

$\displaystyle U_{p}=\frac{g^{2}_{p}}{\gamma_{p}^{c}-2\mu}.$ (19)

Here, $\gamma_{p}^{c}$ is the critical detuning of the quantum phase
transition point in the upper branch. Similarly, the number equation at the
critical point takes the form

$\displaystyle
1=\sum_{\mathbf{k}}\left(1-\frac{A^{\prime}_{k,+}}{2|A^{\prime}_{k,+}|}-\frac{A^{\prime}_{k,-}}{2|A^{\prime}_{k,-}|}\right),$
(20)

where $A^{\prime}_{\mathbf{k},\pm}=\epsilon_{\mathbf{k}}-{\mu}\pm\alpha
k_{\perp}$. From these equations, we see that SOC only affects the quantum
critical point through the chemical potential $\mu$.

Figure 4: Critical detuning $\gamma_{p}^{c}$ for the quantum phase transition
in the upper branch as functions of $g_{p}$ and $\alpha$. Dimensionless
parameters used in these plots are (a) $\alpha=2,U_{p}=0.06$; (b)
$\alpha=5,U_{p}=-0.06$; (c) $g_{p}=0.5,U_{p}=0.06$; and (d)
$g_{p}=0.5,U_{p}=-0.06$.

We show in Fig. 4 the phase transition point in the upper branch as functions
of the scattering parameter $g_{p}$ and the SOC strength. For a fixed
background interaction rate $U_{p}$, the width of the Feshbach resonance
typically narrows with decreasing $g_{p}$. For a positive background
scattering length, the critical detuning can be made to cross the resonance
point when either the resonance width or the SOC strength is tuned [see Fig.
4(a) and 4(c)]. For systems with negative background scattering length,
however, the critical point is always lying on the BEC side of the Feshbach
resonance in any realistic situation. By choosing a narrow resonance with
small SOC strength, this quantum phase transition can be tuned closer to the
resonance points, as can be seen in Fig. 4(b) and 4(d). This observation
suggests that for the experimental observation of this quantum phase
transition, a system with narrow Feshbach resonance under moderate SOC
strength should be preferred.

##  VI SUMMARY

We have studied a spin-orbit coupled ultracold Fermi gas near a Feshbach
resonance by using a two-channel model. We find that under a finite SOC and
with a finite background scattering length, there are in general two branches
of solution for both the two-body and the many-body ground states. This is in
contrast to the conventional BCS-BEC crossover picture, where the background
scattering length is typically neglected; and is different from the case
without SOC, where the upper-branch solution only exists for a positive
background scattering length. As a result, for a negative background
scattering length, the bound state in the open channel is purely SOC induced.
These lead to the interesting situation that a quantum phase transition exists
in the upper branch of the many-body solution. The location of the quantum
phase transition can be tuned by the SOC strength, or by choosing Feshbach
resonances with different resonance widths. In particular, the critical point
of the quantum phase transition can be tuned close to or across the resonance
point, where the Fermi gas is most stable against three-body losses. It is
therefore hopeful that such phase transitions can be observed in experiments.

###### Acknowledgements.

This work is supported by NFRP (2011CB921200, 2011CBA00200), NKBRP
(2013CB922000), NNSF (60921091), NSFC (11105134, 11274009, 11374283), SRFDP
(20113402120022), the Fundamental Research Funds for the Central Universities
(WK2470000006), and the Research Funds of Renmin University of China
(10XNL016, 13XNH123).

## References

  * (1) Y.-J. Lin, K. Jiménez-García, and I. B. Spielman, Nature (London) 471, 83 (2011).
  * (2) P. Wang, Z.-Q. Yu, Z. Fu, J. Miao, L. Huang, S. Chai, H. Zhai, and J. Zhang, Phys. Rev. Lett. 109, 095301 (2012).
  * (3) L. W. Cheuk, A. T. Sommer, Z. Hadzibabic, T. Yefsah, W. S. Bakr, and M. W. Zwierlein, Phys. Rev. Lett. 109, 095302 (2012).
  * (4) R. A. Williams, M. C. Beeler, L. J. LeBlanc, K. Jiménez-García, and I. B. Spielman, Phys. Rev. Lett. 111, 095301 (2013).
  * (5) Z. Fu, L. Huang, Z. Meng, P. Wang, L. Zhang, S. Zhang, H. Zhai, P. Zhang, and J. Zhang, Nat. Phys. 10, 110 (2013).
  * (6) C. Zhang, S. Tewari, R. M. Lutchyn, and S. Das Sarma, Phys. Rev. Lett. 101, 160401 (2008).
  * (7) M. Sato, Y. Takahashi, and S. Fujimoto, Phys. Rev. Lett. 103, 020401 (2009).
  * (8) M. Iskin and A. L. Subasi, Phys. Rev. Lett. 107, 050402 (2011).
  * (9) M. Gong, S. Tewari, and C. Zhang, Phys. Rev. Lett. 107, 195303 (2011).
  * (10) H. Hu, L. Jiang, X.-J. Liu, and H. Pu, Phys. Rev. Lett. 107, 195304 (2011).
  * (11) Z.-Q. Yu and H. Zhai, Phys. Rev. Lett. 107, 195305 (2011).
  * (12) J. P. Vyasanakere, S. Zhang, and V. B. Shenoy, Phys. Rev. B 84, 014512 (2011).
  * (13) H. Hu and X.-J. Liu, Phys. Rev. A 83, 013631 (2011).
  * (14) W. Yi and G.-C. Guo, Phys. Rev. A 84, 031608(R) (2011).
  * (15) L. Dell’Anna, G. Mazzarella, and L. Salasnich, Phys. Rev. A 84, 033633 (2011).
  * (16) J. Zhou, W. Zhang, and W. Yi, Phys. Rev. A 84, 063603 (2011).
  * (17) R. Liao, Y. Yi-Xiang, and W.-M. Liu, Phys. Rev. Lett. 108, 080406 (2012).
  * (18) L. He and X.-G. Huang, Phys. Rev. Lett. 108, 145302 (2012).
  * (19) M. Gong, G. Chen, S. Jia, and C. Zhang, Phys. Rev. Lett. 109, 105302 (2012).
  * (20) W. Yi and W. Zhang, Phys. Rev. Lett. 109, 140402 (2012).
  * (21) L. Han and C. A. R. Sá de Melo, Phys. Rev. A 85 011606(R) (2012).
  * (22) X. Yang and S. Wan, Phys. Rev. A 85, 023633 (2012).
  * (23) P. Zhang, L. Zhang, and W. Zhang, Phys. Rev. A 86, 042707 (2012).
  * (24) P. Zhang, L. Zhang, and Y. Deng, Phys. Rev. A 86 053608 (2012).
  * (25) F. Wu, G.-C. Guo, W. Zhang, and W. Yi, Phys. Rev. Lett. 110, 110401 (2013).
  * (26) C. Qu, Z. Zheng, M. Gong, Y. Xu, L. Mao, X. Zou, G.-C. Guo, and C. Zhang, Nat. Commun. 4, 2710 (2013) .
  * (27) W. Zhang and W. Yi, Nat. Commun. 4, 2711 (2013).
  * (28) L. Dong, L. Jiang, and H. Pu, New J. Phys. 15, 075014 (2013).
  * (29) Z. Zheng, M. Gong, X. Zou, C. Zhang, and G.-C. Guo, Phys. Rev. A 87, 031602(R) (2013).
  * (30) L. Dong, L. Jiang, H. Hu, and H. Pu, Phys. Rev. A 87, 043616 (2013).
  * (31) X.-J. Liu and H. Hu, Phys. Rev. A 87, 051608 (2013).
  * (32) X.-F. Zhou, G.-C. Guo, W. Zhang, and W. Yi, Phys. Rev. A 87, 063606 (2013).
  * (33) M. Iskin and A. L. Subasi, Phys. Rev. A 87, 063627 (2013).
  * (34) M. Iskin, Phys. Rev. A 88, 013631 (2013).
  * (35) V. B. Shenoy, Phys. Rev. A 88, 033609 (2013).
  * (36) F. Wu, G.-C. Guo, W. Zhang, and W. Yi, Phys. Rev. A 88, 043614 (2013).
  * (37) K. Seo, L. Han, and C. A. R. Sá de Melo, arXiv:1301.1353.
  * (38) Y. Xu, C. Qu, M. Gong, and C. Zhang, Phys. Rev. A 89, 013607 (2014).
  * (39) X.-J. Liu, K. T. Law, and T. K. Ng, Phys. Rev. Lett. 112, 086401 (2014).
  * (40) R. Zhang, F. Wu, J.-R. Tang, G.-C. Guo, W. Yi, and W. Zhang, Phys. Rev. A 87, 033629 (2013).
  * (41) J.-X. Cui, X.-J. Liu, G. L. Long, and H. Hu, Phys. Rev. A 86, 053628 (2012).
  * (42) V. B. Shenoy, Phys. Rev. A 89, 043618 (2014).
  * (43) K. Zhang, G. Chen, and S.-T. Jia, Phys. Rev. A 88, 013611 (2013)
  * (44) R. A. Duine and H. T. C. Stoof, Phys. Rep. 396, 115 (2004).
  * (45) W. Yi and L.-M. Duan, Phys. Rev. A 73, 063607 (2006).
  * (46) D. L. Campbell, G. Juzeliūnas, and I. B. Spielman, Phys. Rev. A 84, 025602 (2011).
  * (47) Z. F. Xu and L. You, Phys. Rev. A 85, 043605 (2012).
  * (48) B. M. Anderson, I. B. Spielman, and G. Juzeliūnas, Phys. Rev. Lett. 111, 125301 (2013).
  * (49) Q.-J. Chen, J. Stajic, S. Tan, and K. Levin, Phys. Rep. 412, 1 (2005).