# Field-orientation dependence of quantum phase transitions in the
$\bm{S=1/2}$ triangular-lattice antiferromagnet Ba3CoSb2O9

Kazuki Okada1 Hidekazu Tanaka1<EMAIL_ADDRESS>Nobuyuki Kurita1
Daisuke Yamamoto2 Akira Matsuo3 Koichi Kindo2 1Department of Physics, Tokyo
Institute of Technology, Meguro-ku, Tokyo 152-8551, Japan
2Department of Physics, College of Humanities and Sciences, Nihon University,
Setagaya-ku, Tokyo 156-8550, Japan
3Institute for Solid State Physics, The University of Tokyo, Kashiwa, Chiba
277-8581, Japan

###### Abstract

Ba3CoSb2O9 approximates the two-dimensional spin-1/2 triangular-lattice
Heisenberg antiferromagnet. This compound displays magnetic-field-induced
quantum phase transitions, including the 1/3-magnetization-plateau, but its
magnetization processes for the magnetic field $H$ parallel and perpendicular
to the $c$ axis are different due to the weak easy-plane anisotropy and the
weak interlayer antiferromagnetic exchange interaction. To elucidate how the
quantum phase transitions change between these two field directions, we
measured the field-angle dependence of the magnetization process in Ba3CoSb2O9
using pulsed high magnetic fields. We compared obtained magnetic field–field
angle phase diagram with those obtained by the large-size cluster mean-field
method combined with a scaling scheme and the semiclassical theory. We also
found a narrow 1/3-magnetization plateau and a high-field transition with a
small magnetization jump for $H\,{\parallel}\,c$, not observed in the previous
studies.

###### pacs:

75.10.Jm, 75.45.+j, 75.60.Ej

††preprint: APS/123-QED

##  I Introduction

Usual phase transitions exemplified by the solid-liquid and ferromagnetic-
paramagnetic transitions are induced by temperature variations. The driving
forces are thermal fluctuations. On the other hand, quantum phase transitions
(QPTs) are transitions between different quantum ground states caused by the
continuous variation of interactions, which can be tuned by the magnetic
field, pressure, etc. QPTs are driven by the quantum fluctuation Sachdev ;
Vojta . The $S\,{=}\,1/2$ triangular-lattice Heisenberg antiferromagnet
(TLHAF) exhibits typical QPTs. The ground state of the two-dimensional (2D)
$S\,{=}\,1/2$ TLHAF with the nearest-neighbor exchange interaction at zero
magnetic field has been considered to be an ordered state with the 120∘
structure Huse ; Jolicoeur ; Bernu ; Singh ; White ; Gotze ; Ghioldi .
Although the ground state is qualitatively the same as the classical ground
state, the sublattice moment is reduced to about 40 % of the full moment due
to the quantum fluctuation Huse ; Jolicoeur ; Bernu ; Singh ; White ; Gotze ;
Ghioldi ; Li2 .

In a magnetic field, an up-up-down (UUD) spin structure is stabilized in a
finite field range by the quantum fluctuation. Consequently, the 2D
$S\,{=}\,1/2$ TLHAF exhibits a nonclassical magnetization plateau at one-third
of the saturation magnetization $M_{\rm s}$ Nishimori ; Chubokov ; Nikuni ;
Honecker ; Alicea ; Farnell ; Sakai ; Hotta ; Yamamoto1 ; Starykh2 ; Sellmann
; Coletta . With increasing magnetic field, successive QPTs from the Y-shaped
coplanar state to the UUD state, and to the $2\,{:}\,1$ canted coplanar state
occur Chubokov ; Nikuni ; Honecker ; Alicea ; Farnell ; Sakai ; Hotta ;
Yamamoto1 ; Starykh2 ; Sellmann ; Coletta .

The quantum magnetization plateau characteristic of the 2D $S\,{=}\,1/2$ TLHAF
was experimentally confirmed in Ba3CoSb2O9 Shirata ; Susuki ; Prabhakaran , in
which magnetic Co2+ ions with an effective spin $S\,{=}\,1/2$ at low
temperatures Abragam ; Lines ; Oguchi  form a uniform triangular lattice
Treiber_ZAAC1982 ; Doi . However, the magnetization plateau was observed only
for $H\,{\parallel}\,ab$ and not for $H\,{\parallel}\,c$ Susuki . In addition,
an unexpected high-field phase transition with a small magnetization jump was
observed above the plateau state for $H\,{\parallel}\,ab$ Susuki . This
anisotropic magnetization process and the high-field phase transition were
quantitatively explained by taking into account both the weak XY-like
anisotropic exchange interaction and the weak antiferromagnetic interlayer
exchange interaction Gekht ; Koutroulakis ; Yamamoto2 ; Li . The exchange
anisotropy and antiferromagnetic interlayer exchange interaction also give
rise to the different magnetic field–temperature phase diagrams for
$H\,{\parallel}\,ab$ and $H\,{\parallel}\,c$ Koutroulakis ; Zhou ; Quirion .
Besides Ba3CoSb2O9, 1/3-magnetization plateaux were observed in many quasi-2D
quantum triangular-lattice antiferromagnets such as Cs2CuBr4 Ono1 ; Ono2 ;
Fortune , Ba2CoTeO6 Nick1 , Ba2La2CoTe2O12 Kojima , CsYbSe2 Xing , NaYbSe2
Ranjith ; Ding  with the weak XY-like anisotropy, and Ba3NiSb2O9 Shirata2 ,
Ba2La2NiTe2O12 Saito , Na2BaCo(PO4)2 Li3  with the weak Ising-like anisotropy.
Observing a well-defined magnetization plateau shows that the sample has no
exchange randomness, as it is theoretically shown that the exchange randomness
makes indistinct the magnetization plateau Watanabe .

Figure 1 shows possible spin structures of the $S\,{=}\,1/2$ triangular-
lattice antiferromagnet with the weak XY-like anisotropy and the weak
antiferromagnetic interlayer exchange interaction at zero and finite magnetic
fields for $H\,{\parallel}\,ab$ and $H\,{\parallel}\,c$. The QPT sequence for
$H\,{\parallel}\,ab$ in Ba3CoSb2O9 is (a) $\rightarrow$ (b) $\rightarrow$ (c)
$\rightarrow$ (d) $\rightarrow$ (e) Gekht ; Koutroulakis ; Yamamoto2 ; Li ,
which was confirmed by a neutron diffraction experiment in high magnetic
fields Liu . The transition from the lower high-field coplanar state (d) to
the upper high-field coplanar state (e) is caused by the synergy between the
weak antiferromagnetic interlayer exchange interaction and quantum fluctuation
Gekht ; Koutroulakis ; Yamamoto2 ; Li . For $H\,{\parallel}\,c$, only a
transition from the umbrella structure (f) to the lower-field coplanar
structure (d) was observed in a previous study Susuki , although the theory
predicts a similar high-field transition between the (d) and (e) structures as
observed for $H\,{\parallel}\,ab$ Koutroulakis . Later the high-field
transition for $H\,{\parallel}\,c$ was observed via specific heat measurement
Fortune1 . The high-field transition from the (d) state to the (e) state is
absent for a ferromagnetic interlayer exchange interaction as in the case of
CsCuCl3 Yamamoto2 ; Adachi ; Tazuke ; Nojiri_CsCuCl3 ; Tanaka_JPSJ1992 ;
Motokawa ; Stusser ; Sera ; Yamamoto4 , and the $2\,{:}\,1$ canted coplanar
state is stable up to the saturation.

Figure 1: (Color online) Possible spin structures of the $S\,{=}\,1/2$
triangular-lattice antiferromagnet with the weak XY-like anisotropy and the
weak antiferromagnetic interlayer exchange interaction for
$H\,{\parallel}\,ab$ and $H\,{\parallel}\,c$. Solid and dashed arrows
respectively depict three sublattice spins (${\bm{S}}_{\rm A},{\bm{S}}_{\rm
B},{\bm{S}}_{\rm C}$) and (${\bm{S}}_{\rm A}^{\prime},{\bm{S}}_{\rm
B}^{\prime},{\bm{S}}_{\rm C}^{\prime}$) in neighboring triangular lattices.
(a) Zero-magnetic-field structure, (b) low-field coplanar structure for
$H\,{\parallel}\,ab$, (c) up-up-down (UUD) structure, (d) and (e) lower and
upper high-field coplanar structures, respectively, (f) umbrella structure for
$H\,{\parallel}\,c$, and (g) $\Psi$-structure.

It is not clear how the field-induced QPTs change with varying field
orientation in Ba3CoSb2O9. Using the semiclassical theory, Koutroulakis et al.
Koutroulakis  investigated the field-orientation dependence of field-induced
transitions and obtained a ground-state phase diagram for magnetic field vs.
field orientation. The phase diagram appears nontrivial because the transition
field related to the higher edge field of the 1/3-magnetization plateau for
$H\,{\parallel}\,ab$ increases with the angle between the magnetic field and
the $ab$ plane. However, the transition field connected to the lower edge
field is almost independent of the field orientation, and the critical field
for the high-field transition with a small magnetization jump has a clear
maximum at an intermediate field angle. Recently, Fortune et al. Fortune2
have reported the field-angle dependence of phase transitions in Ba3CoSb2O9
measured via the field dependences of specific heat, thermal conductivity, and
magnetic torque. However, the anomalies of these quantities were broad rather
than sharp. Therefore, there appears to be a certain level of ambiguity in
determining critical fields. It also seems difficult to identify whether these
anomalies arise from the phase transition or crossover.

This paper presents the results of high-field magnetization measurements on
Ba3CoSb2O9 conducted by varying the angle $\theta$ between the magnetic field
and the $c$ axis. We found a narrow 1/3-magnetization plateau and a high-field
transition with a small magnetization jump for $H\,{\parallel}\,c$, which were
not observed in the previous magnetization study Susuki . We compare obtained
magnetic field–field angle ($H\,{-}\,{\theta}$) phase diagram with those
obtained by the large-size cluster mean-field method combined with a scaling
scheme (CMF + S) Yamamoto1 ; Yamamoto2  and the semiclassical theory
Koutroulakis . The obtained $H\,{-}\,{\theta}$ phase diagram is generally in
accordance with those obtained by these theories.

##  II Experimental details

Figure 2: (Color online) Sample holder used in the present high-field
magnetization measurements. A plate-shaped sample is held between two quartz
stages with a slope having an angle of ${\theta}$ to the horizontal plane.

Single crystals of Ba3CoSb2O9 were grown from the melt using a Pt crucible or
Pt tube with one end welded. Ba3CoSb2O9 powder prepared by a procedure
described in a previous paper Shirata  was packed into the Pt crucible or Pt
tube. The temperature of the furnace was decreased from 1700 to 1650 ∘C over
two days. Plate-shaped crystals were extracted. The wide faces have already
been identified to be the $ab$ plane by previous X-ray single-crystal
diffraction and magnetization measurements.

The field-angle dependence of magnetization $M$ was measured at $T\,{=}\,1.8$
K, which is sufficiently lower than the Néel temperature
$T_{\mathrm{N}}\,{=}\,3.8$ K Doi , in magnetic fields up to $\mu_{0}H$ = 7 T
using a SQUID magnetometer (MPMS-XL, Quantum Design) equipped with a sample
rotator. The magnetizations at $T\,{=}\,0.5$ K in magnetic fields up to
$\mu_{0}H$ = 7 T for $H\,{\parallel}\,ab$ and $H\,{\parallel}\,c$ were also
measured using a 3He device (iHelium3, IQUANTUM). The field-angle dependence
of magnetization in pulsed high-field magnetic fields up to 52 T was measured
at $T$ = 1.3 K at the Institute for Solid State Physics, University of Tokyo.
Figure 2 illustrates the sample holder used in the high-field measurement. A
plate-shaped sample with a wide $ab$ plane was held between two quartz stages
with a slope having an angle of ${\theta}$ to the horizontal plane in a heat-
shrinkable tube with an inside diameter of 2.5 mm. The magnetic field was
applied parallel to the cylindrical axis of the holder, i.e.,
$H\,{\parallel}\,c$ when ${\theta}\,{=}\,0^{\circ}$. The absolute value of the
high-field magnetization was calibrated with the magnetization measured by the
SQUID magnetometer.

In the previous study Susuki , approximately ten single crystals were stacked
in a holder. In the present experiment, we used one single crystal to prevent
the dispersion of the crystal orientation. Consequently, sharp magnetization
anomalies indicative of phase transitions were observed, as shown below.
However, the absolute value of magnetization has a certain error level because
the background signal, mainly due to the sample holder, is relatively
enhanced.

##  III Results and discussion

Figure 3: (Color online) Field dependence of the magnetization $M$ (left) and
its field derivative $dM/dH$ (right) of Ba3CoSb2O9 measured at $T\,{=}\,1.3$ K
for (a) $H\,{\parallel}\,ab$ and for (b) $H\,{\parallel}\,c$, respectively,
where the Van Vleck paramagnetism were corrected. Thin solid and dotted lines
are theoretical magnetization curves and their field derivative, respectively,
calculated by the CMF + S with $J_{z}/J\,{=}\,0.75$ and
$J^{\prime}/J\,{=}\,0.05$, where the saturation fields and saturation
magnetizations are normalized by the experimental values. The inset of (b) is
an enlargement around the 1/3-magnetization plateau.  Figure 4: (Color online)
Low-field magnetization process of Ba3CoSb2O9 measured at $T\,{=}\,0.5$ K for
$H\,{\parallel}\,ab$ using a SQUID magnetometer.  Figure 5: (Color online)
Field derivatives of raw magnetization ${\rm d}M/{\rm d}H$ vs magnetic field
$H$ of Ba3CoSb2O9 measured at various field angles ${\theta}$. The data are
shifted upward by multiples of 0.1 (${\mu}_{\rm B}$/T)/Co2+. Vertical arrows
denote transition fields.

Figures 3 (a) and (b) show the magnetization processes in Ba3CoSb2O9 measured
for $H\,{\parallel}\,ab$ and $H\,{\parallel}\,c$, respectively, at
$T\,{=}\,1.3$ K using pulsed magnetic fields Comment . The magnetization data
were corrected for the Van Vleck paramagnetism, which was estimated from the
magnetization slope above the saturation. The transition anomalies in $dM/dH$
for both field directions are much sharper than those observed in the previous
study Susuki . This is because the dispersion of the crystal orientation is
almost absent owing to the use of one single crystal. Field-induced QPTs and
the saturation are identified at $H^{(ab)}_{\rm c1}{=}\,9.8$, $H^{(ab)}_{\rm
c2}{=}\,15.6$, $H^{(ab)}_{\rm c3}{=}\,23.2$, and $H^{(ab)}_{\rm s}{=}\,32.4$ T
for $H\,{\parallel}\,ab$, and at $H^{(c)}_{\rm c1}{=}\,12.0$, $H^{(c)}_{\rm
c2}{=}\,13.6$, $H^{(c)}_{\rm c3}{=}\,22.4$, and $H^{(c)}_{\rm s}{=}\,33.3$ T
for $H\,{\parallel}\,c$. It is particularly notable that a narrow
1/3-magnetization plateau and a transition with a small magnetization jump are
clearly observed for $H\,{\parallel}\,c$ between $H^{(c)}_{\rm c1}$ and
$H^{(c)}_{\rm c2}$, and at $H_{\rm c3}^{(c)}$, respectively, which were not
observed in the previous magnetization study Susuki . This result indicates
that the QPT sequence for $H\,{\parallel}\,c$ is (a) $\rightarrow$ (f)
$\rightarrow$ (c) $\rightarrow$ (d) $\rightarrow$ (e) in the spin structures
shown in Fig. 1 Yamamoto1 ; Sellmann ; Koutroulakis . Note that this QPT
sequence for $H\,{\parallel}\,c$ was also confirmed by a neutron diffraction
experiment in high magnetic fields Liu2 .

Here, we analyze the magnetization processes for $H\,{\parallel}\,ab$ and
$H\,{\parallel}\,c$ based on a model expressed as

$\displaystyle{\cal H}$ $\displaystyle=$ $\displaystyle\sum_{\langle
i,j\rangle}\left[J\left(S_{i}^{x}S_{j}^{x}+S_{i}^{y}S_{j}^{y}\right)+J_{z}S_{i}^{z}S_{j}^{z}\right]$
(1) $\displaystyle+$ $\displaystyle J^{\prime}\sum_{\langle
i,j\rangle^{\prime}}{\bm{S}}_{i}\cdot{\bm{S}}_{j}-g{\mu}_{\rm
B}\sum_{i}{\bm{H}}\cdot{\bm{S}}_{i},$

where the first and second terms are the antiferromagnetic exchange
interactions in a triangular layer and between triangular layers,
respectively. We describe the dominant intralayer exchange interaction by the
XXZ model. However, we express the weak interlayer exchange interaction by the
Heisenberg model for simplification. We calculate the magnetization process in
the ground state using the CMF + S method Yamamoto1 ; Yamamoto2 . Thin solid
and dotted lines in Figs. 3(a) and (b) are theoretical magnetization and its
field derivative, respectively, calculated with $J_{z}/J\,{=}\,0.75$ and
$J^{\prime}/J\,{=}\,0.05$, where the saturation fields and saturation
magnetizations for both field directions are normalized by the experimental
values. For $H\,{\parallel}\,ab$, the agreement between experimental results
and theory is almost perfect. However, for $H\,{\parallel}\,c$, there are some
disagreements between experimental results and theory. Due to the finite
temperature effect, the magnetization has a finite slope in the
1/3-magnetization plateau state, and the magnetization anomaly around the
higher edge field of the plateau is not sharp compared to the theory. The
theory predicts a transition at 30.9 T between the upper high-field coplanar
state and the highly symmetric $\Psi$-state shown in Figs. 1 (e) and (g),
respectively. The $\Psi$-state is stabilized immediately below the saturation
for $H\,{\parallel}\,c$ owing to the XY-like anisotropy and quantum
fluctuation in a specific parameter range Yamamoto1 ; Yamamoto2 . However, no
apparent magnetization anomaly indicative of a transition between the upper
high-field coplanar state and the $\Psi$-state was observed in the present
experiment. Note that the emergence of the $\Psi$-state in the theoretical
prediction for $H\,{\parallel}\,c$ depends largely on the value of the
interlayer interaction $J^{\prime}$, as in the case of $H\,{\parallel}\,ab$
Yamamoto2 . For smaller $J^{\prime}/J$, say $0.025$, the $\Psi$-state below
the saturation is absent, while the overall feature of the experimental
magnetization curve is still well reproduced with the setting of a larger
anisotropy, $J_{z}/J\,{=}\,0.73$. Further measurements are necessary to
confirm the presence or absence of the transition.

Recently, Fortune et al. Fortune2  have reported an additional low-magnetic-
field phase transition at $H\,{\simeq}\,6$ T for $H\,{\parallel}\,a$, which
emerges only below $T\,{=}\,1$ K before reaching the 1/3-magnetization plateau
state. To confirm this transition, we measured the magnetization in Ba3CoSb2O9
for $H\,{\parallel}\,ab$ at $T\,{=}\,0.5$ K in a steady magnetic field up to 7
T using a SQUID magnetometer. We did not identify the field direction in the
$ab$ plane, because the in-plane magnetic anisotropy should be negligible.
Figure 4 shows the magnetization $M$ (left) and its field derivative $dM/dH$
(right) as a function of $H$ at $T\,{=}\,0.5$ K. No anomaly is observed around
$H\,{=}\,6$ T in both the $M$ and $dM/dH$ data. Thus, the origin of the
thermodynamical anomalies at $H\,{\simeq}\,6$ T for $H\,{\parallel}\,a$
Fortune2  is unclear at present.

Figure 6: (Color online) (a) Experimental and (b) theoretical ground-state
phase diagrams parameterized by magnetic field and field angle ${\theta}$
($H\,{-}\,{\theta}$ phase diagram) for Ba3CoSb2O9 obtained by the
magnetization measurements and the calculations using the CMF + S with
$J_{z}/J\,{=}\,0.75$ and $J^{\prime}/J\,{=}\,0.05$, respectively. The
saturation fields in (b) are normalized by the experimental saturation fields.
Solid and dashed curves are guides to the eye, which denote the second-order
and first-order phase transitions, respectively. Phases are labeled in
accordance with Ref. Koutroulakis  as the low-field phase (LF), lower-
intermediate-field phase (LIF), upper-intermediate field (UIF), high-field
phase (HF), and fully polarized phase (FP). $H_{\rm c1}$, $H_{\rm c2}$,
$H_{\rm c3}$, and $H_{\rm s}$ are the critical fields for the LF–LIF, LIF–UIF,
and UIF–HF (LIF–HF) transitions and saturation, respectively. Two
1/3-magnetization plateau (UUD) phases exist for $H\,{\parallel}\,ab$ and
$H\,{\parallel}\,c$, depicted by two vertical red lines. Figure 7: (Color
online) Field dependence of the magnetization $M$ (left) and its field
derivative $dM/dH$ (right) of Ba3CoSb2O9 measured at $T\,{=}\,1.3$ K for
${\theta}\,{=}\,40^{\circ}$ and $60^{\circ}$. The data of $M$ and $dM/dH$ for
${\theta}\,{=}\,60^{\circ}$ are shifted by 2.0 ${\mu}_{\rm B}/{\rm Co}^{2+}$
and 0.2 $({\mu}_{\rm B}/{\rm T})/{\rm Co}^{2+}$, respectively. Thin dotted
lines are $dM/dH$ vs $H$ calculated by the CMF + S with $J_{z}/J\,{=}\,0.75$
and $J^{\prime}/J\,{=}\,0.05$, where the saturation fields and saturation
magnetizations are normalized by the experimental values. Vertical blue arrows
and inverse orange triangles denote experimental and theoretical transition
fields, respectively.

Figure 5 shows field derivatives of magnetization ${\rm d}M/{\rm d}H$ vs
magnetic field $H$ measured at various field angles ${\theta}$. Field-induced
QPTs are clearly observed as sharp anomalies of ${\rm d}M/{\rm d}H$. A
systematic change in transition field can be seen upon varying the field angle
${\theta}$ from 90∘ to 0∘. Figure 6 (a) shows the transition data. We label
the emerging phase in accordance with Ref. Koutroulakis . The low-field (LF)
phase emerges at low magnetic fields and corresponds to the low-field coplanar
state (b) and the umbrella state (f) below the 1/3-magnetization plateau for
$H\,{\parallel}\,ab$ and $H\,{\parallel}\,c$, respectively. The lower-
intermediate-field (LIF) and the upper-intermediate-field (UIF) phases
continuously connect the 1/3-magnetization plateau state (UUD) and the lower-
high-field state (d) for $H\,{\parallel}\,ab$, respectively. The high-field
(HF) phase corresponds to the upper-high-field state (e) for
$H\,{\parallel}\,ab$ and $H\,{\parallel}\,c$. In Fig. 6 (a), $H_{\rm c1}$,
$H_{\rm c2}$, $H_{\rm c3}$, and $H_{\rm s}$ are the critical fields for the
LF–LIF, LIF–UIF, and UIF–HF (LIF–HF) transitions and saturation, respectively.

The lowest critical field $H_{\rm c1}$, which is the lower edge field of the
1/3-magnetization plateau for $H\,{\parallel}\,ab$, slightly increases with
decreasing ${\theta}$ from 90∘. For ${\theta}\,{\geq}\,15^{\circ}$, the
anomaly of ${\rm d}M/{\rm d}H$ at $H_{\rm c1}$ is sharp but continuous, while
for ${\theta}\,{=}\,10^{\circ}$ and 0∘, ${\rm d}M/{\rm d}H$ displays a
$\delta$-function-like peak at $H_{\rm c1}$ indicative of the first-order
transition with a magnetization jump. This result shows that the LF–LIF
transition changes from the second-order to the first-order between 15∘ and
10∘. The change in the nature of the LF–LIF transition can be confirmed by the
microscopic calculation using CMF + S method, as shown in Fig. 6 (b).

In the 1/3-magnetization plateau state for $H\,{\parallel}\,ab$, ${\rm
d}M/{\rm d}H$ exhibits sharp rectangular anomalies at both edge fields $H_{\rm
c1}$ and $H_{\rm c2}$. With decreasing field angle ${\theta}$ from 90∘, the
anomaly of ${\rm d}M/{\rm d}H$ at $H_{\rm c1}$ remains sharp and vertical,
irrespective of ${\theta}$, while the anomaly at $H_{\rm c2}$ is smoothed down
to a cusp, as shown in Fig. 7. This indicates that the 1/3-magnetization
plateau at ${\theta}\,{=}\,90^{\circ}$ changes its shape to a ski-jump ramp
with decreasing field angle ${\theta}$, which is consistent with the
semiclassical theory Koutroulakis , and can also be reproduced by the
microscopic calculation shown in Fig. 7.

Note that the critical field $H_{\rm c2}$ for the LIF–UIF transition, which
corresponds to the higher edge field of the 1/3-magnetization plateau for
$H\,{\parallel}\,ab$, increases with decreasing ${\theta}$ from 90∘, thus
increasing the field range between $H_{\rm c1}$ and $H_{\rm c2}$. Figure 7
shows the experimental and calculated field dependences of $dM/dH$ for
${\theta}\,{=}\,60^{\circ}$ and $40^{\circ}$. The experimental results are
well reproduced by the CMF + S calculations with the same exchange parameters
used for the calculations for $H\,{\parallel}\,ab$ and $H\,{\parallel}\,c$
shown in Figs. 3 (a) and (b). The transition anomaly at $H_{\rm c2}$ becomes
small with decreasing ${\theta}$. The anomaly of experimental $dM/dH$ at
$H_{\rm c2}$ is barely distinguishable at ${\theta}\,{=}\,40^{\circ}$ by
comparing to the theoretical $dM/dH$, which exhibits a small stepwise anomaly
at $H_{\rm c2}$. The calculated transition fields $H_{\rm c2}$ and $H_{\rm
c3}$ are somewhat higher than those observed. As shown in Fig. 6 (b), the
calculated LIF–UIF boundary meets the UIF–HF boundary. However, it is not
clear whether the experimental LIF–UIF boundary has an endpoint or whether it
meets the UIF–HF boundary.

The two plateau states for $H\,{\parallel}\,ab$ and $H\,{\parallel}\,c$ are
not the two ends of a plateau phase. These results are somewhat surprising
because it is naively expected that a plateau phase exists for
$0^{\circ}\,{<}\,{\theta}\,{<}\,90^{\circ}$ that connects the two plateau
states for $H\,{\parallel}\,ab$ and $H\,{\parallel}\,c$, and that its field
range decreases monotonically with decreasing ${\theta}$ from 90∘ to 0∘. It is
considered that the enhanced field range of the LIF phase is attributed to the
XY-like exchange anisotropy, which breaks the rotational symmetry around the
external magnetic field Koutroulakis ; Yamamoto2 . This $U(1)$ symmetry is
conserved only for $H\,{\parallel}\,c$.

The critical field $H_{\rm c3}$ for the UIF–HF (LIF–HF) transition shows a
shallow minimum at ${\theta}\,{=}\,65{-}70^{\circ}$ and a clear maximum at
${\theta}\,{=}\,15{-}20^{\circ}$, as predicted by the semiclassical theory
Koutroulakis . As shown in Fig. 3 (b), the CMF + S calculation for
$H\,{\parallel}\,c$ with $J_{z}/J\,{=}\,0.75$ and $J^{\prime}/J\,{=}\,0.05$
predicts the emergence of the $\Psi$-state between the upper high-field
coplanar state and the fully polarized state. The calculation shows that the
LIF–UIF boundary connects with the transition field between the upper high-
field coplanar state and the $\Psi$-state, as shown in Fig. 6 (b). Further
measurements are necessary to verify the theory. The $\Psi$-state is
stabilized for $H\,{\parallel}\,c$ with the help of the XY-like anisotropy and
quantum fluctuation Yamamoto1 ; Yamamoto2 . Because the XY-like anisotropy in
Ba2La2CoTe2O12 is relatively larger than that in Ba3CoSb2O9 Kojima , we can
expect the emergence of the $\Psi$-state in Ba2La2CoTe2O12.

The high-field LIF–HF transition ($H_{\rm c3}$) is no longer observed in the
${\rm d}M/{\rm d}H$ data at ${\theta}\,{=}\,10^{\circ}$, as shown in Fig. 5.
At ${\theta}\,{=}\,10^{\circ}$, the anomaly assigned to $H_{\rm c2}$ is also
absent. The narrow 1/3-magnetization plateau state at
${\theta}\,{=}\,0^{\circ}$ immediately changes into the shape of a ski-jump
ramp at ${\theta}\,{=}\,10^{\circ}$. The CMF + S calculations show that the
1/3-magnetization plateau state emerges only for ${\theta}\,{<}\,5^{\circ}$,
as shown in Fig. 6 (b), and that in the vicinity of
${\theta}\,{=}\,0^{\circ}$, the transition anomalies for $H_{\rm c2}$ and
$H_{\rm c3}$ become so small that they are difficult to be detected. Thus, the
phase boundaries near ${\theta}\,{=}\,0^{\circ}$ are sensitive to the field
angle ${\theta}$. To investigate the phase boundaries near
${\theta}\,{=}\,0^{\circ}$ in detail, a precise magnetization measurement with
the fine-tuning of the field angle ${\theta}$ is necessary.

In the previous magnetization study Susuki , neither the narrow
1/3-magnetization plateau nor the high-field transition was observed for
$H\,{\parallel}\,c$. In that experiment, about ten single crystals were
stacked in a cylindrical holder. Consequently, a dispersion of crystal
orientations of at most ${\pm}10^{\circ}$ was inevitable. Thus, we deduce that
the narrow 1/3-magnetization plateau and the high-field transition were
smeared out owing to the dispersion of crystal orientations because the fields
and the magnitude of anomalies at $H_{\rm c2}$ and $H_{\rm c3}$ are sensitive
to the field angle ${\theta}$ near ${\theta}\,{=}\,0^{\circ}$.

##  IV Conclusion

The magnetic-field-orientation dependence of QPTs in the quasi-2D
$S\,{=}\,1/2$ triangular-lattice antiferromagnet Ba3CoSb2O9 has been
investigated by magnetization measurements using one single crystal. The
transition anomalies in the field derivative of the magnetization ${\rm
d}M/{\rm d}H$ were sharp and clear because of the absence of the dispersion of
the crystal orientation. We observed a 1/3-magnetization plateau and a high-
field transition with a small magnetization jump for $H\,{\parallel}\,c$,
which were not observed in the previous study Susuki . We obtained a ground-
state $H\,{-}\,{\theta}$ phase diagram of Ba3CoSb2O9 as shown in Fig. 6 (a).
The features of the phase diagram are summarized as follows. The LF–LIF
transition changes from the second-order to the first-order at
${\theta}\,{=}\,10{-}15^{\circ}$ with decreasing field angle ${\theta}$. The
critical field $H_{\rm c2}$ for the LIF–UIF transition increases with
decreasing ${\theta}$; thus, the field range of the LIF phase is enhanced. The
critical field $H_{\rm c3}$ for the UIF(LIF)–HF transition has a clear maximum
at ${\theta}\,{=}\,15{-}20^{\circ}$. The $H\,{-}\,{\theta}$ phase diagram is
generally in accordance with the theoretical phase diagram obtained by the CMF
+ S calculations with $J_{z}/J\,{=}\,0.75$ and $J^{\prime}/J\,{=}\,0.05$, as
shown in Fig. 6 (b). Due to the small XY-like anisotropy and weak
antiferromagnetic interlayer exchange interaction, the phase boundaries
display nontrivial field-angle dependence. It will be of interest to
investigate the $H\,{-}\,{\theta}$ phase diagram by tuning the anisotropy and
interlayer exchange interaction using hydrostatic pressure as recently
performed for CsCuCl3 Sera ; Yamamoto4 .

## Acknowledgment

We would like to thank O. Prokhnenko and X. Liu for showing us their
experimental results of neutron diffraction in high magnetic fields and for
the useful discussion. This work was supported by Grants-in-Aid for Scientific
Research (A) (No. 17H01142) and (C) (No. 19K03711) from Japan Society for the
Promotion of Science.

## References

  * (1) S. Sachdev, Quantum Phase Transitions (Cambridge University Press, 1999).
  * (2) M. Vojta, Quantum phase transitions, Rep. Prog. Phys. 66, 2069 (2003).
  * (3) D. A. Huse and V. Elser, Simple variational wave functions for two-dimensional Heisenberg spin-1/2 antiferromagnets, Phys. Rev. Lett. 60, 2531 (1988).
  * (4) Th. Jolicoeur and J. C. Le Guillou, Spin-wave results for the triangular Heisenberg antiferromagnet, Phys. Rev. B 40, 2727(R) (1989).
  * (5) B. Bernu, P. Lecheminant, C. Lhuillier, and L. Pierre, Exact spectra, spin susceptibilities, and order parameter of the quantum Heisenberg antiferromagnet on the triangular lattice, Phys. Rev. B 50, 10048 (1994).
  * (6) R. R. P. Singh and D. A. Huse, Three-sublattice order in triangular- and Kagomé-lattice spin-half antiferromagnets, Phys. Rev. Lett. 68, 1766 (1992).
  * (7) S. R. White and A. L. Chernyshev, Néel order in square and triangular lattice Heisenberg models, Phys. Rev. Lett. 99, 127004 (2007).
  * (8) O. Götze, J. Richter, R. Zinke, and D. J. J. Farnell, Ground-state properties of the triangular-lattice Heisenberg antiferromagnet with arbitrary spin quantum number $s$, J. Magn. Magn. Mater. 397, 333 (2016).
  * (9) E. A. Ghioldi, M. G. Gonzalez, S. S. Zhang, Y. Kamiya, L. O. Manuel, A. E. Trumper, and C. D. Batista, Dynamical structure factor of the triangular antiferromagnet: Schwinger boson theory beyond mean field, Phys. Rev. B 98, 184403 (2018).
  * (10) Q. Li, H. Li, J. Zhao, H. G. Luo, and Z. Y. Xie, Magnetization of the spin-1/2 Heisenberg antiferromagnet on the triangular lattice, Phys. Rev. B 105, 184418 (2022).
  * (11) H. Nishimori and S. Miyashita, Magnetization process of the spin-1/2 antiferromagnetic Ising-like Heisenberg model on the triangular lattice, J. Phys. Soc. Jpn. 55, 4448 (1986).
  * (12) A. V. Chubokov and D. I. Golosov, Quantum theory of an antiferromagnet on a triangular lattice in a magnetic field, J. Phys.: Condens. Matter 3, 69 (1991).
  * (13) T. Nikuni and H. Shiba, Quantum fluctuations and magnetic structures of CsCuCl3 in high magnetic field, J. Phys. Soc. Jpn. 62, 3268 (1993).
  * (14) A. Honecker, A comparative study of the magnetization process of two-dimensional antiferromagnets, J. Phys.: Condens. Matter 11, 4697 (1999).
  * (15) J. Alicea, A. V. Chubukov, and O. A. Starykh, Quantum stabilization of the 1/3-magnetization plateau in Cs2CuBr4, Phys. Rev. Lett. 102, 137201 (2009).
  * (16) D. J. J. Farnell, R. Zinke, J. Schulenburg, and J. Richter, High-order coupled cluster method study of frustrated and unfrustrated quantum magnets in external magnetic fields, J. Phys.: Condens. Matter 21, 406002 (2009).
  * (17) T. Sakai and H. Nakano, Critical magnetization behavior of the triangular- and kagome-lattice quantum antiferromagnets, Phys. Rev. B 83, 100405(R) (2011).
  * (18) C. Hotta, S. Nishimoto, and N. Shibata, Grand canonical finite size numerical approaches in one and two dimensions: Real space energy renormalization and edge state generation, Phys. Rev. B 87, 115128 (2013).
  * (19) D. Yamamoto, G. Marmorini, and I. Danshita, Quantum phase diagram of the triangular-lattice XXZ model in a magnetic field, Phys. Rev. Lett. 112, 127203 (2014).
  * (20) O. A. Starykh, Unusual ordered phases of highly frustrated magnets: a review, Rep. Prog. Phys. 78, 052502 (2015).
  * (21) D. Sellmann, X. F. Zhang, and S. Eggert, Phase diagram of the antiferromagnetic XXZ model on the triangular lattice, Phys. Rev. B 91, 081104(R) (2015).
  * (22) T. Coletta, T. A. Tóth, K. Penc, and F. Mila, Semiclassical theory of the magnetization process of the triangular lattice Heisenberg model, Phys. Rev. B 94, 075136 (2016).
  * (23) Y. Shirata, H. Tanaka, A. Matsuo, and K. Kindo, Experimental realization of a spin-1/2 triangular-lattice Heisenberg antiferromagnet, Phys. Rev. Lett. 108, 057205 (2012).
  * (24) T. Susuki, N. Kurita, T. Tanaka, H. Nojiri, A. Matsuo, K. Kindo, and H. Tanaka, Magnetization process and collective excitations in the $S=1/2$ triangular-lattice Heisenberg antiferromagnet Ba3CoSb2O9, Phys. Rev. Lett. 110, 267201 (2013).
  * (25) D. Prabhakaran and A. T. Boothroyd, Crystal growth of the triangular-lattice antiferromagnet Ba3CoSb2O9, J. Cryst. Growth 468, 345 (2017).
  * (26) A. Abragam and M. H. L. Pryce, The theory of paramagnetic resonance in hydrated cobalt salts, Proc. R. Soc. (London), A 206, 173 (1951).
  * (27) M. E. Lines, Magnetic properties of CoCl2 and NiCl2, Phys. Rev. 131, 546 (1963).
  * (28) T. Oguchi, Theory of magnetism in CoCl2$\cdot$2H2O, J. Phys. Soc. Jpn. 20, 2236 (1965).
  * (29) U. Treiber and S. Kemmler-Sack, Über die kationenordnung in perowskiten mit hexagonaler BaTiO3-struktur vom typ Ba3BIISb2O9 (BII: Mg, Mn, Co, Ni, Cu, Zn), Z. Anorg. Allg. Chem. 487, 161 (1982).
  * (30) Y. Doi, Y. Hinatsu, and K. Ohoyama, Structural and magnetic properties of pseudo-two-dimensional triangular antiferromagnets Ba3MSb2O9 (M = Mn, Co, and Ni), J. Phys.: Condens. Matter 16, 8923 (2004).
  * (31) R. S. Gekht and I. N. Bondarenko, Triangular antiferromagnets with a layered structure in a uniform field, J. Exp. Theor. Phys. 84, 345 (1997).
  * (32) G. Koutroulakis, T. Zhou, Y. Kamiya, J. D. Thompson, H. D. Zhou, C. D. Batista, and S. E. Brown, Quantum phase diagram of the $S=\frac{1}{2}$ triangular-lattice antiferromagnet Ba3CoSb2O9, Phys. Rev. B 91, 024410 (2015).
  * (33) D. Yamamoto, G. Marmorini, and I. Danshita, Microscopic model calculations for the magnetization process of layered triangular-lattice quantum antiferromagnets, Phys. Rev. Lett. 114, 027201 (2015).
  * (34) M. Li, M. L. Plumer, and G. Quirion, Effects of interlayer and bi-quadratic exchange coupling on layered triangular lattice antiferromagnets, J. Phys.: Condens. Matter 32, 135803 (2020).
  * (35) H. D. Zhou, C. Xu, A. M. Hallas, H. J. Silverstein, C. R. Wiebe, I. Umegaki, J. Q. Yan, T. P. Murphy, J.-H. Park, Y. Qiu, J. R. D. Copley, J. S. Gardner, and Y. Takano, Successive phase transitions and extended spin-excitation continuum in the $S=1/2$ triangular-lattice antiferromagnet Ba3CoSb2O9, Phys. Rev. Lett. 109, 267206 (2012).
  * (36) G. Quirion, M. Lapointe-Major, M. Poirier, J. A. Quilliam, Z. L. Dun, and H. D. Zhou, Magnetic phase diagram of Ba3CoSb2O9 as determined by ultrasound velocity measurements, Phys. Rev. B 92, 014414 (2015).
  * (37) T. Ono, H. Tanaka, H. Aruga Katori, F. Ishikawa, H. Mitamura, and T. Goto, Magnetization plateau in the frustrated quantum spin system Cs2CuBr4, Phys. Rev. B 67, 104431 (2003).
  * (38) T. Ono, H. Tanaka, O. Kolomiyets, H. Mitamura, T. Goto, K. Nakajima, A. Oosawa, Y. Koike, K. Kakurai, J. Klenke, P. Smeibidle, and M. Meißner, Magnetization plateaux of $S\,{=}\,1/2$ two dimensional frustrated antiferromagnet Cs2CuBr4, J. Phys.: Condens. Matter 16, S773 (2004).
  * (39) N. A. Fortune, S. T. Hannahs, Y. Yoshida, T. E. Sherline, T. Ono, H. Tanaka, and Y. Takano, Cascade of magnetic-field-induced quantum phase transitions in a spin-1/2 triangular-lattice antiferromagnet, Phys. Rev. Lett. 102, 257201 (2009).
  * (40) P. Chanlert, N. Kurita, H. Tanaka, D. Goto, A. Matsuo, and K. Kindo, Field-driven successive phase transitions in the quasi-two-dimensional frustrated antiferromagnet Ba2CoTeO6 and highly degenerate classical ground states, Phys. Rev. B 93, 094420 (2016).
  * (41) Y. Kojima, M. Watanabe, N. Kurita, H. Tanaka, A. Matsuo, K. Kindo, and M. Avdeev, Quantum magnetic properties of the spin-1/2 triangular-lattice antiferromagnet Ba2La2CoTe2O12, Phys. Rev. B 98, 174406 (2018).
  * (42) J. Xing, L. D. Sanjeewa, J. Kim, G. R. Stewart, A. Podlesnyak, and A. S. Sefat, Field-induced magnetic transition and spin fluctuations in the quantum spin-liquid candidate CsYbSe2, Phys. Rev. B 100, 220407(R) (2019).
  * (43) K. M. Ranjith, S. Luther, T. Reimann, B. Schmidt, Ph. Schlender, J. Sichelschmidt, H. Yasuoka, A. M. Strydom, Y. Skourski, J. Wosnitza, H. Kühne, Th. Doert, and M. Baenitz, Phys. Rev. B 100, 224417 (2019).
  * (44) L. Ding, P. Manuel, S. Bachus, F. Grußler, P. Gegenwart, J. Singleton, R. D. Johnson, H. C. Walker, D. T. Adroja, A. D. Hillier, and A. A. Tsirlin, Gapless spin-liquid state in the structurally disorder-free triangular antiferromagnet NaYbO2, Phys. Rev. B 100, 144432 (2019).
  * (45) Y. Shirata, H. Tanaka, T. Ono, A. Matsuo, K. Kindo, and H. Nakano, Quantum magnetization plateau in spin-1 triangular-lattice antiferromagnet, J. Phys. Soc. Jpn. 80, 093702 (2011).
  * (46) M. Saito, M. Watanabe, N. Kurita, A. Matsuo, K. Kindo, M. Avdeev, H. O. Jeschke, and H. Tanaka, Successive phase transitions and magnetization plateau in the spin-1 triangular-lattice antiferromagnet, Phys. Rev. B 100, 064417 (2019).
  * (47) N. Li, Q. Huang X. Y. Yue, W. J. Chu, Q. Chen, E. S. Choi, X. Zhao, H. D. Zhou, and X. F. Sun, Possible itinerant excitations and quantum spin state transitions in the effective spin-1/2 triangular-lattice antiferromagnet Na2BaCo(PO4)2, Nat. Commun. 11, 4216 (2020).
  * (48) K. Watanabe, H. Kawamura, H. Nakano, and T. Sakai, Quantum spin-liquid behavior in the spin-1/2 random Heisenberg antiferromagnet on the triangular lattice, J. Phys. Soc. Jpn. 83, 034714 (2014).
  * (49) X. Z. Liu, O. Prokhnenko, D. Yamamoto, M. Bartkowiak, N. Kurita, and H. Tanaka, Microscopic evidence of a quantum magnetization process in the $S\,{=}\,1/2$ triangular-lattice Heisenberg-like antiferromagnet Ba3CoSb2O9, Phys. Rev. B 100, 094436 (2019).
  * (50) N. A. Fortune and S. T. Hannahs, Top-loading small-sample calorimeters for measurements as a function of magnetic field angle, J. Phys.: Conf. Ser. 568, 032008 (2014).
  * (51) K. Adachi, N. Achiwa, and M. Mekata, Helical magnetic structure in CsCuCl3, J. Phys. Soc. Jpn. 49, 545 (1980).
  * (52) Y. Tazuke, H. Tanaka, K. Iio, and K. Nagata, Magnetic susceptibility study of CsCuCl3, J. Phys. Soc. Jpn. 50, 3919 (1981).
  * (53) H. Nojiri, Y. Tokunaga, and M. Motokawa, Magnetic phase transition of helical CsCuCl3 in high magnetic field, J. Phys. (Paris) 49, Suppl. C8, 1459 (1988).
  * (54) H. Tanaka, U. Schotte, and K. D. Schotte, ESR modes in CsCuCl3, J. Phys. Soc. Jpn. 61, 1344 (1992).
  * (55) M. Motokawa, M. Arai, H. Ohta, M. Mino, H. Tanaka, and K. Ubukata, Quantum effect in the high-field phase of CsCuCl3, Physica B 211, 199 (1995).
  * (56) N. Stüßer, U. Shotte, A. Hoser, M. Meschke, M. Meißner, and J. Wosnitza, A neutron diffraction study of the magnetic phases of CsCuCl3 for in-plane fields up to 17 T, J. Phys.: Condens. Matter 14, 5161 (2002).
  * (57) A. Sera, Y. Kousaka, J. Akimitsu, M. Sera, and K. Inoue, Pressure-induced quantum phase transitions in the $S=\frac{1}{2}$ triangular lattice antiferromagnet CsCuCl3, Phys. Rev. B 96, 014419 (2017).
  * (58) D. Yamamoto, T. Sakurai, R. Okuto, S. Okubo, H. Ohta, H. Tanaka, and Y. Uwatoko, Continuous control of classical-quantum crossover by external high pressure in the coupled chain compound CsCuCl3, Nat. Commun. 12, 4263 (2021).
  * (59) N. A. Fortune, Q. Huang, T. Hong, J. Ma, E. S. Choi, S. T. Hannahs, Z. Y. Zhao, X. F. Sun, Y. Takano, and H. D. Zhou, Evolution of magnetic field induced ordering in the layered quantum Heisenberg triangular-lattice antiferromagnet Ba3CoSb2O9, Phys. Rev. B 103, 184425 (2021).
  * (60) Although magnetization data for $H\,{\parallel}\,c$ has already been published in Ref. Liu2 , we show the data again because they are the basis for discussing the evolution of the critical fields with varying the field angle ${\theta}$.
  * (61) X. Z. Liu, O. Prokhnenko, M. Bartkowiak, and A. Gazizulina, D. Yamamoto, A. Matsuo and K. Kindo, K. Okada, N. Kurita, and H. Tanaka, Ground state of $S\,{=}\,1/2$ triangular lattice Heisenberg-like antiferromagnet Ba3CoSb2O9 in an out-of-plane magnetic field, Phys. Rev. B 105, 214433 (2022).


See pages 1-last of paper.pdf