# The magnetic, thermodynamic, and magnetotransport properties of CeGaGe and
PrGaGe single crystals

Daloo Ram  Department of Physics, Indian Institute of Technology, Kanpur
208016, India  Sudip Malick  Department of Physics, Indian Institute of
Technology, Kanpur 208016, India  Zakir Hossain<EMAIL_ADDRESS>Department
of Physics, Indian Institute of Technology, Kanpur 208016, India  Dariusz
Kaczorowski<EMAIL_ADDRESS>Institute of Low Temperature and
Structure Research, Polish Academy of Sciences,ulica Okólna 2, 50-422 Wroclaw,
Poland

###### Abstract

We investigate the physical properties of high-quality single crystals CeGaGe
and PrGaGe using magnetization, heat capacity, and magnetotransport
measurements. Gallium-indium binary flux was used to grow these single
crystals that crystallize in a body-centered tetragonal structure. Magnetic
susceptibility data reveal a magnetic phase transition around 6.0 and 19.4 K
in CeGaGe and PrGaGe, respectively, which is further confirmed by heat
capacity and electrical resistivity data. A number of additional anomalies
have been observed below the ordering temperature in the magnetic
susceptibility data, indicating a complex magnetic structure. The magnetic
measurements also reveal a strong magnetocrystalline anisotropy in both
compounds. Our detailed analysis of the crystalline electric field (CEF)
effect as observed in magnetic susceptibility and heat capacity data suggests
that J = 5/2 multiplet of CeGaGe splits into three doublets, while J = 4
degenerate ground state of PrGaGe splits into five singlets and two doublets.
The estimated energy levels from the CEF analysis are consistent with the
magnetic entropy.

##  I Introduction

The competing interactions resulting from the hybridization of 4f and
conduction electrons in the strongly correlated electron system yields
remarkable physical features such as the Kondo effect, heavy fermion behavior,
intermediate valency, crystalline electric field (CEF) effect, magnetic
ordering, superconductivity, and so on [1, 2, 3, 4, 5, 6, 7, 8]. Therefore,
rare-earth-based intermetallic compounds have always been appealing because
they provide a platform for investigating such exotic physical properties. The
introduction of strong correlation into a nontrivial topological system
results in more intriguing ground states. For instance, Ce3Bi4Pd3 is reported
to be a Weyl-Kondo heavy-fermion semimetal [9, 10], whereas Ce3Bi4Pt3 displays
a topological Kondo insulating state [10, 11]. On the other hand, the
coexistence of relativistic fermions and magnetism in these compounds
manifests anomalous transport [12, 13, 14, 15].

The ternary tetragonal compounds RAlX (R = La$-$Nd and Sm; X = Si and Ge) have
gained attention recently since this class of compounds displays a Weyl
semimetal (WSM) state [16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26].
Interestingly, RAlX crystallizes in two space groups: LaPtSi-type
noncentrosymmetric space group I41md (No. 109) and $\alpha$-ThSi2-type
centrosymmetric space group I41/amd (No. 141). LaAlGe crystallizes in a
noncentrosymmetric structure and exhibits a type-II WSM state as determined by
angle-resolved photoemission spectroscopy [17]. Moreover, magnetic compounds
are more intriguing because they provide a framework for investigating the
interaction between relativistic Weyl fermions and magnetism. For instance,
CeAlSi is a noncollinear ferromagnetic (FM) WSM that exhibits an anisotropic
anomalous Hall effect [23] and PrAlSi is a centrosymmetric ferromagnet
manifesting an anomalous Hall effect for magnetic field applied along $c$ axis
[25]. In addition, the substitution of Ge with Si in PrAlGe results in a
crossover from intrinsic to extrinsic anomalous Hall conductivity [27].
Further, this family of compounds also exhibits strong magnetocrystalline
anisotropy and the CEF effect [22, 23, 24, 25, 26, 28]. Moreover, a number of
recent reports reveal the presence of a WSM state exhibiting complex magnetic
structure and anomalous transport in CeAlSi [23], CeAlGe [29], PrAlSi [25],
NdAlSi [30], and NdAlGe [28, 31].

In this report, we present a thorough investigation of the physical properties
of CeGaGe and PrGaGe single crystals. CeGaGe is a FM Kondo lattice system with
an ordering temperature of 5.5 K as determined in polycrystalline samples [32,
33]. However, to the best of our knowledge there is no report on PrGaGe. Here,
we have grown single crystals of RGaGe (R = Ce and Pr) as well as its
nonmagnetic counterpart, LaGaGe, using the flux method. Powder x-ray
diffraction (XRD) patterns of crushed single crystals revealed that RGaGe (R =
La$-$Pr) crystallizes in a noncentrosymmetric structure with space group
I41md. Magnetic susceptibility data along with the heat capacity and
electrical resistivity data suggest a complex magnetic ordering and CEF effect
in CeGaGe and PrGaGe. Further, both compounds exhibits negative
magnetoresistance near the magnetic ordering temperature.

##  II Experimental details

High-quality single crystals of RGaGe (R = La$-$Pr) were grown using gallium-
indium binary flux. Constituent elements La (99.9%, Alfa Aesar), Ce (99.9%,
Alfa Aesar), Pr (99.9%, Alfa Aesar), Ga (99.9999%, Alfa Aesar), Ge (99.999%,
Alfa Aesar), and In (99.99%, Alfa Aesar) are taken in the molar ratio
R:Ga:Ge:In = 1:2:1:8. The elements were mixed up properly and placed in an
alumina crucible. Next, the crucible was sealed inside quartz tube with
partial argon pressure. The sealed quartz tube was put into a muffle furnace
and heated to 1050 ∘C for 24 h. Then the furnace was slowly cooled down to 500
∘C at a rate of 3 ∘C/h. At this point the excess flux was removed by
centrifuging. Platelike shiny single crystals with a typical dimension of 3
$\times$ 3 $\times$ 0.4 mm3 were obtained as shown in lower insets of bottom
panel of Fig 1.

The structural characterization of obtained crystals was carried out using XRD
method in a PANalytical X’Pert PRO diffractometer with Cu Kα1 radiation. The
chemical compositions of the crystals were checked by energy-dispersive
spectroscopy (EDS) in a JEOL JSM-6010LA electron microscope. Electrical
resistivity and magnetotransport measurements were performed in a Quantum
Design physical property measurement system (PPMS) utilizing the usual four-
probe method. Heat capacity measurements were performed using the relaxation
method in the heat capacity option of PPMS. The magnetic properties were
measured using a Quantum Design magnetic property measurement system.

Figure 1: Powder XRD patterns along with Rietveld refinement of crushed single
crystals (a) LaGaGe, (b) CeGaGe, and (c) PrGaGe recorded at room temperature.
The solid black line and red circles represent the calculated pattern and
experimental data. The blue line shows difference between experimental and
calculated intensities. The olive bars mark the Bragg positions. Inset of (b)
displays the crystal structure of RGaGe. Single crystal XRD patterns are
presented (d) LaGaGe, (e) CeGaGe, and (f) PrGaGe. Insets of (d), (e), and (f)
show optical images of single crystals (lower) and the rocking curves (upper).

##  III Results and discussion

Table 1: The obtained lattice parameters of RGaGe from powder XRD data recorded at room temperature. Lattice parameters | LaGaGe | CeGaGe | PrGaGe
---|---|---|---
$a$ (Å) | 4.3542(2) | 4.2911(5) | 4.2599(4)
$b$ (Å) | 4.3542(2) | 4.2911(5) | 4.2599(4)
$c$ (Å) | 14.581(1) | 14.576(2) | 14.521(2)
$V$ (Å3) | 276.43(3) | 268.39(6) | 263.51(5)

###  III.1 Crystal structure

Powder XRD patterns of crushed crystals, recorded at room temperature, are
shown in Figs. 1(a)$-$1(c). The analysis of the powder XRD patterns using the
Rietveld refinement method suggest all three RGaGe (R = La$-$Pr) compounds
crystallize in a body-centered tetragonal structure with space group I41md
(No. 109) [34]. The estimated lattice parameters as presented in Table I agree
well with the previous report [32]. A schematic diagram of the crystal
structure is displayed in the inset of Fig. 1(b). The structure consists of
four formula unit per unit cell, in which all three atoms (R, Ga, and Ge) have
4a Wyckoff site. There are stacking of R, Ga, and Ge layers along (001)
direction. This structure has two vertical mirror planes ($\sigma_{v}$) but
lacks a horizontal mirror plane ($\sigma_{h}$), resulting in inversion
symmetry breaking. The observed single crystal XRD pattern of RGaGe can be
indexed by (00l) Miller indices, as depicted in Figs. 1(d)$-$1(f), indicating
that the crystallographic $c$ axis is perpendicular to the flat plane of
single crystals. Moreover, the peaks are extremely sharp, suggesting the high
quality of the single crystals. Further, the full width at half maximum (FWHM)
of the rocking curve [see upper insets of Figs. 1(d)$-$1(f)] for the peak
(0012) is about 0.04∘, which is very small, confirming that all single
crystals are of excellent quality. EDS data collected from multiple points and
areas on the surface of crystals reveal expected stoichiometry.

###  III.2 Magnetic properties

Figure 2: The temperature-dependent magnetic susceptibility ($\chi$) measured
at 0.1 T along (a) $H\parallel c$ and (b) $H\parallel ab$ of CeGaGe. The open
and filled circles represent magnetic susceptibility measured in ZFC and FC
configurations, respectively. The $\chi$(T) of PrGaGe for ZFC and FC
configurations was measured at 0.1 T for (d) $H\parallel c$ and (e)
$H\parallel ab$, respectively. The insets (a) and (d) display the temperature-
dependent d$\chi$/dT for FC configuration for CeGaGe and PrGaGe, respectively.
The insets of (b) and (e) depict the inverse magnetic susceptibilities of
CeGaGe and PrGaGe at 0.1 T, respectively, as a function of temperature. The
orange lines represent Curie-Weiss fitting. The temperature-dependent inverse
magnetic susceptibility of (c) CeGaGe and (f) PrGaGe at field 0.1 T. The black
solid lines in (c) and (f) represent the calculated inverse susceptibility
using CEF model.

The temperature-dependent magnetic susceptibilities $\chi_{c}$ ($H\parallel
c$) and $\chi_{ab}$ ($H\parallel ab$) for zero-field cooled (ZFC) and field
cooled (FC) configurations at 0.1 T for both compounds are shown in Figs.
2(a), 2(b), 2(d), and 2(e). The sudden upturn in $\chi_{c}$ around $T_{C}$ =
6.0 K in CeGaGe and $T_{C}$ = 19.4 K in PrGaGe suggest a FM phase transition.
However, $\chi_{ab}$ slowly increases with decreasing temperature down to
$T_{C}$ in both compounds. Interestingly, the ZFC and FC data exhibit a
bifurcation below $T_{C}$ along both crystallographic directions, more
prominent along $H\parallel ab$. Moreover, $\chi_{ab}$ for ZFC drops below
$T_{C}$ in an AFM fashion. The value of $\chi_{c}/\chi_{ab}$ is about 10 above
TC and reaches a value of $\sim$ 105 at 2 K in both compounds, which suggests
strong magnetocrystalline anisotropy in these compounds. A similar temperature
dependence with strong magnetic anisotropy was observed in same family of
compounds RAlX (R = Ce$-$Nd; X = Ge and Si) [23, 24, 25, 26]. In the
magnetically ordered state, $\chi$(T) shows an anomaly around 3.7 K in CeGaGe,
and two anomalies around 11.0 and 16.5 K in PrGaGe, as is evident from the
$d\chi/dT$ data, presented in the insets of Figs. 2(a) and 2(b). The observed
anomalies may be associated with the transition from incommensurate to
commensurate order, which has recently been observed in WSM NdAlSi [30, 26]
and NdAlGe [28, 31]. Therefore, a comprehensive investigation utilizing
neutron diffraction is required to reveal the complex magnetic structure of
CeGaGe and PrGaGe. The inverse magnetic susceptibility data are plotted as a
function of temperature and fitted well above $T_{C}$ with the modified Curie-
Weiss law, $\chi\left(T\right)=\chi_{0}+C/(T-\Theta_{P})$, where C is the
Curie constant, $\Theta_{P}$ is the paramagnetic Curie temperature, and
$\chi_{0}$ is temperature-independent magnetic susceptibility. The insets of
Figs. 2(b) and 2(e) illustrate the fitting of Curie-Weiss law for CeGaGe and
PrGaGe, respectively. The obtained fitting parameters are listed in Table II.
The negative values of $\chi_{0}$ may appear due to the diamagnetic
contribution of sample holder. The positive $\Theta_{P}$ along the c axis
indicates the dominant FM exchange interactions whereas negative $\Theta_{P}$
along the ab plane suggests an AFM coupling. Similar values of $\Theta_{P}$
were also observed in isostructural compounds such as CeAlGe [22], CeAlSi
[23], and PrAlGe [24, 21]. The calculated effective moments $\mu_{eff}$ of
CeGaGe and PrGaGe are very close to Ce3+ (2.54 $\mu_{B}$) and Pr3+ (3.58
$\mu_{B}$) ions, respectively. Below 100 K, the inverse susceptibility
deviates from the Curie-Weiss law and exhibits a hump around 50 K, as depicted
in the insets of Figs. 2(b) and 2(e). Such a hump appears due to the CEF
effect as observed in numerous rare-earth compounds such as CeAgAs2 [4], PrSi
[35], and Pr2Re3Si5 [36].

Table 2: The estimated value of $\chi_{0}$, $\mu_{eff}$, and $\Theta_{P}$ from modified Curie-Weiss fit of CeGaGe and PrGaGe. | CeGaGe | PrGaGe
---|---|---
|  H $\parallel$ c |  H $\parallel$ ab |  H $\parallel$ c |  H $\parallel$ ab
$\chi_{0}$ (10-4emu/mol) | -3.97 | -0.16 | -1.23 | -3.71
$\mu_{eff}$ ($\mu_{B}$) | 2.71 | 2.59 | 3.70 | 3.78
$\Theta_{P}$ (K) | 28.7 | -41.9 | 32.3 | -21.9
Table 3: CEF parameters, energy levels and the corresponding wave functions
for CeGaGe

CEF parameters
---
$B_{2}^{0}$ (K) |  $B_{4}^{0}$ (K) |  $B_{4}^{4}$ (K) |  $B_{6}^{0}$ (K) |  $B_{6}^{4}$ (K)
-6.40 $\pm$ 0.20 | -0.418 $\pm$ 0.035 | 3.32 $\pm$ 0.270 | 0 | 0
$\lambda_{z}$ = 10.0 $\pm$ 1.7 (mol/emu) |  $\lambda_{x,y}$ = 18.7 $\pm$ 1.8 (mol/emu)

Energy levels and wave functions
---
E (K) |  $\lvert+5/2\rangle$ |  $\lvert+3/2\rangle$ |  $\lvert+1/2\rangle$ |  $\lvert-1/2\rangle$ |  $\lvert-3/2\rangle$ |  $\lvert-5/2\rangle$
251.4 $\pm$ 12.6 | -0.38 | 0 | 0 | 0 | -0.92 | 0
251.4 $\pm$ 12.6 | 0 | -0.92 | 0 | 0 | 0 | -0.38
126.9 $\pm$ 7.4 | 0 | 0 | 1 | 0 | 0 | 0
126.9 $\pm$ 7.4 | 0 | 0 | 0 | 1 | 0 | 0
0 | 0.92 | 0 | 0 | 0 | -0.38 | 0
0 | 0 | -0.38 | 0 | 0 | 0 | 0.92

Next, we have analyzed the magnetic susceptibility data using CEF schemes. For
a tetragonal system, the (2J+1) levels of J = 5/2 (Ce3+) split into three
doublets and J = 4 (Pr3+) multiplet splits into two doublets along with five
singlets [37]. The CEF Hamiltonian corresponding to tetragonal site symmetry
and $C_{4v}$ point symmetry is as follows

$\mathcal{H}_{CEF}=B_{2}^{0}O_{2}^{0}+B_{4}^{0}O_{4}^{0}+B_{4}^{4}O_{4}^{4}+B_{6}^{0}O_{6}^{0}+B_{6}^{4}O_{6}^{4}$
(1)

where $B_{p}^{q}$ and $O_{p}^{q}$ are the CEF parameters and the Stevens
operators, respectively [38, 39]. The magnetic susceptibility based on the CEF
scheme, $\chi_{CEF}^{i}$ ($i$ = $x$, $y$, $z$), is given by

$\begin{split}\chi_{CEF}^{i}=&\frac{N_{A}(g_{J}\mu_{B})^{2}}{Z}\bigg{[}\sum_{n}\beta|\langle
n\rvert J_{i}\lvert n\rangle|^{2}e^{-\beta E_{n}}\\\ &+\sum_{n\neq m}|\langle
m\rvert J_{i}\lvert n\rangle|^{2}\frac{e^{-\beta E_{n}}-e^{-\beta
E_{m}}}{E_{m}-E_{n}}\bigg{]},\end{split}$ (2)

where $g_{J}$ is the Landé factor, $Z=\sum_{n}e^{-\beta E_{n}}$, and $\beta$ =
1/$k_{B}T$. $\lvert n\rangle$ is the $n$th eigenfunction and $E_{n}$ is the
corresponding eigenvalue. $J_{i}$ is the component of angular momentum [40].
To analyze the CEF, we have plotted inverse susceptibility as a function of
temperature, subtracting temperature-independent term ($\chi_{0}$) as shown in
Figs. 2(c) and 2(f). The magnetic susceptibility ($\chi_{i}$ -$\chi_{0}^{i}$),
including the molecular field contribution $\lambda_{i}$ and the CEF
contribution, can be expressed as

$(\chi_{i}-\chi_{0}^{i})^{-1}=(\chi_{CEF}^{i})^{-1}-\lambda_{i}$ (3)

The calculated temperature-dependent magnetic susceptibilities along different
crystallographic directions using the above equation, presented in Figs. 2(c)
and 2(f), agree well with the experimental data. The obtained values of CEF
parameters and the energy levels of CeGaGe and PrGaGe are given in Tables III
and IV, respectively. The estimated parameters and energy levels are
comparable to those of other Ce- and Pr-based compounds [41, 42]. We have also
recalculated the CEF parameter $B_{2}^{0}$ directly from the paramagnetic
Curie-Weiss temperatures to check the consistency of our analysis using
following expression [43],

$\Theta_{P}^{c}-\Theta_{P}^{ab}=\frac{3}{10}B_{2}^{0}(2J-1)(2J+3)$ (4)

The calculated value of $B_{2}^{0}$ is -7.3 and -2.4 K for CeGaGe and PrGaGe,
respectively, which are in good agreement with estimated value from the CEF
model fit.

Table 4: CEF parameters, energy levels and the corresponding wave functions
for PrGaGe

CEF parameters
---
$B_{2}^{0}$ (K) |  $B_{4}^{0}$ (K) |  $B_{4}^{4}$ (K) |  $B_{6}^{0}$ (K) |  $B_{6}^{4}$ (K) |  $\lambda_{i}$ (mol/emu)
-2.752 $\pm$ 0.338 | 0.050 $\pm$ 0.012 | 0.106 $\pm$ 0.019 | 0 $\pm$ 0.0002 | 0.006 $\pm$ 0.0018 |  $\lambda_{z}$ = 8.9 $\pm$ 2.1; $\lambda_{x,y}$ = 7.5 $\pm$ 1.9

Energy levels and wave functions
---
E (K) |  $\lvert+4\rangle$ |  $\lvert+3\rangle$ |  $\lvert+2\rangle$ |  $\lvert+1\rangle$ |  $\lvert 0\rangle$ |  $\lvert-1\rangle$ |  $\lvert-2\rangle$ |  $\lvert-3\rangle$ |  $\lvert-4\rangle$
202.8 $\pm$ 29.2 | -0.177 | 0 | 0 | 0 | -0.968 | 0 | 0 | 0 | -0.177
158.9 $\pm$ 23.0 | 0 | 0 | 0 | 0.996 | 0 | 0 | 0 | 0.089 | 0
158.9 $\pm$ 23.0 | 0 | 0.089 | 0 | 0 | 0 | 0.996 | 0 | 0 | 0
77.8 $\pm$ 10.5 | 0 | 0 | -1/$\sqrt{2}$ | 0 | 0 | 0 | -1/$\sqrt{2}$ | 0 | 0
68.9 $\pm$ 10.3 | 0 | 0 | -1/$\sqrt{2}$ | 0 | 0 | 0 | 1/$\sqrt{2}$ | 0 | 0
48.9 $\pm$ 26.1 | -1/$\sqrt{2}$ | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1/$\sqrt{2}$
38.6 $\pm$ 26.2 | 0.685 | 0 | 0 | 0 | -0.250 | 0 | 0 | 0 | 0.685
0 | 0 | 0 | 0 | 0.089 | 0 | 0 | 0 | -0.996 | 0
0 | 0 | -0.996 | 0 | 0 | 0 | 0.089 | 0 | 0 | 0

Furthermore, we measured the isothermal magnetizations $M(H)$ of CeGaGe and
PrGaGe, as shown in Figs. 3(a) and 3(b), respectively. Magnetization along
$H\parallel c$ increases sharply and saturates at 0.15 and 0.35 T for CeGaGe
and PrGaGe, respectively. The saturated value of magnetization is 1.55
$\mu_{B}$ for CeGaGe and 2.95 $\mu_{B}$ for PrGaGe, which is less than the
value of free Ce3+ ($gJ\mu_{B}$ = 2.16 $\mu_{B}$) and Pr3+ ($gJ\mu_{B}$ = 3.20
$\mu_{B}$) ions. Such low value of saturation magnetization may be due to the
presence of CEF effect [23, 24, 21]. Whereas, magnetization along $H\parallel
ab$ is substantially lower and does not saturate even at 7 T. Such a variation
in magnetization in different directions indicates strong magnetocrystalline
anisotropy. Moreover, $M(H)$ data also indicate that the c axis is the
magnetic easy axis as magnetization saturates easily at low applied field. A
small hysteresis is observed in both measured directions in the magnetization
data, supporting the FM nature in CeGaGe and PrGaGe. We have also determined
the isothermal magnetization considering the CEF scheme using the expression

$M_{i}=\frac{g_{J}\mu_{B}}{Z}\sum_{n}|\langle n\rvert J_{i}\lvert
n\rangle|e^{-\beta E_{n}}\\\ $ (5)

The eigenvalue and associated eigenfunction of the above expression can be
calculated by diagonalizing the given total Hamiltonian

$\mathcal{H}=\mathcal{H}_{CEF}-g_{j}\mu_{B}J_{i}(H+\lambda_{i}M_{i}),$ (6)

where the first term of the Hamiltonian is defined in Eq. (1). The second and
third terms of the above Hamiltonian are the Zeeman and the molecular field
term, respectively. The calculated magnetizations of CeGaGe and PrGaGe are
represented in Figs. 3(a) and 3(b) by solid black lines obtained by solving
Eqs. (5) and (6). The above calculation roughly reproduces the magnetization
as we neglected the exchange interaction in the Hamiltonian. However, this
calculation perfectly recognizes the easy and hard axes of magnetization.
Furthermore, we have estimated the saturation magnetization ($M^{sat}$) using
the Zeeman term [40, 44]

$\mathcal{H}_{x,z}=\textit{g}_{J}\mu_{B}\langle\Gamma_{mix,1}\rvert\textit{J}_{x,z}\lvert\Gamma_{mix,1}\rangle\textit{B}_{z}$
(7)

where $\Gamma_{mix,1}$ is the ground state wave function. The estimated and
observed value of saturation magnetization is presented in Table V. A slight
discrepancy between the computed and observed numbers is due to the use of the
simplified CEF model [36].

Table 5: The calculated saturation magnetizations using Eq. (7) and observed value at 1.7 K and 7 T. | CeGaGe | PrGaGe
---|---|---
| Estimated | Observed |  Estimated | Observed
$M_{\parallel c}^{sat}(\mu_{B})$ | 1.64 | 1.63 | 2.38 | 2.96
$M_{\perp c}^{sat}(\mu_{B})$ | 0.68 | 0.72 | 0 | 0.74
Figure 3: The isothermal magnetization of (a) CeGaGe and (b) PrGaGe was
measured at 1.7 K as a function of the applied magnetic fields. Insets of (a)
and (b) show a zoomed view of low fields magnetization for the $H\parallel c$
of CeGaGe and PrGaGe, respectively. The black lines are the result of the CEF
calculations.

###  III.3 Specific heat

The temperature-dependent heat capacity (Cp) of single crystalline RGaGe (R =
La$-$Pr) at a constant pressure is presented in Figs. 4(a)$-$4(c),
respectively. A sharp anomaly at 6.0 and 19.4 K for CeGaGe and PrGaGe,
respectively in the Cp data confirms the bulk magnetic ordering as observed in
magnetic susceptibility data. The heat capacity reaches an expected value of
Dulong-Petit limits $C_{p}$ = 3nR = 74.83 J/mol K for all three compounds,
where n is the number of atoms in formula unit and R is the universal gas
constant. The Cp(T) of the nonmagnetic LaGaGe can be expressed well by
considering the Debye ($C_{D}$) and Einstein ($C_{E}$) modes of heat capacity
as presented by the formula

$C_{p}(T)=\gamma T+mC_{D}(T)+(1-m)C_{E}(T)$ (8)

where $m$ is the weight factor and $\gamma$ is the Sommerfeld coefficient.
$C_{D}$(T) and $C_{E}$(T) are defined as

$C_{D}(T)=9nR\left(\frac{T}{\Theta_{D}}\right)^{3}\int_{0}^{\Theta_{D}/T}\frac{x^{4}e^{x}}{(e^{x}-1)^{2}}dx,$
(9)

and

$C_{E}(T)=3nR\left(\frac{\Theta_{E}}{T}\right)^{2}\frac{e^{\Theta_{E}/T}}{(e^{\Theta_{E}/T}-1)^{2}},$
(10)

where $\Theta_{D}$ and $\Theta_{E}$ are the Debye and Einstein temperatures,
respectively [45, 46]. The obtained fitting parameters are $\Theta_{D}$ = 295
K, $\Theta_{E}$ = 100 K and $m$ = 0.76. We have estimated the $\gamma$ from Cp
data of LaGaGe by applying the formula $C_{p}/T=\gamma+\beta T^{2}$ in low-
temperature regime (2 K $\leq T\leq$ 7 K). The linear fit of $C_{p}/T$ vs
$T^{2}$ curve as shown in the inset of Fig. 4(a) gives $\gamma$ = 2(2) mJ/mol
K2 and $\beta$ = 0.44(1) mJ/mol K4. The estimated value $\gamma$ is comparable
to the previous report [32]. Furthermore, to estimate $\gamma$ for CeGaGe and
PrGaGe, we have fitted Cp data in the magnetically ordered state as presented
in the inset of Figs. 4(b) and 4(c) using the following formula

$C_{p}(T)=\gamma T+\beta T^{3}+\delta T^{3/2}e^{-\Delta/T}.$ (11)

The last term of the above equation is the contribution to the Cp due to the
spin-wave for a ferromagnet with an energy gap $\Delta$ in the magnon spectrum
[47]. To perform the above fitting we have used same $\beta$ obtained from
LaGaGe, which reduces the fitting parameter. So-obtained fitting parameters
are listed in Table VI. The estimated value of $\gamma$ for CeGaGe and PrGaGe
are comparable to other FM compounds such as CeIr2B2 [48], Pr2Rh2Ga [49] and
Ce11Pd4In9 [50]. However, it is less than typical Kondo/heavy fermion
compounds like CeIn3 [51] and PrV2Al20 [52].

Figure 4: The temperature-dependent heat capacity of single crystals (a)
LaGaGe, (b) CeGaGe, and (c) PrGaGe. The red solid line in (a) represents the
fitting of the Debye plus Einstein model. The inset of (a) shows linear
fitting to $C_{p}/T$ vs $T^{2}$ at low temperatures. The purple solid line in
the insets of (b) and (c) is the fitting of Eq. 11 to the Cp data below TC.
The temperature dependence of magnetic contribution to Cp of (d) CeGaGe and
(e) PrGaGe. The solid black line shows the fitting of $C_{m}(T)$ data using
different levels CEF scheme. (f) The magnetic entropy of CeGaGe and PrGaGe
calculated as a function of temperature.

The magnetic contribution to the Cp for CeGaGe and PrGaGe was calculated by
subtracting the Cp of the nonmagnetic reference LaGaGe, assuming that the
phonon contribution in CeGaGe and PrGaGe is approximately equal to that of
LaGaGe. The temperature dependence of magnetic heat capacity [$C_{m}(T)$] is
presented in Figs. 4(d) and 4(e). In addition to the prominent peak at low
temperatures due to the magnetic phase transition, a broad hump around 60 and
30 K for CeGaGe and PrGaGe is seen in $C_{m}(T)$, respectively. The broad peak
can be attributed to a Schottky-type anomaly, which arises from the CEF-
induced splitting of energy levels Ce3+ and Pr3+. The generalized formula of
Schottky contribution to the heat capacity for multi-level CEF scheme is given
by

$\begin{split}C_{Sch}(T)=&\left(\dfrac{R}{T^{2}}\right)\bigg{[}\sum_{i}g_{i}e^{-\Delta_{i}/T}\sum_{i}g_{i}{\Delta_{i}}^{2}e^{-\Delta_{i}/T}\\\
&-\left(\sum_{i}g_{i}\Delta_{i}e^{-\Delta_{i}/T}\right)^{2}\bigg{]}\left(\sum_{i}g_{i}e^{-\Delta_{i}/T}\right)^{-2},\end{split}$
(12)

where $g_{i}$ is the degeneracy of the ith state with $\Delta_{i}$ energy gap
splitting [47]. Here, we have used the same CEF parameters obtained from CEF
analysis of magnetic susceptibility data to reproduce the Schottky anomaly
observed in $C_{m}$(T). Interestingly, the calculated Schottky anomaly matches
well with the experimental data for both compounds, as presented in Figs. 4(d)
and 4(e). This justifies the validity of the CEF parameters and the energy
level splittings of both compounds.

Figure 5: Temperature-dependent electrical resistivity of (a) CeGaGe and (b) PrGaGe measured in between 2$-$300 K. The insets show the fitting of Eq. 13 in the magnetically ordered state. (c) The temperature dependence of electrical resistivity of CeGaGe measured under external magnetic fields. The MR of (d) CeGaGe and (e) PrGaGe as a function of the magnetic field for different temperatures. Table 6: Obtained fitting parameters of low temperature Cp data of CeGaGe and PrGaGe using Eq. (11). |  $\gamma$ (mJ/mol K2) | $\delta~{}(J/molK^{5/2})$ |  $\Delta$ (K)
---|---|---|---
CeGaGe | 13 | 1.54 | 3.90
PrGaGe | 35 | 0.58 | 14.42

Next, we have calculated magnetic entropy using the expression
$S_{m}(T)=\int\frac{C_{m}}{T}dT$. The magnetic entropy as a function of
temperature for CeGaGe and PrGaGe is presented in Fig. 4(f). According to CEF
analysis, six energy levels in CeGaGe split into three doublets, resulting in
three plateaus in $S_{m}$(T) near Rln2, Rln4, and Rln6. In case of PrGaGe, the
$S_{m}$(T) reaches a value of 7.9 J/mol K at $T_{C}$, which lies between a
value of magnetic entropy of ground state Rln2 and Rln4. Therefore, a strong
thermal population of the excited levels is indicated, and the first excited
level is probably slightly higher in energy than $T_{C}$. The total magnetic
entropy Rln6 for CeGaGe and Rln9 for PrGaGe is released at 252 and 210 K,
respectively, indicating the overall splitting of CEF levels within this
energy window. The observed $S_{m}$(T) is consistent with our CEF analysis.

###  III.4 Magnetotransport

The electrical resistivity $\rho(T)$ measured within the ab plane in CaGaGe
and PrGaGe single crystals in the temperature range of 2$-$300 K is shown in
Figs. 5(a) and 5(b), respectively. The $\rho(T)$ of CeGaGe and PrGaGe show
metallic behavior down to 2 K with a sharp anomaly around 6.0 and 19.4 K,
respectively. The low-temperature anomaly is associated with the magnetic
phase transition where resistivity drops rapidly due to reduction of spin
scattering as systems order magnetically. The residual resistivity ratio (RRR)
of CeGaGe and PrGaGe is $\sim$ 2.1 and $\sim$ 1.5, respectively, which is
small compared to typical metals. Such value of RRR may appear due to site
disorder between Ga and Ge. Similar RRR has also been observed in RAlX (R =
La$-$Nd and X = Ge, Si) family for site disorder between Al and Ge/Si [22, 23,
24]. On the other hand, $\rho(T)$ of CeGaGe exhibits a broad hump from 220 to
50 K, may arise from the CEF splitting, which is also evident from
thermodynamic measurement. Similarly a concave curvature is observed in the
$\rho(T)$ data of PrGaGe. Such behavior of $\rho(T)$ is commonly seen in rare-
earth compounds like CeAuGa3 [53], Ce2NiSi3 [7] and Pr7Ru3 [54]. For both
compounds, the $\rho(T)$ in the magnetically order state can be described
using the following expression

$\rho(T)=\rho_{0}+AT^{2}\text{exp}\left(-\frac{\Delta}{T}\right),$ (13)

where $\rho_{0}$ is the residual resistivity, and the second term is the
contribution to resistivity due to FM magnon. $A$ is related to the strength
of electron-magnon scattering, and $\Delta$ is the energy gap in the magnon
spectrum [55]. The so-obtained value of fitting parameters from Eq. 13 are
listed in Table VII. The estimated value of $\Delta$ is very close to that
obtained from the heat capacity data, confirming consistency of our analysis.

Table 7: The estimated fitting parameters $\rho_{0}$, $A$ and $\Delta$ from the resistivity data using Eq. (13). |  $\rho_{0}$ ($\mu$$\Omega$ cm) |  $A$ ($\mu$$\Omega$ cm/K2) |  $\Delta$ (K)
---|---|---|---
CeGaGe | 81.3 | 0.45 | 3.33
PrGaGe | 134.0 | 0.05 | 13.36

The $\rho(T)$ of CeGaGe and PrGaGe under various external magnetic fields is
presented in Figs. 5(b) and 5(c). The low temperature anomaly in $\rho$(T) due
to magnetic ordering for both compounds is getting smeared out by the applied
field, and the magnitude of electrical resistivity is reduced near $T_{C}$.
The field dependence of MR of CeGaGe and PrGaGe at various temperatures for
$H\parallel c$ are represented in Figs. 5(d) and 5(e), respectively. The MR is
defined as [$\rho(H)-\rho(0)]/\rho(0)$, where $\rho(H)$ and $\rho(0)$ are the
resistivities in the presence and absence of a magnetic field, respectively.
The MR of both compounds becomes negative close to the magnetic ordering
temperature and remains negative throughout the paramagnetic region. The
maximum value of MR reaches about -6.5 % and -3.8 % for CeGaGe and PrGaGe,
respectively. The negative MR for both samples near $T_{C}$ results from the
suppression of spin scattering by the applied magnetic field, which is the
typical MR behavior for an FM [56, 57]. Interestingly, at low temperatures (T
$<T_{C}$), the MR for both samples in a weak field regime is negative, but as
the field intensity increases, the MR rises and becomes positive. The magnetic
moments become polarized at low applied fields ($\sim$ 0.4 T), and MR becomes
negative due to suppression of spin disorder; however, as the field strength
increases, the Lorentz force becomes dominant, and MR starts to increase, as
observed in PrAlGe [24] and CeIr2B2 [48].

##  IV Summary

We have studied the magnetic, thermodynamic, and magnetotransport properties
of RGaGe (R = Ce and Pr) single crystals synthesized using gallium-indium
flux. The powder XRD patterns indicated that these crystals crystallize in a
body-centered tetragonal structure with space group I41md (No. 109). The
temperature-dependent magnetic susceptibility and resistivity measurements
show a magnetic phase transition in CeGaGe and PrGaGe at $T_{C}$ = 6.0 and
19.4 K, respectively. Further, heat capacity data shows a sharp peak at the
magnetic transition, which confirms the bulk nature of the magnetic ordering.
The temperature and field-dependent magnetization data reveal strong magnetic
anisotropy and the presence of CEF in both compounds. The CEF analysis for
inverse magnetic susceptibility and heat capacity data of CeGaGe and PrGaGe
reveal that J = 5/2 degenerate ground state of Ce3+ ion splits into three
doublets, whereas J = 4 multiplet of Pr3+ ion splits into two doublets and
five singlets. The field-dependent MR for both compounds is positive at low
temperatures and high fields, it switches to negative near the magnetic
transition temperature and remains negative in the paramagnetic region.

##  V Acknowledgment

We acknowledge IIT Kanpur and the Department of Science and Technology, India,
[Order No. DST/NM/TUE/QM-06/2019 (G)] for financial support. We thank Suman
Sanki for valuable discussion. D.K. acknowledges financial support from the
National Science Centre (Poland) under Research Grant No. 2021/41/B/ST3/01141.

## References

  * Wang  _et al._ [2019] L. Wang, C. Wang, Z. Liu, J. Cheng, S. Miao, Y. Song, Y. Shi, and Y.-f. Yang, Phys. Rev. B 100, 085122 (2019).
  * Das  _et al._ [2018] D. Das, D. Gnida, Ł. Bochenek, A. Rudenko, M. Daszkiewicz, and D. Kaczorowski, Scientific Reports 8, 16703 (2018).
  * Steglich  _et al._ [1979] F. Steglich, J. Aarts, C. D. Bredl, W. Lieke, D. Meschede, W. Franz, and H. Schäfer, Phys. Rev. Lett. 43, 1892 (1979).
  * Mondal  _et al._ [2018] R. Mondal, R. Bapat, S. K. Dhar, and A. Thamizhavel, Phys. Rev. B 98, 115160 (2018).
  * Anand  _et al._ [2007] V. K. Anand, Z. Hossain, G. Behr, G. Chen, M. Nicklas, and C. Geibel, Journal of Physics: Condensed Matter 19, 506205 (2007).
  * Mathur  _et al._ [1998] N. D. Mathur, F. M. Grosche, S. R. Julian, I. R. Walker, D. M. Freye, R. K. W. Haselwimmer, and G. G. Lonzarich, Nature 394, 39 (1998).
  * Malick  _et al._ [2019] S. Malick, D. Das, and Z. Hossain, Journal of Magnetism and Magnetic Materials 482, 108 (2019).
  * Opletal  _et al._ [2022] P. Opletal, E. Duverger–Nédellec, K. Miliyanchuk, S. Malick, Z. Hossain, and J. Custers, Journal of Alloys and Compounds 927, 166941 (2022).
  * Lai  _et al._ [2018] H.-H. Lai, S. E. Grefe, S. Paschen, and Q. Si, Proceedings of the National Academy of Sciences 115, 93 (2018).
  * Dzsaber  _et al._ [2017] S. Dzsaber, L. Prochaska, A. Sidorenko, G. Eguchi, R. Svagera, M. Waas, A. Prokofiev, Q. Si, and S. Paschen, Phys. Rev. Lett. 118, 246601 (2017).
  * Campbell  _et al._ [2019] D. J. Campbell, Z. E. Brubaker, C. Roncaioli, P. Saraf, Y. Xiao, P. Chow, C. Kenney-Benson, D. Popov, R. J. Zieve, J. R. Jeffries, and J. Paglione, Phys. Rev. B 100, 235133 (2019).
  * Hirschberger  _et al._ [2016] M. Hirschberger, S. Kushwaha, Z. Wang, Q. Gibson, S. Liang, C. A. Belvin, B. A. Bernevig, R. J. Cava, and N. P. Ong, Nature materials 15, 1161 (2016).
  * Singha  _et al._ [2019] R. Singha, S. Roy, A. Pariari, B. Satpati, and P. Mandal, Phys. Rev. B 99, 035110 (2019).
  * Wolgast  _et al._ [2013] S. Wolgast, C. Kurdak, K. Sun, J. W. Allen, D.-J. Kim, and Z. Fisk, Phys. Rev. B 88, 180405(R) (2013).
  * Cheng  _et al._ [2021] E. Cheng, W. Xia, X. Shi, H. Fang, C. Wang, C. Xi, S. Xu, D. C. Peets, L. Wang, H. Su, L. Pi, W. Ren, X. Wang, N. Yu, Y. Chen, W. Zhao, Z. Liu, Y. Guo, and S. Li, Nature Communications 12, 6970 (2021).
  * Su  _et al._ [2021] H. Su, X. Shi, J. Yuan, Y. Wan, E. Cheng, C. Xi, L. Pi, X. Wang, Z. Zou, N. Yu, W. Zhao, S. Li, and Y. Guo, Phys. Rev. B 103, 165128 (2021).
  * Xu  _et al._ [2017] S.-Y. Xu, N. Alidoust, G. Chang, H. Lu, B. Singh, I. Belopolski, D. S. Sanchez, X. Zhang, G. Bian, H. Zheng, M.-A. Husanu, Y. Bian, S.-M. Huang, C.-H. Hsu, T.-R. Chang, H.-T. Jeng, A. Bansil, T. Neupert, V. N. Strocov, H. Lin, S. Jia, and M. Z. Hasan, Science Advances 3, e1603266 (2017).
  * Cao  _et al._ [2022] W. Cao, Y. Su, Q. Wang, C. Pei, L. Gao, Y. Zhao, C. Li, N. Yu, J. Wang, Z. Liu, Y. Chen, G. Li, J. Li, and Y. Qi, Chinese Physics Letters 39, 047501 (2022).
  * Zhao  _et al._ [2022] J. Zhao, W. Liu, A. Rahman, F. Meng, L. Ling, C. Xi, W. Tong, Y. Bai, Z. Tian, Y. Zhong, Y. Hu, L. Pi, L. Zhang, and Y. Zhang, New Journal of Physics 24, 013010 (2022).
  * Puphal  _et al._ [2020] P. Puphal, V. Pomjakushin, N. Kanazawa, V. Ukleev, D. J. Gawryluk, J. Ma, M. Naamneh, N. C. Plumb, L. Keller, R. Cubitt, E. Pomjakushina, and J. S. White, Phys. Rev. Lett. 124, 017202 (2020).
  * Destraz  _et al._ [2020] D. Destraz, L. Das, S. S. Tsirkin, Y. Xu, T. Neupert, J. Chang, A. Schilling, A. G. Grushin, J. Kohlbrecher, L. Keller, P. Puphal, E. Pomjakushina, and J. S. White, npj Quantum Materials 5, 5 (2020).
  * Hodovanets  _et al._ [2018] H. Hodovanets, C. J. Eckberg, P. Y. Zavalij, H. Kim, W.-C. Lin, M. Zic, D. J. Campbell, J. S. Higgins, and J. Paglione, Phys. Rev. B 98, 245132 (2018).
  * Yang  _et al._ [2021] H.-Y. Yang, B. Singh, J. Gaudet, B. Lu, C.-Y. Huang, W.-C. Chiu, S.-M. Huang, B. Wang, F. Bahrami, B. Xu, J. Franklin, I. Sochnikov, D. E. Graf, G. Xu, Y. Zhao, C. M. Hoffman, H. Lin, D. H. Torchinsky, C. L. Broholm, A. Bansil, and F. Tafti, Phys. Rev. B 103, 115143 (2021).
  * Meng  _et al._ [2019] B. Meng, H. Wu, Y. Qiu, C. Wang, Y. Liu, Z. Xia, S. Yuan, H. Chang, and Z. Tian, APL Materials 7, 051110 (2019).
  * Lyu  _et al._ [2020] M. Lyu, J. Xiang, Z. Mi, H. Zhao, Z. Wang, E. Liu, G. Chen, Z. Ren, G. Li, and P. Sun, Phys. Rev. B 102, 085143 (2020).
  * Wang  _et al._ [2022] J.-F. Wang, Q.-X. Dong, Z.-P. Guo, M. Lv, Y.-F. Huang, J.-S. Xiang, Z.-A. Ren, Z.-J. Wang, P.-J. Sun, G. Li, and G.-F. Chen, Phys. Rev. B 105, 144435 (2022).
  * Yang  _et al._ [2020] H.-Y. Yang, B. Singh, B. Lu, C.-Y. Huang, F. Bahrami, W.-C. Chiu, D. Graf, S.-M. Huang, B. Wang, H. Lin, D. Torchinsky, A. Bansil, and F. Tafti, APL Materials 8, 011111 (2020).
  * Yang  _et al._ [2023] H.-Y. Yang, J. Gaudet, R. Verma, S. Baidya, F. Bahrami, X. Yao, C.-Y. Huang, L. DeBeer-Schmitt, A. A. Aczel, G. Xu, H. Lin, A. Bansil, B. Singh, and F. Tafti, Phys. Rev. Mater. 7, 034202 (2023).
  * He  _et al._ [2023] X. He, Y. Li, H. Zeng, Z. Zhu, S. Tan, Y. Zhang, C. Cao, and Y. Luo, Science China Physics, Mechanics & Astronomy 66, 237011 (2023).
  * Gaudet  _et al._ [2021] J. Gaudet, H.-Y. Yang, S. Baidya, B. Lu, G. Xu, Y. Zhao, J. A. Rodriguez-Rivera, C. M. Hoffmann, D. E. Graf, D. H. Torchinsky, P. Nikolić, D. Vanderbilt, F. Tafti, and C. L. Broholm, Nature Materials 20, 1650 (2021).
  * Dhital  _et al._ [2023] C. Dhital, R. L. Dally, R. Ruvalcaba, R. Gonzalez-Hernandez, J. Guerrero-Sanchez, H. B. Cao, Q. Zhang, W. Tian, Y. Wu, M. D. Frontzek, S. K. Karna, A. Meads, B. Wilson, R. Chapai, D. Graf, J. Bacsa, R. Jin, and J. F. DiTusa, arXiv preprint arXiv:2302.05596 (2023).
  * Dhar  _et al._ [1993] S. Dhar, S. Pattalwar, and R. Vijayaraghavan, Physica B: Condensed Matter 186-188, 491 (1993).
  * Patil and Dhar [1996] N. Patil and S. Dhar, Physica B: Condensed Matter 223-224, 359 (1996).
  * Puphal  _et al._ [2019] P. Puphal, C. Mielke, N. Kumar, Y. Soh, T. Shang, M. Medarde, J. S. White, and E. Pomjakushina, Phys. Rev. Materials 3, 024204 (2019).
  * Das  _et al._ [2014] P. K. Das, A. Bhattacharyya, R. Kulkarni, S. K. Dhar, and A. Thamizhavel, Phys. Rev. B 89, 134418 (2014).
  * Sanki  _et al._ [2022] S. Sanki, V. Sharma, S. Sasmal, V. Saini, G. Dwari, B. B. Maity, R. Kulkarni, R. Prakash Pandeya, R. Mondal, A. Lakshan, S. Ramakrishnan, P. Pratim Jana, K. Maiti, and A. Thamizhavel, Phys. Rev. B 105, 165134 (2022).
  * Walter [1984] U. Walter, Journal of Physics and Chemistry of Solids 45, 401 (1984).
  * Stevens [1952] K. W. H. Stevens, Proceedings of the Physical Society. Section A 65, 209 (1952).
  * Hutchings [1964] M. Hutchings,  Solid State Physics, 16, 227 (1964).
  * Banda  _et al._ [2018] J. Banda, B. K. Rai, H. Rosner, E. Morosan, C. Geibel, and M. Brando, Phys. Rev. B 98, 195120 (2018).
  * Van Hieu  _et al._ [2007] N. Van Hieu, T. Takeuchi, H. Shishido, C. Tonohiro, T. Yamada, H. Nakashima, K. Sugiyama, R. Settai, T. D. Matsuda, Y. Haga, M. Hagiwara, K. Kindo, S. Araki, Y. Nozue, and Y. Ōnuki, Journal of the Physical Society of Japan 76, 064702 (2007).
  * Nakano  _et al._ [2010] Y. Nakano, F. Honda, T. Takeuchi, K. Sugiyama, M. Hagiwara, K. Kindo, E. Yamamoto, Y. Haga, R. Settai, H. Yamagami, and Y. Ōnuki, Journal of the Physical Society of Japan 79, 024702 (2010).
  * Wang [1971] Y.-L. Wang, Physics Letters A 35, 383 (1971).
  * Jin  _et al._ [2022] H. Jin, W. Cai, J. Coles, J. R. Badger, P. Klavins, S. Deemyad, and V. Taufour, Phys. Rev. B 106, 075131 (2022).
  * Malick  _et al._ [2022] S. Malick, J. Singh, A. Laha, V. Kanchana, Z. Hossain, and D. Kaczorowski, Phys. Rev. B 105, 045103 (2022).
  * Ram  _et al._ [2023] D. Ram, J. Singh, M. K. Hooda, O. Pavlosiuk, V. Kanchana, Z. Hossain, and D. Kaczorowski, Phys. Rev. B 107, 085137 (2023).
  * Gopal [1966] E. S. R. Gopal,  _Specific heats at low temperatures_ (Plenum, New York, 1966).
  * Prasad  _et al._ [2012] A. Prasad, V. K. Anand, U. B. Paramanik, Z. Hossain, R. Sarkar, N. Oeschler, M. Baenitz, and C. Geibel, Phys. Rev. B 86, 014414 (2012).
  * Sahu [2021] B. Sahu, arXiv preprint arXiv:2102.04106 (2021).
  * Das and Kaczorowski [2019] D. Das and D. Kaczorowski, Journal of Magnetism and Magnetic Materials 471, 315 (2019).
  * Shishido  _et al._ [2010] H. Shishido, T. Shibauchi, K. Yasu, T. Kato, H. Kontani, T. Terashima, and Y. Matsuda, Science 327, 980 (2010).
  * Tsujimoto  _et al._ [2014] M. Tsujimoto, Y. Matsumoto, T. Tomita, A. Sakai, and S. Nakatsuji, Phys. Rev. Lett. 113, 267001 (2014).
  * Lv and Liu [2022] B. Lv and B. Liu, Nuclear Analysis 1, 100002 (2022).
  * Krolak  _et al._ [2022] S. Krolak, H. Swiatek, K. Gornicka, M. Winiarski, W. Xie, R. Cava, and T. Klimczuk, Journal of Alloys and Compounds 929, 167279 (2022).
  * Kaczorowski and Szytuła [2014] D. Kaczorowski and A. Szytuła, Journal of Alloys and Compounds 614, 186 (2014).
  * Anand  _et al._ [2012] V. K. Anand, D. T. Adroja, and A. D. Hillier, Phys. Rev. B 85, 014418 (2012).
  * Nallamuthu  _et al._ [2017] S. Nallamuthu, A. Dzubinska, M. Reiffers, J. R. Fernandez, and R. Nagalakshmi, Physica B: Condensed Matter 521, 128 (2017).


#  EMVLight: A Decentralized Reinforcement Learning Framework for Efficient
Passage of Emergency Vehicles

Haoran Su, 1 Yaofeng Desmond Zhong, 2 Biswadip Dey, 2 Amit Chakraborty 2
This work is done through his internship with Siemens Corporation, Technology.

###### Abstract

Emergency vehicles (EMVs) play a crucial role in responding to time-critical
events such as medical emergencies and fire outbreaks in an urban area. The
less time EMVs spend traveling through the traffic, the more likely it would
help save people’s lives and reduce property loss. To reduce the travel time
of EMVs, prior work has used route optimization based on historical traffic-
flow data and traffic signal pre-emption based on the optimal route. However,
traffic signal pre-emption dynamically changes the traffic flow which, in
turn, modifies the optimal route of an EMV. In addition, traffic signal pre-
emption practices usually lead to significant disturbances in traffic flow and
subsequently increase the travel time for non-EMVs. In this paper, we propose
EMVLight, a decentralized reinforcement learning (RL) framework for
simultaneous dynamic routing and traffic signal control. EMVLight extends
Dijkstra’s algorithm to efficiently update the optimal route for the EMVs in
real time as it travels through the traffic network. The decentralized RL
agents learn network-level cooperative traffic signal phase strategies that
not only reduce EMV travel time but also reduce the average travel time of
non-EMVs in the network. This benefit has been demonstrated through
comprehensive experiments with synthetic and real-world maps. These
experiments show that EMVLight outperforms benchmark transportation
engineering techniques and existing RL-based signal control methods.

##  1 Introduction

Emergency vehicles (EMVs) are vehicles including ambulances, fire trucks, and
police cars, which respond to critical events such as medical emergencies,
fire disasters, and criminal activities. Emergency response time is the key
indicator of a city’s emergency management capability. Reducing response time
saves lives and prevents property loss. For instance, the survivor rate from a
sudden cardiac arrest without treatment drops 7% - 10% for every second
elapsed, and there is barely any chance to survive after 8 minutes. EMV travel
time, the time interval for an EMV to travel from a rescue station to an
incident site, accounts for a major portion of the emergency response time.
However, overpopulation and urbanization have been exacerbating road
congestion, making it more and more challenging to reduce the average EMV
travel time. Records (Analytics 2021) have shown that even with a decline in
average emergency response time, the average EMV travel time increases from
7.2 minutes in 2015 to 10.1 minutes in 2021 in New York City. Needless to say,
there is a severe urgency and significant benefit for shortening the average
EMV travel time on increasingly crowded roads.

Existing works have studied strategies to reduce the travel time of EMVs by
route optimization and traffic signal pre-emption. Route optimization usually
refers to the search for a time-based shortest path. The traffic network
(e.g., city road map) is modeled as a graph with intersections as nodes and
road segments between intersections as edges. Based on the time a vehicle
needs to travel through each edge (road segment), route optimization
calculates an optimal route such that an EMV can travel from the rescue
station to the incident site in the least amount of time. In addition, as the
EMV needs to be as fast as possible, the law in most places requires non-EMVs
to yield to emergency vehicles sounding sirens, regardless of the traffic
signals at intersections. Even though this practice gives the right-of-way to
EMVs, it poses safety risks for vehicles and pedestrians at the intersections.
To address this safety concern, existing methods have also studied traffic
signal pre-emption which refers to the process of deliberately altering the
signal phases at each intersection to prioritize EMV passage.

However, as the traffic condition constantly changes, an optimal route
returned by route optimization can potentially become suboptimal as an EMV
travels through the network. Moreover, traffic signal pre-emption has a
significant impact on the traffic flow, which would change the fastest route
as well. Thus, the optimal route should be updated with real-time traffic flow
information, i.e., the route optimization should be solved in a dynamic (time-
dependent) way. As an optimal route can change as an EMV travels through the
traffic network, the traffic signal pre-emption would need to adapt
accordingly. In other words, the subproblems of route optimization and traffic
signal pre-emption are coupled and should be solved simultaneously in real-
time. Existing approaches does not address this coupling.

In addition, most of the existing models on emergency vehicle service have a
single objective of reducing the EMV travel time. As a result, their traffic
signal control strategies have an undesirable effect of increasing the travel
time of non-EMVs, since only EMV passage is optimized. In this paper, we aim
to perform route optimization and traffic signal pre-emption to not only
reduce EMV travel time but also to reduce the average travel time of non-EMVs.
In particular, we address the following two key challenges:

  * •

How to dynamically route an EMV to a destination under time-dependent traffic
conditions in a computationally efficient way? As the congestion level of each
road segment changes over time, the routing algorithm should be able to update
the remaining route as the EMV passes each intersection. Running the shortest-
path algorithm each time the EMV passes through an intersection is not
efficient. A computationally efficient dynamic routing algorithm is desired.

  * •

How to coordinate traffic signals to not only reduce EMV travel time but
reduce the average travel time of non-EMVs as well? To reduce EMV travel time,
only the traffic signals along the route of the EMV need to be altered.
However, to further reduce average non-EMV travel time, traffic signals in the
whole traffic network need to be operated cooperatively.

To tackle these challenges, we propose EMVLight, a decentralized multi-agent
reinforcement learning framework with a dynamic routing algorithm to control
traffic signal phases for efficient EMV passage. Our experimental results
demonstrate that EMVLight outperforms traditional traffic engineering methods
and existing RL methods under two metrics - EMV travel time and the average
travel time of all vehicles - on different traffic configurations.

##  2 Related Work

Conventional routing optimization and traffic signal pre-emption for EMVs.
Although, in reality, routing and pre-emption are coupled, the existing
methods usually solve them separately. Many of the existing approaches
leverage Dijkstra’s shortest path algorithm to get the optimal route (Wang,
Ma, and Yang 2013; Mu, Song, and Liu 2018; Kwon, Kim, and Betts 2003; Jotshi,
Gong, and Batta 2009). An A* algorithm for ambulance routing has been proposed
by Nordin et al. (2012). However, as this line of work assumes that the routes
and traffic conditions are fixed and static, they fail to address the dynamic
nature of real-world traffic flows. Another line of work has considered the
change of traffic flows over time. Ziliaskopoulos and Mahmassani (1993) have
proposed a shortest-path algorithm for time-dependent traffic networks, but
the travel time associated with each edge at each time step is assumed to be
known in prior. Musolino et al. (2013) propose different routing strategies
for different times in a day (e.g., peak/non-peak hours) based on traffic
history data at those times. However, in the problem of our consideration,
routing and pre-emption strategies can significantly affect the travel time
associated with each edge during the EMV passage, and the existing methods
cannot deal with this kind of real-time changes. Haghani, Hu, and Tian (2003)
formulated the dynamic shortest path problem as a mixed-integer programming
problem. Koh et al. (2020) have used RL for real-time vehicle navigation and
routing. However, both of these studies have tackled a general routing
problem, and signal pre-emption and its influence on traffic have not been
modeled.

Once an optimal route for the EMV has been determined, traffic signal pre-
emption is deployed. A common pre-emption strategy is to extend the green
phases of green lights to let the EMV pass each intersection along a fixed
optimal route (Wang, Ma, and Yang 2013; Bieker-Walz and Behrisch 2019).
Asaduzzaman and Vidyasankar (2017) have proposed pre-emption strategies for
multiple EMV requests.

Please refer to Lu and Wang (2019) and Humagain et al. (2020) for a thorough
survey of conventional routing optimization and traffic signal pre-emption
methods. We would also like to point out that the conventional methods
prioritize EMV passage and have significant disturbances on the traffic flow
which increases the average non-EMV travel time.

RL-based traffic signal control. Traffic signal pre-emption only alters the
traffic phases at the intersections where an EMV travels through. However, to
reduce congestion, traffic phases at nearby intersections also need to be
changed cooperatively. The coordination of traffic signals to mitigate traffic
congestion is referred to as traffic signal control which has been addressed
by leveraging deep RL in a growing body of work. Many of the existing
approaches use Q-learning (Abdulhai, Pringle, and Karakoulas 2003; Prashanth
and Bhatnagar 2010; Wei et al. 2019a, b; Zheng et al. 2019; Chen et al. 2020).
Zang et al. (2020) leverage meta-learning algorithms to speed up Q-learning
for traffic signal control. Another line of work has used actor-critic
algorithms for traffic signal control (El-Tantawy, Abdulhai, and Abdelgawad
2013; Aslani, Mesgari, and Wiering 2017; Chu et al. 2019). Xu et al. (2021)
propose a hierarchical actor-critic method to encourage cooperation between
intersections. Please refer to Wei et al. (2019c) for a review on traffic
signal control methods. However, these RL-based traffic control methods focus
on reducing the congestion in the traffic network and are not designed for EMV
pre-emption. In contrast, our RL framework is built upon state-of-the-art
ideas such as max pressure and is designed to reduce both EMV travel time and
overall congestion.

##  3 Preliminaries

Figure 1: Traffic movements illustration and an example pressure calculation
for incoming lane #2.

######  Definition 1 (traffic map, link, lane)

A traffic map can be represented by a graph $G(\mathcal{V},\mathcal{E})$, with
intersections as nodes and road segments between intersections as edges. We
refer to a one-directional road segment between two intersections as a link. A
link has a fixed number of lanes, denoted as $h(l)$ for lane $l$. Fig. 1 shows
8 links and each link has 2 lanes.

######  Definition 2 (Traffic movements)

A traffic movement $(l,m)$ is defined as the traffic traveling across an
intersection from an incoming lane $l$ to an outgoing lane $m$. The
intersection shown in Fig. 1 has 24 permissible traffic movements. The set of
all permissible traffic movements of an intersection is denoted as
$\mathcal{M}$.

######  Definition 3 (Traffic signal phase)

A traffic signal phase is defined as the set of permissible traffic movements.
As shown in Fig. 2, an intersection with 4 links has 8 phases.

Figure 2: _Top_ : 8 signal phases; _Left_ : phase #2 illustration; _Right_ :
phase #5 illustration.

######  Definition 4 (Pressure of an incoming lane)

The pressure of an incoming lane $l$ measures the unevenness of vehicle
density between lane $l$ and corresponding out going lanes in permissible
traffic movements. The vehicle density of a lane is $x(l)/x_{max}(l)$, where
$x(l)$ is the number of vehicles on lane $l$ and $x_{max}(l)$ is the vehicle
capacity on lane $l$, which is related to the length of a lane. Then the
pressure of an incoming lane $l$ is

$w(l)=\left|\frac{x(l)}{x_{max}(l)}-\sum_{\\{m|(l,m)\in\mathcal{M}\\}}\frac{1}{h(m)}\frac{x(m)}{x_{max}(m)}\right|,$
(1)

where $h(m)$ is the number of lanes of the outgoing link which contains $m$.
In Fig. 1, $h(m)=2$ for all the outgoing lanes. An example for Eqn. (1) is
shown in Fig. 1.

######  Definition 5 (Pressure of an intersection)

The pressure $P$ of an intersection is the average of the pressure of all
incoming lanes.

The pressure of an intersection indicates the unevenness of vehicle density
between incoming and outgoing lanes in an intersection. Intuitively, reducing
the pressure leads to more evenly distributed traffic, which indirectly reduce
congestion and average travel time of vehicles.

##  4 Dynamic Routing

Dijkstra’s algorithm is an algorithm that finds shortest path between a given
node and every other nodes in a graph, which has been used for EMV routing.
The EMV travel time along each link is estimated based on the number of
vehicles on that link. We refer to it as the _intra-link travel time._
Dijkstra’s algorithm takes as input the traffic graph, the intra-link travel
time and a destination, and can return the time-based shortest path as well as
estimated travel time from each intersection to the destination. The latter is
usually referred to as the _estimated time of arrival_ (ETA) of each
intersection.

However, traffic conditions are constantly changing and so does EMV travel
time along each link. Moreover, EMV pre-emption techniques alters traffic
signal phases, which will significantly change the traffic condition as the
EMV travels. The pre-determined shortest path might become congested due to
stochasticity and pre-emption. Thus, updating the optimal route dynamically
can facilitate EMV passage. In theory we can run Dijkstra’s algorithm
frequently as the EMV travels through the network to take into account the
updated EMV intra-link travel time, but this is inefficient.

To achieve dynamics routing, we extend Dijkstra’s algorithm to efficiently
update the optimal route based on the updated intra-link travel times. As
shown in Algorithm 1, first a prepopulation process is carried out where a
(static) Dijkstra’s algorithm is run to get the ETA from each intersection to
the destination. For each intersection, the next intersection along the
shortest path is also calculated and stored. We assume this process can be
done before the EMV starts to travel. This is reasonable since a sequence of
processes, including call-taker processing, are performed before the EMVs are
dispatched. Once the pre-population process is finished, we can update
$\mathsf{ETA}$ and $\mathsf{Next}$ for each intersection efficiently in
parallel, since the update only depends on information of neighboring
intersections. Please see Appendix for how intra-link travel time is estimated
in real time.

Input :

$G=(\mathcal{V},\mathcal{E})$ | traffic map as a graph
---|---
$T^{t}=[T_{ij}^{t}]$ | intra-link travel time at time $t$
$i_{d}$ | index of the destination

Output :

$\mathsf{ETA}^{t}=[\mathsf{ETA}^{t}_{i}]$ | ETA of each intersection
---|---
$\mathsf{Next}^{t}=[\mathsf{Next}^{t}_{i}]$ | next intersection to go
| from each intersection

/* pre-population */

1 $\mathsf{ETA}^{0},\mathsf{Next}^{0}$ $=$ Dijkstra$(G,T^{0},i_{d})$;

/* dynamic routing */

2 for  _$t=0\to T$_ do

3 foreach  _$i\in\mathcal{V}$_ do (in parallel)

4
$\mathsf{ETA}_{i}^{t+1}\leftarrow\min_{(i,j)\in\mathcal{E}}(\mathsf{ETA}_{j}^{t}+T_{ji}^{t})$;

5
$\mathsf{Next}_{i}^{t+1}\leftarrow\arg\min_{\\{j|(i,j)\in\mathcal{E}\\}}(\mathsf{ETA}_{j}^{t}+T_{ji}^{t}$);

Algorithm 1 Dynamic Dijkstra’s for EMV routing

###### Remark 1

In static Dijkstra’s algorithm, the shortest path is obtained by repeatedly
query the $\mathsf{Next}$ attribute of each node from the origin until we
reach the destination. In our dynamic Dijkstra’s algorithm, since the shortest
path changes, at a intersection $i$, we only care about the immediate next
intersection to go to, which is exactly $\mathsf{Next}_{i}$.

##  5 Reinforcement Learning Formulation

While dynamic routing directs the EMV to the destination, it does not take
into account the possible waiting times for red lights at the intersections.
Thus, traffic signal pre-emption is also required for the EMV to arrive at the
destination in the least amount of time. However, since traditional pre-
emption only focuses on reducing the EMV travel time, the average travel time
of non-EMVs can increase significantly. Thus, we set up traffic signal control
for efficient EMV passage as a decentralized RL problem. In our problem, an RL
agent controls the traffic signal phases of an intersection based on local
information. Multiple agents coordinate the control signal phases of
intersections cooperatively to (1) reduce EMV travel time and (2) reduce the
average travel time of non-EMVs. First we design 3 agent types. Then we
present agent design and multi-agent interactions.

###  5.1 Types of agents for EMV passage

When an EMV is on duty, we distinguish 3 types of traffic control agents based
on EMV location and routing (Fig. 3). An agent is a _primary pre-emption
agent_ $i_{p}$ if an EMV is on one of its incoming links. The agent of the
next intersection $i_{s}=\mathsf{Next}_{i_{p}}$ is refered to as a _secondary
pre-emption agent_. The rest of the agents are _normal agents_. We design
these types since different agents have different local goals, which is
reflected in their reward designs.

###  5.2 Agent design

  * •

State: The state of an agent $i$ at time $t$ is denoted as $s^{t}_{i}$ and it
includes the number of vehicles on each outgoing lanes and incoming lanes, the
distance of the EMV to the intersection, the estimated time of arrival
($\mathsf{ETA}$), and which link the EMV will be routed to ($\mathsf{Next}$),
i.e.,

$s^{t}_{i}=\\{x^{t}(l),x^{t}(m),d^{t}_{\text{EMV}}[L_{ji}],\mathsf{ETA}^{t^{\prime}}_{i},\mathsf{Next}^{t^{\prime}}_{i}\\},$
(2)

where $L_{ji}$ represents the links incoming to intersection $i$, and with a
slight abuse of notation $l$ and $m$ denote the set of incoming and outgoing
lanes, respectively. For the intersection shown in Fig. 1,
$d^{t}_{\text{EMV}}$ is a vector of four elements. For primary pre-emption
agents, one of the elements represents the distance of EMV to the intersection
in the corresponding link. The rest of the elements are set to -1. For all
other agents, $d^{t}_{\text{EMV}}$ are padded with -1.

  * •

Action: Prior work has focused on using phase switch, phase duration and phase
itself as actions. In this work, we define the action of an agent as one of
the 8 phases in Fig. 2; this enables more flexible signal patterns as compared
to the traditional cyclical patterns. Due to safety concerns, once a phase has
been initiated, it should remain unchanged for a minimum amount of time, e.g.
5 seconds. Because of this, we set our MDP time step length to be 5 seconds to
avoid rapid switch of phases.

  * •

Reward: PressLight has shown that minimizing the pressure is an effective way
to encourage efficient vehicle passage, we adopt similar idea for normal
agents. For secondary pre-emption agents we additionally encourage less
vehicle on the link where the EMV is about to enter in order to encourage
efficient EMV passage. For primary pre-emption agents, we simply assign a unit
penalty at each time step to encourage fast EMV passage. Thus, depending on
the agent type, the local reward for agent $i$ at time $t$ is

$r_{i}^{t}=\begin{cases}-P_{i_{p}}^{t}&i\notin\\{i_{p},i_{s}\\},\\\ -\beta
P_{i_{s}}^{t}-\frac{1-\beta}{|L_{i_{p}i_{s}}|}\sum\limits_{l\in
L_{i_{p}i_{s}}}\frac{x(l)}{x_{max}(l)}&i=i_{s},\\\ -1&i=i_{p}.\end{cases}$ (3)

Figure 3: Three types of agents.

Justification of agent design. The quantities in local agent state can be
obtained at each intersection using various technologies. Numbers of vehicles
on each lane $(x^{t}(l),x^{t}(m))$ can be obtained by vehicle detection
technologies, such as inductive loop (Gajda et al. 2001) based on the hardware
installed underground. The distance of the EMV to the intersection
$d^{t}_{EMV}[L_{ji}]$ can be obtained by _vehicle-to-infrastructure_
technologies such as VANET(Buchenscheit et al. 2009), which broadcasts the
real-time position of a vehicle to an intersection. Prior work by Wang et al.
(2013) and Noori, Fu, and Shiravi (2016) have explored these technologies for
traffic signal pre-emption.

The dynamic routing algorithm (Algorithm 1) can provide
$(\mathsf{ETA},\mathsf{Next})$ for each agent at every time step. However, due
to the stochastic nature of traffic flows, updating the route too frequently
might confuse the EMV driver, since the driver might be instructed a new
route, say, every 5 seconds. There are many ways to ensure reasonable
frequency. One option is to inform the driver only once while the EMV is
travels in a single link. We implement it by updating the state of an RL agent
$(\mathsf{ETA}^{t^{\prime}}_{i},\mathsf{Next}^{t^{\prime}}_{i})$ at the time
step when the EMV travels through half of a link. For example, if the EMV
travels through a link to agent $i$ from time step 11 to 20 in constant speed,
then dynamic routing information in $s_{i}^{16}$ to $s_{i}^{20}$ are the same,
which is $(\mathsf{ETA}_{i}^{15},\mathsf{Next}_{i}^{15})$, i.e.,
$t^{\prime}=15$.

As for the reward design, one might wonder how an agent can know its type. As
we assume an agent can observe the state of its neighbors, agent type can be
inferred from the observation. This will become clearer below.

###  5.3 Multi-agent Advantage Actor-critic

We adopt a multi-agent advantage actor-critic (MA2C) framework similar to Chu
et al. (2019). The difference is that our local state includes dynamic routing
information and our local reward encourages efficient passage of EMV. Here we
briefly introduce the MA2C framework. Please refer to Chu et al. (2019) for
additional details.

In a multi-agent network $G(\mathcal{V},\mathcal{E})$, the neighborhood of
agent $i$ is denoted as $\mathcal{N}_{i}=\\{j|ji\in\mathcal{E}\textrm{ or
}ij\in\mathcal{E}\\}$. The local region of agent $i$ is
$\mathcal{V}_{i}=\mathcal{N}_{i}\cup i$. We define the distance between two
agents $d(i,j)$ as the minimum number of edges that connect them. For example,
$d(i,i)=0$ and $d(i,j)=1,\forall j\in\mathcal{N}_{i}$. In MA2C, each agent
learns a policy $\pi_{\theta_{i}}$ (actor) and the corresponding value
function $V_{\phi_{i}}$ (critic), where ${\theta_{i}}$ and ${\phi_{i}}$ are
learnable neural network parameters of agent $i$.

Local Observation. In an ideal setting, agents can observe the states of every
other agent and leverage this global information to make a decision. However,
this is not practical in our problem due to communication latency and will
cause scalability issues. We assume agents can observe its own state and the
states of its neighbors, i.e.,
$s^{t}_{\mathcal{V}_{i}}=\\{s^{t}_{j}|j\in\mathcal{V}_{i}\\}$. The agents feed
this observation to its policy network $\pi_{\theta_{i}}$ and value network
$V_{\phi_{i}}$.

Fingerprint. In multi-agent training, each agent treats other agents as part
of the environment, but the policy of other agents are changing over time.
Foerster et al. (2017) introduce _fingerprints_ to inform agents about the
changing policies of neighboring agents in multi-agent Q-learning. Chu et al.
(2019) bring fingerprints into MA2C. Here we use the probability simplex of
neighboring policies
$\pi^{t-1}_{\mathcal{N}_{i}}=\\{\pi^{t-1}_{j}|j\in\mathcal{N}_{i}\\}$ as
fingerprints, and include it into the input of policy network and value
network. Thus, our policy network can be written as
$\pi_{\theta_{i}}(a_{i}^{t}|s^{t}_{\mathcal{V}_{i}},\pi^{t-1}_{\mathcal{N}_{i}})$
and value network as
$V_{\phi_{i}}(\tilde{s}^{t}_{\mathcal{V}_{i}},\pi^{t-1}_{\mathcal{N}_{i}})$,
where $\tilde{s}^{t}_{\mathcal{V}_{i}}$ is the local observation with spatial
discount factor, which is introduced below.

Spatial Discount Factor and Adjusted Reward. MA2C agents cooperatively
optimize a global cumulative reward. We assume the global reward is
decomposable as $r_{t}=\sum_{i\in\mathcal{V}}r^{t}_{i}$, where $r^{t}_{i}$ is
defined in Eqn. (3). Instead of optimizing the same global reward for every
agent, Chu et al. (2019) propose a spatial discount factor $\alpha$ to let
each agent pay less attention to rewards of agents far away. The adjusted
reward for agent $i$ is

$\tilde{r}_{i}^{t}=\sum_{d=0}^{D_{i}}\Big{(}\sum_{j\in\mathcal{V}|d(i,j)=d}(\alpha)^{d}r^{t}_{j}\Big{)},$
(4)

where $D_{i}$ is the maximum distance of agents in the graph from agent $i$.
When $\alpha>0$, the adjusted reward include global information, it seems this
is in contradiction to the local communication assumption. However, since
reward is only used for offline training, global reward information is
allowed. Once trained, the RL agents can control traffic signal without
relying on global information.

Temporal Discount Factor and Return. The local return $\tilde{R}^{t}_{i}$ is
defined as the cumulative adjusted reward
$\tilde{R}^{t}_{i}:=\sum_{\tau=t}^{T}\gamma^{\tau-t}\tilde{r}^{\tau}_{i}$,
where $\gamma$ is the temporal discount factor and $T$ is the length of an
episode. we can estimate the local return using value function,

$\tilde{R}^{t}_{i}=\tilde{r}^{t}_{i}+\gamma
V_{\phi_{i}^{-}}(\tilde{s}^{t+1}_{\mathcal{V}_{i}},\pi^{t}_{\mathcal{N}_{i}}|\pi_{\theta_{-i}^{-}}),$
(5)

where $\phi_{i}^{-}$ means parameters $\phi_{i}$ are frozen and
$\theta_{-i}^{-}$ means the parameters of policy networks of all other agents
are frozen.

Network architecture and training. As traffic flow data are spatial temporal,
we leverage a long-short term memory (LSTM) layer along with fully connected
(FC) layers for policy network (actor) and value network (critic). Our multi-
agent actor-critic training pipeline is similar to that in Chu et al. (2019).
We provide neural architecture details, policy loss expression, value loss
expression as well as a training pseudocode in the Appendix.

##  6 Experimentation

In this section, we demonstrate our RL framework using Simulation of Urban
MObility (SUMO) (Lopez et al. 2018) SUMO is an open-source traffic simulator
capable of simulating both microscopic and macroscopic traffic dynamics,
suitable for capturing the EMV’s impact on the regional traffic as well as
monitoring the overall traffic flow. A pipeline is established between the
proposed RL framework and SUMO, i.e., the agents collects observations from
SUMO and preferred signal phases are fed back into SUMO.

###  6.1 Datasets and Maps Descriptions

We conduct the following experiments based on both synthetic and real-world
map.

##### Synthetic $\text{Grid}_{5\times 5}$

We synthesize a $5\times 5$ traffic grid, where intersections are connected
with bi-directional links. Each link contains two lanes. We design 4
configurations, listed in Table 1. The origin (O) and destination (D) of the
EMV are labelled in Fig. 4. The traffic for this map has a time span of 1200s.
We dispatch the EMV at $t=600s$ to ensure the roads are compacted when it
starts travel.

Figure 4: _Left_ : the synthetic $\text{grid}_{5\times 5}$. _Right_ : an intersection illustration in SUMO, the teal area are inductive loop detected area. Origin and destination for EMV are labeled. Config | Traffic Flow (veh/lane/hr) | Origin | Destination
---|---|---|---
Non-peak | Peak
1 | 200 | 240 | N,S | E,W
2 | 160 | 320
3 | 200 | 240 | Randomly
4 | 160 | 320 | generated
Table 1: Configuration for Synthetic $\text{Grid}_{5\times 5}$. Peak flow is
assigned from 400s to 800s and non-peak flow is assigned out of this period.
For Config. 1 and 2, the vehicles enter the grid from North and South, and
exit toward East and West.

##### $\text{Manhattan}_{16\times 3}$

This is a $16\times 3$ traffic network extracted from Manhattan Hell’s Kitchen
area (Fig. 5) and customized for demonstrating EMV passage. In this traffic
network, intersections are connected by 16 one-directional streets and 3 one-
directional avenues. We assume each avenue contains four lanes and each street
contains two lanes so that the right-of-way of EMVs and pre-emption can be
demonstrated. The traffic flow for this map is generated from open-source NYC
taxi data. Both the map and traffic flow data are publicly
available.111https://traffic-signal-control.github.io/ The origin and
destination of EMV are set to be far away as shown in Fig. 5

Figure 5: Manhattan map: a 16-by-3 traffic network in Hell’s Kitchen area. Origin and destination for the EMV dispatching are labeled. Method  | EMV Travel Time $T_{\textrm{EMV}}$ [s] | Average Travel Time $T_{\textrm{avg}}$ [s]
---|---|---
Config 1 | Config 2 | Config 3 | Config 4 | $\text{Manhattan}_{16\times 3}$ | Config 1 | Config 2 | Config 3 | Config 4 | $\text{Manhattan}_{16\times 3}$
FT w/o EMV | N/A | N/A | N/A | N/A | N/A | 353.43 | 371.13 | 314.25 | 334.10 | 1649.64
W + Static + FT | 257.20 | 272.00 | 259.20 | 243.80 | 487.20 | 372.19 | 389.13 | 342.49 | 355.05 | 1811.03
W + Static + MP | 255.00 | 269.00 | 261.20 | 245.40 | 461.80 | 349.38 | 352.54 | 307.91 | 322.68 | 708.13
W + Static + CL | 281.20 | 286.20 | 289.80 | 277.80 | 492.20 | 503.35 | 524.26 | 488.12 | 509.55 | 2013.54
W + Static + PL | 276.00 | 282.20 | 271.40 | 275.00 | 476.00 | 358.18 | 369.45 | 332.98 | 338.95 | 1410.76
W + dynamic + FT | 229.60 | 231.20 | 228.60 | 227.20 | 442.20 | 370.09 | 393.40 | 330.13 | 345.50 | 1699.30
W + dynamic + MP | 226.20 | 234.60 | 224.20 | 217.60 | 438.80 | 345.45 | 348.43 | 313.26 | 325.72 | 721.32
W + dynamic + CL | 273.40 | 269.60 | 281.00 | 270.80 | 450.20 | 514.29 | 536.78 | 502.12 | 542.63 | 1987.86
W + dynamic + PL | 251.20 | 257.80 | 247.00 | 268.80 | 436.20 | 359.31 | 342.59 | 340.11 | 349.20 | 1412.12
EMVLight | 198.60 | 192.20 | 199.20 | 196.80 | 391.80 | 322.40 | 318.76 | 301.90 | 321.02 | 681.23
Table 2: Performance comparison of different methods evaluated in the four
configurations of the synthetic traffic grid as well as Manhattan Map. For
both metrics, the lower value indicates better performance. The lowest values
are highlighted in bold. The average travel time of Manhattan map (1649.64) is
retrieved from data.

###  6.2 Baselines

Due to the lack of existing RL methods for efficient EMV passage, we select
traditional methods and RL methods for each subproblem and combine them to set
up baselines.

For traffic signal pre-emption, the most intuitive and widely used approach is
extending green light period for EMV passage at each intersection which
results in a _Green Wave_ (Corman et al. 2009). Walabi (W) (Bieker-Walz and
Behrisch 2019) is an effective rule-based method that implemented Green Wave
for EMVs in SUMO environment. We integrate Walabi with combinations of routing
and traffic signal control strategies introduced below as baselines.

_Routing baselines:_

  * •

Static routing is performed when EMV is dispatched and the route remains fixed
as the EMV travels. We adopt A* search as the baseline since it is an powerful
extension to the Dijkstra’s shortest path algorithm and is used in many real-
time applications because of its optimality. 222Our implementation of A*
search employs a Manhattan distance as the heuristic function.

  * •

Dynamic routing relies on real-time information of traffic conditions. To set
up the baseline, we run A* every 50s as EMV travels. This is because running
the full A* to update optimal route is not as efficient as our proposed
dynamic Dijkstra’s algorithm.

_Traffic signal control baselines:_

  * •

Fixed Time (FT): Cyclical fixed time traffic phases with random offset (Roess,
Prassas, and McShane 2004) is a policy that split all phases with an
predefined green ratio. It is the default strategy in real traffic signal
control.

  * •

Max Pressure (MP): The state-of-the-art (SOTA) network-level signal control
strategy based on pressure (Varaiya 2013). It aggressively select the phase
with maximum pressure to smooth congestion.

  * •

Coordinated Learner (CL): A Q-learning based coordinator which directly learns
joint local value functions for adjacent intersections (Van der Pol and
Oliehoek 2016).

  * •

PressLight (PL): A RL method aiming to optimize the pressure at each
intersection(Wei et al. 2019a).

###  6.3 Results

We evaluate performance of models under two metrics: _EMV travel time_ , which
reflects routing and pre-emption ability, and _average travel time_ , which
indicates the ability of traffic signal control for efficient vehicle passage.
The performance of our EMVLight and the baselines in both the synthetic and
the Manhattan map is shown in Table 2. The results of all methods are averaged
over five independent runs and RL methods are tested with random seeds. We
observe that EMVLight outperforms all baseline models under both metrics.

In terms of EMV travel time $T_{\textrm{EMV}}$, the dynamic routing baseline
performs better than static routing baselines. This is expected since dynamic
routing considers the time-dependent nature of traffic conditions and update
optimal route accordingly. EMVLight further reduces EMV travel time by 18% in
average as compared to dynamic routing baselines. This advantage in
performance can be attributed to the design of secondary pre-emption agents.
This type of agents learn to “reserve a link” by choosing signal phases that
help clear the vehicles in the link to encourage high speed EMV passage (Eqn.
(3)).

As for average travel time $T_{\textrm{avg}}$, we first notice that the
traditional pre-emption technique (W+Static+FT) indeed increases the average
travel time by around 10% as compared to a traditional Fix Time strategy
without EMV (denoted as “FT w/o EMV” in Table 2), thus decreasing the
efficiency of vehicle passage. Different traffic signal control strategies
have a direct impact on overall efficiency. Fixed Time is designed to handle
steady traffic flow. Max Pressure, as a SOTA traditional method, outperforms
Fix Time and, surprisingly, outperforms both RL baselines in terms of overall
efficiency. This shows that pressure is an effective indicator for reducing
congestion and this is why we incorporate pressure in our reward design.
Coordinate Learner performs the worst probably because its reward is not based
on pressure. PressLight doesn’t beat Max Pressure because it has a reward
design that focuses on smoothing vehicle densities along a major direction,
e.g. an arterial. Grid networks with the presence of EMV make PressLight less
effective. Our EMVLight improves its pressure-based reward design to encourage
smoothing vehicle densities of all directions for each intersection. This
enable us to achieve an advantage of 5% over our best baselines (Max
Pressure).

#### Ablation study on pressure and agent types

We propose three types of agents and design their rewards (Eqn. (3)) based on
our improved pressure definition and heuristics. In order to see how our
improved pressure definition and proposed special agents influence the
results, we (1) replace our pressure definition by that defined in PressLight,
(2) replace secondary pre-emption agents with normal agents and (3) replace
primary pre-emption agents with normal agents.

Ablations | (1) | (2) | (3) | EMVLight
---|---|---|---|---
$T_{\text{EMV}}$ [s] | 197 | 289 | 320 | 199
$T_{\text{avg}}$ [s] | 361.05 | 347.13 | 359.62 | 322.40
Table 3: Ablation study on pressure and agent types. Experiments are conducted
on the Config 1 synthetic $\text{grid}_{5\times 5}$.

Table 3 shows the results of these ablations: (1) PressLight-style pressure
(see Appendix) yields a slightly smaller EMV travel time but significantly
increases the average travel time; (2) Without secondary pre-emption agents,
EMV travel time increases by 45% since almost no “link reservation” happened;
(3) Without primary pre-emption agents, EMV travel time increases
significantly, which shows the importance of pre-emption.

#### Ablation study on fingerprint

In multi-agent RL, fingerprint has been shown to stabilize training and enable
faster convergence. In order to see how fingerprint affects training in
EMVLight, we remove the fingerprint design, i.e., policy and value networks
are changed from
$\pi_{\theta_{i}}(a_{i}^{t}|s^{t}_{\mathcal{V}_{i}},\pi^{t-1}_{\mathcal{N}_{i}})$
and
$V_{\phi_{i}}(\tilde{s}^{t}_{\mathcal{V}_{i}},\pi^{t-1}_{\mathcal{N}_{i}})$ to
$\pi_{\theta_{i}}(a_{i}^{t}|s^{t}_{\mathcal{V}_{i}})$ and
$V_{\phi_{i}}(\tilde{s}^{t}_{\mathcal{V}_{i}})$, respectively. Fig. 6 shows
the influence of fingerprint on training. With fingerprint, the reward
converges faster and suffers from less fluctuation, confirming the
effectiveness of fingerprint.

Figure 6: Reward convergence with and without fingerprint. Experiments are
conducted on Config 1 synthetic $\text{grid}_{5\times 5}$.

##  7 Conclusion

In this paper, we proposed a decentralized reinforcement learning framework,
EMVLight, to facilitate the efficient passage of EMVs and reduce traffic
congestion at the same time. Leveraging the multi-agent A2C framework, agents
incorporate dynamic routing and cooperatively control traffic signals to
reduce EMV travel time and average travel time of non-EMVs. Evaluated on both
synthetic and real-world map, EMVLight significantly outperforms the existing
methods. Future work will explore more realistic microscopic interaction
between EMV and non-EMVs, efficient passage of multiple EMVs and closing the
sim-to-real gap.

## References

  * Abdulhai, Pringle, and Karakoulas (2003) Abdulhai, B.; Pringle, R.; and Karakoulas, G. J. 2003.  Reinforcement learning for true adaptive traffic signal control.  _Journal of Transportation Engineering_ , 129(3): 278–285.
  * Analytics (2021) Analytics, N. 2021.  End-To-End Response Times.  https://www1.nyc.gov/site/fdny/about/resources/data-and-analytics/end-to-end-response-times.page.
  * Asaduzzaman and Vidyasankar (2017) Asaduzzaman, M.; and Vidyasankar, K. 2017.  A Priority Algorithm to Control the Traffic Signal for Emergency Vehicles.  In _2017 IEEE 86th Vehicular Technology Conference (VTC-Fall)_ , 1–7.
  * Aslani, Mesgari, and Wiering (2017) Aslani, M.; Mesgari, M. S.; and Wiering, M. 2017.  Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events.  _Transportation Research Part C: Emerging Technologies_ , 85: 732–752.
  * Bieker-Walz and Behrisch (2019) Bieker-Walz, L.; and Behrisch, M. 2019.  Modelling green waves for emergency vehicles using connected traffic data.  _EPiC Series in Computing_ , 62: 1–11.
  * Buchenscheit et al. (2009) Buchenscheit, A.; Schaub, F.; Kargl, F.; and Weber, M. 2009.  A VANET-based emergency vehicle warning system.  In _2009 IEEE Vehicular Networking Conference (VNC)_ , 1–8. IEEE.
  * Chen et al. (2020) Chen, C.; Wei, H.; Xu, N.; Zheng, G.; Yang, M.; Xiong, Y.; Xu, K.; and Li, Z. 2020\.  Toward A Thousand Lights: Decentralized Deep Reinforcement Learning for Large-Scale Traffic Signal Control.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 34(04): 3414–3421.
  * Chu et al. (2019) Chu, T.; Wang, J.; Codecà, L.; and Li, Z. 2019.  Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control.  _IEEE Transactions on Intelligent Transportation Systems_.
  * Corman et al. (2009) Corman, F.; D’Ariano, A.; Pacciarelli, D.; and Pranzo, M. 2009.  Evaluation of green wave policy in real-time railway traffic management.  _Transportation Research Part C: Emerging Technologies_ , 17(6): 607–616.
  * El-Tantawy, Abdulhai, and Abdelgawad (2013) El-Tantawy, S.; Abdulhai, B.; and Abdelgawad, H. 2013.  Multiagent reinforcement learning for integrated network of adaptive traffic signal controllers (MARLIN-ATSC): methodology and large-scale application on downtown Toronto.  _IEEE Transactions on Intelligent Transportation Systems_ , 14(3): 1140–1150.
  * Foerster et al. (2017) Foerster, J.; Nardelli, N.; Farquhar, G.; Afouras, T.; Torr, P. H.; Kohli, P.; and Whiteson, S. 2017.  Stabilising experience replay for deep multi-agent reinforcement learning.  In _International conference on machine learning_ , 1146–1155. PMLR.
  * Gajda et al. (2001) Gajda, J.; Sroka, R.; Stencel, M.; Wajda, A.; and Zeglen, T. 2001.  A vehicle classification based on inductive loop detectors.  In _IMTC 2001. Proceedings of the 18th IEEE Instrumentation and Measurement Technology Conference. Rediscovering Measurement in the Age of Informatics_ , volume 1, 460–464. IEEE.
  * Haghani, Hu, and Tian (2003) Haghani, A.; Hu, H.; and Tian, Q. 2003.  An optimization model for real-time emergency vehicle dispatching and routing.  In _82nd annual meeting of the Transportation Research Board, Washington, DC_. Citeseer.
  * Humagain et al. (2020) Humagain, S.; Sinha, R.; Lai, E.; and Ranjitkar, P. 2020.  A systematic review of route optimisation and pre-emption methods for emergency vehicles.  _Transport reviews_ , 40(1): 35–53.
  * Jotshi, Gong, and Batta (2009) Jotshi, A.; Gong, Q.; and Batta, R. 2009.  Dispatching and routing of emergency vehicles in disaster mitigation using data fusion.  _Socio-Economic Planning Sciences_ , 43(1): 1 – 24.
  * Koh et al. (2020) Koh, S.; Zhou, B.; Fang, H.; Yang, P.; Yang, Z.; Yang, Q.; Guan, L.; and Ji, Z. 2020\.  Real-time deep reinforcement learning based vehicle navigation.  _Applied Soft Computing_ , 96: 106694.
  * Kwon, Kim, and Betts (2003) Kwon, E.; Kim, S.; and Betts, R. 2003.  Route-based dynamic preemption of traffic signals for emergency vehicle operations.  In _Transportation Research Board 82nd Annual MeetingTransportation Research Board_.
  * Lopez et al. (2018) Lopez, P. A.; Behrisch, M.; Bieker-Walz, L.; Erdmann, J.; Flötteröd, Y.-P.; Hilbrich, R.; Lücken, L.; Rummel, J.; Wagner, P.; and Wießner, E. 2018.  Microscopic traffic simulation using sumo.  In _2018 21st International Conference on Intelligent Transportation Systems (ITSC)_ , 2575–2582. IEEE.
  * Lu and Wang (2019) Lu, L.; and Wang, S. 2019.  Literature Review of Analytical Models on Emergency Vehicle Service: Location, Dispatching, Routing and Preemption Control.  In _2019 IEEE Intelligent Transportation Systems Conference (ITSC)_ , 3031–3036.
  * Mu, Song, and Liu (2018) Mu, H.; Song, Y.; and Liu, L. 2018.  Route-Based Signal Preemption Control of Emergency Vehicle.  _Journal of Control Science and Engineering_ , 2018: 1–11.
  * Musolino et al. (2013) Musolino, G.; Polimeni, A.; Rindone, C.; and Vitetta, A. 2013.  Travel time forecasting and dynamic routes design for emergency vehicles.  _Procedia-Social and Behavioral Sciences_ , 87: 193–202.
  * Noori, Fu, and Shiravi (2016) Noori, H.; Fu, L.; and Shiravi, S. 2016.  A connected vehicle based traffic signal control strategy for emergency vehicle preemption.  In _Transportation Research Board 95th Annual Meeting_ , 16-6763.
  * Nordin et al. (2012) Nordin, N. A. M.; Zaharudin, Z. A.; Maasar, M. A.; and Nordin, N. A. 2012.  Finding shortest path of the ambulance routing: Interface of A-star algorithm using C programming.  In _2012 IEEE Symposium on Humanities, Science and Engineering Research_ , 1569–1573. IEEE.
  * Prashanth and Bhatnagar (2010) Prashanth, L.; and Bhatnagar, S. 2010.  Reinforcement learning with function approximation for traffic signal control.  _IEEE Transactions on Intelligent Transportation Systems_ , 12(2): 412–421.
  * Roess, Prassas, and McShane (2004) Roess, R. P.; Prassas, E. S.; and McShane, W. R. 2004.  _Traffic engineering_.  Pearson/Prentice Hall.
  * Su et al. (2021) Su, H.; Shi, K.; Chow, J. Y. J.; and Jin, L. 2021.  Dynamic Queue-Jump Lane for Emergency Vehicles under Partially Connected Settings: A Multi-Agent Deep Reinforcement Learning Approach.  arXiv:2003.01025.
  * Van der Pol and Oliehoek (2016) Van der Pol, E.; and Oliehoek, F. A. 2016.  Coordinated deep reinforcement learners for traffic light control.  _Proceedings of Learning, Inference and Control of Multi-Agent Systems (at NIPS 2016)_.
  * Varaiya (2013) Varaiya, P. 2013.  Max pressure control of a network of signalized intersections.  _Transportation Research Part C: Emerging Technologies_ , 36: 177–195.
  * Wang, Ma, and Yang (2013) Wang, J.; Ma, W.; and Yang, X. 2013.  Development of degree-of-priority based control strategy for emergency vehicle preemption operation.  _Discrete dynamics in nature and society_ , 2013.
  * Wang et al. (2013) Wang, Y.; Wu, Z.; Yang, X.; and Huang, L. 2013.  Design and implementation of an emergency vehicle signal preemption system based on cooperative vehicle-infrastructure technology.  _Advances in Mechanical Engineering_ , 5: 834976.
  * Wei et al. (2019a) Wei, H.; Chen, C.; Zheng, G.; Wu, K.; Gayah, V.; Xu, K.; and Li, Z. 2019a.  Presslight: Learning max pressure control to coordinate traffic signals in arterial network.  In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, 1290–1298.
  * Wei et al. (2019b) Wei, H.; Xu, N.; Zhang, H.; Zheng, G.; Zang, X.; Chen, C.; Zhang, W.; Zhu, Y.; Xu, K.; and Li, Z. 2019b.  Colight: Learning network-level cooperation for traffic signal control.  In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management_ , 1913–1922.
  * Wei et al. (2019c) Wei, H.; Zheng, G.; Gayah, V.; and Li, Z. 2019c.  A Survey on Traffic Signal Control Methods.  _arXiv preprint arXiv:1904.08117_.
  * Xu et al. (2021) Xu, B.; Wang, Y.; Wang, Z.; Jia, H.; and Lu, Z. 2021.  Hierarchically and Cooperatively Learning Traffic Signal Control.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, 669–677.
  * Zang et al. (2020) Zang, X.; Yao, H.; Zheng, G.; Xu, N.; Xu, K.; and Li, Z. 2020.  MetaLight: Value-Based Meta-Reinforcement Learning for Traffic Signal Control.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 34(01): 1153–1160.
  * Zheng et al. (2019) Zheng, G.; Xiong, Y.; Zang, X.; Feng, J.; Wei, H.; Zhang, H.; Li, Y.; Xu, K.; and Li, Z. 2019.  Learning phase competition for traffic signal control.  In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management_ , 1963–1972.
  * Ziliaskopoulos and Mahmassani (1993) Ziliaskopoulos, A. K.; and Mahmassani, H. S. 1993.  Time-dependent, shortest-path algorithm for real-time intelligent vehicle highway system applications.  In _Transportation Research Record 1408_ , 94–100.

##  Appendix S1 Pressure Definition Comparison

Here we present the key difference in pressure definition between our work and
PressLight (Wei et al. 2019a).

###  S1.1 Pressure in PressLight

PressLight assumes that traffic movements are lane-to-lane, i.e., vehicles in
one lane can only move into a particular lane in a link. Because of the lane-
to-lane assumption, in PressLight, the pressure is defined per movement.
PressLight defines the pressure of a movement as the difference of the vehicle
density between an incoming lane $l$ and the outgoing lane $m$, i.e.,

$w^{*}(l,m)=\frac{x(l)}{x_{max}(l)}-\frac{x(m)}{x_{max}(m)},$

PressLight then defines the pressure of an intersection $i$ as the absolute
value of the sum of pressure of movements of intersection $i$, i.e.,

$P^{*}_{i}=|\sum_{(l,m)\in\mathcal{M}_{i}}w^{*}(l,m)|,$

where $\mathcal{M}_{i}$ is the set of permissible traffic movements of
intersection $i$.

###  S1.2 Pressure in our work

EMVLight assumes a lane-to-link style traffic movement as vehicles can enter
either lane on the target link, see Fig 1. To present the pressure of an
intersection, we first define the pressure of an incoming lane as (Definition
4)

$w(l)=\left|\frac{x(l)}{x_{max}(l)}-\sum_{\\{m|(l,m)\in\mathcal{M}\\}}\frac{1}{h(m)}\frac{x(m)}{x_{max}(m)}\right|.$

The pressure of an intersection in EMVLight is defined as the average of the
pressure of all incoming lanes (Definition 5),

$P_{i}=\frac{1}{|\mathcal{I}_{i}|}\sum_{l\in\mathcal{I}_{i}}w(l),$

where $\mathcal{I}_{i}$ represents the set of all incoming lanes of
intersection $i$.

###  S1.3 Comparison

The first difference between the two definitions is that $w^{*}(l,m)$ can be
both positive or negative, but $w(l)$ can only take positive values that
measures the unevenness of the vehicle density in the incoming lane and that
of the corresponding outgoing lanes. We take the absolute value since the
direction of pressure is irrelevant here, and the goal of each agent is to
minimize this unevenness. The second difference is that at the intersection
level, $P^{*}_{i}$ takes a sum but $P_{i}$ takes an average. The average is
more suitable for our purpose since it scales the pressure down and the unit
penalty for normal agents would be relatively large as compared to rewards for
pre-emption agents (Eqn. (3)). This design puts the efficient passage of EMV
vehicles at the top priority. Our experimentation results indicate the
proposed pressure design produces a more robust reward signal during training
and outperforms PressLight in congestion reduction.

##  Appendix S2 Intralink EMV travel time

The intra-link traffic pattern with the presence of an EMV on duty is
complicated and is under-explored in the current literature. For simplicity,
here we demonstrate a simple intra-link traffic model for a link with 2 lanes.
The model can be easily extended for multiple lanes.

In a two-lane link, the EMV takes a lane and the non-EMVs on the other lane
usually slows down or entirely stop. Some non-EMVs ahead of the EMV find pull-
over spots in the other lane and park there. Those that cannot find a parking
spot continue to drive in front of the EMV, potentially blocking the EMV
passage (Su et al. 2021). In this study, we propose a meso-scopic model to
estimate the intra-link travel time of an EMV.

Figure S.1: Normal traffic state.

Normally, the traffic flow of a link is modeled by a fundamental diagram, see
Fig S.1. This diagram depicts a simplified relationship between the flow rate,
i.e. number of vehicles passing within the unit amount of the time, and number
of vehicles on a link. The max number of vehicles $H$ indicates the capacity
of this link. The critical number of vehicles $F$ indicates the boundary
differentiates the non-congested state and congested state. When the number of
vehicles is smaller than $F$, all vehicles are traveling at the free flow
speed, which is represented by the slope of $OB$. When number of vehicles is
larger than $F$, vehicles are slowing down and traffic flows declines since
the link is now congested. The max flow is attained when the number of
vehicles is at $F$. The travel speed of vehicles in a congested state $Q$ is
obtained by the slope of $OQ$.

Figure S.2: Traffic state during EMV pre-emption.

During the EMV pre-emption, the original traffic flow relationship pictured in
black are diverted into two parts, representing two lanes respectively. The
green line represents the traffic conditions of the pre-emption lane, i.e. the
lane where the EMV is traveling, and the orange line represents the other
lane. During pre-emption, part of the vehicles originally travelling in front
of the EMV pull over onto the adjacent lane, resulting a significant decrease
in the max capacities of the pre-emption lane. Meanwhile, because vehicles can
park onto the curbs, orange line depicts a larger maximum capacity $I$ than
normal.

Regarding the max flow, the adjacent lane has a smaller max flow as the
vehicles on this lane are required to slow down when EMV is on duty. The pre-
emption lane obtains a max capacity when there is one vehicle on the lane,
i.e. the EMV itself. Under this circumstance, the flow rate is equivalent as
the free flow speed of the EMV. However, when there are vehicles remaining in
front of the EMV, the travel speed of the EMV might be slowed down, but still
higher than the free flow speeds of the non-EMVs. Furthermore, the travel
speed of EMV has a discrete value corresponding to the number of non-EMVs
blocking. For example, pre-emption lane has a traffic state represented by
$M$, and the travel speed of the EMV is obtained as the slope of $M$. We use
this simple model, especially the green plot to estimate the intra-link travel
time of EMV in a link as a function of number of vehicles in that link.

We calibrate this model with the SUMO environment. Intuitively, the travel
speed of the EMV is affected more by the number of vehicles on the pre-emption
lane than their positions. The reason behind is since the ETA is frequently
updated every $\Delta T$ seconds ($\Delta T$ in our experiment), and the
estimated ETA would eventually converge. Imagine there are vehicles very far
away from the EMV and about to leave the intersections at $t$, their presence
would not slow down the approaching EMV. Therefore, when they have left the
intersection at $t+1$, we have an updated number of non-EMVs count and updated
travel speed estimation.

Input :

$T$ | maximum time step of an episode
---|---
$N_{\mathrm{bs}}$ | batch size
$\eta_{\theta}$ | learning rate for policy networks
$\eta_{\phi}$ | learning rate for value networks
$\alpha$ | spatial discount factor
$\gamma$ | (temporal) discount factor
$\lambda$ | regularizer coefficient

Output :

$\\{\phi_{i}\\}_{i\in\mathcal{V}}$ | learned parameters in value networks
---|---
$\\{\theta_{i}\\}_{i\in\mathcal{V}}$ | learned parameters in policy networks

1 initialize $\\{\phi_{i}\\}_{i\in\mathcal{V}}$,
$\\{\theta_{i}\\}_{i\in\mathcal{V}}$, $k\leftarrow 0$,
$B\leftarrow\varnothing$; initialize SUMO, $t\leftarrow 0$, get
$\\{s^{0}_{i}\\}_{i\in\mathcal{V}}$;

2 repeat

/* generate trajectories */

3 foreach  _$i\in\mathcal{V}$_ do (in parallel)

4 sample $a^{t}_{i}$ from $\pi^{t}_{i}$;

5 receive $\tilde{r}^{t}_{i}$ and $s^{t+1}_{i}$;

6

7 $B\leftarrow
B\cup\\{(s_{i}^{t},\pi_{i}^{t},a_{i}^{t},s_{i}^{t+1},r_{i}^{t})_{i\in\mathcal{V}}\\}$;

8 $t\leftarrow t+1$, $k\leftarrow k+1$;

9 if  _$t==T$_ then

10 initialize SUMO, $t\leftarrow 0$, get $\\{s^{0}_{i}\\}_{i\in\mathcal{V}}$;

11

/* update actors and critics */

12 if  _$k==N_{\mathrm{bs}}$_ then

13 foreach  _$i\in\mathcal{V}$_ do (in parallel)

14 calculate $\tilde{r}^{t}_{i}$ (Eqn. (4)), $\tilde{R}^{t}_{i}$ (Eqn. (5));

15 $\phi_{i}\leftarrow\phi_{i}-\eta_{\phi}\nabla\mathcal{L}_{v}(\phi_{i})$;

16
$\theta_{i}\leftarrow\theta_{i}-\eta_{\theta}\nabla\mathcal{L}_{p}(\theta_{i})$;

17

18 $k\leftarrow 0,B\leftarrow\varnothing$;

19

20until  _Convergence_ ;

Algorithm 2 Multi-agent A2C Training

##  Appendix S3 Training Details

###  S3.1 Value loss function

With a batch of data
$B=\\{(s_{i}^{t},\pi_{i}^{t},a_{i}^{t},s_{i}^{t+1},r_{i}^{t})_{i\in\mathcal{V}}^{t\in\mathcal{T}}\\}$,
each agent’s value network is trained by minimizing the difference between
bootstrapped estimated value and neural network approximated value

$\mathcal{L}_{v}(\phi_{i})=\frac{1}{2|B|}\sum_{B}\Big{(}\tilde{R}^{t}_{i}-V_{\phi_{i}}(\tilde{s}^{t}_{\mathcal{V}_{i}},\pi^{t-1}_{\mathcal{N}_{i}})\Big{)}^{2}.$
(S.1)

###  S3.2 Policy loss function

Each agent’s policy network is trained by minimizing its policy loss

$\displaystyle\mathcal{L}_{p}(\theta_{i})=-$
$\displaystyle\frac{1}{|B|}\sum_{B}\bigg{(}\log\pi_{\theta_{i}}(a_{i}^{t}|s^{t}_{\mathcal{V}_{i}},\pi^{t-1}_{\mathcal{N}_{i}})\tilde{A}^{t}_{i}$
(S.2)
$\displaystyle-\lambda\sum_{a_{i}\in\mathcal{A}_{i}}\pi_{\theta_{i}}\log\pi_{\theta_{i}}(a_{i}|s^{t}_{\mathcal{V}_{i}},\pi^{t-1}_{\mathcal{N}_{i}})\bigg{)},$
(S.3)

where
$\tilde{A}^{t}_{i}=\tilde{R}^{t}_{i}-V_{\phi_{i}^{-}}(\tilde{s}^{t}_{\mathcal{V}_{i}},\pi^{t-1}_{\mathcal{N}_{i}})$
is the estimated advantage which measures how much better the action
$a^{t}_{i}$ is as compared to the average performance of the policy
$\pi_{\theta_{i}}$ in the state $s_{i}^{t}$. The second term is a
regularization term that encourage initial exploration, where
$\mathcal{A}_{i}$ is the action set of agent $i$. For an intersection as shown
in Fig. 1, $\mathcal{A}_{i}$ contains 8 traffic signal phases.

###  S3.3 Training algorithm

Algorithm 2 shows the multi-agent A2C training process.

##  Appendix S4 Implementation Details

###  S4.1 Implementation details for synthetic $\text{grid}_{5\times 5}$

  * •

dimension of $s^{t}_{\mathcal{V}_{i}}$: $5\times(8+8+4+2)=110$

  * •

dimension of $\tilde{s}^{t}_{\mathcal{V}_{i}}$: $5\times(8+8+4+2)=110$

  * •

dimension of $\pi^{t-1}_{\mathcal{N}_{i}}$: $4\times 8=32$

  * •

Policy network
$\pi_{\theta_{i}}(a_{i}^{t}|s^{t}_{\mathcal{V}_{i}},\pi^{t-1}_{\mathcal{N}_{i}})$:
concat[$110\xrightarrow[]{\textrm{FC}}128$ReLu,
$32\xrightarrow[]{\textrm{FC}}64$ReLu] $\xrightarrow[]{}64$LSTM
$\xrightarrow[]{\textrm{FC}}8$Softmax

  * •

Value network
$V_{\phi_{i}}(\tilde{s}^{t}_{\mathcal{V}_{i}},\pi^{t-1}_{\mathcal{N}_{i}})$:
concat[$110\xrightarrow[]{\textrm{FC}}128$ReLu,
$32\xrightarrow[]{\textrm{FC}}64$ReLu] $\xrightarrow[]{}64$LSTM
$\xrightarrow[]{\textrm{FC}}1$Linear

  * •

Each link is $200m$. The free flow speed of the EMV is $12m/s$ and the free
flow speed for non-EMVs is $6m/s$.

  * •

Temporal discount factor $\gamma$ is $0.99$ and spatial discount factor
$\alpha$ is $0.90$.

  * •

Initial learning rates $\eta_{\phi}$ and $\eta_{\theta}$ are both 1e-3 and
they decay linearly. Adam optimizer is used.

  * •

MDP step length $\Delta t=5s$ and for secondary pre-emption reward weight
$\beta$ is $0.5$.

  * •

Regularization coefficient is $0.01$.

###  S4.2 Implementation details for $\text{Manhattan}_{16\times 3}$

The implementation is similar to the synthetic network implementation, with
the following differences:

  * •

Initial learning rates $\eta_{\phi}$ and $\eta_{\theta}$ are both 5e-4.

  * •

Since the avenues and streets are both one-directional, the number of actions
of each agent are adjusted accordingly.