Institute of Physics, Jan Kochanowski University, ul. Uniwersytecka 7, 25-406,
Kielce, Poland
Institute of Radiation Problems, Ministry of Science and Education,
B.Vahabzade St. 9, AZ1143, Baku, Azerbaijan

# The role of pomeron states in the Glueball Resonance Gas

E. Trottiins:x S. Jafarzadeins:xins:y
trottienrico@gmail.comshahriyar.jzade@gmail.comins:xins:xins:yins:y

###### Abstract

In this note, we present our recent analyses of the thermodynamic properties
of the glueball resonance gas. We observe that the dominant contribution to
the thermodynamic quantities, such as pressure, trace anomaly, and entropy, is
coming from the free glueball gas with the states of positive charge
conjugation (i.e., pomeron). A comparison of pomeron states obtained from LQCD
and functional methods within the glueball resonance gas model is also
presented.

##  1 Introduction

Quantum chromodynamics (QCD) can be described in the high-temperature regime
by a perturbative quark-gluon plasma, while in the low-temperature regime, by
a gas of weakly interacting hadrons. The two regimes are separated by the
pseudo-critical temperature $T_{c}$, which represents the confinement-
deconfinement cross-over transition of QCD whose value is still under debate
[1, 2, 3, 4, 5]. In the case of Yang-Mills (YM) (QCD without quarks), the
$T_{c}$ represents the transition (of the first order) between the bound
states of gluons (glueballs) and gas of gluons at low and high temperatures,
respectively.

At finite temperature, several approaches have been developed to study QCD [6,
7, 8, 9]. One typically compares the outcomes of the lattice QCD (LQCD) to the
hadron resonance gas model (HRG) below $T_{c}$. This procedure can be repeated
in the YM case. Instead of HRG, one has a subset of it, that we call a
Glueball Resonance Gas (GRG) [10].

The study on the pomeron and odderon trajectories suggests the existence of
glueballs [11, 12]. We observe that, within the GRG, the contribution provided
by the pomeron, $J^{P+}$, states is predominant, since the contribution to the
thermodynamic (TD) quantities, e.g., pressure and trace anomaly, is dominated
by the lightest resonances ($0^{++}$ and $2^{++}$).

We have studied the GRG in [10] by considering the mass spectra of glueballs
from Refs. [13, 14, 15]. In this work, we compare the results for the TD
quantities coming from the pomeron states obtained from the LQCD [15] and the
functional method [16].

##  2 Results

The TD of the GRG can be described by using the total, dimensionless energy
density $\hat{\epsilon}$

$\hat{\epsilon}=\sum_{i=1}^{N}\dfrac{(2J_{i}+1)}{T^{4}}\int_{0}^{\infty}\dfrac{k^{2}}{2\pi^{2}}\dfrac{\sqrt{k^{2}+m_{i}^{2}}}{\exp[\dfrac{\sqrt{k^{2}+m_{i}^{2}}}{T}\Biggr{]}-1}dk\text{
,}$ (1)

and the pressure $\hat{p}$:

$\hat{p}=-\sum_{i=1}^{N}\dfrac{(2J_{i}+1)}{T^{3}}\int_{0}^{\infty}\dfrac{k^{2}}{2\pi^{2}}\ln{\Big{(}1-e^{-\frac{\sqrt{k^{2}+m_{i}^{2}}}{T}}\Big{)}}dk\text{
,}$ (2)

where $J_{i}$ is the total spin of the $i$-th state. In the description of the
GRG, other two quantities are also relevant, i.e., the dimensionless trace
anomaly $\hat{I}$ and the entropy density $\hat{s}$:

$\hat{I}=\hat{\epsilon}-3\hat{p}\,,\quad\hat{s}=\hat{p}+\hat{\epsilon}\text{
.}$ (3)

These quantities play a central role in Ref. [10], where the comparison
between the LQCD TD data from Ref. [3] and GRG model constructed the lattice
spectra from [13, 14, 15] was performed. The results have shown that the GRG
with the most recent lattice work on the glueball masses [15] better describes
LQCD TD data. In Fig. 1, this comparison is shown for the normalized pressure,
with the addition of the statistical errors (not present in Ref. [10]).

Figure 1: Pressure of the GRG as a function of the temperature for three
different sets of the LQCD mass spectra [13, 14, 15], compared to the same
quantity evaluated in Ref. [3].

Even when considering the errors in the pressure, we confirm the masses from
Ref. [15] are favored. We remind that the TD results provided in Ref. [3] are
given in functions of $T/T_{c}$; thus, the same quantities calculated from the
GRG model must be presented for this ratio. However, lattice parameters used
to calculate the mass spectra in [13, 14, 15] imply different $T_{c}$ values
for each line in Fig 1. For more details, see Ref. [10].

In Ref [10], we included the excited states by using Regge trajectory fitted
from the glueballs with the quantum numbers
$J^{PC}=0^{++},2^{++},0^{-+},2^{-+},1^{+-}$ using the spectrum of Ref. [15].
The results do not significantly change, even with the addition of states up
to radial quantum number $n=10$. As we see from Fig. 2, among these states,
the ones with positive charge conjugation provide the main contribution to the
pressure, while those with C=-1 have a negligible effect up to the vicinity of
$T_{c}$.

Figure 2: Pressure obtained in Ref. [3] compared to the GRG by considering the
spectrum of Ref. [15] and all excited stated up to $n=10$ (red), the pomeron
($C=+$) states (black) and the odderon ($C=-$) states (cyan).

As an additional task, we compare the (favored) masses of [15] with the mass
predictions obtained with functional methods [16]. The comparison is possible
by neglecting the odderon contribution from [15]. In Table 1, the masses that
will be used in the GRG model are reported.

$n\,J^{PC}$ | M[MeV] |  $n\,J^{PC}$ | M[MeV]
---|---|---|---
|  Huber et al. [16] |  A & T [15] |  |  Huber et al. [16] |  A & T [15]
$\textbf{1\, 0}^{++}$ | 1850(130) | 1653(26) | $\textbf{1\,0}^{-+}$ | 2580(180) | 2561(40)
$\textbf{2\,0}^{++}$ | 2570(210) | 2842(40) | $\textbf{2\,0}^{-+}$ | 3870(120) | 3540(80)
$\textbf{1\,2}^{++}$ | 2610(180) | 2376(32) | $\textbf{1\,2}^{-+}$ | 2740(140) | 3070(60)
$\textbf{2\,2}^{++}$ | 3640(240) | 3300(50) | $\textbf{2\,2}^{-+}$ | 4300(190) | 3970(70)
$\textbf{1\,3}^{++}$ | 3370(50) | 3740(70) | $\textbf{1 4}^{++}$ | 4140(30) | 3690(80)
Table 1: The pomeron spectra reported in Refs. [15, 16].

In Refs. [15, 16], the values are reported using the same value of the lattice
parameter $r_{0}^{-1}=418(5)$ MeV. By considering the relation between $T_{c}$
and $r_{0}$ [17]:

$T_{c}=1.26(7)\cdot 0.614(2)\cdot r_{0}^{-1}\text{ ,}$ (4)

we obtain the common value $T_{c}=323\pm 18$ MeV. Three TD values (pressure,
trace anomaly and entropy) are shown in Fig. 3. One can see that the values
obtained using the GRG model with the masses from [16] are lower than those
from [15]. This is due mainly to the effect of the lightest states since a
slight increase in the mass of the $i$-th glueball is reflected in a sizeable
decrease in the TD quantities. However, in both cases, a slight discrepancy is
still present with the lattice TD results [3] (which include the total,
odderon, and pomeron contribution).

Figure 3: Pressure (top, left), trace anomaly (top, right), and entropy
(bottom) as a function of $T/T_{c}$. The green dashed, and magenta continuous,
plots (from the GRG model with the spectra (pomerons only) from [15] and [16]
respectively) are reported with errors due to the mass uncertainties. The
comparison is done with the lattice data from [3].

##  3 Conclusions

We revisited the results reported in Ref. [10] by considering the statistical
errors in the mass spectrum, confirming that GRG with the glueball spectrum
from [15] better describes the LQCD date from [3]. Among the states, the ones
with positive charge conjugation (pomeron) provide the dominant contribution.
Along this line, we present the comparison between the LQCD spectra of [15]
with the one observed by functional methods [16], and we evaluate TD
quantities for both pomeron spectra within GRG.

###### Acknowledgements.

The authors thank F. Giacosa for previous publications [10, 18] and for useful
comments. The authors acknowledge financial support through the project
AKCELERATOR ROZWOJU Uniwersytetu Jana Kochanowskiego w Kielcach (Development
Accelerator of the Jan Kochanowski University of Kielce), co-financed by the
European Union under the European Social Fund, with no.
POWR.03.05.00-00-Z212/18. The work of Shahriyar Jafarzade is partially
supported by the Polish National Science Centre (NCN) through the OPUS project
2019/33/B/ST2/00613.

## References

  * [1] M. Panero, Phys. Rev. Lett. 103 (2009), 232001 doi:10.1103/PhysRevLett.103.232001 [arXiv:0907.3719 [hep-lat]].
  * [2] B. Lucini and M. Panero, Phys. Rept. 526 (2013), 93-163 doi:10.1016/j.physrep.2013.01.001 [arXiv:1210.4997 [hep-th]].
  * [3] S. Borsanyi, G. Endrodi, Z. Fodor, S. D. Katz and K. K. Szabo, JHEP 07 (2012), 056 doi:10.1007/JHEP07(2012)056 [arXiv:1204.6184 [hep-lat]].
  * [4] M. Caselle, L. Castagnini, A. Feo, F. Gliozzi and M. Panero, JHEP 06 (2011), 142 doi:10.1007/JHEP06(2011)142 [arXiv:1105.0359 [hep-lat]].
  * [5] M. Caselle, A. Nada and M. Panero, JHEP 07 (2015), 143 [erratum: JHEP 11 (2017), 016] doi:10.1007/JHEP07(2015)143 [arXiv:1505.01106 [hep-lat]].
  * [6] A. Pilaftsis and D. Teresi, Nucl. Phys. B 874 (2013) no.2, 594-619 doi:10.1016/j.nuclphysb.2013.06.004 [arXiv:1305.3221 [hep-ph]].
  * [7] A. Koenigstein, M. J. Steil, N. Wink, E. Grossi, J. Braun, M. Buballa and D. H. Rischke, Phys. Rev. D 106 (2022) no.6, 065012 doi:10.1103/PhysRevD.106.065012 [arXiv:2108.02504 [cond-mat.stat-mech]].
  * [8] W. Broniowski, F. Giacosa and V. Begun, Phys. Rev. C 92 (2015) no.3, 034905 doi:10.1103/PhysRevC.92.034905 [arXiv:1506.01260 [nucl-th]].
  * [9] S. Samanta and F. Giacosa, Phys. Rev. D 107 (2023) no.3, 036001 doi:10.1103/PhysRevD.107.036001 [arXiv:2110.14752 [hep-ph]].
  * [10] E. Trotti, S. Jafarzade and F. Giacosa, Eur. Phys. J. C 83 (2023) no.5, 390 doi:10.1140/epjc/s10052-023-11557-0 [arXiv:2212.03272 [hep-ph]].
  * [11] I. Szanyi, L. Jenkovszky, R. Schicker and V. Svintozelskyi, Nucl. Phys. A 998 (2020), 121728 doi:10.1016/j.nuclphysa.2020.121728 [arXiv:1910.02494 [hep-ph]].
  * [12] A. A. Godizov, Eur. Phys. J. C 76 (2016) no.7, 361 doi:10.1140/epjc/s10052-016-4229-z [arXiv:1604.01689 [hep-ph]].
  * [13] Y. Chen, A. Alexandru, S. J. Dong, T. Draper, I. Horvath, F. X. Lee, K. F. Liu, N. Mathur, C. Morningstar and M. Peardon, et al. Phys. Rev. D 73 (2006), 014516 doi:10.1103/PhysRevD.73.014516 [arXiv:hep-lat/0510074 [hep-lat]].
  * [14] H. B. Meyer, [arXiv:hep-lat/0508002 [hep-lat]].
  * [15] A. Athenodorou and M. Teper, JHEP 11 (2020), 172 doi:10.1007/JHEP11(2020)172 [arXiv:2007.06422 [hep-lat]].
  * [16] M. Q. Huber, C. S. Fischer and H. Sanchis-Alepuz, Eur. Phys. J. C 81 (2021) no.12, 1083 [erratum: Eur. Phys. J. C 82 (2022), 38] doi:10.1140/epjc/s10052-021-09864-5 [arXiv:2110.09180 [hep-ph]].
  * [17] M. Gockeler, R. Horsley, A. C. Irving, D. Pleiter, P. E. L. Rakow, G. Schierholz and H. Stuben, Phys. Rev. D 73 (2006), 014513 doi:10.1103/PhysRevD.73.014513 [arXiv:hep-ph/0502212 [hep-ph]].
  * [18] F. Giacosa, A. Pilloni and E. Trotti, Eur. Phys. J. C 82 (2022) no.5, 487 doi:10.1140/epjc/s10052-022-10403-z [arXiv:2110.05582 [hep-ph]].


# Where is the base of the Transition Region? Evidence from TRACE, SDO, IRIS
and ALMA observations 11footnotemark: 1

Costas E. Alissandrakis<EMAIL_ADDRESS>Department of Physics, University of
Ioannina, GR45110 Ioannina, Greece

###### Abstract

Classic solar atmospheric models put the Chromosphere-Corona Transition Region
( CCTR) at $\sim 2$ Mm above the $\tau_{5000}=1$ level, whereas radiative MHD
(rMHD) models place the CCTR in a wider range of heights. However,
observational verification is scarce. In this work we review and discuss
recent results from various instruments and spectral domains. In SDO and TRACE
images spicules appear in emission in the 1600, 1700 and 304 Å bands and in
absorption in the EUV bands; the latter is due to photo-ionization of H i and
He i, which increases with wavelength. At the shortest available AIA
wavelength and taking into account that the photospheric limb is $\sim 0.34$
Mm above the $\tau_{5000}=1$ level, we found that CCTR emission starts at
$\sim 3.7$ Mm; extrapolating to $\lambda=0$, where there is no chromospheric
absorption, we deduced a height of $3.0\pm 0.5$ Mm, which is above the value
of 2.14 Mm of the Avrett and Loeser model. Another indicator of the extent of
the chromosphere is the height of the network structures. Height differences
produce a limbward shift of features with respect to the position of their
counterparts in magnetograms. Using this approach, we measured heights of
$0.14\pm 0.04$ Mm (at 1700 Å), $0.31\pm 0.09$ Mm (at 1600 Å) and $3.31\pm
0.18$ Mm (at 304 Å) for the center of the solar disk. A previously reported
possible solar cycle variation is not confirmed. A third indicator is the
position of the limb in the UV, where IRIS observations of the Mg ii triplet
lines show that they extend up to $\sim 2.1$ Mm above the 2832 Å limb, while
AIA/SDO images give a limb height of $1.4\pm 0.2$ Mm (1600 Å) and $5.7\pm 0.2$
Mm (304 Å). Finally, ALMA mm-$\lambda$ full-disk images provide useful
diagnostics, though not very accurate, due to their relatively low resolution;
values of $2.4\pm 0.7$ Mm at 1.26 mm and $4.2\pm 2.5$ Mm at 3 mm were
obtained. Putting everything together, we conclude that the average
chromosphere extends higher than homogeneous models predict, but within the
range of rMHD models..

###### keywords:

Sun, Chromosphere-corona transition region, Solar atmospheric models

††journal: Advances in Space Research

##  1 Introduction

The direct measurement of the height of the solar emission in various spectral
regions is very important for the computation and verification of atmospheric
models. In physical terms, the formation height of the radiation is determined
by the local conditions and radiative transfer. We note that in classic
empirical atmospheric models the height is not a basic parameter, but it is
derived from the optical depth and the absorption coefficient (see, e.g.,
Zirin, 1966).). The formation height of the emission at a particular
wavelength is computed from the emissivity and opacity, which depend on the
atmospheric model (see, e.g. Figure 1 of Vernazza et al., 1976).

Fig. 1:  Left: The electron temperature as a function of height (red line)
according to model C7 of Avrett & Loeser, 2008. The contribution to the
continuum intensity is plotted in blue for the short radio wavelengths (below
the red horizontal line) and the extreme ultraviolet wavelengths, marked at
the right. Adapted from Fig. 2 of Avrett & Loeser, 2008.. Right: Probability
density function of the temperature as function of height from Bifrost
simulations of an enhanced network region. The vertical dashed line is at 2.14
Mm. Adapted from the lower panel of Fig. 10 of Carlsson et al., 2016.

Classical semi-empirical models assume that the solar atmosphere is
horizontally homogeneous and in hydrostatic equilibrium, with an additional ad
hoc turbulent pressure to set the extent of the chromosphere; they ignore mass
motions and the associated dynamic pressure, as well as Lorentz forces which
produce the rich horizontal structure that is more prominent in the upper
layers. Multi-component models take into the account differences between the
network and internetwork regions, but still ignore the radiative transfer in
the horizontal direction. Nevertheless, they represent a reasonable
approximation of the one-dimensional structure of the solar atmosphere. Such
models put the base of the Chromosphere-Corona Transition Region ( CCTR) at a
height of about 2 000 km (i.e. a mere 3′′) above the $\tau_{\rm 5000}=1$ level
(e.g. Vernazza et al., 1981; Fontenla et al., 1990, 1991, 1993; Avrett &
Loeser, 2008; Avrett et al., 2015; also left panel of Fig. 1). This is well
below the top of spicules, which protrude from the chromosphere into the
corona, to a height that may exceed 15′′ (e.g. Alissandrakis et al., 2018).

Thanks to the development of fast numerical computations, a number of
sophisticated tools, such as the Bifrost radiative magnetohydrodynamics (rMHD)
code (Gudiksen et al., 2011) and the STockholm inversion Code (STiC, de la
Cruz Rodríguez et al., 2019) have been developed for solar atmospheric
modeling. Such models demonstrate well the complex structure and the dynamics
of the upper layers of the solar atmosphere. An example of the deduced
temperature structure is given in the right panel of Fig. 1, from a simulation
of an enhanced network region by Carlsson et al. (2016). It is interesting
that transition region temperatures, from various features, appear in an
extended range of heights that go beyond 2.14 Mm (dashed line). Although this
particular model may not be representative of the average QS, which includes
inter-network regions, it is still indicative. Nevertheless, the classic
models still provide a reference picture of the solar atmosphere.

Observational evidence for the position of the base of the CCTR came
originally from eclipse data, and is based on the relative timing between the
instance that the lunar disk covers the photosphere and the instance at which
coronal lines appear after the occultation of the chromosphere by the moon
(Zirin, 1966). Subsequently, Avrett & Loeser (2008) placed the top of the
chromosphere at 2140 km, roughly consistent with the maximum intensity of the
10833 Å He i and the the 5875 Å D3 lines.

The observable quantity most directly associated with the height of formation
is the solar radius. Such measurements were compiled by Rozelot et al. (2015)
for the optical and extreme ultraviolet (EUV) and by by Menezes & Valio (2017)
for the microwave range (see also Battaglia et al., 2017 for a measurement in
X-rays). We note that direct measurements beyond the limb in the EUV and the
X-rays are difficult, due to the low spatial resolution of the older space-
born instruments. Daw et al. (1995) used the Normal Incidence X-ray Telescope
(NIXT) to measure the position of the limb at 63.5 Å with a spatial resolution
of $\sim 1$′′, while Zhang et al. (1998) used the Extreme-Ultraviolet Imaging
Telescope (EIT) aboard the Solar and Heliospheric Observatory, (SoHO) at 171,
195, 284 and 304 Å with $\sim 3$′′ resolution.

Although both the Transition Region and Coronal Explorer (TRACE) and the Solar
Dynamics Observatory (SDO) have sufficient resolution (of the order of 1′′),
the measurement requires precise relative pointing among white light (WL)
observations and observations in the EUV channels. In a recent work
(Alissandrakis & Valentino, 2019b, hereafter Paper I; see also Alissandrakis &
Valentino, 2019a for a correction), we used the transits of Mercury and Venus
to obtain accurate pointing information and, subsequently, we measured the
height of the limb with respect to the photosphere from TRACE and Atmospheric
Imaging Assembly (AIA) images. The pointing problem does not exist in the case
of the Interface Region Imaging Spectrograph (IRIS), where the position of the
photospheric limb can be measured directly in the far wing of the k line, and
Alissandrakis et al. (2018) gave values of the limb height for a number of
spectral lines and continua with respect to the limb at 2832 Å.

On the solar disk, height differences translate to shifts between features
observed at different wavelengths. Such measurements were performed by
Alissandrakis (2019), hereafter paper II, where the height of network
structures seen in AIA images at 1600 Å, 1700 Å and 304Å̇ with respect to the
associated magnetic structures HMI images was computed.

Fig. 2: TRACE images near the limb in polar projection, in white light (top),
the 1600 Å band and the 171 Å band, after pointing correction. The dark disk
is Mercury. From Alissandrakis & Valentino (2019b), reprinted by permission
from: Springer Nature, Solar Physics, ©2019.

At the other side of the spectrum, microwave and mm-$\lambda$ observations
near the limb give important information on the CCTR height and structure. In
this range the opacity, and hence the limb height, increases with wavelength,
but the situation is complicated by the low resolution of full-disk
observations and the presence of spicules. A compilation of results from
Atacama Large Millimeter/submillimeter Array (ALMA) full-disk images and from
AIA UV bands was given in Table 4 of Alissandrakis et al. (2020). According to
that, mm-$\lambda$ radiation forms between the levels of 1600 Å and 304 Å
emission, while limb heights of $2.4\pm 0.7$ Mm and $4.2\pm 2.5$ Mm were
reported at 1.26 mm and 3 mm respectively.

An important point to note is that the white light limb is above the
$\tau_{5000}=1$ level, which is the reference point for zero height. Athay
(1976) gave a height difference of 340 km in his Table I-1, whereas Thuillier
et al. (2011) gave similar values: 314 to 346 km at 4000 Å, depending on the
model used (their Table 5), with a slight increase with wavelength.
Consequently, a correction must be applied to the measured values in order to
compare them with model predictions.

In Section 2 of this work we present a brief summary of our previous results.
Subsequently, in Sect. 3 we investigate the variation of the network height
during the solar cycle and in Sect. 4 we discuss the combined results.

Fig. 3: The azimuthally averaged intensity, as a function of distance from the
white-light limb for the three TRACE wavelength bands shown in Fig. 2. Curves
in red show the rms of the respective average values, which is a measure of
the azimuthal intensity variation at each radial position. Adapted from
Alissandrakis & Valentino (2019b).

##  2 Review of previous results

###  2.1 Measurements of the location of the CCTR

Beyond the photospheric limb, spicules are well visible in the TRACE UV images
(Alissandrakis et al., 2005) and in the SDO 1600 Å, 1700 Å and 304 Å band
images; they are also seen as absorbing features in the EUV images of both
instruments, where they give a rugged appearance to the lower boundary of the
corona near the limb. This is demonstrated in Fig. 2, which shows TRACE images
near the limb, corrected for pointing using the position of Mercury, in the
white light (WL), 1600 and 171 Å bands.

Fig.3 shows plots of the azimuthally averaged intensity for the three
wavelength bands displayed in Fig. 2. The position of the photospheric limb,
used as a reference for zero height, can be easily measured from the
inflection point of the corresponding curve. In the 1600 Å emission, which
comes from the lower chromosphere around the temperature minimum, the curve
shows limb brightening that peaks slightly above the photospheric limb,
whereas the 171 Å emission, formed in the CCTR, the peak is about 6 Mm above
the WL limb. The inflection point of the intensity curve before the peak
defines the inner limb, and corresponds to the height where the spicule forest
becomes transparent, so that emission form the back side of the plane of the
sky can reach the observer.

Measurements of the inner limb position in all AIA high-temperature bands (94,
131, 171, 193, 211 and 335 Å) were performed in Paper I, after small pointing
and image scale corrections derived from the position of Venus during its
transit in June 5-6, 2012. The height of the inner limb measured from the
$\tau_{5000}=1$ level is plotted as a function of wavelength in Fig. 4, for
regions around the north, south, east and west limbs.

The position of the inner limb places an upper limit to the height of the base
of the CCTR, as the presence of spicules does not allow a direct measurement
of the average height of the top of the homogeneous chromosphere. This upper
limit, deduced from the minimum height of the inner limb at 94 Å, is about 3.7
Mm above the $\tau_{5000}=1$ level.

Spicule absorption at wavelengths shorter than the hydrogen Lyman-continuum
head is attributed to photoionization, principally of neutral hydrogen (H i)
and neutral or singly-ionized helium (He i and He ii), with the absorption
coefficient increasing with wavelength; this is in conformity with our results
of Fig. 4, which show that the height of the inner limb increases with
wavelength. Therefore, a better estimate of the height of the CCTR can be
obtained from the extrapolation of the regression line of the average inner
limb height versus wavelength plot (Fig. 4) to to low $\lambda$ values, where
there is negligible chromospheric absorption. This gives a height of ($3.0\pm
0.5$) Mm, where the height of the WL limb with respect to the $\tau_{5000}=1$
level has been taken into account; this range is above the value of 2.140 Mm
used by Avrett & Loeser (2008).

Fig. 4: The height of the inner limb with respect to the $\tau_{5000}=1$ level
for the high temperature AIA channels as a function of wavelength. The
straight lines show the result of a linear regression. From Alissandrakis &
Valentino (2019a), reprinted by permission from: Springer Nature, Solar
Physics, ©2019.

We note that the linear fit of the variation of height with wavelength in Fig.
4 is a reasonable approximation, as a full computation of the height as a
function of wavelength requires knowledge of H and He densities and degrees of
ionization that we do not have. We further note that 212 Å is below the
ionization limit of He ii, and that the greater value of height in this band
could be attributed to the associated opacity jump; however, our computations
showed that the opacity jump is small, 6% to 15%, for reasonable degrees of He
ionization of 10% and 20% respectively (see also Table 3 in Anzer & Heinzel,
2005).

###  2.2 Measurements of the height of network structures

In Paper II, we measured the shift of network structures in AIA 1600, 1700 and
304 Å images with respect to the corresponding features in HMI magnetograms,
at several distances from the disc center; the shift was measured by cross-
correlation, after dividing each image into 12 sectors, 30∘ wide; each sector
was further divided into a number of zones, 60′′ wide, from just inside the
limb down to a distance of about 0.25 solar radii from the center of the disk.
The absolute value of the longitudinal magnetic field was used for the cross-
correlation.

Fig. 5: The radial shift of network structures with respect to the
corresponding magnetic features as a function of position angle for three
locations on the solar disk. The distance from the center of the disk in solar
radii, $r$, and the corresponding cosine of the heliocentric angle, $\mu$, are
marked above each panel. The horizontal lines mark the shift averaged over the
position angle. Black asterisks and full lines are for 1700 Å, red circles and
dotted lines for 1600 Å, and blue crosses and dash-dotted lines for 304 Å.
Adapted from paper II.

Representative results are shown in Fig. 5. There is no systematic variation
with position angle, which points against any pole-equator differences; thus
the scatter of the data points reflects measurement errors. It is obvious that
the emission in the AIA 304 Å band forms well above the that in the 1600 Å
band, and that the latter forms slightly above the 1700 Å band. As expected,
the shifts are larger near the limb. Similar results were obtained by Rutten
(2020).

Fig. 6: The height of network emission, plotted as a function of distance from
the center of the disk (left column) and as a function of the cosine of the
heliocentric angle, $\mu$. The reference wavelength and the image wavelength
are marked above each panel. Solid lines show the result of linear regression
for 304 Å and a quadratic regression for the other wavelengths. Adapted from
paper II.

The network height, derived from the shifts, is plotted in Fig. 6, as a
function of distance from the center of the disk (left column) and as a
function of the cosine of the heliocentric angle, $\mu$, (right column). The
plots show a well-defined increase of the network height from the center of
the disk to the limb, in all three wavelength bands. This is apparently due to
opacity effects, as the radiation near the limb is formed higher than near the
disk center. The lowest height is that of the 1700 Å network, just 0.14 Mm
above the HMI continuum level at the disk center, whereas the 304 Å network
height is 3.3 Mm, again at the disk center.

###  2.3 Limb position from IRIS data

As mentioned in the introduction, Alissandrakis et al. (2018) measured the
limb height for a number of spectral lines and continua from IRIS spectra,
using as reference the position of the limb at 2832 Å. This is the wavelength
of maximum intensity in the near ultraviolet (NUV) IRIS spectral band, located
in the far red wing of the k line, but still below the local continuum level.
A plot of the height as a function of wavelength for the three spectral
windows of IRIS NUV band is shown in Fig. 7; it was computed from averaged
spectra, corrected for light scattered beyond the limb.

Fig. 7: The height of the limb as a function of wavelength for the three
spectral windows in the NUV spectral band of IRIS. The height was measured
with respect to the limb position at 2832 Å. From Alissandrakis et al. (2018),
reprinted by permission from: Springer Nature, Solar Physics, ©2018.

In addition to the Mg ii k and h lines at 2796.32 and 2803.58 Å respectively,
many weaker lines extend above the height of the local continuum limb in this
plot. Of particular interest are the Mg ii triplet lines at 2791.59 and
2798.79 Å, not only because they extend more than 2′′ above the 2832 Å limb,
but also because their variation with time does not reveal any spicular
structure, which is very prominent in the k and h lines (Fig. 8). This may
suggest the presence of an interspicular region in the chromosphere and/or
that spicules are formed above the height of 2′′, the chromosphere being more
homogeneous up to that height.

Fig. 8: The intensity as a function of wavelength and time in the region of k
and h lines, at a distance of 2′′ above the limb. The width of the cuts is
0.5′′. The arrows point to the Mg ii triplet lines. From Alissandrakis et al.
(2018), reprinted by permission from: Springer Nature, Solar Physics, ©2018.

##  3 Variation of the network height with the solar cycle

In this section we investigate further a possible solar cycle variation of the
network height, reported in paper II, based on the analysis of 6 observations.
To this end, we expanded the data set by adding 7 more dates, so that we now
cover the full interval of SDO observations from 2010 to the present, with
about one observation per year.

Fig. 9: The height network structures as a function of time for a period
slightly longer than a solar cycle, from SDO data. Horizontal lines mark the
average value.

Fig. 9 shows the network height as a function of time, extrapolated to the
solar limb, for the 1600 Å and 304Å̇ network structures with respect to
continuum magnetic features, as well as that of the 304Å̇ network with respect
to 1600 Å. As in paper II, the values that were obtained by comparison to HMI
magnetograms were corrected for the 0.34 Mm shift between magnetic and
intensity features near the limb. We note that variations are small, with no
apparent association with the solar cycle.

Table 1: Network formation height and limb position with respect to white light Wavelength | Disk center | Near the limb | Limb position
---|---|---|---
Å | Mm | Mm | Mm
1700 | $0.14\pm 0.04$ | $0.24\pm 0.11$ | $0.72\pm 0.04$
1600 | $0.31\pm 0.09$ | $0.50\pm 0.08$ | $1.13\pm 0.25$
304 | $3.31\pm 0.18$ | $4.06\pm 0.20$ | $9.25\pm 2.62$

As the extended data set gives more precise values for the network height, we
present the new results in Table 1, where we have added a column with the
position of the (outer) limb measured in paper I. We note that the latter is
clearly above the level of the network near the limb, an effect that should be
attributed to spicules. For reference, we note that Fossum & Carlsson (2005)
computed the formation height of the TRACE 1700 and 1600 Å wavelength bands at
0.36 and 0.43 Mm, with widths of 0.325 and 0.185 Mm respectively.

##  4 Discussion and conclusions

The measurement of the position of the CCTR is based on the implicit
assumption that the notion of the quiet sun (QS) is valid. We know very well
that the quiet sun is not that quiet, with a multitude of small scale
transient events and with structures like spicules extending into the corona,
having their own CCTR. Such phenomena can be considered as perturbations on
top of the QS background; alternatively, the QS can be considered as the
spatial, over the horizontal dimension, and temporal average of the actual
physical conditions.

From the observational point of view, as measurements in different spectral
bands are required, an important issue is the precision of the image scale.
This effect is minimized when measurements are performed with the same
instrument, e.g. with TRACE, IRIS or the AIA/HMI combination in SDO. It goes
without saying that accurate pointing is required and that position
measurements with respect to the WL limb are less susceptible to image scale
errors than those with respect to the center of the solar disk.

Among the observations discussed in this work, the one that uses the position
of the inner limb in EUV AIA images, developed in paper I, provides the most
direct measurement of the height of the base of the transition region. The
principal uncertainty of this approach is the correction for spicular
absorption; without this correction we have an upper limit of 3.7 Mm, which
drops to 3 Mm after the correction is applied. This is clearly above the
classical value of $\sim 2$ Mm, but is is well within the range predicted by
rMHD models, such as that of Carlsson et al. (2016).

In order to put in the same framework the measurements of the formation height
of the network, we should consider that the network emission is not formed at
the same level as the average QS emission. Multi-component atmospheric models
were first introduced by Vernazza et al. (1981), to describe specific
atmospheric structures, such as the network and inter-network regions. Such
models place the CCTR of each component at a different height, for example in
the models of Fontenla et al. (1993) the base of the CCTR for the network is
at 2.25 Mm, while it is at 2.02 Mm for the inter-network and 2.20 Mm for the
average QS. These differences are small and the formation height differences
in various spectral bands are expected to be also small. Moreover, these
models predict too much network/internetwork contrast at mm-$\lambda$,
compared to ALMA results (Alissandrakis et al., 2020), which implies that
network/internetwork differences are even smaller than the model predictions.
We may thus consider that the network shift results are a fair approximation
of the average QS formation height. In any case, we should bear in mind that
the radiation forms in an extended height range, as shown in the left panel of
Fig. 1.

Direct measurements of the height of the solar limb in lines and continua that
do not show spicular structure, such as the the Mg ii triplet lines, place a
lower limit to the height of the chromosphere. Still, as the region of
formation of the radiation has a certain vertical extent, the position of the
limb may represent the upper bound of this region rather than the average. In
IRIS measurements, the uncertainty comes from the fact that the height is
measured with respect to the quasi-continuum at 2832 Å, which should be formed
slightly above the true continuum.

The situation is more complicated in lines and continua that show spicules. In
such cases the height of the limb reflects the average height of the spicule
forest rather than that of the homogeneous chromosphere. Measurements with
ALMA fall in this group, where, in addition, the low spatial resolution of
full-disk images limits the accuracy of the measurements.

Fig. 10: The height of the network emission (top) and the position of the limb
(bottom) in Mm with respect to $\tau_{5000}=1$ level from various data sets
discussed in this article. The position of the base of the CCTR from paper I,
as well as that of the temperature minimum according to the model of Fontenla
et al. (1993) are also marked.

With all of the above considerations in mind, we have put together all the
relevant observational results in Fig. 10, where zero height is at the level
of $\tau_{5000}=1$. The top panel shows the network height near disk center
and near the limb from Table 1, assuming that the network forms at the same
height as the average quiet sun and adding the 0.34 Mm difference between the
$\tau_{5000}=1$ and the limb to the network height near the limb. In the same
plot we included the limb position from paper I, for reference; for 304 Å we
put the height of the peak intensity, rather than that of the outer limb. As
noted in Sect. 3, the limb is well above the network, due to spicules. Another
interesting remark is that, even at the center of the solar disk, the 304 Å
network is above the base of the CCTR. We note the AIA 304 Å channel is
dominated by the He ii lines which form around $T_{e}=50\,000$ K, but this is
very close to the top of the chromosphere (about 0.01 Mm higher in the C7
model), as the CCRT is very steep in this temperature range. This might imply
that the CCTR is higher than estimated in paper I, but a more plausible
explanation is that it is due to the effect of spicules, which have a very
large optical depth at 304 Å, as evidenced in limb images where they extend
much higher than their counterparts in other wavelengths (see, e.g. Fig. 9 in
paper I). Their high optical depth makes them opaque on the disk, so that what
we see there is not the network, but rather the base of spicules that are
located on top of the network structures.

In the bottom panel of Fig. 10 we see that the Mg ii triplet lines are
comfortably below the base of the CCTR, and above the 1700/1600 Å limb. At
mm-$\lambda$, the limb at 3 mm is above, while at 1.3 mm it is below the CCTR;
we should note that, due to the low resolution of the full-disk images, the
values have a large uncertainty: $\pm 2.5$ and $\pm 1.$4 Mm respectively. We
add that numerical simulations under local thermodynamic equilibrium by
Martínez-Sykora et al. (2020) gave average heights of 0.9 Mm (with a standard
deviation of 0.7 Mm) for 1.2 mm and 1.8 Mm (standard deviation of 1 Mm) at 3
mm, whereas computations by the same authors under non-equilibrium hydrogen
ionization gave greater heights and a much smaller height difference: $2.67\pm
1.08$ Mm for 1.2 mm emission and $2.78\pm 1.09$ Mm for 3 mm.

All things considered, the height of the base of the CCTR reported in paper I,
$3.0\pm 0.5$ Mm, which is consistent with the other observational results
presented here, is definitely above the height predicted by classic
homogeneous models and broadly consistent with the predictions of rMHD models.
Possible pole-equator differences, together with the effect of coronal holes,
require further investigation. In paper I we found a pole/equator height ratio
from CCTR images of just $1.08\pm 0.02$, while we confirmed the prolate shape
of the chromosphere, reported by Auchère et al. (1998), in the 304 Å images;
no pole/equator effect was found in the network height. As for solar cycle
variations, our extended data set did not confirm the possible variation of
the the network height reported in paper II. Concerning solar cycle variations
of the height of the CCTR, the question is open.

##  5 Acknowledgments

The data used in his work were retrieved from the data bases of TRACE, SDO,
IRIS and ALMA. The author is grateful to all those that worked for the
development and operation of the instruments and made the data freely
available to the community. Thanks are also due to the reviewers for their
comments that helped the author to improve the quality of this article.

##  6 References

## References

Alissandrakis, C. E. (2019). Measurement of the Height of the Chromospheric
Network Emission from Solar Dynamics Observatory Images (Paper II). Solar
Phys., 294(11), 161. doi:10.1007/s11207-019-1552-1. 381 arXiv:1911.00758.

Alissandrakis, C. E., Nindos, A., Bastian, T. S. et al. (2020). Modeling the
quiet Sun cell and network emission with ALMA. Astron. Astrophys., 640, A57.
doi:10.1051/0004-6361/202038461. arXiv:2006.09886.

Alissandrakis, C. E., & Valentino, A. (2019a). Erratum: Correction to:
Structure of the Transition Region and the Low Corona from TRACE and SDO
Observations Near the Limb. Solar Phys., 294(10), 146.
doi:10.1007/s11207-019-1545-0.

Alissandrakis, C. E., & Valentino, A. (2019b). Structure of the Transition
Region and the Low Corona from TRACE and SDO Observations Near the Limb (Paper
I). Solar Phys., 294(7), 96. doi:10.1007/s11207-019-1486-7. arXiv:1906.09497.

Alissandrakis, C. E., Vial, J. C., Koukras, A. et al. (2018). IRIS
Observations of Spicules and Structures Near the Solar Limb. Solar Phys.,
293(2), 20. doi:10.1007/s11207-018-1242-4. arXiv:1801.02082.

Alissandrakis, C. E., Zachariadis, T., & Gontikakis, C. (2005). Trace
Observations of Solar Spicules Beyond the Limb in Ly$\alpha$ and Civ. In D.
Danesy, S. Poedts, A. de Groof, & J. Andries (Eds.), The Dynamic Sun:
Challenges for Theory and Observations. volume 11 of ESA Special Publication.
Published on CDROM, id. 54.1.

Anzer, U., & Heinzel, P. (2005). On the Nature of Dark Extreme Ultraviolet
Structures Seen by SOHO/EIT and TRACE. Astrophys. J., 622(1), 714–721.
doi:10.1086/427817.

Athay, R. G. (1976). The solar chromosphere and corona: Quiet sun. volume 53
of Astronomy Space Science Library Series. doi:10.1007/978-94-010-1715-2.

Auchère, F., Boulade, S., Koutchmy, S. et al. (1998). The prolate solar
chromosphere. Astron. Astrophys., 336, L57–L60.

Avrett, E., Tian, H., Landi, E. et al. (2015). Modeling the Chromosphere of a
Sunspot and the Quiet Sun. Astrophys. J., 811(2), 87.
doi:10.1088/0004-637X/811/2/87.

Avrett, E. H., & Loeser, R. (2008). Models of the Solar Chromosphere and
Transition Region from SUMER and HRTS Observations: Formation of the Extreme-
Ultraviolet Spectrum of Hydrogen, Carbon, and Oxygen. Astrophys. J. Suppl.,
175(1), 229–276. doi:10.1086/523671.

Battaglia, M., Hudson, H. S., Hurford, G. J. et al. (2017). The Solar X Ray
Limb. Astrophys. J., 843(2), 123. doi:10.3847/1538-4357/aa76da.
arXiv:1705.11044.

Carlsson, M., Hansteen, V. H., Gudiksen, B. V. et al. (2016). A publicly
available simulation of an enhanced network region of the Sun. Astron.
Astrophys., 585, A4. doi:10.1051/0004-6361/201527226. arXiv:1510.07581.

Daw, A., Deluca, E. E., & Golub, L. (1995). Observations and Interpretation of
Soft X-Ray Limb Absorption Seen by the Normal Incidence X-Ray Telescope.
Astrophys. J., 453, 929–944. doi:10.1086/176453.

de la Cruz Rodríguez, J., Leenaarts, J., Danilovic, S. et al. (2019). STiC: A
multiatom non-LTE PRD inversion code for full-Stokes solar observations.
Astron. Astrophys., 623, A74. doi:10.1051/0004-6361/201834464.
arXiv:1810.08441.

Fontenla, J. M., Avrett, E. H., & Loeser, R. (1990). Energy Balance in the
Solar Transition Region. I. Hydrostatic Thermal Models with Ambipolar
Diffusion. Astrophys. J., 355, 700–718. doi:10.1086/168803.

Fontenla, J.M., Avrett, E. H., & Loeser, R. (1991). Energy Balance in the
Solar Transition Region. II. Effects of Pressure and Energy Input on
Hydrostatic Models. Astrophys. J., 377, 712–725. doi:10.1086/170399.

Fontenla, J. M., Avrett, E. H., & Loeser, R. (1993). Energy Balance in the
Solar Transition Region. III. Helium Emission in Hydrostatic, Constant-
Abundance Models with Diffusion. Astrophys. J., 406, 319–345.
doi:10.1086/172443.

Fossum, A., & Carlsson, M. (2005). Response Functions of the Ultraviolet
Filters of TRACE and the Detectability of High-Frequency Acoustic Waves.
Astrophys. J., 625(1), 556–562. doi:10.1086/429614.

Gudiksen, B. V., Carlsson, M., Hansteen, V. H. et al. (2011). The stellar
atmosphere simulation code Bifrost. Code description and validation. Astron.
Astrophys., 531, A154. doi:10.1051/0004-6361/201116520. arXiv:1105.6306.

Martínez-Sykora, J., De Pontieu, B., de la Cruz Rodriguez, J. et al. (2020).
The Formation Height of Millimeter-wavelength Emission in the Solar
Chromosphere. Astrophys. J. Letters, 891(1), L8. doi:10.3847/2041-8213/ab75ac.
arXiv:2001.10645.

Menezes, F., & Valio, A. (2017). Solar Radius at Subterahertz Frequencies and
Its Relation to Solar Activity. Solar Phys., 292(12), 195.
doi:10.1007/s11207-017-1216-y. arXiv:1712.06771.

Rozelot, J. P., Kosovichev, A., & Kilcik, A. (2015). Solar Radius Variations:
An Inquisitive Wavelength Dependence. Astrophys. J., 812(2), 91.
doi:10.1088/0004-637X/812/2/91.

Rutten, R. J. (2020). SolO campfires in SDO images. arXiv e-prints.
arXiv:2009.00376.

Thuillier, G., Claudel, J., Djafer, D. et al. (2011). The Shape of the Solar
Limb: Models and Observations. Solar Phys., 268(1), 125–149.
doi:10.1007/s11207-010-9664-7.

Vernazza, J. E., Avrett, E. H., & Loeser, R. (1976). Structure of the solar
chromosphere. II. The underlying photosphere and temperature-minimum region.
Astrophys. J. Suppl., 30, 1–60. doi:10.1086/190356.

Vernazza, J. E., Avrett, E. H., & Loeser, R. (1981). Structure of the solar
chromosphere. III. Models of the EUV brightness components of the quiet sun.
Astrophys. J. Suppl., 45, 635–725. doi:10.1086/190731.

Zhang, J., White, S. M., & Kundu, M. R. (1998). The Height Structure of the
Solar Atmosphere from the Extreme-Ultraviolet Perspective. Astrophys. J.
Letters, 504(2), L127–L130. doi:10.1086/311587. arXiv:astro-ph/9807175

Zirin, H. (1966). The solar atmosphere. Waltham, Mass.: Blaisdell.


#  TopoNav: Topological Navigation for Efficient Exploration in Sparse Reward
Environments

Jumman Hossain1, Abu-Zaher Faridee1, Nirmalya Roy1, Jade Freeman2, Timothy
Gregory2, Theron T. Trout3<EMAIL_ADDRESS>1Department of
Information Systems, University of Maryland, Baltimore County, USA  2DEVCOM
Army Research Lab, USA, 3Stormfish Scientific Corp.  1{jumman.hossain,
faridee1<EMAIL_ADDRESS>2{jade.l.freeman2.civ,
<EMAIL_ADDRESS>
###### Abstract

Autonomous robots exploring unknown areas face a significant challenge:
navigating effectively without prior maps and with limited external feedback.
This challenge intensifies in sparse reward environments, where traditional
exploration techniques often fail. In this paper, we introduce TopoNav, a
novel framework that empowers robots to overcome these constraints and achieve
efficient, adaptable, and goal-oriented exploration. TopoNav’s fundamental
building blocks are active topological mapping, intrinsic reward mechanisms,
and hierarchical objective prioritization. Throughout its exploration, TopoNav
constructs a dynamic topological map that captures key locations and pathways.
It utilizes intrinsic rewards to guide the robot towards designated sub-goals
within this map, fostering structured exploration even in sparse reward
settings. To ensure efficient navigation, TopoNav employs the Hierarchical
Objective-Driven Active Topologies framework, enabling the robot to prioritize
immediate tasks like obstacle avoidance while maintaining focus on the overall
goal. We demonstrate TopoNav’s effectiveness in simulated environments that
replicate real-world conditions. Our results reveal significant improvements
in exploration efficiency, navigational accuracy, and adaptability to
unforeseen obstacles, showcasing its potential to revolutionize autonomous
exploration in a wide range of applications, including search and rescue,
environmental monitoring, and planetary exploration.

###### Index Terms:

Topology, Navigation, Sparse Reward, Reinforcement Learning

††publicationid: pubid: This work has been partially supported by U.S. Army
Grant #W911NF2120076, ONR Grant #N00014-23-1-2119

##  I Introduction

Figure 1:  TopoNav Navigation Strategies: The navigation starts at the Start
node (green circle). The robot’s path unfolds through a series of intermediate
points, labeled A, B, and C, each represented by blue circles. These points
signify crucial waypoints or decision points. The robot traverses from the
Start to Point A via a straightforward path, then moves to Point B over a
complex, dashed-line path, indicating intricate navigation challenges. A
direct, solid-line path leads from Point B to the Goal (red diamond),
symbolizing the final destination. An alternative, dotted-line path from Point
C to the Goal presents a more challenging route, possibly chosen under
specific conditions. The presence of obstacles (black triangles), adds
complexity to the navigation, requiring the robot to dynamically adapt its
route. This graph encapsulates the robot’s strategic navigation from start to
finish, highlighting its decision-making and adaptability in a simulated
environment with varied navigational challenges.

Autonomous navigation in unknown and dynamic environments remains a formidable
challenge in robotics. Traditional navigation systems heavily rely on pre-
existing maps or continuous feedback from the environment. However, in many
real-world scenarios, such as disaster response or extraterrestrial
exploration, these resources are often unavailable or unreliable [1, 2].
Typically, such a map is either topological or metric. While metric maps are
often reconstructed by optimizing geometric constraints between landmarks and
sensor poses with classic simultaneous localization and mapping (SLAM),
topological maps have recently attracted attention in visual navigation tasks
due to the simplicity, flexibility, scalability, and interpretability [3]. In
comparison to metric maps, topological maps serve as a more efficient tool for
planning and are also utilized by mammals for navigating through space. RL-
based navigation approaches [4], have been used for robot navigation in
crowded scenes with dynamic obstacles. However, they are mostly limited to
navigation on flat surfaces and do not account for outdoor sparse reward
environments. To address this limitation, in this paper, we propose TopoNav, a
framework designed to overcome the limitations of current methods in sparse
reward environments. TopoNav integrates dynamic topological mapping with a
hierarchical objective-driven approach, supported by advanced RL techniques.
This framework constructs an active map as the robot explores, identifying key
locations and potential paths. The hierarchical system within TopoNav
prioritizes objectives, allowing the robot to navigate effectively by
balancing short-term tasks, such as obstacle avoidance, with long-term
exploration goals. TopoNav also employs intrinsic rewards to foster
exploration, addressing the issue of sparse external feedback. By providing
internal incentives for reaching sub-goals, TopoNav encourages thorough
exploration, enhancing the robot’s understanding of its environment. This
approach not only improves navigation efficiency but also adapts to changes in
the environment, making it highly suitable for complex, real-world
applications.

We evaluate TopoNav in simulated environments that replicate real-world
conditions, demonstrating its superiority over traditional methods in terms of
exploration efficiency, navigational accuracy, and adaptability [3, 5]. The
results indicate that TopoNav offers a significant advancement in autonomous
robotic exploration, with potential applications extending to various fields,
including search and rescue, environmental monitoring, and space exploration
[6, 7]. Our key contributions are as follows:

  * •

Active Topological Mapping: We propose an active topological mapping approach
where the robot progressively builds a map with nodes representing key
locations and edges denoting navigable paths. This map is continuously updated
as the robot explores, enabling efficient path planning and adaptation to
environmental changes.

  * •

Intrinsic Rewards for Structured Exploration: In sparse reward environments,
reaching sub-goals within the map triggers intrinsic rewards, guiding the
robot towards a thorough and structured exploration strategy. This alleviates
the need for pre-defined reward functions, making TopoNav applicable to
diverse and unpredictable environments.

  * •

Hierarchical Objective Prioritization: Hierarchical Objective Prioritization
enables TopoNav to prioritize immediate tasks like obstacle avoidance while
keeping the final goal in sight. This hierarchical objective system ensures
focused and efficient exploration, preventing the robot from getting
sidetracked by irrelevant distractions.

  * •

Experimental Validation of Superior Performance: We evaluate TopoNav in
diverse simulated environments, showcasing its superior performance compared
to established baselines like PPO and TRPO. Our results demonstrate TopoNav’s
ability to achieve faster completion times, higher success rates in
challenging scenarios, and efficient online adaptation to dynamic
environments.

##  II Related work

In this section, we discuss existing work on robot navigation with topological
mapping.

###  II-A Metric Map-based Navigation

Metric maps, with their detailed spatial granularity, remain paramount for
tasks demanding pinpoint accuracy. Recent works augment this precision with
the power of deep learning. Deep SLAM frameworks [8] harness convolutional
neural networks (CNNs) to extract landmarks and features from camera data,
leading to robust and scalable metric map construction. This enhanced
perception empowers robots to navigate complex environments with unprecedented
accuracy. Moreover, uncertainty-aware exploration with deep Gaussian processes
[9] integrates metric maps with probabilistic modeling, allowing robots to
prioritize areas with high information gain potential, balancing exploration
and exploitation effectively. Deep reinforcement learning also finds its place
in metric-based exploration, with spatial memory-based deep Q-learning [10]
enabling robots to learn an exploration policy that prioritizes novel areas
while navigating efficiently. However, despite these advancements, the
computational demands and sensitivity to sensor errors inherent in metric maps
remain demanding concerns.

###  II-B Topological Mapping in Robot Navigation

The use of topological maps for robotic exploration and navigation has gained
significant advancements due to their efficiency, adaptability, and robustness
to limited information [3, 5]. Existing approaches leverage diverse sensor
modalities like RGB images, depth maps, poses, and LiDAR scans to build these
maps [11]. Some employ data-intensive reinforcement learning (RL) techniques
for map construction, often in conjunction with imitation learning, leading to
accurate but computationally expensive models [12]. However, their dependence
on metric information and reliance on large datasets limit their applicability
in sparse reward environments [11]. Random walk approaches offer a simpler
solution, but they often result in inefficient map exploration.

###  II-C Hybrid Map-based Navigation

Hybrid maps, seeking to synergize the strengths of both metric and topological
representations, continue to evolve. Metric-topological co-SLAM [13] proposes
a framework that simultaneously builds both metric and topological maps,
enabling concurrent localization, mapping, and exploration, particularly
beneficial for dynamic environments. Learning hybrid maps with differentiable
neural rendering [14] utilizes differentiable neural rendering to create a
hybrid map that combines metric and topological representations. This allows
robots to efficiently generate panoramic views and plan paths within the map.
Finally, exploration with learned hybrid map representations combines metric
maps with learned latent topological representations. This empowers robots to
plan fine-grained motions using the metric map while utilizing the topological
representation for efficient exploration across large areas. However,
efficient algorithms for data fusion and integration remain crucial challenges
within the hybrid map domain.

TopoNav, our proposed framework, overcomes some of the limitations of existing
approaches by embracing intrinsic rewards and dynamic awareness. It constructs
and dynamically updates a lightweight topological map, guiding the robot
towards designated sub-goals within this map even in environments with sparse
external rewards. This structured exploration is driven by intrinsic
motivations while remaining focused on the overall objective through a
hierarchical objective-driven approach. This combination leads to efficient
exploration, adaptability to dynamic environments, and robustness to delayed
or absent external rewards, paving the way for successful robot navigation in
diverse and challenging contexts.

##  III Problem Formulation

The primary challenge addressed by TopoNav is to navigate and explore
efficiently in environments where the reward structure is sparse and feedback
is minimal. The formulation of this problem involves constructing a
mathematical model that captures the dynamics of the environment, the robot’s
sensory inputs, and the decision-making process.

###  III-A Environment and State Space

The environment is represented as a state space $S$, where each state $s\in S$
encapsulates the robot’s perception of its surroundings. This includes
positional data, sensory inputs, and any relevant environmental features.

###  III-B Topological Map Construction

TopoNav constructs a topological map $M=G(N,E)$, where $N$ represents nodes
correspTopoNavonding to significant locations or features, and $E$ represents
edges denoting navigable paths. Each node $n_{i}\in N$ is associated with a
feature vector $f_{i}$ derived from the robot’s sensors. Edges
$(n_{i},n_{j})\in E$ are established based on a connectivity function
$C(n_{i},n_{j})$, dependent on factors like proximity and visibility.

$\displaystyle M$ $\displaystyle=G(N,E)$ $\displaystyle n_{i}$
$\displaystyle=f_{i},\;\forall n_{i}\in N$ $\displaystyle(n_{i},n_{j})$
$\displaystyle\in E\iff C(n_{i},n_{j})$

###  III-C Hierarchical Objective System

A hierarchical objective system is employed to prioritize goals based on their
relevance to the current state and exploration strategy. The system
dynamically selects objectives from a set $O=\\{o_{1},o_{2},\ldots,o_{n}\\}$.
The relevance of each objective $o_{i}$ in state $s$ is evaluated using a
function $E(s,o_{i})$.

$O(s)=\text{sort}(\\{(o_{i},E(s,o_{i}))\mid o_{i}\in O\\})$

###  III-D Policy Network and Decision Making

The decision-making process is governed by a policy network
$\pi_{\theta}(a|s)$, parameterized by $\theta$. The network outputs a
probability distribution over actions $a\in A$ given the current state $s$.
The goal is to maximize the expected cumulative reward.

$\max_{\theta}E_{\pi_{\theta}}[\sum_{t=0}^{T}\gamma^{t}R(s_{t},a_{t})]$

where $\gamma$ is the discount factor, and $R(s_{t},a_{t})$ is the reward
function.

Figure 2:  Overview of TopoNav System Architecure

1 Input: Environment, $Env$

2 Input: Primary objective function, $\mathcal{O}_{P}$

3 Input: Set of secondary objective functions, $\mathcal{O}_{S}$

4 Input: Intrinsic reward scaling factor, $\alpha$

5 Input: Perception scaling factor, $\beta$

6 Input: Policy (initially random or pre-trained), $\pi$

7 Output: Optimized policy, $\pi$

8 Procedure  _TopoNav_

9 Set the state $s$ to the initial state

10 Initialize policy network parameters)

11 for  _each episode_ do

12 while  _$s$ is not a terminal state_ do

13 Compute the active topology $\mathcal{M}(s)$ using the Active Topology
Mapping Module

14 Based on $\mathcal{M}(s)$ and $\mathcal{O}_{P}$, select a dominant
objective using the Hierarchical Objective Selector (Algorithm 2)

15 Choose action $a$ using policy $\pi$ (e.g., epsilon-greedy)

16 Execute action $a$ in $Env$, observe new state $s^{\prime}$ and external
reward $r_{ext}$

17 Compute the perceived reward using Equation 9

18 Update the policy network parameters using $r_{perceived}$

19 Set $s=s^{\prime}$

20Reset the environment

21return the optimized policy $\pi$

Algorithm 1 TopoNav with Hierarchical Objective-Driven Active Topologies

1Define the set of objectives $O=\\{o_{1},o_{2},\ldots,o_{n}\\}$

2 Initialize evaluation function $E(s,o_{i})$ for each objective $o_{i}\in O$

3 Procedure  _PrioritizeObjectives(_$s$ , $O$_)_

4 Create an empty priority list $P(s)$

for  _each objective $o_{i}\in O$_ do

5 Evaluate objective importance $E(s,o_{i})$ in the current state $s$

6 Append $(o_{i},E(s,o_{i}))$ to $P(s)$

7Sort $P(s)$ based on the evaluated importance of objectives

8 return top-ranked objective from $P(s)$

Algorithm 2 Hierarchical Objective Prioritization for TopoNav

##  IV TopoNav: Proposed Approach

TopoNav employs a hierarchical reinforcement learning strategy to navigate and
explore environments with sparse reward structures. The system is divided into
two primary decision-making layers: (i) the high-level controller and (ii) the
low-level controller.

###  IV-A High-Level Controller

The high-level controller is responsible for setting strategic goals and is
informed by a dynamically constructed topological map $M$ of the environment.

####  IV-A1 Topological Map Construction

$M=G(N,E)$ (1)

where $N$ represents nodes (significant locations) and $E$ denotes edges
(navigable paths).

####  IV-A2 Objective Setting

High-level objectives are set based on the current state of the map and
overall exploration goals.

$o_{H}=\mathcal{O}_{H}(M,s)$ (2)

where $o_{H}$ is the high-level objective, $\mathcal{O}_{H}$ is the high-level
objective function, and $s$ is the current state.

###  IV-B Low-Level Controller

The low-level controller executes tactical actions to achieve the high-level
objectives. It operates based on immediate sensory inputs and environmental
dynamics.

$a=\pi_{L}(o_{H},s)$ (3)

where $a$ is the action, $\pi_{L}$ is the low-level policy, $o_{H}$ is the
current high-level objective, and $s$ is the state.

###  IV-C Hierarchical Integration

The two controllers work collaboratively, with the high-level controller
guiding the strategic direction and the low-level controller handling the
immediate navigation.

$\text{Update}(\pi_{H},\pi_{L},r)$ (4)

Both $\pi_{H}$ (high-level policy) and $\pi_{L}$ (low-level policy) are
updated based on the reward $r$, which is a function of both intrinsic and
extrinsic factors.

###  IV-D Intrinsic Reward Functions

Intrinsic rewards are crucial for encouraging exploration and efficient
navigation, especially in sparse reward environments.

####  IV-D1 Novelty-Based Reward

The Novelty-Based Reward function is designed to encourage exploration by
utilizing Shannon entropy, $H(N(s))$, as a measure of uncertainty in
visitation patterns. It is defined as:

$R_{\text{novelty}}(s)=\alpha\cdot\exp(-\beta\cdot H(N(s)))$ (5)

where $\alpha$ and $\beta$ are scaling factors. This function incentivizes the
robot to explore new or less frequented areas within the environment,
enhancing overall exploration efficiency.

####  IV-D2 Information Gain Reward

This function quantifies the value of information gained through transitions
in the environment. It employs mutual information between the topological maps
of successive states:

$R_{\text{info}}(s,s^{\prime})=\gamma\cdot(\text{MI}(M(s^{\prime}),M(s))-\text{MI}(M(s),M(s^{\prime})))$
(6)

where MI represents mutual information, and $\gamma$ is a scaling factor. This
reward encourages actions that lead to a significant change in the robot’s
understanding of the environment.

####  IV-D3 Curiosity-Driven Reward

The Curiosity-Driven Reward promotes exploration of outcomes that deviate from
the robot’s predictions. It is formulated as:

$R_{\text{curiosity}}(s,a)=\delta\cdot\left|P(s^{\prime}\mid
s,a)-P_{\text{model}}(s^{\prime}\mid s,a)\right|\cdot(1-C(s,a))$ (7)

where $P_{\text{model}}$ is the model’s prediction confidence, $C(s,a)$ is a
safety penalty factor, and $\delta$ is a scaling factor.

####  IV-D4 Path Optimization Reward

This function aims to optimize the robot’s path by balancing efficiency and
obstacle avoidance. It combines distance and obstacle considerations:

$R_{\text{path}}(s,a)=-\lambda\cdot(D(s,a)+\kappa\cdot O(s,a))$ (8)

where $D(s,a)$ represents the distance to the target, $O(s,a)$ is the obstacle
penalty, and $\lambda,\kappa$ are scaling factors. This reward ensures
efficient and safe navigation within the environment.

The overall perceived reward function, $R_{\text{perceived}}$, in TopoNav
combines the external reward with the above various intrinsic rewards to guide
the robot’s decision-making process in complex environments. The function is
formulated as:

$\displaystyle R_{\text{perceived}}(s,a)=R_{\text{ext}}(s,a)$
$\displaystyle+w_{\text{novelty}}\cdot R_{\text{novelty}}(s)$ $\displaystyle+$
(9) $\displaystyle w_{\text{info}}\cdot R_{\text{info}}(s,s^{\prime})$
$\displaystyle+w_{\text{curiosity}}\cdot R_{\text{curiosity}}(s,a)$
$\displaystyle+$ $\displaystyle w_{\text{path}}\cdot R_{\text{path}}(s,a)$

where $R_{\text{ext}}(s,a)$ represents the external reward from the
environment, and $R_{\text{novelty}}(s)$, $R_{\text{info}}(s,s^{\prime})$,
$R_{\text{curiosity}}(s,a)$, and $R_{\text{path}}(s,a)$ are the intrinsic
rewards for novelty, information gain, curiosity, and path optimization,
respectively. The coefficients $w_{\text{novelty}}$, $w_{\text{info}}$,
$w_{\text{curiosity}}$, and $w_{\text{path}}$ control the relative importance
of each intrinsic reward component, allowing for a balanced integration of
different motivational factors.

###  IV-E Goal and Sub-Goal Setting

TopoNav employs a dynamic and hierarchical approach to goal setting, essential
for effective navigation in complex environments. This section outlines how
TopoNav sets and adapts its high-level goals and sub-goals based on the
evolving understanding of the environment.

####  IV-E1 High-Level Goal Setting

High-level goals in TopoNav are derived from the mission objectives and the
robot’s understanding of the environment, as represented by the topological
map $T$. The primary aim is to identify areas of interest or unexplored
regions within the environment. This process can be mathematically represented
as:

$G_{\text{high}}=h(M,T)$ (10)

where $G_{\text{high}}$ are the high-level goals, $M$ denotes the mission
objectives, and $T$ is the topological map. The function $h$ is responsible
for translating the objectives and map information into actionable goals.

####  IV-E2 Sub-Goal Identification

Sub-goals are smaller, actionable objectives that guide the robot’s immediate
actions. It involves breaking down complex objectives into smaller, more
immediate tasks. For example, if the primary goal is to explore a specific
area, a sub-goal might be navigating to the entrance of that area. The
identification of sub-goals is influenced by the robot’s current state and
real-time sensory inputs, ensuring that the navigation is responsive to
immediate environmental conditions. This process is influenced by the robot’s
current state $s$ and real-time sensory inputs $o$, and can be formulated as:

$G_{\text{sub}}=f(G_{\text{high}},T,s,o)$ (11)

where $G_{\text{sub}}$ represents the sub-goals, and $f$ is the function
determining these goals based on the current context.

####  IV-E3 Adapting to the Environment

TopoNav is designed to adapt its goals dynamically in response to new
information or environmental changes. As the robot explores the environment
and updates its map $T_{\text{new}}$, both the high-level goals and sub-goals
are revised accordingly:

$G_{\text{high\\_new}}=g(G_{\text{high}},T_{\text{new}})$ (12)
$G_{\text{sub\\_new}}=f^{\prime}(G_{\text{high\\_new}},T_{\text{new}},s_{\text{new}},o_{\text{new}})$
(13)

These equations represent the updated high-level goals $G_{\text{high\\_new}}$
and sub-goals $G_{\text{sub\\_new}}$, ensuring that the navigation strategy
remains aligned with the current state of the environment.

##  V Experiments and Results

((a))

((b))

((c))

Figure 3: Performance comparison of TopoNav with PPO and TRPO policies in
three scenarios: (a) Goal-reaching (b) Obstacle avoidance, and (c) Navigating
on uneven terrains

We explain our implementation and experimental details using a Husky robot in
the ARL Unity simulation environment.

###  V-A Implementation Details

Our RL network is implemented in PyTorch and trained using simulated terrains
with a Clearpath Husky robot in ROS Noetic and Unity Simulation framework,
which allows for testing in realistic outdoor environments with diverse
terrain and objects. The simulated Husky robot is mounted with a Velodyne
VLP16 3D LiDAR. The network is trained in a workstation with a 10th-generation
Intel Core i9-10850K processor and an NVIDIA GeForce RTX 3090 GPU.

####  V-A1 Network Architecture

The TopoNav agent’s policy is parameterized by a neural network
$\pi_{\theta}(a|s)$, where $\theta$ represents the network’s parameters. The
network is structured with an input layer designed to match the dimensionality
of the state space, denoted by $|S|$. Two fully connected layers with a
rectified linear unit (ReLU) activation function follow this. The output layer
employs a softmax function to yield a probability distribution over the action
space. This architecture facilitates the agent’s decision-making process by
mapping environmental states to optimal actions.

####  V-A2 Training

For training the agent, we use the Proximal Policy Optimization (PPO)
algorithm. Since TopoNav uses a hierarchical framework where high-level
policies output sub-goals that are then achieved using lower-level policies,
PPO can be efficient for optimizing these lower-level policies due to its
sample efficiency and stability. The training process is carried out through
repeated interactions with the environment across $N$ episodes, where a
terminal state or a predetermined maximum step count bounds each episode. We
employ the Adam optimizer with a learning rate of $\alpha=1\times 10^{-3}$ to
update the network’s weights.

####  V-A3 Policy Update

The network parameters $\theta$ are updated at each training step using the
perceived reward signal $r_{\text{perceived}}$. This update aims to maximize
the expected return and drive the agent towards more effective exploration and
navigation strategies.

###  V-B Testing Scenario

We evaluate our topology based navigation framework and compare it with prior
methods in three scenarios (see Fig. 3). The environment is inherited from
some of our previous works [15, 16, 17]

  * •

Scenario 1 (Goal Reaching): In this scenario, the robot is placed in an
obstacle-free outdoor environment with an objective of reaching a given goal
location.

  * •

Scenario 2 (Obstacle Avoidance): In this scenario, the robot is placed in an
outdoor area with static obstacles such as trees, walls, and sharp hills. The
goal is to reach the destination without colliding with obstacles.

  * •

Scenario 3 (Navigating on Uneven Terrains): In this scenario, the robot is
placed in a highly elevated outdoor terrain where it might flip over or
experience instability in some regions. The objective is to navigate to a goal
location while avoiding steep elevations in the environment.

###  V-C Evaluation Metrics

TopoNav requires a comprehensive set of metrics to evaluate its performance.
These metrics address TopoNav’s efficiency, agility, and adaptability in
discovering and navigating through outdoor environments.

####  V-C1 Metrics for Efficient Exploration

  * •

Area Coverage ($C_{\text{area}}$): This metric evaluates the extent of the
environment explored by TopoNav. It is defined as:

$C_{\text{area}}=\left(\frac{U_{\text{new}}}{U_{\text{total}}}\right)\times
100$ (14)

where $U_{\text{new}}$ is the newly explored area, and $U_{\text{total}}$ is
the total size of the environment. Higher values indicate more efficient
exploration.

  * •

Distance to Unexplored ($D_{\text{unexplored}}$): This metric reflects
TopoNav’s inclination towards unexplored regions. It is calculated as:

$D_{\text{unexplored}}=\text{avg}(\min(d_{u})\,\text{for}\,u\in
U_{\text{unexplored}})$ (15)

where $d_{u}$ is the distance from the robot to each point in the unexplored
region $U_{\text{unexplored}}$. Smaller values signify a focus on exploring
new areas

  * •

Information Gain ($I_{\text{gain}}$): TopoNav’s knowledge accumulation is
quantified by this metric. It is estimated as:

$I_{\text{gain}}=H(T_{\text{new}})-H(T_{\text{old}})$ (16)

where $H(T)$ is the entropy of the topological map $T$. An increase in
$I_{\text{gain}}$ indicates enhanced understanding of the environment.

####  V-C2 Metrics for Adaptive Navigation

  * •

Success Rate ($R_{\text{success}}$): This metric measures the robot’s ability
to reach its goals:

$R_{\text{success}}=\left(\frac{G_{\text{reached}}}{G_{\text{total}}}\right)\times
100$ (17)

where $G_{\text{reached}}$ is the number of successfully reached goals, and
$G_{\text{total}}$ is the total number of goals attempted.

  * •

Path Efficiency ($E_{\text{path}}$): TopoNav’s path optimality is evaluated
using this metric:

$E_{\text{path}}=\frac{D_{\text{actual}}}{D_{\text{shortest}}}$ (18)

where $D_{\text{actual}}$ is the length of the path taken, and
$D_{\text{shortest}}$ is the shortest possible distance to the goal.

Metric | Scenario 1 | Scenario 2 | Scenario 3
---|---|---|---
Area Coverage (C_area) | $88\%$ | $75\%$ | $70\%$
Distance to Unexplored |  |  |
(D_unexplored) |  $30$ meters |  $21$ meters |  $18$ meters
Success Rate (R_success) | $89\%$ | $82\%$ | $75\%$
Path Efficiency (E_path) | $1.15$ | $1.25$ | $1.35$

TABLE I: Comparative Analysis of TopoNav Performance in Different Scenario. It
presents a comparison of TopoNav’s performance metrics in three different
scenarios (described in Section V-B)

.

###  V-D Performance Analysis

In Figure 3, we present a comparison of TopoNav against PPO and TRPO
algorithms. The graph shows TopoNav’s reward trends for goal-reaching (a),
obstacle avoidance (b), and uneven terrain navigation (c). TopoNav
consistently performs well, learning steadily over time. It shows promise in
obstacle-rich environments and excels on complex surfaces, surpassing TRPO and
matching PPO’s performance levels. Moreover, Table I shows TopoNav’s
outstanding performance from Scenario 1 to 3, with area coverage from $88\%$
to $70\%$ and success rate from $89\%$ to $75\%$. These metrics reflect the
system’s increasing challenges with environmental complexity. Moreover, path
efficiency decreased to 70% in scenario 3, indicating room for optimization in
navigating complex terrains. This evidence points to TopoNav’s strong
potential for versatile and effective navigation tasks in varying conditions.

##  VI Conclusion, Limitations, and Future Directions

We present TopoNav framework for autonomous navigation, especially in sparse
reward environments, through its integration of hierarchical reinforcement
learning with dynamic topological mapping. However, challenges remain,
particularly in its scalability to real-world scenarios and reliance on
precise sensor data. In addition to improvements in sensor fusion techniques
to enhance performance in a variety of conditions, we intend to test it
extensively in the real world to further evaluate and improve its
capabilities. Optimizing computational efficiency remains a key objective to
broaden TopoNav’s applicability across various platforms. Addressing these
areas will be crucial in enhancing TopoNav’s performance in real-world
applications and promoting its widespread adoption in autonomous navigation.

## References

  * [1] Fatih Abut, Mehmet Fatih Akay, and Mohammad Abdeh.  Autonomous navigation in search and rescue simulated environment using deep reinforcement learning.  Balkan Journal of Electrical and Computer Engineering, 9(2), 2021\.
  * [2] Autonomous navigation of rovers for planetary exploration.  IFAC Proceedings Volumes, 31(21):83–87, 1998.  14th IFAC Symposium on Automatic Control in Aerospace 1998, Seoul, Korea, 24-28 August 1998.
  * [3] Nikolay Savinov, Alexey Dosovitskiy, and Vladlen Koltun.  Semi-parametric topological memory for navigation.  arXiv preprint arXiv:1803.00653, 2018.
  * [4] Han Hu, Kaicheng Zhang, Aaron Hao Tan, Michael Ruan, Christopher Agia, and Goldie Nejat.  A sim-to-real pipeline for deep reinforcement learning for autonomous robot navigation in cluttered rough terrain.  IEEE Robotics and Automation Letters, 6(4):6569–6576, 2021.
  * [5] Nuri Kim, Obin Kwon, Hwiyeon Yoo, Yunho Choi, Jeongho Park, and Songhwai Oh.  Topological semantic graph memory for image-goal navigation.  In Conference on Robot Learning, pages 393–402. PMLR, 2023.
  * [6] Yugang Liu and Goldie Nejat.  Multirobot cooperative learning for semiautonomous control in urban search and rescue applications.  Journal of Field Robotics, 33(4):512–536, 2016.
  * [7] Andrew Thoesen and Hamid Marvi.  Planetary surface mobility and exploration: A review.  Current Robotics Reports, 2(3):239–249, 2021.
  * [8] Rana Azzam, Tarek Taha, Shoudong Huang, and Yahya Zweiri.  A deep learning framework for robust semantic slam.  In 2020 Advances in Science and Engineering Technology International Conferences. IEEE, 2020.
  * [9] Dawei Sun, Mohammad Javad Khojasteh, Shubhanshu Shekhar, and Chuchu Fan.  Uncertain-aware safe exploratory planning using gaussian process and neural control contraction metric.  In Learning for Dynamics and Control, pages 728–741. PMLR, 2021\.
  * [10] Lei Tai and Ming Liu.  Towards cognitive exploration through deep reinforcement learning for mobile robots.  arXiv preprint arXiv:1610.01733, 2016.
  * [11] Edward Beeching, Jilles Dibangoye, Olivier Simonin, and Christian Wolf.  Learning to plan with uncertain topological maps.  In European Conference on Computer Vision, pages 473–490. Springer, 2020.
  * [12] Devendra Singh Chaplot, Ruslan Salakhutdinov, Abhinav Gupta, and Saurabh Gupta.  Neural topological slam for visual navigation.  In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12872–12881, 2020.
  * [13] Jose-Luis Blanco, Juan-Antonio Fernández-Madrigal, and Javier Gonzalez.  A new approach for large-scale localization and mapping: Hybrid metric-topological slam.  In Proceedings 2007 IEEE International Conference on Robotics and Automation, pages 2061–2067. IEEE, 2007.
  * [14] Obin Kwon, Jeongho Park, and Songhwai Oh.  Renderable neural radiance map for visual navigation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9099–9108, 2023.
  * [15] Jumman Hossain, Abu-Zaher Faridee, Nirmalya Roy, Anjan Basak, and Derrik E Asher.  Covernav: Cover following navigation planning in unstructured outdoor environment with deep reinforcement learning.  In 2023 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS), pages 127–132. IEEE, 2023.
  * [16] Emon Dey, Jumman Hossain, Nirmalya Roy, and Carl Busart.  Synchrosim: An integrated co-simulation middleware for heterogeneous multi-robot system.  In 2022 18th International Conference on Distributed Computing in Sensor Systems (DCOSS), pages 334–341. IEEE, 2022.
  * [17] Avi Spector, Wanying Zhu, Jumman Hossain, and Nirmalya Roy.  Simulated forest environment and robot control framework for integration with cover detection algorithms.  In 2022 IEEE/ACM International Conference on Big Data Computing, Applications and Technologies (BDCAT), pages 277–283. IEEE, 2022.