# Iterative Sparse Recovery based Passive Localization in Perceptive Mobile
Networks

Lei Xie and Shenghui Song  Dept. of ECE, The Hong Kong University of Science
and Technology, Hong Kong Email: eelxie<EMAIL_ADDRESS>
###### Abstract.

Perceptive mobile networks (PMNs) were proposed to integrate sensing
capability into current cellular networks where multiple sensing nodes (SNs)
can collaboratively sense the same targets. Besides the active sensing in
traditional radar systems, passive sensing based on the uplink communication
signals from mobile user equipment may play a more important role in PMNs,
especially for targets with weak electromagnetic wave reflection, e.g.,
pedestrians. However, without the properly designed active sensing waveform,
passive sensing normally suffers from low signal to noise power ratio (SNR).
As a result, most existing methods require a large number of data samples to
achieve an accurate estimate of the covariance matrix for the received
signals, based on which a power spectrum is constructed for localization
purposes. Such a requirement will create heavy communication workload for PMNs
because the data samples need to be transferred over the network for
collaborative sensing. To tackle this issue, in this paper we leverage the
sparse structure of the localization problem to reduce the searching space and
propose an iterative sparse recovery (ISR) algorithm that estimates the
covariance matrix and the power spectrum in an iterative manner. Experiment
results show that, with very few samples in the low SNR regime, the ISR
algorithm can achieve much better localization performance than existing
methods.

Perceptive mobile network, passive sensing, direct localization, sparse
recovery

††conference: ACM MobiCom Workshop on Integrated Sensing and Communication
Systems; 24-28 October, 2022; Sydney, Australia

##  1\. Introduction

With the development of innovative applications such as autonomous driving,
sensing becomes an important service for future wireless networks. To this
end, the recently proposed integrated sensing and communication (ISAC)
provides a promising platform to exploit the synergy between sensing and
communication. As a special type of ISAC system, perceptive mobile networks
(PMNs) were proposed to add sensing capability to cellular networks (Xie et
al., 2022; Xie and Song, 2022; Xie et al., 2022b; Zhang et al., 2021) without
interfering with the communication service. The adoption of millimeter wave
(mmWave) by 5G and beyond systems further enables hardware and software reuse
between sensing and communication.

Given the existing uplink communication from the user equipment (UEs) to the
base stations, passive sensing, which utilizes uplink signals for sensing
purposes instead of actively transmitting probing signals, demonstrates its
advantage for targets with weak electromagnetic wave reflection, e.g.,
pedestrians. Furthermore, without the need to transmit sensing signals,
passive sensing is more energy-efficient and causes less latency than active
sensing. However, without the well-designed waveform in active sensing, the
received signal to noise power ratio (SNR) for passive sensing is normally
low. The situation is even worse in the mmWave band due to the very high
pathloss and omni-directional uplink signals.

There are two types of localization methods for passive sensing. The most
popular one is the indirect localization method which has two steps. In the
first step, some intermediate parameters of the targets, e.g., distance, time-
difference-of-arrival (TDOA), and angle of arrival (AOA) are estimated. In the
second step, the localization of the target is achieved based on the
intermediate parameters. A closed-form localization solution based on weighted
least squares estimation was proposed in (Noroozi and Sebt, 2015). The authors
of (Noroozi et al., 2017) modelled the localization based on intermediate
parameters as a quadratically constrained quadratic programming (QCQP)
problem, which is non-convex and thus hard to solve. To address this issue,
the QCQP problem is reformulated as a linearly constrained quadratic
programming (LCQP) problem, and solved by the iterative constrained weighted
least squares method. On the other hand, the localization performance also
suffers from outlier samples (Xie et al., 2022a). To handle this issue, (Liang
et al., 2016) proposed a robust localization method by exploiting the maximum
correntropy criterion. The authors of (Liu et al., 2020) investigated
predictive beamforming to track vehicles by exploiting the dual-functional
radar-communication (DFRC) technique, where the positions of the vehicles are
recovered based on time delay and Doppler frequency. However, the indirect
methods may lose information when estimating intermediate parameters in the
first step, which causes poor localization performance especially with low
SNR. Furthermore, there is an intrinsic issue for indirect localization
methods with multiple targets, i.e., the matching of the intermediate
parameters with the locations of different targets (Tzoreff and Weiss, 2017).
In particular, even if the intermediate parameters are perfectly estimated,
the final localization results may be wrong due to the mismatching issues.

Another class of passive localization methods estimates the target positions
directly without the intermediate parameters estimation, and is called direct
localization. Compared with the indirect methods, direct localization is
implemented at the signal level and can achieve much better performance,
especially with low SNR (Tzoreff and Weiss, 2017). The conventional direct
localization methods located the targets based on the power spectrum estimated
from the sample covariance matrix (SCM) (Tzafri and Weiss, 2016; Zhao et al.,
2020). The authors of (Tzafri and Weiss, 2016) proposed to locate the targets
based on the minimum variance distortionless response (MVDR) method, and
assumed the sample covariance matrix (SCM) to be block-diagonal. The authors
of (Zhao et al., 2020) proposed to exploit the beam-space (BS) of the
covariance matrix, but the estimation of the covariance matrix by the SCM-
based method requires a large number of samples. As a result, the above two
methods cannot achieve accurate localization performance when the number of
samples is limited. This situation is worse in PMNs, because collaborative
sensing requires the transferring of data samples from the sensing nodes (SNs)
to the central processing unit (CPU), and the large number of samples will
cause high communication workload for the network. This motivates us to design
a communication-efficient localization method with limited data samples.

The existing direct localization methods require many samples to guarantee a
satisfactory estimate of the covariance matrix of the received signals such
that the power spectrum constructed based on the covariance matrix is accurate
enough. To reduce the requirement on data samples, in this paper, we propose
an iterative sparse recovery (ISR) method leveraging the sparse representation
of the localization problem. In particular, among a large number of possible
locations, only a few can hold the targets. As a result, the localization
problem, i.e., the estimation of the power spectrum, does not require a large
number of samples. Furthermore, by building up the connection between the
covariance matrix of the received signal and the power spectrum of different
locations, we propose an iterative method where the covariance matrix and the
power spectrum are estimated in an iterative manner. In this way, the
estimation of the covariance matrix and the construction of the power spectrum
can be improved iteratively, leading to better estimation performance
especially with limited samples. Compared with the existing methods, the ISR
algorithm can achieve high-resolution localization performance with a few
samples (even one), which is critical to reduce the communication workload of
PMNs.

##  2\. Preliminary

In this section, we present the system model of the proposed multi-target
passive localization problem and introduce the existing methods in the
literature.

###  2.1. Signal Model

Consider a PMN where $L$ SNs collaboratiely monitor $K$ targets, where $K$ is
unknown111In general, the number of targets $K$ can be estimated by Akaike
information criterion and Baysian information criterion (Stoica and Selen,
2004).. Depending on the network architecture, the SNs can be base stations
(Liu et al., 2018), remote radio units (RRUs) (Zhang et al., 2021), or target
monitoring terminals (TMTs) (Xie et al., 2022; Xie and Song, 2022). Assume SNs
are synchronized to the same clock and are equipped with a uniform linear
array (ULA) of $N_{R}$ antennas and the targets, e.g., UEs, only have one
antenna. In this paper, we consider passive localization based on the uplink
communication signals from the UEs in an orthogonal frequency division
multiple (OFDM) system (Song et al., 2011). In particular, different UEs
transmit their signals at different carrier-frequencies within a given
frequency band.

Let $\mathbf{u}_{l}$ and $\mathbf{p}_{k}$ denote the coordinate vectors of the
$l$th SN and the $k$th target, respectively. Then, the steering vector of the
$l$-th SN towards the $k$th target can be given by

(1)
$\mathbf{a}_{l}(\mathbf{p}_{k})=\frac{1}{\sqrt{N_{R}}}\left[1,e^{j\frac{2\pi
d}{\lambda}\mathbf{k}_{k}^{T}\mathbf{e}_{l}},\cdots,e^{j\frac{2\pi
d}{\lambda}(N_{R}-1)\mathbf{k}_{k}^{T}\mathbf{e}_{l}}\right]^{T},$

where $(\cdot)^{T}$ denotes the transpose operator, while $d$ and $\lambda$
represent the inner spacing of the ULA and the wavelength, respectively. The
vector

(2)
$\mathbf{k}_{k}=\frac{\mathbf{u}_{l}-\mathbf{p}_{k}}{\|\mathbf{u}_{l}-\mathbf{p}_{k}\|_{2}}$

denotes the direction vector from the $k$th target to the $l$th SN and
$\mathbf{e}_{l}$ represents the unit vector parallel to the line formed by all
antennas of the ULA.

For ease of implementation, we do not require the SNs to know the exact
carrier frequencies of all UEs, but instead they only know the central carrier
and the bandwidth of the OFDM systems. As a result, each SN only performs
down-conversion on the received signal with the central carrier and the
resulting baseband signal of the $l$-th SN is denoted by $\mathbf{y}_{l}(t)$.
Then, the $l$-th SN samples its received signal and obtains

(3)
$\mathbf{y}_{l}(n)=\sum_{k=1}^{K}\sqrt{N_{R}}\alpha_{l,k}\mathbf{a}_{l}(\mathbf{p}_{k})s_{k}(n)+\mathbf{v}_{l}(n),n=1,2,\cdots,N_{s}$

where $N_{s}$ denotes the number of samples per symbol and the $n$th sample of
the transmit signal, $s_{k}(n)$, is assumed to have zero mean and unit
variance. Here, $\alpha_{l,k}$ denotes the complex channel coefficient from
the $k$-th target to the $l$-th SN, which is modeled as a Gaussian random
variable (RV) with zero mean and variance $\sigma_{l,k}^{2}$.
$\mathbf{v}_{l}(n)$ represents the additive white Gaussian noise with
covariance matrix $\sigma_{v}^{2}\mathbf{I}$. Note that synchronization
between different uplink signals is not required.

It can be validated that $\mathbf{y}_{l}(n)$ follows a zero-mean Gaussian
distribution with covariance matrix

(4)
$\mathbf{R}_{l}={E}\left(\mathbf{y}_{l}(n)\mathbf{y}_{l}^{H}(n)\right)=\sum_{k=1}^{K}N_{R}\sigma_{l,k}^{2}\mathbf{a}_{l}(\mathbf{p}_{k})\mathbf{a}_{l}^{H}(\mathbf{p}_{k})+\sigma_{v}^{2}\mathbf{I},$

where $(\cdot)^{H}$ denotes the conjugate transpose operator. By stacking the
received signals of different SNs into a vector, we have

(5)
$\mathbf{y}(n)=\left[\mathbf{y}_{1}^{T}(n),\mathbf{y}_{2}^{T}(n),\cdots,\mathbf{y}_{L}^{T}(n)\right]^{T}=\sum_{k=1}^{K}\mathbf{A}_{k}\mathbf{x}_{k}(n)+\mathbf{v}(n),$

where

(6)
$\mathbf{A}_{k}=\left[\begin{matrix}\mathbf{a}_{1}(\mathbf{p}_{k})&\mathbf{0}&\cdots&\mathbf{0}\\\
\mathbf{0}&\mathbf{a}_{2}(\mathbf{p}_{k})&\cdots&\mathbf{0}\\\
\vdots&\vdots&\ddots&\vdots\\\
\mathbf{0}&\mathbf{0}&\cdots&\mathbf{a}_{L}(\mathbf{p}_{k})\\\
\end{matrix}\right]\in\mathbb{C}^{N_{R}L\times L},$

denotes the steering matrix from $L$ SNs towards the $k$th target,

(7)
$\mathbf{x}_{k}(n)=\left[\alpha_{1,k},\alpha_{2,k},\cdots,\alpha_{L,k}\right]^{T}\sqrt{N_{R}}s_{k}(n)\in\mathbb{C}^{L\times
1},$

and

(8)
$\mathbf{v}(n)=\left[\mathbf{v}_{1}^{T}(n),\mathbf{v}_{2}^{T}(n),\cdots,\mathbf{v}_{L}^{T}(n)\right]^{T}\in\mathbb{C}^{N_{R}L\times
1}.$

We assume that $\\{\mathbf{x}_{k}(n)\\}_{k=1}^{K}$ and $\mathbf{v}(n)$ are
independent and statistically stationary throughout the observation period.
Then, the covariance matrix of $\mathbf{y}(n)$ is given as

(9) $\displaystyle\mathbf{R}={E}\left(\mathbf{y}(n)\mathbf{y}(n)^{H}\right)$
$\displaystyle=\sum_{k=1}^{K}\mathbf{A}_{k}\mathbf{\Lambda}_{k}\mathbf{A}_{k}^{H}+\sigma_{v}^{2}\mathbf{I},$

where
$\mathbf{\Lambda}_{k}\triangleq\mathbb{E}\left(\mathbf{x}_{k}(n)\mathbf{x}_{k}^{H}(n)\right)$.

###  2.2. Direct Localization Algorithms

Given the received signals at the SNs, the objective of localization is to
determine the locations of the $K$ targets. For that purpose, we may
discretize the concerned area into a grid of $N_{G}$ searching points
$\\{\bar{\mathbf{p}}_{i}\\}_{i=1}^{N_{G}}$. The direct localization methods
generate a power spectrum for the whole grid with an estimated power value for
each searching point. If one target is present at one searching point, there
is supposed to be a peak in the power spectrum.

Due to the difficulty to directly estimate the transmit power, one option is
to estimate the received signal power from a given searching point. For that
purpose, we want to collect the energy transmitted from a specific location,
but suppress the interference from other locations. To this end, the MVDR
method is a good candidate, because it was originally proposed as a
beamforming technology to minimize the output power of interference and noise
while guaranteeing a fixed gain for the desired signal (Capon, 1969; Xie et
al., 2020). The authors of (Tzafri and Weiss, 2016) chose MVDR as the combiner
and utilized the output power of the combiner as the estimated power, where
the MVDR spectrum is obtained as

(10)
$\begin{split}P_{\text{MVDR},i}&=\frac{1}{\sum_{l=1}^{L}\mathbf{a}_{l}^{H}(\bar{\mathbf{p}}_{i})\widehat{\mathbf{R}}_{\text{SCM},l}^{-1}\mathbf{a}_{l}(\bar{\mathbf{p}}_{i})},i=1,...,N_{G}\end{split}$

with
$\widehat{\mathbf{R}}_{\text{SCM},l}=\frac{1}{N_{s}}\sum_{n=1}^{N_{s}}\mathbf{y}_{l}(n)\mathbf{y}_{l}^{H}(n)$
representing the SCM of the $l$th SN. Note that the MVDR method estimates the
covariance matrix by exploiting the block-diagonal structure

(11)
$\widehat{\mathbf{R}}_{\text{B-SCM}}=\left[\begin{matrix}\widehat{\mathbf{R}}_{\text{SCM},1}&\mathbf{0}&\cdots&\mathbf{0}\\\
\mathbf{0}&\widehat{\mathbf{R}}_{\text{SCM},2}&\cdots&\mathbf{0}\\\
\vdots&\vdots&\ddots&\vdots\\\
\mathbf{0}&\mathbf{0}&\cdots&\widehat{\mathbf{R}}_{\text{SCM},L}\\\
\end{matrix}\right].$

This assumption may not hold in practice, but can reduce the computational
complexity of the algorithm.

The authors of (Zhao et al., 2020) proposed to construct the spectrum by
exploiting the beam-space (BS) of the covariance matrix and the estimated
spectrum is given by

(12)
$P_{\text{BS},i}=\frac{1}{\det\left(\mathbf{A}_{i}^{H}\left(\sum\limits_{j=KL+1}^{LN_{R}}\mathbf{u}_{j}\mathbf{u}_{j}^{H}\right)\mathbf{A}_{i}\right)},i=1,...,N_{G}$

where $\mathbf{u}_{j}$ denotes the $j$-th eigen-vector of the covariance
matrix

(13)
$\widehat{\mathbf{R}}_{\text{SCM}}=\frac{1}{N_{s}}\sum_{n=1}^{N_{s}}\mathbf{y}(n)\mathbf{y}^{H}(n).$

However, when the number of samples is small, the estimation accuracy of
$\widehat{\mathbf{R}}_{\text{B-SCM}}$ and $\widehat{\mathbf{R}}_{\text{SCM}}$
will be low. Therefore, the spectrum constructed based on
$\widehat{\mathbf{R}}_{\text{B-SCM}}$ and $\widehat{\mathbf{R}}_{\text{SCM}}$
will not be accurate, causing bad localization performance. In the following,
we propose the ISR method which achieves better localization performance with
small number of samples even in the low SNR regime.

##  3\. Iterative Sparse Recovery

Given the number of targets $K$ is usually much smaller than the number of
searching points $N_{G}$, we propose an iterative localization algorithm by
exploiting the sparse structure. Like most direct localization algorithms, the
proposed method will estimate a power spectrum for the grid, based on which
localization is performed. However, this is done in an iterative manner.

###  3.1. Maximum Likelihood Estimation for the Power Spectrum

With the searching grid $\\{\bar{\mathbf{p}}_{i}\\}_{i=1}^{N_{G}}$, (5) can be
reformulated as

(14)
$\mathbf{y}(n)=\sum_{i=1}^{N_{G}}\mathbf{A}_{i}\bar{\mathbf{x}}_{i}(n)+\mathbf{v}(n),$

where $\mathbf{A}_{i}$ denotes the steering matrix from $L$ SNs to a given
searching point $\bar{\mathbf{p}}_{i}$ as defined in (6), and

(15)
$\bar{\mathbf{x}}_{i}(n)=\left\\{\begin{matrix}\mathbf{x}_{k}(n),&\bar{\mathbf{p}}_{i}=\mathbf{p}_{k},\\\
\mathbf{0},&\bar{\mathbf{p}}_{i}\neq\mathbf{p}_{k}.\\\ \end{matrix}\right.$

Note that (14) utilized the sparse structure.

Different from the existing methods, the ISR algorithm directly utilizes the
received signal power from $\bar{\mathbf{p}}_{i}$ to build the power spectrum,
i.e.,

(16)
$P_{i}\triangleq\frac{1}{N_{R}}\mathrm{Tr}\left(\mathbf{\Lambda}_{i}\right).$

However, $\mathbf{\Lambda}_{i}$ is unknown, and the maximum likelihood
estimation (MLE) of $\mathbf{\Lambda}_{i}$ is given by

(17)
$\widehat{\mathbf{\Lambda}}_{i}=\frac{1}{N_{s}}\sum_{n=1}^{N_{s}}\bar{\mathbf{x}}_{i}(n)\bar{\mathbf{x}}_{i}^{H}(n),$

for which we need to estimate $\bar{\mathbf{x}}_{i}(n)$.

###  3.2. Maximum Likelihood Estimation for $\bar{\mathbf{x}}_{i}(n)$

Assuming $\bar{\mathbf{p}}_{i}$ is the target of interest (TOI), we rewrite
(3) as

(18)
$\mathbf{y}(n)=\mathbf{A}_{i}\bar{\mathbf{x}}_{i}(n)+\mathbf{i}(n)+\mathbf{v}(n),$

where $\mathbf{i}(n)=\sum_{j\neq i}\mathbf{A}_{j}\bar{\mathbf{x}}_{j}(n)$
denotes the interference from other searching points. Then, the covariance
matrix for the interference and noise is given as

(19) $\displaystyle\mathbf{R}_{IN,i}$
$\displaystyle=\mathbb{E}\left(\mathbf{i}(n)\mathbf{i}^{H}(n)\right)=\mathbf{R}-\mathbf{A}_{i}\mathbf{\Lambda}_{i}\mathbf{A}_{i}^{H}.$

The MLE of $\bar{\mathbf{x}}_{i}(n)$ can be constructed as a weighted least
squares (WLS) problem (Li and Stoica, 1996)

(20)
$\hat{\mathbf{x}}_{i}(n)=\arg\min_{\mathbf{x}}\left(\mathbf{y}(n)-\mathbf{A}_{i}\mathbf{x}\right)^{H}\mathbf{R}_{IN,i}^{-1}\left(\mathbf{y}(n)-\mathbf{A}_{i}\mathbf{x}\right),$

whose solution is given as

(21)
$\hat{\mathbf{x}}_{i}(n)=\left(\mathbf{A}_{i}^{H}\mathbf{R}_{IN,i}^{-1}\mathbf{A}_{i}\right)^{-1}\mathbf{A}_{i}^{H}\mathbf{R}_{IN,i}^{-1}\mathbf{y}(n).$

By substituting (19) into (21) and using the matrix inverse lemma, we have

(22) $\displaystyle\hat{\mathbf{x}}_{i}(n)$
$\displaystyle=\left(\mathbf{A}_{i}^{H}\mathbf{R}^{-1}\mathbf{A}_{i}\right)^{-1}\mathbf{A}_{i}^{H}\mathbf{R}^{-1}\mathbf{y}(n).$

However, (22) requires $\mathbf{R}$, which is unknown.

###  3.3. Estimation of $\mathbf{R}$

The estimation of $\mathbf{R}$ is given as

(23)
$\widehat{\mathbf{R}}=\sum_{i=1}^{N_{G}}\mathbf{A}_{i}\widehat{\mathbf{\Lambda}}_{i}\mathbf{A}_{i}^{H}+\sigma_{v}^{2}\mathbf{I},$

where we assume the noise power $\sigma_{v}^{2}$ is perfectly obtained. Note
that $\widehat{\mathbf{\Lambda}}_{i}$ is the original variable we want to
estimate in (17).

###  3.4. ISR Algorithm

We need $\widehat{\mathbf{\Lambda}}_{i}$ to estimate the spectrum for
localization purposes. It follows from (17) that
$\widehat{\mathbf{\Lambda}}_{i}$ depends on $\bar{\mathbf{x}}_{i}(n)$. Then,
$\hat{\mathbf{x}}_{i}(n)$ depends on $\mathbf{R}$ as shown in (22). Finally,
(23) gives the relation between $\widehat{\mathbf{R}}$ and
$\widehat{\mathbf{\Lambda}}_{i}$. Therefore, we propose an iterative algorithm
to find a fixed point for $\hat{\mathbf{x}}_{i}(n)$,
$\widehat{\mathbf{\Lambda}}_{i}$, and $\widehat{\mathbf{R}}$ by updating them
in a cyclic order, i.e.,

(24)
$\cdots\to\widehat{\mathbf{R}}^{(t)}\to\left\\{\hat{\mathbf{x}}_{i}^{(t+1)}(n)\right\\}_{n=1}^{N_{s}}\to\widehat{\mathbf{\Lambda}}_{i}^{(t+1)}\to\widehat{\mathbf{R}}^{(t+1)}\to\cdots.$

where $\hat{\mathbf{x}}_{i}^{(t)}(n)$, $\widehat{\mathbf{\Lambda}}_{i}^{(t)}$,
and $\widehat{\mathbf{R}}^{(t)}$ denote $\hat{\mathbf{x}}_{i}(n)$,
$\widehat{\mathbf{\Lambda}}_{i}$, and $\widehat{\mathbf{R}}$ at the $t$-th
iteration, respectively. The initialization of $\widehat{\mathbf{R}}$ is given
by the block-diagonal SCM as defined in (11). The proposed ISR algorithm is
summarized in Algorithm 1.

1 Input: The samples $\\{\mathbf{y}(n)\\}_{n=1}^{N_{s}}$, the searching grid
$\left\\{\bar{\mathbf{p}}_{1},\bar{\mathbf{p}}_{2},\cdots,\bar{\mathbf{p}}_{N_{G}}\right\\}$,
the noise power $\sigma_{v}^{2}$.

2Initialize: $t=0$, $\widehat{\mathbf{R}}^{(0)}$ based on (11).

Repeat:

  1. (1)

Update $\hat{\mathbf{x}}_{i}^{(t+1)}(n),n=1,\cdots,N_{s},i=1,\cdots,N_{G},$
based on (22), i.e.,

$\hat{\mathbf{x}}_{i}^{(t+1)}(n)=\left(\mathbf{A}_{i}^{H}\left(\widehat{\mathbf{R}}^{(t)}\right)^{-1}\mathbf{A}_{i}\right)^{-1}\mathbf{A}_{i}^{H}\left(\widehat{\mathbf{R}}^{(t)}\right)^{-1}\mathbf{y}(n).$

  2. (2)

Update $\widehat{\mathbf{\Lambda}}_{i}^{(t+1)}$ based on (17), i.e.,

$\widehat{\mathbf{\Lambda}}_{i}^{(t+1)}=\frac{1}{N_{s}}\sum_{n=1}^{N_{s}}\hat{\mathbf{x}}_{i}^{(t+1)}(n)\left(\hat{\mathbf{x}}_{i}^{(t+1)}(n)\right)^{H}.$

  3. (3)

Update $\widehat{\mathbf{R}}^{(t+1)}$ based on (23), i.e.,

$\widehat{\mathbf{R}}^{(t+1)}=\sum_{i=1}^{N_{G}}\mathbf{A}_{i}\widehat{\mathbf{\Lambda}}_{i}^{(t+1)}\mathbf{A}_{i}^{H}+\sigma_{v}^{2}\mathbf{I}.$

  4. (4)

Update the spectrum

$P^{(t+1)}(\hat{\mathbf{p}}_{i})=\frac{1}{N_{R}}\mathrm{Tr}\left(\widehat{\mathbf{\Lambda}}_{i}^{(t+1)}\right).$

  5. (5)

$t\leftarrow t+1$.

Until some termination condition is met.

Algorithm 1 The proposed ISR algorithm

Remark: We now compare the ISR algorithm with the MVDR (10) and BS (12) based
methods.

  1. (1)

The MVDR method assumes the block-diagonal covariance structure in (11), which
reduces the computational complexity at the cost of worse performance.
Furthermore, it estimates the received signal power from one searching point
based on the output of the MVDR combiner, which may not be optimal.

  2. (2)

The BS method estimates $\mathbf{R}$ by the conventional SCM method (13),
which is theoretically optimal but requires a large number of samples. As a
result, the estimation performance will suffer from limited number of samples.

  3. (3)

The proposed IRS method utilizes sparse recovery technique to represent the
received signal in a lower dimension and reduces the searching space.
Furthermore, by connecting three key parameters, $\hat{\mathbf{x}}_{i}(n)$,
$\widehat{\mathbf{\Lambda}}_{i}$, and $\widehat{\mathbf{R}}$, the proposed
method iteratively improves the estimation accuracy even with very few
samples, achieving a communication-efficient localization.

Figure 1. Illustration of scenario 1.

##  4\. Simulation

In this section, we demonstrate the performance of the proposed algorithm by
simulation. We consider an OFDMA systems where the transmitted signals from
different targets are assumed to be single-carrier signals with different
frequencies. The signal-to-noise ratio (SNR) is defined as
$\mathrm{SNR}=\frac{1}{LK}\sum_{l=1}^{L}\sum_{k=1}^{K}\frac{\sigma_{l,k}^{2}}{\sigma_{v}^{2}}.$
For comparison purpose, we consider two scenarios.

###  4.1. Scenario 1: Linear Monitoring Area

We begin with a simple case where two SNs and two targets are located on the
same line, as illustrated in Fig. 1. Assume the two SNs are located at $(5,0)$
and $(15,0)$, where all coordinates are in meters. The monitoring area is the
line from $(0,0)$ to $(20,0)$ with step 0.1. The height of the received
antenna arrays is 6 meters and each SN is equipped with $N_{R}=64$ antennas.
The users transmit OFDM signals on a typical 802.11ac WiFi standard with
$\Delta f=312.5kHz$ and $|s_{k}(n)|=1$.

Figure 2. Comparison of the power spectrum with 2 targets.

Fig. 2 shows the estimated power spectrum for the two targets located at
$(7.8,0)$ and $(8.0,0)$, which are close to each other. The SNR is set as $-5$
dB and the number of samples is $N_{s}=2$. It can be observed from Fig. 2 that
the ISR algorithm achieves better resolution than the MVDR and BS methods. In
particular, the MVDR-based method only has one peak corresponding to the
target at $(8,0)$, while the BS method can not differentiate the two targets.

Figure 3. Comparison of the power spectrum with 8 targets.

Fig. 3 shows the power spectrum for eight targets uniformly distributed
between $(2,0)$ and $(9,0)$. The SNR is set as $\mathrm{SNR}=0$ dB and the
number of samples is $N_{s}=8$. We can observe from Fig. 3 that the ISR
algorithm achieves good localization performance, while the MVDR and BS
methods are not able to differentiate all targets. This is because the number
of samples is not sufficient for the MVDR and BS methods to recover the
subspace of all targets from the SCM.

Figure 4. Comparison of the MSE versus the number of samples.

To quantify the estimation accuracy, we utilize the mean squared error (MSE),
i.e.,
$\text{MSE}=\frac{1}{N_{mc}}\sum_{n=1}^{N_{mc}}\left\|\hat{\mathbf{p}}_{n}-\mathbf{p}\right\|^{2}$
as the performance metric, where $\hat{\mathbf{p}}_{n}$ denotes the estimated
position at the $n$-th Monte-Carlo trial and $N_{mc}$ represents the number of
Monte-Carlo trials. Here we set $N_{R}=36$, $\text{SNR}=0$dB, $N_{mc}=10^{4}$
and consider four targets, which are uniformly located from $(2,0)$ to
$(5,0)$. The localization is achieved by finding the highest peak in the power
spectrum. Fig. 4 compares the MSE of different methods versus the number of
samples. It can be observed that the ISR algorithm can achieve better accuracy
than the other two methods, especially when $N_{s}\geq 2$. How to improve the
performance with $N_{s}=1$ would be left as future work.

###  4.2. Scenario 2: Square Monitoring Area

Figure 5. Illustration of scenario 2.

Next, we consider a square monitoring area with 4 SNs, as illustrated in Fig.
5. The SNs are placed at $(0,10)$, $(10,20)$, $(20,10)$, and $(10,0)$, and
equipped with 64 antennas. The searching area is from $1$ to $19$ with step
$0.5$ in both $x$ and $y$ directions. Two targets are placed at $(3.5,13.5)$
and $(3.5,14.5)$ and the SNR is set as $3$ dB.

(a)  (b)  (c)

Figure 6. Comparison of the power spectrum with 2 targets. (a) MVDR; (b) BS;
(c) ISR.

Fig. 6 shows the power spectrum estimated by three different methods with
$N_{s}=2$. It can be observed that there are peaks at the positions
corresponding the actual targets for all three methods. However, the peaks for
MVDR are week and will be submerged by other false peaks. The BS method
achieves better performance than MVDR, but the ISR algorithm can clearly form
two peaks in the power spectrum with a clean background.

(a)  (b)  (c)

Figure 7. Comparison of the power spectrum with 8 targets. (a) MVDR; (b) BS;
(c) ISR.

Finally, we consider a more complex scenario with 8 targets, which are
uniformly located from $(2.5,2.5)$ to $(16.5,16.5)$. The SNR is set as $5$ dB
and $N_{s}=16$. We can observe from Fig. 7 that some targets can not be
detected by the MVDR and BS methods, whereas the ISR algorithm can still
provide good localization performance.

##  5\. Conclusion

This paper considered the direct passive localization in perceptive mobile
networks, where the localization of several targets is collaboratively
achieved by several sensing nodes based on the uplink communication signals.
Different from existing methods, the proposed ISR algorithm exploited the
sparse structure of the problem and improved the estimation accuracy for the
covariance matrix and the power spectrum in an iterative manner. Experiment
results showed the better localization performance of the proposed ISR
algorithm than existing methods. The work in this paper demonstrated the
advantage of the ISR algorithm and provided a communication-efficient multi-
target localization method for future perceptive mobile networks.

## Acknowledgements

We are grateful to the anonymous reviewers for their valuable and constructive
comments. This work was supported by the HKUST-BDR Joint Research Institute
under Grant OKT22EG04.

## References

  * (1)
  * Capon (1969) J. Capon. 1969.  High-resolution frequency-wavenumber spectrum analysis.  _Proc. IEEE_ 57, 8 (1969), 1408–1418.
  * Li and Stoica (1996) Jian Li and Petre Stoica. 1996.  An adaptive filtering approach to spectral estimation and SAR imaging.  _IEEE Trans. Signal Process._ 44, 6 (1996), 1469–1484.
  * Liang et al. (2016) Junli Liang, Dong Wang, Li Su, Badong Chen, H Chen, and Hing Cheung So. 2016\.  Robust MIMO radar target localization via nonconvex optimization.  _Signal Process._ 122 (2016), 33–38.
  * Liu et al. (2018) Fan Liu, Christos Masouros, Ang Li, Huafei Sun, and Lajos Hanzo. 2018.  MU-MIMO Communications With MIMO Radar: From Co-Existence to Joint Transmission.  _IEEE Trans. Wirel. Commun._ 17, 4 (2018), 2755–2770.
  * Liu et al. (2020) Fan Liu, Weijie Yuan, Christos Masouros, and Jinhong Yuan. 2020\.  Radar-assisted predictive beamforming for vehicular links: Communication served by sensing.  _IEEE Trans. Wirel. Commun._ 19, 11 (2020), 7704–7719.
  * Noroozi et al. (2017) Ali Noroozi, Amir Hosein Oveis, and Mohammad Ali Sebt. 2017\.  Iterative target localization in distributed MIMO radar from bistatic range measurements.  _IEEE Signal Process. Lett._ 24, 11 (2017), 1709–1713.
  * Noroozi and Sebt (2015) Ali Noroozi and Mohammad Ali Sebt. 2015.  Target localization from bistatic range measurements in multi-transmitter multi-receiver passive radar.  _IEEE Signal Process. Lett._ 22, 12 (2015), 2445–2449.
  * Song et al. (2011) S.H. Song, GL Chen, and Khaled B Letaief. 2011.  Localized or interleaved? A tradeoff between diversity and CFO interference in multipath channels.  _IEEE Trans. Wirel. Commun._ 10, 9 (2011), 2829–2834.
  * Stoica and Selen (2004) P. Stoica and Y. Selen. 2004.  Model-order selection: a review of information criterion rules.  _IEEE Signal Processing Magazine_ 21, 4 (2004), 36–47.
  * Tzafri and Weiss (2016) Liran Tzafri and Anthony J. Weiss. 2016.  High-Resolution Direct Position Determination Using MVDR.  _IEEE Trans. Wirel. Commun._ 15, 9 (2016), 6449–6461.
  * Tzoreff and Weiss (2017) Elad Tzoreff and Anthony J. Weiss. 2017.  Expectation-maximization algorithm for direct position determination.  _Signal Process._ 133 (2017), 32–39.
  * Xie et al. (2022a) Lei Xie, Zishu He, Jun Tong, Tianle Liu, Jun Li, and Jiangtao Xi. 2022a.  Regularized Covariance Estimation for Polarization Radar Detection in Compound Gaussian Sea Clutter.  _IEEE Trans. Geosci. Remote Sens._ 60 (2022), 1–16.
  * Xie et al. (2020) Lei Xie, Zishu He, Jun Tong, and Wei Zhang. 2020\.  A recursive angle-Doppler channel selection method for reduced-dimension space-time adaptive processing.  _IEEE Trans. Aerosp. Electron. Syst._ 56, 5 (2020), 3985–4000.
  * Xie and Song (2022) Lei Xie and S.H. Song. 2022\.  Networked Sensing with AI-Empowered Environment Estimation: Exploiting Macro-Diversity and Array Gain in Perceptive Mobile Networks.  _arXiv preprint arXiv:2205.11331_ (2022).
  * Xie et al. (2022b) Lei Xie, S.H. Song, Yonina C Eldar, and Khaled B Letaief. 2022b.  Collaborative Sensing in Perceptive Mobile Networks: Opportunities and Challenges.  _arXiv preprint arXiv:2205.15805_ (2022).
  * Xie et al. (2022) Lei Xie, Peilan Wang, S.H. Song, and Khaled B Letaief. 2022\.  Perceptive Mobile Network with Distributed Target Monitoring Terminals: Leaking Communication Energy for Sensing.  _IEEE Trans. Wirel. Commun._ Early Access (2022).
  * Zhang et al. (2021) Andrew Zhang, Md. Lushanur Rahman, Xiaojing Huang, Yingjie Jay Guo, Shanzhi Chen, and Robert W. Heath. 2021\.  Perceptive Mobile Networks: Cellular Networks With Radio Vision via Joint Communication and Radar Sensing.  _IEEE Veh. Technol. Mag._ 16, 2 (2021), 20–30.
  * Zhao et al. (2020) Hanying Zhao, Ning Zhang, and Yuan Shen. 2020.  Beamspace direct localization for large-scale antenna array systems.  _IEEE Trans. Signal Process._ 68 (2020), 3529–3544.


@Stringcvpr = Proc. of the IEEE/CVF Conference on Computer Vision and Pattern
Recognition (CVPR) @Stringcvprw = Proc. of the IEEE/CVF Conference on Computer
Vision and Pattern Recognition Workshops (CVPRW) @Stringeccv = Proc. of the
European Conference on Computer Vision (ECCV) @Stringeccvw = Proc. of the
European Conference on Computer Vision Workshops (ECCVW) @Stringiccv = Proc.
of the IEEE/CVF International Conference on Computer Vision (ICCV)
@Stringiccvw = Proc. of the IEEE/CVF International Conference on Computer
Vision Workshops (ICCVW) @Stringnips = Proc. of Advances in Neural Information
Processing Systems (NeurIPS) @Stringnipsw = Proc. of Advances in Neural
Information Processing Systems Workshops @Stringicml = Proc. of the
International Conference on Machine Learning (ICML) @Stringiclr = Proc. of the
International Conference on Learning Representations (ICLR) @Stringicpr =
Proc. of the IAPR International Conference on Pattern Recognition (ICPR)
@Stringtdv = Proc. of the International Conference on 3D Vision (3DV)
@Stringicra = Proc. of the IEEE International Conference on Robotics and
Automation (ICRA) @Stringiros = Proc. of the IEEE/RSJ International Conference
on Intelligent Robots and Systems (IROS) @Stringcorl = Proc. of the
International Conference on Robot Learning (CoRL) @Stringemnlp = Proc. of the
International Conference on Empirical Methods in Natural Language Processing
(EMNLP) @Stringnaacl = Proc. of the Conference of the North American Chapter
of the Association for Computational Linguistics (NAACL) @Stringaaai = Proc.
of the AAAI Conference on Artificial Intelligence @Stringijcai = Proc. of the
International Joint Conference on Artificial Intelligence (IJCAI) @Stringuai =
Proc. of the Conference on Uncertainty in Artificial Intelligence (UAI)
@Stringicaps = Proc. of the International Conference on Automated Planning and
Scheduling (ICAPS) @Stringecai = Proc. of the European Conference on
Artificial Intelligence (ECAI)

@inproceedingsp_cloud, author=Ben Salah, Imeen and Kramm, Sebastien and
Demonceaux, Cédric and Vasseur, Pascal, booktitle=Proc. of the IEEE
International Conference on Intelligent Transportation Systems (ITSC),
title=Summarizing large scale 3D point cloud for navigation tasks, year=2017

@articlechen2019learning, title=Learning exploration policies for navigation,
author=Chen, Tao and Gupta, Saurabh and Gupta, Abhinav, journal=iclr,
year=2019

@inproceedingsnarasimhan2020semantic, title=Seeing the un-scene: Learning
amodal semantic maps for room navigation, author=Narasimhan, Medhini and
Wijmans, Erik and Chen, Xinlei and Darrell, Trevor and Batra, Dhruv and
Parikh, Devi and Singh, Amanpreet, booktitle=eccv, year=2020

@inproceedingsbeeching2020topological, title=Learning to plan with uncertain
topological maps, author=Beeching, Edward and Dibangoye, Jilles and Simonin,
Olivier and Wolf, Christian, booktitle=eccv, year=2020

@inproceedingsgupta2017cognitive, title=Cognitive mapping and planning for
visual navigation, author=Gupta, Saurabh and Davidson, James and Levine,
Sergey and Sukthankar, Rahul and Malik, Jitendra, booktitle=cvpr, year=2017

@inproceedingstrpo, title=Trust region policy optimization, author=Schulman,
John and Levine, Sergey and Abbeel, Pieter and Jordan, Michael and Moritz,
Philipp, booktitle=icml, year=2015

@inproceedingssac, title=Soft actor-critic: Off-policy maximum entropy deep
reinforcement learning with a stochastic actor, author=Haarnoja, Tuomas and
Zhou, Aurick and Abbeel, Pieter and Levine, Sergey, booktitle=icml, year=2018

@inproceedingsa3c, title=Asynchronous methods for deep reinforcement learning,
author=Mnih, Volodymyr and Badia, Adria Puigdomenech and Mirza, Mehdi and
Graves, Alex and Lillicrap, Timothy and Harley, Tim and Silver, David and
Kavukcuoglu, Koray, booktitle=icml, year=2016

@articleppo, title=Proximal Policy Optimization algorithms, author=Schulman,
John and Wolski, Filip and Dhariwal, Prafulla and Radford, Alec and Klimov,
Oleg, journal=arXiv preprint arXiv:1707.06347, year=2017

@inproceedingswijmans2019dd, title=DD-PPO: Learning near-perfect pointgoal
navigators from 2.5 billion frames, author=Wijmans, Erik and Kadian, Abhishek
and Morcos, Ari and Lee, Stefan and Essa, Irfan and Parikh, Devi and Savva,
Manolis and Batra, Dhruv, booktitle=iclr, year=2019

@articleslam, title=Learning to explore using active neural slam,
author=Chaplot, Devendra Singh and Gandhi, Dhiraj and Gupta, Saurabh and
Gupta, Abhinav and Salakhutdinov, Ruslan, journal=arXiv preprint
arXiv:2004.05155, year=2020

@inproceedingschaplot2020neural, title=Neural Topological SLAM for Visual
Navigation, author=Devendra Singh Chaplot and Ruslan Salakhutdinov and Abhinav
Gupta and Saurabh Gupta, booktitle = cvpr, year=2020

@inproceedingspredoccupancy, title=Occupancy anticipation for efficient
exploration and navigation, author=Ramakrishnan, Santhosh K and Al-Halah, Ziad
and Grauman, Kristen, booktitle=eccv, year=2020

@inproceedingsye2020auxiliary, title=Auxiliary Tasks Speed Up Learning
PointGoal Navigation, author=Ye, Joel and Batra, Dhruv and Wijmans, Erik and
Das, Abhishek, booktitle=corl, year=2020

@articlecpca, title=Neural predictive belief representations, author=Guo,
Zhaohan Daniel and Azar, Mohammad Gheshlaghi and Piot, Bilal and Pires,
Bernardo A. and Munos, Rémi, journal=arXiv preprint arXiv:1811.06407,
year=2018

@inproceedingsid, title=Curiosity-driven exploration by self-supervised
prediction, author=Pathak, Deepak and Agrawal, Pulkit and Efros, Alexei A. and
Darrell, Trevor, booktitle=icml, year=2017

@inproceedingsmultion, title=MultiON: Benchmarking semantic map memory using
multi-object navigation, author=Wani, Saim and Patel, Shivansh and Jain, Unnat
and Chang, Angel X. and Savva, Manolis, booktitle=nips, year=2020

@articlerearrangement, title=Rearrangement: A challenge for embodied ai,
author=Batra, Dhruv and Chang, Angel X and Chernova, Sonia and Davison, Andrew
J and Deng, Jia and Koltun, Vladlen and Levine, Sergey and Malik, Jitendra and
Mordatch, Igor and Mottaghi, Roozbeh and others, journal=arXiv preprint
arXiv:2011.01975, year=2020

@articleinteractive, title=Interactive gibson benchmark: A benchmark for
interactive navigation in cluttered environments, author=Xia, Fei and Shen,
William B and Li, Chengshu and Kasimbeg, Priya and Tchapmi, Micael Edmond and
Toshev, Alexander and Martín-Martín, Roberto and Savarese, Silvio,
journal=IEEE Robotics and Automation Letters, volume=5, number=2,
pages=713–720, year=2020

@inproceedingskrantz2020beyond, title=Beyond the nav-graph: Vision-and-
language navigation in continuous environments, author=Krantz, Jacob and
Wijmans, Erik and Majumdar, Arjun and Batra, Dhruv and Lee, Stefan,
booktitle=eccv, year=2020 @inproceedingscampari2022online, title=Online
Learning of Reusable Abstract Models for Object Goal Navigation,
author=Campari, Tommaso and Lamanna, Leonardo and Traverso, Paolo and
Serafini, Luciano and Ballan, Lamberto, booktitle=cvpr, year=2022

@articledeitke2022retrospectives, title=Retrospectives on the Embodied AI
Workshop, author=Deitke, Matt and Batra, Dhruv and Bisk, Yonatan and Campari,
Tommaso and Chang, Angel X. and Chaplot, Devendra Singh and Chen, Changan and
D’Arpino, Claudia Pérez and Ehsani, Kiana and Farhadi, Ali and others,
journal=arXiv preprint arXiv:2210.06849, year=2022

@articleeaisurvey, title=Deep learning for embodied vision navigation: A
survey, author=Zhu, Fengda and Zhu, Yi and Lee, Vincent and Liang, Xiaodan and
Chang, Xiaojun, journal=arXiv preprint arXiv:2108.04097, year=2021

@inproceedingschang2017matterport, title=Matterport3D: Learning from RGB-D
Data in Indoor Environments, author=Chang, Angel and Dai, Angela and
Funkhouser, Thomas and Halber, Maciej and Niessner, Matthias and Savva,
Manolis and Song, Shuran and Zeng, Andy and Zhang, Yinda, booktitle=tdv,
year=2017

@inproceedingssavva2019habitat, title=Habitat: A platform for embodied AI
research, author=Savva, Manolis and Kadian, Abhishek and Maksymets, Oleksandr
and Zhao, Yili and Wijmans, Erik and Jain, Bhavana and Straub, Julian and Liu,
Jia and Koltun, Vladlen and Malik, Jitendra and others, booktitle=iccv,
year=2019

@inproceedingsgibson, title=Gibson Env: Real-World Perception for Embodied
Agents, author=Xia, Fei and Zamir, Amir R. and He, Zhiyang and Sax, Alexander
and Malik, Jitendra and Savarese, Silvio, booktitle=cvpr, year=2018

@articlesim2real, title=Sim2real predictivity: Does evaluation in simulation
predict real-world performance?, author=Kadian, Abhishek and Truong, Joanne
and Gokaslan, Aaron and Clegg, Alexander and Wijmans, Erik and Lee, Stefan and
Savva, Manolis and Chernova, Sonia and Batra, Dhruv, journal=IEEE Robotics and
Automation Letters, volume=5, number=4, pages=6670–6677, year=2020,
publisher=IEEE

@inproceedingsjain2019two, title=Two body problem: Collaborative visual task
completion, author=Jain, Unnat and Weihs, Luca and Kolve, Eric and Rastegari,
Mohammad and Lazebnik, Svetlana and Farhadi, Ali and Schwing, Alexander G and
Kembhavi, Aniruddha, booktitle=Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition, pages=6689–6699, year=2019

@inproceedingsjain2020cordial, title=A cordial sync: Going beyond marginal
policies for multi-agent embodied tasks, author=Jain, Unnat and Weihs, Luca
and Kolve, Eric and Farhadi, Ali and Lazebnik, Svetlana and Kembhavi,
Aniruddha and Schwing, Alexander, booktitle=European Conference on Computer
Vision, pages=471–490, year=2020, organization=Springer

@articlesurvey1, title=A survey on human-aware robot navigation,
author=Möller, Ronja and Furnari, Antonino and Battiato, Sebastiano and Härmä,
Aki and Farinella, Giovanni Maria, journal=Robotics and Autonomous Systems,
volume=145, pages=103837, year=2021, publisher=Elsevier

@inproceedingspartsey2022mapping, title=Is Mapping Necessary for Realistic
PointGoal Navigation?, author=Partsey, Ruslan and Wijmans, Erik and Yokoyama,
Naoki and Dobosevych, Oles and Batra, Dhruv and Maksymets, Oleksandr,
booktitle=cvpr, year=2022 @inproceedingsramrakhya2022habitat, title=Habitat-
Web: Learning Embodied Object-Search Strategies from Human Demonstrations at
Scale, author=Ramrakhya, Ram and Undersander, Eric and Batra, Dhruv and Das,
Abhishek, booktitle=cvpr, year=2022

@articlexia2020interactive, title=Interactive gibson benchmark: A benchmark
for interactive navigation in cluttered environments, author=Xia, Fei and
Shen, William B and Li, Chengshu and Kasimbeg, Priya and Tchapmi, Micael
Edmond and Toshev, Alexander and Martín-Martín, Roberto and Savarese, Silvio,
journal=IEEE Robotics and Automation Letters, volume=5, number=2,
pages=713–720, year=2020, publisher=IEEE

@inproceedingsperez2021robot, title=Robot navigation in constrained pedestrian
environments using reinforcement learning, author=Pérez-D’Arpino, Claudia and
Liu, Can and Goebel, Patrick and Martín-Martín, Roberto and Savarese, Silvio,
booktitle=icra, year=2021 @inproceedingsramakrishnan2022poni, title=PONI:
Potential Functions for ObjectGoal Navigation with Interaction-free Learning,
author=Ramakrishnan, Santhosh K. and Chaplot, Devendra Singh and Al-Halah,
Ziad and Malik, Jitendra and Grauman, Kristen, booktitle=cvpr, year=2022
@articleramakrishnan2021habitat, title=Habitat-matterport 3D dataset (HM3D):
1000 large-scale 3D environments for embodied AI, author=Ramakrishnan,
Santhosh K. and Gokaslan, Aaron and Wijmans, Erik and Maksymets, Oleksandr and
Clegg, Alex and Turner, John and Undersander, Eric and Galuba, Wojciech and
Westbury, Andrew and Chang, Angel X. and others, journal=arXiv preprint
arXiv:2109.08238, year=2021 @articledeitke2022procthor, title=Procthor: Large-
scale Embodied AI using procedural generation, author=Deitke, Matt and
VanderBilt, Eli and Herrasti, Alvaro and Weihs, Luca and Salvador, Jordi and
Ehsani, Kiana and Han, Winson and Kolve, Eric and Farhadi, Ali and Kembhavi,
Aniruddha and others, journal=arXiv preprint arXiv:2206.06994, year=2022
@miscchallenge, title=IGibson Challenge 2022, author=,
url=https://svl.stanford.edu/igibson/challenge.html, journal=Igibson Challenge
2022

@articlekolve2017ai2, title=Ai2-thor: An interactive 3d environment for visual
ai, author=Kolve, Eric and Mottaghi, Roozbeh and Han, Winson and VanderBilt,
Eli and Weihs, Luca and Herrasti, Alvaro and Gordon, Daniel and Zhu, Yuke and
Gupta, Abhinav and Farhadi, Ali, journal=arXiv preprint arXiv:1712.05474,
year=2017 @inproceedingsshen2021igibson, title=iGibson 1.0: a simulation
environment for interactive tasks in large realistic scenes, author=Shen,
Bokui and Xia, Fei and Li, Chengshu and Martín-Martín, Roberto and Fan, Linxi
and Wang, Guanzhi and Pérez-D’Arpino, Claudia and Buch, Shyamal and
Srivastava, Sanjana and Tchapmi, Lyne and others, booktitle=iros, year=2021

@articleyokoyama2021learning, title=Learning Robust Agents for Visual
Navigation in Dynamic Environments: The Winning Entry of iGibson Challenge
2021, author=Yokoyama, Naoki and Luo, Qian and Batra, Dhruv and Ha, Sehoon,
journal=iros, year=2022

@articlepirk2022protocol, title=A Protocol for Validating Social Navigation
Policies, author=Pirk, Sören and Lee, Edward and Xiao, Xuesu and Takayama,
Leila and Francis, Anthony and Toshev, Alexander, journal=arXiv preprint
arXiv:2204.05443, year=2022

@articleduan2022survey, title=A survey of embodied ai: From simulators to
research tasks, author=Duan, Jiafei and Yu, Samson and Tan, Hui Li and Zhu,
Hongyuan and Tan, Cheston, journal=IEEE Transactions on Emerging Topics in
Computational Intelligence, year=2022, publisher=IEEE

@incollectionorca, title=Reciprocal n-body collision avoidance, author=Berg,
Jur van den and Guy, Stephen J and Lin, Ming and Manocha, Dinesh,
booktitle=Robotics Research, pages=3–19, year=2011, publisher=Springer

@inproceedingsrvo, title=Reciprocal velocity obstacles for real-time multi-
agent navigation, author=Van den Berg, Jur and Lin, Ming and Manocha, Dinesh,
booktitle=icra, year=2008

@inproceedingsfreeze, title=Unfreezing the robot: Navigation in dense,
interacting crowds, author=Trautman, Peter and Krause, Andreas,
booktitle=iros, year=2010

@articlempdyn, title=Probabilistically safe motion planning to avoid dynamic
obstacles with uncertain motion patterns, author=Aoude, Georges S. and Luders,
Brandon D. and Joseph, Joshua M. and Roy, Nicholas and How, Jonathan P.,
journal=Autonomous Robots, volume=35, number=1, pages=51–76, year=2013,
publisher=Springer

@inproceedingssocialforces, title=Social-aware robot navigation in urban
environments, author=Ferrer, Gonzalo and Garrell, Anais and Sanfeliu, Alberto,
booktitle=Proc. of thr European Conference on Mobile Robots, year=2013

@inproceedingscadrl, title=Decentralized non-communicating multiagent
collision avoidance with deep reinforcement learning, author=Chen, Yu Fan and
Liu, Miao and Everett, Michael and How, Jonathan P., booktitle=icra, year=2017

@inproceedingssocialattention, author = Changan Chen and Yuejiang Liu and Sven
Kreiss and Alexandre Alahi, title = Crowd-Robot Interaction: Crowd-aware Robot
Navigation with Attention-based Deep Reinforcement Learning, booktitle = icra,
year = 2019

@inproceedingssa-cadrl, title=Socially aware motion planning with deep
reinforcement learning, author=Chen, Yu Fan and Everett, Michael and Liu, Miao
and How, Jonathan P., booktitle=iros, year=2017

@articlesocial-graph, title=Deep Reinforcement Learning Based on Social
Spatial–Temporal Graph Convolution Network for Crowd Navigation, author=Lu,
Yazhou and Ruan, Xiaogang and Huang, Jing, journal=Machines, volume=10,
number=8, pages=703, year=2022, publisher=MDPI

@inproceedingssocialim, title=Socially compliant navigation through raw depth
inputs with generative adversarial imitation learning, author=Tai, Lei and
Zhang, Jingwei and Liu, Ming and Burgard, Wolfram, booktitle=icra, year=2018

@articlelearned, title=Deep-learned collision avoidance policy for distributed
multiagent navigation, author=Long, Pinxin and Liu, Wenxi and Pan, Jia,
journal=IEEE Robotics and Automation Letters, volume=2, number=2,
pages=656–663, year=2017, publisher=IEEE

@articlesurvey, title=Core challenges of social robot navigation: A survey,
author=Mavrogiannis, Christoforos and Baldini, Francesca and Wang, Allan and
Zhao, Dapeng and Trautman, Pete and Steinfeld, Aaron and Oh, Jean,
journal=arXiv preprint arXiv:2103.05668, year=2021

@inproceedingsmousavian2019visual, title=Visual representations for semantic
target driven navigation, author=Mousavian, Arsalan and Toshev, Alexander and
Fišer, Marek and Košecká, Jana and Wahid, Ayzaan and Davidson, James,
booktitle=icra, year=2019

@inproceedingsRamakrishnan2018sidekick, title=Sidekick policy learning for
active visual exploration, author=Ramakrishnan, Santhosh K and Grauman,
Kristen, booktitle=Proceedings of the European Conference on Computer Vision
(ECCV), pages=413–430, year=2018

@inproceedingsfang2019scene, title=Scene Memory Transformer for Embodied
Agents in Long-Horizon Tasks, author=Fang, Kuan and Toshev, Alexander and Fei-
Fei, Li and Savarese, Silvio, booktitle=cvpr, year=2019 ha
@articlehirose2019deep, title=Deep Visual MPC-Policy Learning for Navigation,
author=Hirose, Noriaki and Xia, Fei and Martín-Martín, Roberto and Sadeghian,
Amir and Savarese, Silvio, journal=arXiv preprint arXiv:1903.02749, year=2019

@inproceedingsshen2019situational, title=Situational Fusion of Visual
Representation for Visual Navigation, author=Shen, William B and Xu, Danfei
and Zhu, Yuke and Guibas, Leonidas J and Fei-Fei, Li and Savarese, Silvio,
booktitle=iccv, year=2019

@inproceedingswu2019bayesian, title=Bayesian Relational Memory for Semantic
Visual Navigation, author=Wu, Yi and Wu, Yuxin and Tamar, Aviv and Russell,
Stuart and Gkioxari, Georgia and Tian, Yuandong, booktitle=iccv, year=2019

@articlechang2020semantic, title=Semantic Visual Navigation by Watching
YouTube Videos, author=Chang, Matthew and Gupta, Arjun and Gupta, Saurabh,
journal=arXiv preprint arXiv:2006.10034, year=2020

@inproceedingschaplot2020object, author = Chaplot, Devendra Singh and Gandhi,
Dhiraj Prakashchand and Gupta, Abhinav and Salakhutdinov, Ruslan, title =
Object Goal Navigation using Goal-Oriented Semantic Exploration, booktitle =
nips, year = 2020

@inproceedingsolga2020, author = Zhiwei Deng Karthik Narasimhan Olga
Russakovsky, title = Evolving Graphical Planner: Contextual Global Planning
for Vision-and-Language Navigation, booktitle = nips, year = 2020

@inproceedingssax2019learning, title=Learning to Navigate Using Mid-Level
Visual Priors, author=Sax, Alexander and Zhang, Jeffrey O and Emi, Bradley and
Zamir, Amir R. and Savarese, Silvio and Guibas, Leonidas and Malik, Jitendra,
booktitle=corl, year=2020

@inproceedingsgupta2017cognitive, title=Cognitive mapping and planning for
visual navigation, author=Gupta, Saurabh and Davidson, James and Levine,
Sergey and Sukthankar, Rahul and Malik, Jitendra, booktitle=cvpr, year=2017

@articlegupta2017unifying, title=Unifying map and landmark based
representations for visual navigation, author=Gupta, Saurabh and Fouhey, David
and Levine, Sergey and Malik, Jitendra, journal=arXiv preprint
arXiv:1712.08125, year=2017

@articlechen2019learning, title=Learning exploration policies for navigation,
author=Chen, Tao and Gupta, Saurabh and Gupta, Abhinav, journal=arXiv preprint
arXiv:1903.01959, year=2019

@inproceedingscampari2020exploiting, title=Exploiting Scene-Specific Features
for Object Goal Navigation, author=Campari, Tommaso and Eccher, Paolo and
Serafini, Luciano and Ballan, Lamberto, booktitle=eccvw, year=2020

@inproceedingskumar2018visual, title=Visual memory for robust path following,
author=Kumar, Ashish and Gupta, Saurabh and Fouhey, David and Levine, Sergey
and Malik, Jitendra, booktitle=Advances in Neural Information Processing
Systems, pages=765–774, year=2018

@inproceedingszamir2018taskonomy, title=Taskonomy: Disentangling task transfer
learning, author=Zamir, Amir R. and Sax, Alexander and Shen, William and
Guibas, Leonidas J. and Malik, Jitendra and Savarese, Silvio, booktitle=cvpr,
year=2018

@articleanderson2018evaluation, title=On evaluation of embodied navigation
agents, author=Anderson, Peter and Chang, Angel and Chaplot, Devendra Singh
and Dosovitskiy, Alexey and Gupta, Saurabh and Koltun, Vladlen and Kosecka,
Jana and Malik, Jitendra and Mottaghi, Roozbeh and Savva, Manolis and Zamir,
Amir R., journal=arXiv preprint arXiv:1807.06757, year=2018

@inproceedingsvaswani2017attention, title=Attention is all you need,
author=Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob
and Jones, Llion and Gomez, Aidan and Kaiser, Lukasz and Polosukhin, Illia,
booktitle=nips, year=2017

@articlekadian2019we, title=Are we making real progress in simulated
environments? measuring the sim2real gap in embodied visual navigation,
author=Kadian, Abhishek and Truong, Joanne and Gokaslan, Aaron and Clegg,
Alexander and Wijmans, Erik and Lee, Stefan and Savva, Manolis and Chernova,
Sonia and Batra, Dhruv, journal=arXiv preprint arXiv:1912.06321, year=2019

@inproceedingschaplot2020learning, title=Learning To Explore Using Active
Neural SLAM, author=Chaplot, Devendra Singh and Gandhi, Dhiraj and Gupta,
Saurabh and Gupta, Abhinav and Salakhutdinov, Ruslan, booktitle=iclr,
year=2019

@articlesong2016ssc, author = Song, Shuran and Yu, Fisher and Zeng, Andy and
Chang, Angel X and Savva, Manolis and Funkhouser, Thomas, title = Semantic
Scene Completion from a Single Depth Image, journal = Proceedings of 30th IEEE
Conference on Computer Vision and Pattern Recognition, year = 2017,

@inproceedingsanderson2018vision, title=Vision-and-language navigation:
Interpreting visually-grounded navigation instructions in real environments,
author=Anderson, Peter and Wu, Qi and Teney, Damien and Bruce, Jake and
Johnson, Mark and Sünderhauf, Niko and Reid, Ian and Gould, Stephen and van
den Hengel, Anton, booktitle=cvpr, year=2018

@articlereplica19arxiv, title = The Replica Dataset: A Digital Replica of
Indoor Spaces, author = Julian Straub and Thomas Whelan and Lingni Ma and
Yufan Chen and Erik Wijmans and Simon Green and Jakob J. Engel and Raul Mur-
Artal and Carl Ren and Shobhit Verma and Anton Clarkson and Mingfei Yan and
Brian Budge and Yajie Yan and Xiaqing Pan and June Yon and Yuyang Zou and
Kimberly Leon and Nigel Carter and Jesus Briales and Tyler Gillingham and
Elias Mueggler and Luis Pesqueira and Manolis Savva and Dhruv Batra and Hauke
M. Strasdat and Renzo De Nardi and Michael Goesele and Steven Lovegrove and
Richard Newcombe , journal = arXiv preprint arXiv:1906.05797, year = 2019

@articlekavraki1996probabilistic, title=Probabilistic roadmaps for path
planning in high-dimensional configuration spaces, author=Kavraki, Lydia E and
Svestka, Petr and Latombe, J-C and Overmars, Mark H, journal=IEEE transactions
on Robotics and Automation, volume=12, number=4, pages=566–580, year=1996,
publisher=IEEE

@bookcanny1988complexity, title=The complexity of robot motion planning,
author=Canny, John, year=1988, publisher=MIT press

@articlelavalle2001rapidly, title=Rapidly-exploring random trees: Progress and
prospects, author=LaValle, Steven M and Kuffner, James J, journal=Algorithmic
and computational robotics: new directions, number=5, pages=293–308, year=2001

@articlemirowski2016learning, title=Learning to navigate in complex
environments, author=Mirowski, Piotr and Pascanu, Razvan and Viola, Fabio and
Soyer, Hubert and Ballard, Andrew J and Banino, Andrea and Denil, Misha and
Goroshin, Ross and Sifre, Laurent and Kavukcuoglu, Koray and others,
journal=arXiv preprint arXiv:1611.03673, year=2016

@inproceedingslample2017playing, title=Playing FPS games with deep
reinforcement learning, author=Lample, Guillaume and Chaplot, Devendra Singh,
booktitle=aaai, year=2017

@articlesavva2017minos, title=MINOS: Multimodal indoor simulator for
navigation in complex environments, author=Savva, Manolis and Chang, Angel X.
and Dosovitskiy, Alexey and Funkhouser, Thomas and Koltun, Vladlen,
journal=arXiv preprint arXiv:1712.03931, year=2017

@inproceedingsmnih2016asynchronous, title=Asynchronous methods for deep
reinforcement learning, author=Mnih, Volodymyr and Badia, Adria Puigdomenech
and Mirza, Mehdi and Graves, Alex and Lillicrap, Timothy and Harley, Tim and
Silver, David and Kavukcuoglu, Koray, booktitle=icml, year=2016

@articlepaszke2017automatic, title=Automatic differentiation in pytorch,
author=Paszke, Adam and Gross, Sam and Chintala, Soumith and Chanan, Gregory
and Yang, Edward and DeVito, Zachary and Lin, Zeming and Desmaison, Alban and
Antiga, Luca and Lerer, Adam, year=2017

@articleschulman2017proximal, title=Proximal policy optimization algorithms,
author=Schulman, John and Wolski, Filip and Dhariwal, Prafulla and Radford,
Alec and Klimov, Oleg, journal=arXiv preprint arXiv:1707.06347, year=2017

@articlekaelbling1998planning, title=Planning and acting in partially
observable stochastic domains, author=Kaelbling, Leslie Pack and Littman,
Michael L and Cassandra, Anthony R, journal=Artificial intelligence,
volume=101, number=1-2, pages=99–134, year=1998, publisher=Elsevier

@articlebatra2020objectnav, title=ObjectNav Revisited: On Evaluation of
Embodied Agents Navigating to Objects, author=Batra, Dhruv and Gokaslan, Aaron
and Kembhavi, Aniruddha and Maksymets, Oleksandr and Mottaghi, Roozbeh and
Savva, Manolis and Toshev, Alexander and Wijmans, Erik, journal=arXiv preprint
arXiv:2006.13171, year=2020

@articleyang2018visual, title=Visual semantic navigation using scene priors,
author=Yang, Wei and Wang, Xiaolong and Farhadi, Ali and Gupta, Abhinav and
Mottaghi, Roozbeh, journal=arXiv preprint arXiv:1810.06543, year=2018

@inproceedingswortsman2019learning, title=Learning to learn how to learn:
Self-adaptive visual navigation using meta-learning, author=Wortsman, Mitchell
and Ehsani, Kiana and Rastegari, Mohammad and Farhadi, Ali and Mottaghi,
Roozbeh, booktitle=Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition, pages=6750–6759, year=2019

@articlewu2018learning, title=Learning and planning with a semantic model,
author=Wu, Yi and Wu, Yuxin and Tamar, Aviv and Russell, Stuart and Gkioxari,
Georgia and Tian, Yuandong, journal=arXiv preprint arXiv:1809.10842, year=2018

@articlemnih2015human, title=Human-level control through deep reinforcement
learning, author=Mnih, Volodymyr and Kavukcuoglu, Koray and Silver, David and
Rusu, Andrei A and Veness, Joel and Bellemare, Marc G and Graves, Alex and
Riedmiller, Martin and Fidjeland, Andreas K and Ostrovski, Georg and others,
journal=Nature, volume=518, number=7540, pages=529–533, year=2015,
publisher=Nature Publishing Group

@articlezhou2018semantic, title=Semantic understanding of scenes through the
ade20k dataset, author=Zhou, Bolei and Zhao, Hang and Puig, Xavier and Xiao,
Tete and Fidler, Sanja and Barriuso, Adela and Torralba, Antonio,
journal=International Journal on Computer Vision, year=2018

@inproceedingssethian1996fast, title=Fast-marching level-set methods for
three-dimensional photolithography development, author=Sethian, James A,
booktitle=Optical Microlithography IX, volume=2726, pages=262–272, year=1996,
organization=International Society for Optics and Photonics

@incollectionstachniss2016, author = Cyrill Stachniss and John J. Leonard and
Sebastian Thrun, OPTeditor = Bruno Siciliano and Oussama Khatib, title =
Simultaneous Localization and Mapping, booktitle = Springer Handbook of
Robotics, pages = 1153–1176, publisher = Springer, year = 2016

@articledatta2020integrating, title=Integrating Egocentric Localization for
More Realistic Point-Goal Navigation Agents, author=Datta, Samyak and
Maksymets, Oleksandr and Hoffman, Judy and Lee, Stefan and Batra, Dhruv and
Parikh, Devi, booktitle=corl, year=2020

@articlesmith2005development, author = Smith, Linda and Gasser, Michael, title
= The Development of Embodied Cognition: Six Lessons from Babies, year = 2005,
issue_date = January 2005, publisher = MIT Press, address = Cambridge, MA,
USA, volume = 11, number = 1–2, journal = Artificial Life, month = jan, pages
= 13–30

@InProceedingscartillier2020semantic, author = Vincent Cartillier and Zhile
Ren and Neha Jain and Stefan Lee and Irfan Essa and Dhruv Batra, title =
Semantic MapNet: Building Allocentric SemanticMaps and Representations from
Egocentric Views, booktitle = aaai, year = 2021

@inproceedingshe2016deep, title=Deep residual learning for image recognition,
author=He, Kaiming and Zhang, Xiangyu and Ren, Shaoqing and Sun, Jian,
booktitle=Proceedings of the IEEE conference on computer vision and pattern
recognition, pages=770–778, year=2016

@inproceedingslin2014microsoft, title=Microsoft coco: Common objects in
context, author=Lin, Tsung-Yi and Maire, Michael and Belongie, Serge and Hays,
James and Perona, Pietro and Ramanan, Deva and Dollár, Piotr and Zitnick, C
Lawrence, booktitle=European conference on computer vision, pages=740–755,
year=2014, organization=Springer

@inproceedingshe2017mask, title=Mask r-cnn, author=He, Kaiming and Gkioxari,
Georgia and Dollár, Piotr and Girshick, Ross, booktitle=Proceedings of the
IEEE international conference on computer vision, pages=2961–2969, year=2017

@inproceedingseqa_matterport, title=Embodied Question Answering in
Photorealistic Environments with Point Cloud Perception, author=Erik Wijmans
and Samyak Datta and Oleksandr Maksymets and Abhishek Das and Georgia Gkioxari
and Stefan Lee and Irfan Essa and Devi Parikh and Dhruv Batra, booktitle=cvpr,
year=2019

@inproceedingszhu2017target, title=Target-driven visual navigation in indoor
scenes using deep reinforcement learning, author=Zhu, Yuke and Mottaghi,
Roozbeh and Kolve, Eric and Lim, Joseph J. and Gupta, Abhinav and Fei-Fei, Li
and Farhadi, Ali, booktitle=icra, year=2017

@inproceedingsserafini2019learning, title=Learning abstract planning domains
and mappings to real world perceptions, author=Serafini, Luciano and Traverso,
Paolo, booktitle=Proc. of the International Conference of the Italian
Association for Artificial Intelligence (AI*IA), year=2019

@inproceedingsku2020room, title=Room-Across-Room: Multilingual Vision-and-
Language Navigation with Dense Spatiotemporal Grounding, author=Ku, Alexander
and Anderson, Peter and Patel, Roma and Ie, Eugene and Baldridge, Jason,
booktitle=emnlp, year=2020

@inproceedingslamanna_AAAI2021, title=On-line Learning of Planning Domains
from Sensor Data in PAL: Scaling up to Large State Spaces, author=Lamanna,
Leonardo and Gerevini, Alfonso and Saetti, Alessandro and Serafini, Luciano
and Traverso, Paolo, booktitle=aaai, year=2021

@articleaineto_AIJ2019, title=Learning action models with minimal
observability, author=Aineto, Diego and Celorrio, Sergio Jiménez and Onaindia,
Eva, journal=Artificial Intelligence, volume=275, pages=104–137, year=2019

@bookghallab2016, author = M. Ghallab and D. S. Nau and P. Traverso, title =
Automated Planning and Acting, publisher = Cambridge University Press, year =
2016

@bookghallab2004, author = M. Ghallab and D. S. Nau and P. Traverso, title =
Automated Planning - Theory and Practice, publisher = Elsevier, year = 2004

@booksutton98, author = R. S. Sutton and A. G. Barto, title = Reinforcement
learning - an introduction, series = Adaptive computation and machine
learning, publisher = MIT Press, year = 1998

@bookgeffner2013, author = H. Geffner and B. Bonet, title = A Concise
Introduction to Models and Methods for Automated Planning, series = Synthesis
Lectures on Artificial Intelligence and Machine Learning, publisher = Morgan &
Claypool Publishers, year = 2013

@inproceedingscoeeyes2018, author = J. D. Co-Reyes and Y. Liu and A. Gupta and
B. Eysenbach and P. Abbeel and S. Levine, title = Self-Consistent Trajectory
Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddings,
booktitle = ICML 2018, year = 2018

@inproceedingscoeeyes2018short, author = J. D. Co-Reyes and Y. Liu and A.
Gupta and B. Eysenbach and P. Abbeel and S. Levine, title = Self-Consistent
Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory
Embeddings, booktitle = ICML 2018, pages = 1008–1017, year = 2018

@inproceedingsdonadello2017, author = I. Donadello and L. Serafini and A. S.
d’Avila Garcez, title = Logic Tensor Networks for Semantic Image
Interpretation, booktitle = ijcai, year = 2017

@inproceedingsdeisenroth2011, author = M. Peter Deisenroth and C. Edward
Rasmussen and D. Fox, title = Learning to Control a Low-Cost Manipulator using
Data-Efficient Reinforcement Learning, booktitle = Robotics: Science and
Systems VII, University of Southern California, Los Angeles, CA, USA, June
27-30, 2011, year = 2011

@inproceedingsyang2018, author=Yang, Fangkai and Lyu, Daoming and Liu, Bo and
Gustafson, Steven, title = PEORL: Integrating Symbolic Planning and
Hierarchical Reinforcement Learning for Robust Decision-Making, booktitle =
ijcai, year = 2018

@inproceedingslyu2019, author = D. Lyu and F. Yang and B. Liu and S.
Gustafson, title = SDRL: Interpretable and Data-efficient Deep Reinforcement
Learning Leveraging Symbolic Planning, booktitle = aaai, year = 2019

@inproceedingsparr97, author = Parr, Ronald and Russell, Stuart, title =
Reinforcement Learning with Hierarchies of Machines, booktitle = nips, year =
1997

@articleleonetti2016, author = M. Leonetti and L. Iocchi and P. Stone, title =
A synthesis of automated planning and reinforcement learning for efficient,
robust decision-making, journal = Artif. Intell., volume = 241, pages =
103–130, year = 2016

@inproceedingszhang2017, author = S. Zhang and P. Khandelwal and P. Stone,
title = Dynamically Constructed (PO)MDPs for Adaptive Robot Planning,
booktitle = AAAI, pages = 3855–3863, year = 2017

@inproceedingssutton90, author = Richard S. Sutton, title = Integrated
Architectures for Learning, Planning, and Reacting Based on Approximating
Dynamic Programming, booktitle = Machine Learning, Proceedings of the Seventh
International Conference on Machine Learning, Austin, Texas, USA, June 21-23,
1990, pages = 216–224, year = 1990 @inproceedingsabbeel2006, author = Pieter
Abbeel and Morgan Quigley and Andrew Y. Ng, title = Using inaccurate models in
reinforcement learning, booktitle = ICML, year = 2006

@articlecresswell2013, author = Stephen Cresswell and Thomas Leo McCluskey and
Margaret Mary West, title = Acquiring planning domain models using LOCM,
journal = Knowledge Eng. Review, volume = 28, number = 2, pages = 195–213,
year = 2013

@articleingrand2017, author = Félix Ingrand and Malik Ghallab, title =
Deliberation for autonomous robots: A survey, journal = Artif. Intell., volume
= 247, pages = 10–44, year = 2017

@articlekaelbling96, author = Leslie Pack Kaelbling and Michael L. Littman and
Andrew W. Moore, title = Reinforcement Learning: A Survey, journal = Journal
of Artificial Intelligence Research, volume = 4, pages = 237–285, year = 1996

@inproceedingsbogomolov2014, author = Sergiy Bogomolov and Daniele Magazzeni
and Andreas Podelski and Martin Wehrle, title = Planning as Model Checking in
Hybrid Domains, booktitle = AAAI, pages = 2228–2234, year = 2014

@articlehenaff2017, author = Mikael Henaff and William F. Whitney and Yann
LeCun, title = Model-Based Planning in Discrete Action Spaces, journal = CoRR,
volume = abs/1705.07177, year = 2017

@ARTICLEhenaff2017ArXiv, author = Henaff, M. and Whitney, W. F. and LeCun, Y.,
title = ”Model-Based Planning with Discrete and Continuous Actions”, journal =
ArXiv e-prints, archivePrefix = ”arXiv”, eprint = 1705.07177, primaryClass =
”cs.AI”, keywords = Computer Science - Artificial Intelligence, year = 2017

@inproceedingsgregory2016, author = Peter Gregory and Stephen Cresswell, title
= Domain Model Acquisition in the Presence of Static Relations in the LOP
System, booktitle = ijcai, year = 2016

@inproceedingsmccluskey2009, author = Thomas Leo McCluskey and Stephen
Cresswell and N. Elisabeth Richardson and Margaret Mary West, title =
Automated Acquisition of Action Knowledge, booktitle = ICAART, year = 2009

@inproceedingsmourao2012, author = Kira Mourão and Luke S. Zettlemoyer and
Ronald P. A. Petrick and Mark Steedman, title = Learning STRIPS Operators from
Noisy and Incomplete Observations, booktitle = uai, year = 2012

@inproceedingszhuo2013, author = Hankz Hankui Zhuo and Subbarao Kambhampati,
title = Action-Model Acquisition from Noisy Plan Traces, booktitle = ijcai,
year = 2013

@inproceedingsmehta2011, author = Neville Mehta and Prasad Tadepalli and Alan
Fern, title = Autonomous Learning of Action Models for Planning, booktitle =
NIPS, year = 2011

@bookbishop2006, author = Bishop, Christopher M., title = Pattern Recognition
and Machine Learning (Information Science and Statistics), year = 2006,
publisher = Springer-Verlag,

@inproceedingsasai2018, author = Masataro Asai and Alex Fukunaga, title =
Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic
Boundary, booktitle = aaai, year = 2018

@inproceedingsasai2019, author = Masataro Asai, title = Unsupervised Grounding
of Plannable First-Order Logic Representation from Images, booktitle = icaps,
year = 2019

@inproceedingskurutach2018, author = Hanard Kurutach and Aviv Tamar and Ge
Yang and Stuart Russell and Pieter Abbeel, title = Learning Plannable
Representations with Causal InfoGAN, booktitle = nips, year = 2018

@articlestulp2008, author = Freek Stulp and Michael Beetz, title = Refining
the Execution of Abstract Actions with Learned Action Models, journal = J.
Artif. Intell. Res., volume = 32, pages = 487–523, year = 2008

@inproceedingsblochliger2018, author = Fabian Blöchliger and Marius Fehr and
Marcin Dymczyk and Thomas Schneider and Roland Siegwart, title = Topomap:
Topological Mapping and Navigation Based on Visual SLAM Maps, booktitle =
icra, year = 2018

@inproceedingsmaliah2014, author = Shlomi Maliah and Ronen I. Brafman and Erez
Karpas and Guy Shani, title = Partially Observable Online Contingent Planning
Using Landmark Heuristics, booktitle = icaps, year = 2014

@inproceedingspandey2018, author = Binda Pandey and Jussi Rintanen, title =
Planning for Partial Observability by SAT and Graph Constraints, booktitle =
ICAPS, pages = 190–198, year = 2018

@articleferns2012metrics, title=Metrics for Markov decision processes with
infinite state spaces, author=Ferns, Norman and Panangaden, Prakash and
Precup, Doina, journal=arXiv preprint arXiv:1207.1386, year=2012

@articleyang2007, author = Q Yang and K Wu and Y Jang, title = Learning action
models from plan examples using weighted MAX-SAT, journal = Artif. Intell.,
volume = 171, pages = 107–143, year = 2007

@articleross2011, author = Stéphane Ross and Joelle Pineau and Brahim Chaib-
draa and Pierre Kreitmann, title = A Bayesian Approach for Learning and
Planning in Partially Observable Markov Decision Processes, journal = Journal
of Machine Learning Research, volume = 12, pages = 1729–1770, year = 2011

@inproceedingsabel2018, author = David Abel and Dilip Arumugam and Lucas
Lehnert and Michael L. Littman, title = State Abstractions for Lifelong
Reinforcement Learning, booktitle = ICML, year = 2018

@articleabel2017, author = David Abel and D. Ellis Hershkowitz and Michael L.
Littman, title = Near Optimal Behavior via Approximate State Abstraction,
journal = CoRR, volume = abs/1701.04113, year = 2017

@bookotterlo2009, author = Martijn van Otterlo, title = The Logic of Adaptive
Behavior - Knowledge Representation and Algorithms for Adaptive Sequential
Decision Making under Uncertainty in First-Order and Relational Domains,
series = Frontiers in Artificial Intelligence and Applications, volume = 192,
publisher = IOS Press, year = 2009

@articlezhuo2010, author = Hankz Hankui Zhuo and Qiang Yang and Derek Hao Hu
and Lei Li, title = Learning complex action models with quantifiers and
logical implications, journal = Artificial Intelligence, volume = 174, number
= 18, pages = 1540–1569, year = 2010

@articlezhuo2014, author = Hankz Hankui Zhuo and Qiang Yang, title = Action-
model acquisition for planning via transfer learning, journal = Artificial
Intelligence, volume = 212, pages = 80–103, year = 2014

@inproceedingsschmillOC2000, author = Matthew D. Schmill and Tim Oates and
Paul R. Cohen, title = Learning Planning Operators in Real-World, Partially
Observable Environments, booktitle = ICAPS, pages = 246–253, year = 2000

@articlepasula2007, author = Hanna M. Pasula and Luke S. Zettlemoyer and
Leslie Pack Kaelbling, title = Learning Symbolic Models of Stochastic Domains,
journal = J. Artif. Intell. Res., volume = 29, pages = 309–352, year = 2007

@inproceedingszettlemoyer2005, author = Luke S. Zettlemoyer and Hanna Pasula
and Leslie Pack Kaelbling, title = Learning Planning Rules in Noisy Stochastic
Worlds, booktitle = AAAI, year = 2005

@inproceedingspasula2004, author = Hanna Pasula and Luke S. Zettlemoyer and
Leslie Pack Kaelbling, title = Learning Probabilistic Relational Planning
Rules, booktitle = ICAPS, year = 2004

@inproceedingszheng2018, author = Wei Zheng and Bo Wu and Hai Lin, title =
POMDP Model Learning for Human Robot Collaboration, booktitle = 57th IEEE
Conference on Decision and Control, CDC 2018, Miami, FL, USA, December 17-19,
2018, pages = 1156–1161, year = 2018

@inproceedingskatt2017, author = Sammie Katt and Frans A. Oliehoek and
Christopher Amato, title = Learning in POMDPs with Monte Carlo Tree Search,
booktitle = ICML, year = 2017

@articlekonidaris2018, author = George Konidaris and Leslie Pack Kaelbling and
Tomás Lozano-Pérez, title = From Skills to Symbols:Learning Symbolic
Representations for Abstract High-Level Planning, journal = J. Artif. Intel.
Res, year = 2018

@articleslaney2001, author = John K. Slaney and Sylvie Thiébaux, title =
Blocks World revisited, journal = Artif. Intell., volume = 125, number = 1-2,
pages = 119–153, year = 2001

@inproceedingsLi2006TowardsAU, title=Towards a Unified Theory of State
Abstraction for MDPs, author=Lihong Li and Thomas J. Walsh and Michael L.
Littman, booktitle=ISAIM, year=2006

@inproceedingsryan2002, author = Malcolm R. K. Ryan, title = Using Abstract
Models of Behaviours to Automatically Generate Reinforcement Learning
Hierarchies, booktitle = icml, year = 2002

@misclecun-mnisthandwrittendigit-2010, author = LeCun, Yann and Cortes,
Corinna, title = MNIST handwritten digit database, url =
http://yann.lecun.com/exdb/mnist/, year = 2010

@inproceedingsdoshivelez09, author = Finale Doshi-Velez, title = The Infinite
Partially Observable Markov Decision Process, booktitle = nips, year = 2009

@inproceedingsgarmelo16, Author = Garnelo, Marta and Arulkumaran, Kai and
Shanahan, Murray, Title = Towards Deep Symbolic Reinforcement Learning,
booktitle = nipsw, Year = 2016

@articleALP, author = Luciano Serafini and Paolo Traverso, title = Incremental
Learning of Discrete Planning Domains from Continuous Perceptions, journal =
CoRR, volume = abs/1903.05937, year = 2019

@inproceedingsbonet2019, author = Blai Bonet and Hector Geffner, title =
Learning First-Order Symbolic Representations for Planning from the Structure
of the State Space, booktitle = ecai, year = 2019

@inproceedingsserafini2019, author = Luciano Serafini and Paolo Traverso,
title = Learning Abstract Planning Domains and Mappings to Real World
Perceptions, booktitle = AI*IA 2019 - Advances in Artificial Intelligence,
series = Lecture Notes in Computer Science, volume = 11946, pages = 461–476,
publisher = Springer, year = 2019

@TECHREPORT McDermott:98-PDDL, Author = Drew McDermott and Malik Ghallab and
A. Howe and Craig A. Knoblock and A. Ram and M. Veloso and Daniel S. Weld and
David E. Wilkins, Title = PDDL—The Planning Domain Definition Language,
Institution = Yale Center for Computational Vision and Control, Number = DCS
TR-1165, Address = New Haven, Connecticut, year = 1998

@articleDBLP:journals/ai/FikesN71, author = Richard Fikes and Nils J. Nilsson,
title = STRIPS: A New Approach to the Application of Theorem Proving to
Problem Solving, journal = Artificial Intelligence, volume = 2, number = 3/4,
pages = 189–208, year = 1971, OPTurl =
https://doi.org/10.1016/0004-3702(71)90010-5, OPTdoi =
10.1016/0004-3702(71)90010-5, OPTtimestamp = Wed, 14 Nov 2018 10:50:00 +0100,
OPTbiburl = https://dblp.org/rec/journals/ai/FikesN71.bib, OPTbibsource = dblp
computer science bibliography, https://dblp.org

@articleDBLP:journals/jair/Helmert06, author = Malte Helmert, title = The Fast
Downward Planning System, journal = J. Artif. Intell. Res., volume = 26, pages
= 191–246, year = 2006, OPTurl = https://doi.org/10.1613/jair.1705, OPTdoi =
10.1613/jair.1705, OPTtimestamp = Mon, 21 Jan 2019 15:01:17 +0100, OPTbiburl =
https://dblp.org/rec/journals/jair/Helmert06.bib, OPTbibsource = dblp computer
science bibliography, https://dblp.org

@articleDBLP:journals/aim/McDermott00, author = Drew V. McDermott, title = The
1998 AI Planning Systems Competition, journal = AI Magazine, volume = 21,
number = 2, pages = 35–55, year = 2000, OPTurl =
https://doi.org/10.1609/aimag.v21i2.1506, OPTdoi = 10.1609/aimag.v21i2.1506,
OPTtimestamp = Mon, 20 Jul 2020 17:21:29 +0200, OPTbiburl =
https://dblp.org/rec/journals/aim/McDermott00.bib, OPTbibsource = dblp
computer science bibliography, https://dblp.org

@articleDBLP:journals/aim/Bacchus01, author = Fahiem Bacchus, title = The AIPS
’00 Planning Competition, journal = AI Magazine, volume = 22, number = 3,
pages = 47–56, year = 2001, OPTurl =
http://www.aaai.org/ojs/index.php/aimagazine/article/view/1571, OPTtimestamp =
Wed, 29 Dec 2010 17:08:37 +0100, OPTbiburl =
https://dblp.org/rec/journals/aim/Bacchus01.bib, OPTbibsource = dblp computer
science bibliography, https://dblp.org

@articleDBLP:journals/corr/abs-1106-5998, author = Maria Fox and Derek Long,
title = The 3rd International Planning Competition: Results and Analysis,
journal = CoRR, volume = abs/1106.5998, year = 2011, url =
http://arxiv.org/abs/1106.5998, OPTarchivePrefix = arXiv, OPTeprint =
1106.5998, OPTtimestamp = Mon, 13 Aug 2018 16:46:36 +0200, OPTbiburl =
https://dblp.org/rec/journals/corr/abs-1106-5998.bib, OPTbibsource = dblp
computer science bibliography, https://dblp.org

@articlechen2019audio, title=Audio-visual embodied navigation, author=Chen,
Changan and Jain, Unnat and Schissler, Carl and Gari, Sebastia Vicenc Amengual
and Al-Halah, Ziad and Ithapu, Vamsi Krishna and Robinson, Philip and Grauman,
Kristen, journal=environment, volume=97, pages=103, year=2019

@articlejiang2018rednet, title=Rednet: Residual encoder-decoder network for
indoor rgb-d semantic segmentation, author=Jiang, Jindong and Zheng, Lunan and
Luo, Fei and Zhang, Zhijun, journal=arXiv preprint arXiv:1806.01054, year=2018

@inproceedingsdas2018embodied, title=Embodied question answering, author=Das,
Abhishek and Datta, Samyak and Gkioxari, Georgia and Lee, Stefan and Parikh,
Devi and Batra, Dhruv, booktitle=cvpr, year=2018

@articlebatra2020rearrangement, title=Rearrangement: A challenge for embodied
AI, author=Batra, Dhruv and Chang, Angel X and Chernova, Sonia and Davison,
Andrew J and Deng, Jia and Koltun, Vladlen and Levine, Sergey and Malik,
Jitendra and Mordatch, Igor and Mottaghi, Roozbeh and others, journal=arXiv
preprint arXiv:2011.01975, year=2020

@inproceedingsye2021auxiliary, title=Auxiliary Tasks and Exploration Enable
ObjectNav, author=Ye, Joel and Batra, Dhruv and Das, Abhishek and Wijmans,
Erik, booktitle=iccv, year=2021

@inproceedingswang2019reinforced, title=Reinforced cross-modal matching and
self-supervised imitation learning for vision-language navigation,
author=Wang, Xin and Huang, Qiuyuan and Celikyilmaz, Asli and Gao, Jianfeng
and Shen, Dinghan and Wang, Yuan-Fang and Wang, William Yang and Zhang, Lei,
booktitle=cvpr, year=2019

@inproceedingszhu2020vision, title=Vision-language navigation with self-
supervised auxiliary reasoning tasks, author=Zhu, Fengda and Zhu, Yi and
Chang, Xiaojun and Liang, Xiaodan, booktitle=cvpr, year=2020

@inproceedingsdugas2022navdreams, title=NavDreams: Towards Camera-Only RL
Navigation Among Humans, author=Dugas, Daniel and Andersson, Olov and
Siegwart, Roland and Chung, Jen Jen, booktitle=2022 IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS), pages=2504–2511,
year=2022, organization=IEEE

@articlegao2022evaluation, title=Evaluation of socially-aware robot
navigation, author=Gao, Yuxiang and Huang, Chien-Ming, journal=Frontiers in
Robotics and AI, volume=8, pages=420, year=2022, publisher=Frontiers

@articlekarnan2022socially, title=Socially compliant navigation dataset
(SCAND): A large-scale dataset of demonstrations for social navigation,
author=Karnan, Haresh and Nair, Anirudh and Xiao, Xuesu and Warnell, Garrett
and Pirk, Sören and Toshev, Alexander and Hart, Justin and Biswas, Joydeep and
Stone, Peter, journal=IEEE Robotics and Automation Letters, volume=7,
number=4, pages=11807–11814, year=2022, publisher=IEEE

@articlerudenko2020thor, title=Thör: Human-robot navigation data collection
and accurate motion trajectories dataset, author=Rudenko, Andrey and Kucner,
Tomasz P and Swaminathan, Chittaranjan S and Chadalavada, Ravi T and Arras,
Kai O and Lilienthal, Achim J, journal=IEEE Robotics and Automation Letters,
volume=5, number=2, pages=676–682, year=2020, publisher=IEEE

@articlemartin2021jrdb, title=JRDB: A dataset and benchmark of egocentric
robot visual perception of humans in built environments, author=Martin-Martin,
Roberto and Patel, Mihir and Rezatofighi, Hamid and Shenoi, Abhijeet and Gwak,
JunYoung and Frankel, Eric and Sadeghian, Amir and Savarese, Silvio,
journal=IEEE transactions on pattern analysis and machine intelligence,
year=2021, publisher=IEEE

@inproceedingsguzzi2013human, title=Human-friendly robot navigation in dynamic
environments, author=Guzzi, Jérôme and Giusti, Alessandro and Gambardella,
Luca M and Theraulaz, Guy and Di Caro, Gianni A, booktitle=ICRA, year=2013