# Finding a Landing Site on an Urban Area: A Multi-Resolution Probabilistic
Approach

Barak Pinkovich, Boaz Matalon, Ehud Rivlin and Hector Rotstein This work was
partially supported by Hyundai NGV.B. Pinkovich and E. Rivlin are with the
Faculty of Computer Science, Technion Israel Institute of Technology,
<EMAIL_ADDRESS><EMAIL_ADDRESS>Matalon is with Rafael
Advanced Defense Systems Ltd<EMAIL_ADDRESS>Rotstein is with the
Faculty of Computer Science, Technion Israel Institute of Technology and with
Rafael Advanced Defense Systems Ltd<EMAIL_ADDRESS>
###### Abstract

This paper considers the problem of finding a landing spot for a drone in a
dense urban environment. The conflicting requirement of fast exploration and
high resolution is solved using a _multi-resolution_ approach, by which visual
information is collected by the drone at decreasing altitudes so that spatial
resolution of the acquired images increases monotonically. A probability
distribution is used to capture the uncertainty of the decision process for
each terrain patch. The distributions are updated as information from
different altitudes is collected. When the confidence level for one of the
patches becomes larger than a pre-specified threshold, suitability for landing
is declared. One of the main building blocks of the approach is a semantic
segmentation algorithm that attaches probabilities to each pixel of a single
view. The decision algorithm combines these probabilities with _a priori_ data
and previous measurements to obtain the best estimates. Feasibility is
illustrated by presenting a number of examples generated by a realistic
closed-loop simulator.

##  I Introduction

As opposed to conventional aircraft that take-off and land from designated and
controlled areas outside city limits, future commercial drones are expected to
operate smoothly in crowded urban environments. Consequently, the static and
well-defined zones delimited for landing and take off will be replaced by
dynamic and opportunistic areas within cities. For instance, one of the
challenges of future delivery or transportation drones is to solve the ”last-
mile” problem, for which an autonomous drone must find a place to land having
the following characteristics:

  1. 1.

Be relatively close to the intended destination. These places cannot be
limited to pre-determined areas like heliports, sports fields, or similar.

  2. 2.

Be appropriate for the drone size and weight.

  3. 3.

Be appropriate for landing under harsh (or relatively harsh) flight conditions
compatible with the drone’s flying capabilities.

  4. 4.

Pose no safety concerns to itself, other vehicles, or living beings in the
environment.

This paper describes a multi-resolution, probabilistic approach for searching
for a landing place in a dense urban environment like the one illustrated in
Fig.1. The approach is multi-resolution since a visual sensor from decreasing
altitudes observes the urban environment, hence generating a sequence of
images with monotonically increasing spatial resolution. It is also
probabilistic in the sense that confidence levels of different regions of the
urban region are computed based on a-priori knowledge and the result of
observations.

Figure 1: Simulated urban environment in which the drone attempts to land

The probabilistic viewpoint provides connections with extensive literature,
including search theory and Bayes-based decision making. For example, [1]
provides a survey on Bayesian networks’ usage for intelligent autonomous
vehicles decision-making with no focus on specific missions. Similarly, [2]
describes spacecraft autonomy challenges for future space missions, in which
real-time autonomous decision-making and human-robotic cooperation must be
considered. In a related autonomous spacecraft mission, [3] studies the
selection of an appropriate landing site for an autonomous spacecraft on an
exploration mission. The problem is formulated so that three main variables
are defined on which to select the landing site: terrain safety, engineering
factors (spacecraft’s descendent trajectory, velocity, and available fuel),
and the site pre-selected by using available a-priori information. The
approach was tested by using a dynamics and spacecraft simulator for entry,
descent, and landing.

The problem considered here is also somewhat related with _forced landing_.
This is because a UAV may need to decide the most suitable forced landing
sites autonomously, usually from a list of known candidates [4]. In that work,
references were made to the specifications for a forced landing system laid
out in a NASA technical report essentially consisting of three main criteria:
risk to the civilian population, reachability, and probability of a safe
landing. The emphasis is on public safety, where human life and property are
more important than the UAV airframe and payload. Specifications were included
in a Multi-Criteria Decision Making (MCDM) Bayesian network. See [5] for an
application of the model to a real-life example. Initial designs for UAVs’
autonomous decision systems to select emergency landing sites in a vehicle
fault scenario are also considered in citeding2016initial. The overall design
consists of two main components: pre-planning and real-time optimization. In
the pre-planning component, the system uses offline information such as
geographical and population data to generate landing loss maps over the
operating environment. In the real-time component, onboard sensor data is used
to update a probabilistic risk assessment for potential landing areas.

Another related field of interest is that of _search and rescue_ , a
challenging task as it usually involves a large variety of scenarios that
require a high level of autonomy and versatile decision-making capabilities. A
formal framework casting the search problem as a decision between hypotheses
using current knowledge was introduced in [6]. The search task was defined as
follows: given a detector model (i.e., detection error probabilities for false
alarms and miss detections) and the prior belief that the target exists in the
search region, determine the evolution of the belief that the target is
present in the search region, as a function of the observations made until
time $t$. The belief evolution is computed via a recursive Bayesian expression
that provides a compact and efficient way to update the belief function at
every time step as the searcher observes a sequence of unexplored and/or
previously visited cells in the search region. After generating a method for
computing the belief evolution for a sequence of imperfect observations, the
authors investigate the search control policy\strategy. In the context of an
area search problem, [7] investigates the uncertainty associated with a
typical search problem to answer a crucial question: how many image frames
would be required by a camera onboard a vehicle to classify a target as
”detected” or ”undetected” with uncertain prior information on the target’s
existence. The paper presents a formulation that incorporates uncertainty
using Beta distributions to create robust search actions. As shown below,
these ideas are highly related to our approach.

##  II Problem Formulation

Suppose a drone needs to find a landing place in an urban area $\cal{A}$. For
simplicity, consider $\cal{A}$ to be planar, with an attached coordinate
system $\\{W\\}$ such that $\cal{A}$ lies on the $x-y$ plane, measuring the
altitude along the $z$-direction. Buildings and other constructions are
modeled as occupied volumes over $\cal{A}$. As an example, the area considered
in this paper will be 1 Km by 1 Km square. The drone can fly at different
altitudes $h$ while collecting measurements using a monocular camera and a
visual sensor with range capabilities. Examples of the latter are an RGB-D
sensor, a Lidar, or a couple of stereo cameras. Let ${\cal{A}}_{h}$ be the
plane parallel to $\cal{A}$ at an altitude $h$ onto which $\cal{A}$’s relevant
characteristics can be mapped. For instance, a no-fly zone $\cal U$ in
$\cal{A}$ (e.g., the base of a building ) will be mapped onto the
corresponding subset ${\cal{U}}_{h}$ in ${\cal{A}}_{h}$ (at least if $h$ is
smaller than the corresponding building’s altitude).

Figure 2: The area of interest $\cal{A}$, the plane ${\cal A}_{h}$ at an
altitude $h$ and the mapping of an obstacle ${\cal U}_{h}$

The mechanical structure of the drone, the size of its propellers, and the
flying conditions (e.g., wind) constrain the minimum dimensions of the landing
site on which the drone can safely land. Conservatively, $\cal{A}$ will be
divided into a grid of identical cells $c_{ij}$, so that
$\cup_{ij}c_{ij}=\cal{A}$, and each $c_{ij}$ is in principle a landing cite
candidate. Note that this division is in-line with search theory (see, e.g.,
[8]) but stems from a different motivation: it is not a unit area being
explored but the smallest area of interest. As discussed next, this will
impact our developments in several ways.

The images taken by a camera on the drone at time instant $t$ will be a
function of the pose $p_{t}$ and the camera’s field-of-view. Note that the
camera’s orientation can differ from that of the drone by a relative rotation
between the two. Assuming that the FOV is fixed and known, let $I_{t}(h)$ be
the camera’s image at time $t$ and $F_{t}(h)$ be the camera’s corresponding
footprint, namely the 3D structure mapped onto the image plane. At time $t$
the drone will have a unique pose $p_{t}$, but the dependence on the altitude
is specifically considered in the notation; this is because the altitude
scales the resolution and the footprint: for smaller $h$, one gets better
resolution at the cost of a smaller footprint. The structure $F_{t}(h)$ is
built on a collection of cells
$C_{t}(h)=\bigcup_{\\{j,k\\}\in\\{J_{t}(h),K_{t}(h)\\}}c_{ij}\subset\cal{A}$.

In the absence of additional constraints, the drone could overfly $\cal A$ at
a relatively low altitude $h_{min}$ searching for an appropriate cell $c_{ij}$
on which to land. However, at this altitude, the camera’s _footprint_
$C_{t}(h_{min})$ will include a relatively small number of cells $c_{ij}$ and
hence the drone would spend a potentially prohibitively amount of time/energy
exploring the whole $\cal A$. On the other hand, the altitude can be selected
to be the maximum allowable by regulations, say $h_{max}$, resulting in as
large as possible footprints. In an extreme case, $C_{t}(h_{max})=\cal{A}$.
This maximizes the area subtended by a single image and minimizes the
exploration time, but will give rise to an image resolution that cannot
guarantee the safety of landing, e.g., will not resolve relatively small
obstacles. The trade-off between altitude and resolution is solved in this
work by considering a multi-resolution approach: the exploration will start at
high altitude, say $h_{1}$, looking for the largest possible subset of
$\cal{A}$ that appears to be _feasible_ for landing, say $L_{1}$.
Subsequently, the drone will reduce its altitude to $h_{2}$ and re-explore
$L_{1}$ with the higher resolution resulting from $h_{2}<h_{1}$. This process
results in a sequence of ${\cal A}\supset L_{1}\supset\cdots\supset L_{N}$
that will eventually converge to a collection of one or more safe landing
places.

Figure 3: The plane at a given altitude is divided into small cells

Figures 1 to 3 illustrate this scenario.

###  II-A A Probabilistic Model

In classical search theory [8], finding a target is often formulated as a
decision problem by defining a set of binary variables:

${\cal H}_{ij}\doteq\left\\{\begin{array}[]{ll}1&\mbox{if $c_{ij}$ has a
target}\\\ 0&\mbox{otherwise}\end{array}\right.$ (1)

This can be extended to the case of interest by defining:

${\cal H}_{ij}\doteq\left\\{\begin{array}[]{ll}1&\mbox{if $c_{ij}$ is
appropriate for landing}\\\ 0&\mbox{otherwise}\end{array}\right.$ (2)

In the presence of uncertainty, each cell is likely to be suitable for landing
with some probability $K_{ij}$ referred to as the _fitness for landing_.
Clearly, if a cell is appropriate for landing, then $K_{ij}=1$, and if it is
not, then $K_{ij}=0$. In real-world scenarios, prior knowledge about the
fitness for landing can be based on using some kind of maps or 3D models that
can be imprecise or outdated. Consequently, uncertainty in $K_{ij}$ needs to
be incorporated in the model. Probably the simplest way to model the decision
problem would be to introduce a Binary distribution and say that the
probability of $K_{ij}=1$ is $p$ and $K_{ij}=0$ is $1-p$. However, as observed
by [7], this model fails to capture the _uncertainty_ of the information, and
instead, it is preferable to use a Beta distribution for describing the prior
knowledge together with its underlying uncertainty. Note that the Beta
distribution and the Binomial and Bernoulli distributions form a _conjugate
pair_ , so that if a Bernoulli distribution can model the sensor, then the
observation of new data changes only the parameters of the prior, while the
conjugacy property ensures that the posterior is in the same class (i.e.,
Beta). This is a critical property when propagating the belief on the fitness
for landing in a Bayesian framework. Using these models allows simplifying the
decision problem into a binary outcome as defined in Eq. 2.

The Beta distribution is defined as,

$Pr\left(K_{ij}\left|\alpha,\beta\right.\right)=\dfrac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}K_{ij}^{\alpha-1}(1-K_{ij})^{\beta-1}$
(3)

Where $0<K_{ij}<1$ and $\Gamma(\alpha)$ is the gamma function defined as,

$\Gamma(\alpha)=\int_{0}^{\infty}x^{\alpha-1}e^{{-x}}\,dx$ (4)

When $\alpha$ is an integer $\Gamma(\alpha)=(\alpha-1)!$ . The parameters
$\alpha$ and $\beta$ can be considered prior ”successes” and ”failures”. The
Beta distribution is somewhat similar to the Binomial distribution. The main
difference is that, whereas the random variable is ${\cal H}_{ij}$ and the
parameter is $K_{ij}$ in the Binomial distribution, the random variable is
$K_{ij}$ and the parameters are $\alpha$ and $\beta$ in the Beta distribution.

Bayes theorem is often used in search theory [8, 6, 7] to update the
aggregated belief (e.g., posterior distribution), which is proportional to the
_likelihood_ function times the prior distribution:

$Pr\left(K_{ij}\left|S_{ij}^{N},\alpha,\beta\right.\right)\propto
Pr\left(S_{ij}^{N}\left|,K_{ij}\right.\right)Pr\left(K_{ij}\left|\alpha,\beta\right.\right)$
(5)

Here $S_{ij}^{N}=\sum_{n=1}^{N}{\cal H}_{ij}^{n}$ is the number of successes
in $N$ Bernoulli trials.
$Pr\left(K_{ij}\left|S_{ij}^{N},\alpha,\beta\right.\right)$ is the posterior
distribution for $K_{ij}$ given the number of successes.
$Pr\left(S_{ij}^{N}\left|K_{ij}\right.\right)$ is the _likelihood function_
and $Pr\left(K_{ij}\left|\alpha,\beta\right.\right)$ is the prior distribution
for $K_{ij}$. In [6] the likelihood distribution is given by a Binomial
distribution series of $N$ observations:

$Pr\left(S_{ij}^{N}\left|K_{ij}\right.\right)=\binom{S_{ij}^{N}}{N}K_{ij}^{S_{ij}^{N}}(1-K_{ij})^{N-S_{ij}^{N}}$
(6)

Note that the underlying assumption is that the Bernoulli distribution
provides an appropriate statistical model for the sensor used. This assumption
is more or less natural when considering a series of noisy images. In our
case, simple image processing algorithms are replaced by a more complex _meta_
-sensor: fitness is computed by a semantic segmentation algorithm that
associates for each pixel on a given image the suitability for landing on the
corresponding cell on the ground. An appropriate model for this process is
discussed next.

###  II-B A Correlated Detection Model

The probability framework is motivated by the limitations of the sensors used
for establishing whether a given cell is appropriate for landing or not. As
mentioned above, the main limitations are the camera resolution, the
environmental conditions limiting visibility, and possibly scene dynamics. The
probability estimated by a sensor that a given cell is appropriate for landing
may vary according to altitude, with lower altitudes having higher confidence
levels111Up to certain point depending on the sensor..

In [7] the authors assumed independent Bernoulli trials when detecting a
target in recurrent visits. Independent trials are acceptable when the
condition of the experiment does not change. However, in this research, our
multi-resolution approach implies that when observing cell $c_{ij}$ at
different altitude levels, one cannot assume uncorrelated measurements between
different levels. At each level, the experiment’s condition changes (e.g.,
different resolution), and if a landing place exists, then it is expected that
the rate of success will depend on previous trials and will increase when the
level of details increases while descending toward cell $c_{ij}$.

Generalizing the Binomial distribution typically involves modifying either the
assumption of constant ”success” probability and/or the assumption of
independence between trials in the underlying Bernoulli process. The approach
to generalizing the Binomial distribution in this research follows the
Generalized Bernoulli Distribution (GBD) model [9] by relaxing the assumption
of independence between trials. The GBD model was further considered in
statistics literature [10, 11, 12, 13]) with the aim to obtain its central
limit theorems, including the strong law of large numbers and the law of the
iterated logarithm for partial sums.

Consider a Bernoulli process $\\{{\cal H}_{ij}^{n},\ n\geq 1\\}$ in which the
random variables ${\cal H}_{ij}^{n}$ are correlated so that the success
probability for the trial conditional on all the previous trials depends on
the total number of successes achieved to that point. More precisely, for some
$0<K_{ij}<1$,

$Pr\left({\cal H}_{ij}^{n+1}\left|{\cal
F}_{ij}^{n}\right.\right)=(1-\theta_{ij}^{n})K_{ij}+\theta_{ij}^{n}\frac{S_{ij}^{n}}{n}$
(7)

Where $0\leq\theta_{ij}^{n}\leq 1$ are dependence parameters,
$S_{ij}^{N}=\sum_{n=1}^{N}{\cal H}_{ij}^{n}$ for $N\geq 1$ and ${\cal
F}_{ij}^{N}=\sigma({\cal H}_{ij}^{1},\cdots,\ {\cal H}_{ij}^{N})$. If ${\cal
H}_{ij}^{1}$ has a Bernoulli distribution with parameter $K_{ij}$, it follows
that ${\cal H}_{ij}^{1},\ {\cal H}_{ij}^{2},\cdots$ are identically
distributed Bernoulli random variables.

By replacing the Binomial distribution in Eq.5 with the GBD at each altitude
$h_{n}$, the aggregated belief that a cell $c_{ij}$ is suitable for landing
given the number of successes will be proportional to the product of prior
distribution and the altitude correlation-based distribution.

##  III Simulation

In order to develop and test the probability multiple-resolution approach, a
simulation environment was created using AirSim [14], a drones and cars
simulator built on the Unreal Engine [15] AirSim is an open-source, cross-
platform simulator for physically and visually realistic simulations. It is
developed as an Unreal plug-in that can be integrated into any Unreal
environment. Within AirSim, a drone can be controlled using a Python/C++ API;
for our project’s requirements, the drone can be configured similarly to a
real drone in terms of dynamics, sensor data, and computer interface. The
drone can be flown in the simulation environment from one way-point to another
while acquiring data from the sensors defined in the platform. For the current
configuration, the simulator computes images taken by a downwards-looking
camera and a GPS/inertial navigation system.

The simulator has two main purposes:

  1. 1.

Test the overall multi-resolution approach as the Unit Under Test (UUT). In
this mode, the simulator functions as the Hardware In The Loop (HIL) tester’s
data generator. The data generated by the simulator is streamed into the
drone’s mission computer; the system processes the data and computes the next
coordinate to which the drone flies. Note that in this case, the simulator
drives the real-time functioning of the closed-loop system.

  2. 2.

Generating off-line data for the search algorithm. As mentioned above, the
drone can be flown using the API around the map at different scenarios and
heights while generating data at pre-determined rates. Typical data consists
of RGB images, segmented images, and inertial navigation data, forming a
probabilistic model analysis and validation data set.

The 3-D model used in the simulator was the Brushify - Urban Buildings Pack
[16], purchased from the Unreal marketplace. Figures 4 and 5 show a simple
example of RGB and corresponding segmentation images for the cameras simulated
on the drone. The images highlight the observation that objects that occupy a
cell are hardly detectable from a high altitude (e.g., phone booth), while
when descending, the gathered information allows the algorithm to detect these
objects and decide that the drone cannot land in these specific cells.

|
---|---
Figure 4: High altitude urban scene captured with a downward looking camera. On the left, a simulated image. On the right, the corresponding segmented scene. |
---|---
Figure 5: Low altitude urban scene captured with a downward looking camera. On
the left, a simulated image. On the right, the corresponding segmented scene.

##  IV Analysis and Preliminary Results

Obtaining some prior knowledge about the urban scene is necessary for having a
probability model. For such purpose, a labeled 3-D Digital Surface Model (DSM)
was generated using the simulation by using a $2x2[m]$ cell resolution.

Figure 6: Top View of the Digital Surface Model with chosen labels

The DSM, shown in Fig.6, allows choosing the parameters of the prior
distribution for each cell given the label of that cell. The labels that were
chosen to be represented with initial probabilities were such that they were
visible from a high altitude and may be considered appropriate for landing or
not when descending. Fig.7 shows the Beta distribution’s parameters for each
label. These parameters were chosen to give some knowledge on an appropriate
(or not) place to land. Still, there is sufficient uncertainty in the prior’s
belief to allow some degree of freedom to change the values and the new belief
with new observations.

Figure 7: Prior Beta Distribution For Chosen Labels

The $\alpha$ and $\beta$ parameters in Eq.3 are stored for each cell $c_{ij}$
in a 2-D model of the urban scene as illustrated in Fig.8. To determine if
cell $c_{ij}$ contains a place to land, the probability $Pr(K_{ij}>\kappa)$ is
calculated:

$Pr(K_{ij}>\kappa)=\int_{\kappa}^{1}Pr(K_{ij})\,dK$ (8)

For instance, Fig.9 shows the prior belief that a landing place exists for
$Pr(K_{ij}>0.5)$. Only sidewalks are somewhat appropriate for landing even for
$\kappa=0.5$, and the belief can easily be changed when new observations are
obtained.

|
---|---
Figure 8: Prior $\alpha$ and $\beta$ parameters for each cell in the urban
scene Figure 9: Prior Landing Probability For $\kappa=0.5$

The downwards-looking camera mounted on the drone provides color images that
need to be converted into information on how much each cell is suitable for
landing. Clearly, this relation may be highly complex. In recent years, deep
neural networks have shown to be successful for various Computer-Vision
applications, including the kind of Semantic Segmentation problems relevant to
our purpose. Consequently, we chose to employ the semantic segmentation
network BiSeNet [17], which fuses two information paths - context path and
spatial path. The context allows information from distant pixels to affect a
pixel’s classification at the cost of reduced spatial resolution. In contrast,
the spatial path maintains fine details by limiting the number of down-
sampling operations. This net also provides a reasonable compromise between
segmentation accuracy and compute requirements. Other network architectures,
potentially more complex and accurate, will be explored in future work.

The network was trained and validated on images taken by the camera while
flying in the urban environment at different altitudes. The model uses the
labels in Fig.6 for training, and during inference, the model predicts
probability scores (summing to 1) for the different categories. Each category
is also assigned with an a-priori weight, representing how much this category
is suitable for landing (e.g. weight[sidewalk] = 0.8 and weight[building] =
0). To obtain a final score $p_{mn}$ for each image pixel, we take a weighted
average of the categories‘ probabilities using the pre-defined weights.
$p_{mn}$, which can vary between $0$ to $1$, describes the probability that a
landing site exists based on the observed data. Using the 6-DOF of the drone,
the image footprint, i.e., pixels coordinates projected on the ground, were
transformed to a world coordinate system to be associated with each cell
$c_{ij}$ in the grid. The outcome of a Bernoulli trial for success or failure
is given by counting $N_{p}$, the number of pixels associated to cell $c_{ij}$
that pass $p_{mn}>0.5$ and are relative to $N_{pc}$, the total number of
pixels associated with $c_{ij}$. If the relative amount is greater than
$0.99$, then $c_{ij}$ holds a successful trial.

$N_{p}=\sum_{m,n\in i,j}^{N_{pc}}\\{p_{mn}>0.5\\}$ (9) ${\cal
H}_{ij}\doteq\left\\{\begin{array}[]{ll}1,&\mbox{
$\dfrac{N_{p}}{N_{pc}}>0.99$}\\\ 0,&\mbox{otherwise}\end{array}\right.$ (10)

###  IV-A A Single Altitude Bayesian Update

The Bayesian update was tested for several flight scenarios.Suppose now that
the drone flies and takes images at a constant altitude and that the semantic
segmentation network analyzes images. Each cell $c_{ij}$ belonging to an image
footprint is associated with the corresponding projected pixels, and a
Bernoulli trial is performed according to Eq.10. The trial is performed on
each cell only once to prevent added correlation effects at that altitude. The
outcome of the single-trial is added to the $\alpha$, and $\beta$ values
previously selected as the prior (shown in Fig.7, 8) and then integrated
numerically according to Eq.8. Figures 10 and 11 show the updating stage at
different time instances and altitudes.

|
---|---
Figure 10: Bernoulli trial update - each cell is updated only once |
---|---
Figure 11: Bernoulli trial update - each cell is updated only once

###  IV-B An Altitude-Based Bernoulli Trials Distribution

In order to study the GBD model, an experiment with Bernoulli trials at
different altitudes was performed. There are several objects placed on the
ground at different locations around the urban scene. The experiment was
planned so that a single object is selected for the drone to descend upon at
different locations. At each location, images are taken as input for the
semantic segmentation network. On each output of the network, a Bernoulli
trial is performed on a single cell according to Eq.10, so that at each
altitude that an image is taken, there is a single success or failure output
on a given cell $c_{ij}$. Fig.12, 13 show the input and output at selected
altitudes in a single location. A phone booth was selected for the drone to
descend upon. There are $31$ locations around the urban scene with the phone
booth placed on the sidewalk. A cell in the world coordinate system was
selected at each location so that the phone booth occupies partially or the
entire cell. The expected outcome would be that the phone booth will be
partially detected at high altitudes, and the selected cell would be detected
as fit for landing. In contrast, more details will be detected when
descending, and the cell will be detected as unfit for landing.

Fig.14 shows the histogram for the $31$ locations with altitude-based trials.
We can see that the aforementioned expected behavior is indeed observed under
160 meters approximately. Peculiarly, above this altitude, the success
frequency diminishes markedly. This may be explained by the fact that there
were no images from these altitudes in the training set, making the net’s
prediction unreliable. It should be noted that even in lower altitudes, its
prediction can be noisy due to various reasons, particularly the training set
being not diverse enough. We expect that more diversity in the training set
will yield greater reliability of the net, which in turn will result in a
better fit for the model.

|
---|---
Figure 12: Semantic segmentation input - telephone booth partially occupies the cell |
---|---
Figure 13: Semantic segmentation output - telephone booth partially occupies
the cell  Figure 14: an Altitude-Based Bernoulli Trials Histogram

##  V Conclusions and Further Work

This paper has presented a multi-resolution probabilistic approach for finding
an appropriate landing site for a drone in a dense urban environment. The
approach uses a-priori data (e.g., a map or a DSM) to estimate the fitness for
landing probability distribution for each cell on which the environment is
divided. Distribution and not probabilities are used in an attempt to model
the uncertainty of the data. Subsequently, the data collected by a visual
sensor and processed by a semantic segmentation neural net is used to update
the distribution using Bayesian networks. In order to do that, the probability
of success is factored into the results obtained by the net. Images are
captured at different altitudes in an attempt to solve the trade-off between
image quality, including spatial resolution, context, and others. After
presenting theoretical aspects, the simulation environment in which the
approach was developed and tested is detailed, and the experiments conducted
for validation are described. The overall approach is shown to produce the
desired results, at least for the simulation environment in which it was
tested.

Further work is currently underway in three main directions. Firstly, we would
like to establish some success criteria for the procedure. For instance, we
would like to develop bounds to enable more accurate ways of analyzing our
results . Secondly, we would like to generate more realistic images on which
the semantic segmentation can be trained and tested. Lastly, we would like to
test the approach on actual data and conduct a flight-test to achieve real-
life validation.

## Acknowledgement

We would like to thank Distinguished Professor Daniel Wehis, head of the
Technion Autonomous System Program (TASP) for his continuous support.

## References

  * [1] R. D. d. L. Torres, M. Molina, and P. Campoy, “Survey of bayesian networks applications to intelligent autonomous vehicles,” arXiv preprint arXiv:1901.05517, 2019.
  * [2] J. A. Starek, B. Açıkmeşe, I. A. Nesnas, and M. Pavone, “Spacecraft autonomy challenges for next-generation space missions,” in Advances in Control System Technology for Aerospace Applications, pp. 1–48, Springer, 2016.
  * [3] N. Serrano, “A bayesian framework for landing site selection during autonomous spacecraft descent,” in 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5112–5117, IEEE, 2006.
  * [4] M. Coombes, W.-H. Chen, and P. Render, “Site selection during unmanned aerial system forced landings using decision-making bayesian networks,” Journal of Aerospace Information Systems, pp. 491–495, 2016.
  * [5] J. Ding, C. J. Tomlin, L. R. Hook, and J. Fuller, “Initial designs for an automatic forced landing system for safer inclusion of small unmanned air vehicles into the national airspace,” in 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC), pp. 1–12, IEEE, 2016.
  * [6] T. H. Chung and J. W. Burdick, “A decision-making framework for control strategies in probabilistic search,” in Proceedings 2007 IEEE International Conference on Robotics and Automation, pp. 4386–4393, IEEE, 2007\.
  * [7] L. F. Bertuccelli and J. How, “Robust uav search for environments with imprecise probability maps,” in Proceedings of the 44th IEEE Conference on Decision and Control, pp. 5680–5685, IEEE, 2005.
  * [8] L. D. Stone, Theory of optimal search.  Elsevier, 1976.
  * [9] Z. Drezner and N. Farnum, “A generalized binomial distribution,” Communications in Statistics-Theory and Methods, vol. 22, no. 11, pp. 3051–3063, 1993.
  * [10] Z. Drezner, “On the limit of the generalized binomial distribution,” Communications in Statistics—Theory and Methods, vol. 35, no. 2, pp. 209–221, 2006.
  * [11] B. James, K. James, and Y. Qi, “Limit theorems for correlated bernoulli random variables,” Statistics & probability letters, vol. 78, no. 15, pp. 2339–2345, 2008.
  * [12] L. Wu, Y. Qi, and J. Yang, “Asymptotics for dependent bernoulli random variables,” Statistics & Probability Letters, vol. 82, no. 3, pp. 455–463, 2012.
  * [13] Y. Zhang and L. Zhang, “Limit theorems for dependent bernoulli variables with statistical inference,” Communications in Statistics-Theory and Methods, vol. 46, no. 4, pp. 1551–1559, 2017.
  * [14] S. Shah, D. Dey, C. Lovett, and A. Kapoor, “Airsim: High-fidelity visual and physical simulation for autonomous vehicles,” in Field and service robotics, pp. 621–635, Springer, 2018.
  * [15] “Unreal engine.” https://www.unrealengine.com.  Accessed: 2021-03-09.
  * [16] “Brushify - urban buildings pack.” https://www.unrealengine.com/marketplace/en-US/product/brushify-urban-buildings-pack/.
  * [17] C. Yu, J. Wang, C. Peng, C. Gao, G. Yu, and N. Sang, “Bisenet: Bilateral segmentation network for real-time semantic segmentation,” in Proceedings of the European conference on computer vision (ECCV), pp. 325–341, 2018.