# Towards 3D Metric GPR Imaging Based on DNN Noise Removal and Dielectric
Estimation

Jinglun Feng1, Liang Yang1, Jizhong Xiao1∗ 1The CCNY Robotics Lab, Electrical
Engineering Department, The City College of New York, New York, USA
email(jfeng1, lyang1<EMAIL_ADDRESS>∗Corresponding author.) Financial
support for this study was provided by NSF grant IIP-1915721, and by the U.S.
Department of Transportation, Office of the Assistant Secretary for Research
and Technology (USDOT/OST-R) under Grant No. 69A3551747126 through INSPIRE
University Transportation Center (http://inspire-utc.mst.edu) at Missouri
University of Science and Technology. J Xiao has significant financial
interest in InnovBot LLC, a company involved in R&D and commercialization of
the technology.

###### Abstract

Ground Penetrating Radar (GPR) is one of the most important non-destructive
evaluation (NDE) devices to detect subsurface objects (i.e., rebars, utility
pipes) and reveal the underground scene. The two biggest challenges in GPR-
based inspection are the GPR data collection and subsurface target imaging. To
address these challenges, we propose a robotic solution that automates the GPR
data collection process with a free motion pattern. It facilitates the 3D
metric GPR imaging by tagging the pose information with GPR measurement in
real time. We also introduce a deep neural network (DNN) based GPR data
analysis method which includes a noise removal segmentation module to clear
the noise in GPR raw data and a DielectricNet to estimate the dielectric value
of subsurface media in each GPR B-scan data. We use both the field and
synthetic data to verify the proposed method. Experimental results demonstrate
that our proposed method can achieve better performance and faster processing
speed in GPR data collection and 3D GPR imaging than other methods.

###### Index Terms:

Back-projection (BP) algorithm, ground penetrating radar (GPR), deep neural
network (DNN), none-destructive evaluation (NDE).

##  I Introduction

Ground Penetrating Radar is widely used in non-destructive evaluation/testing
(NDE/NDT), field archaeology investigation, infrastructure inspection, and
measurements. GPR works by sending a pulse of polarized high-frequency
electromagnetic (EM) wave into the subsurface media. EM wave attenuates as it
travels in media and reflects when it encounters a material change. GPR
antenna would thus record the strength and traveled time of each reflected
pulse. The received signal is called an _A-scan_. When GPR antenna surveys
over a subsurface object, it produces a series of A-scans at different
positions, and the ensemble of A-scans forms a _B-scan_. The B-scan contains
the hyperbolic feature which would indicate the location of the target.

However, there are two major challenges to reveal underground objects using
GPR. First, it is time-consuming and tedious to manually scan a large area for
detailed mapping since current commercial GPR devices can only move forward to
trigger survey wheel encoder but cannot make turns. It causes the GPR data to
be collected along X-Y grid lines, making the data collection vary from person
to person and inconsistent from time to time. Second, the current GPR imaging
techniques are easily affected by several constraints, i.e., _background data
noise_ , _permittivity of the surrounding media_ , etc. [1, 2]. Thus, it is
very important to automate the GPR data collection procedure, tag accurate
pose information with each GPR sample, and propose an efficient metric GPR
imaging method to allow non-professional people to understand the result of
reconstructed subsurface objects.

To reach this purpose, many recent researches have devoted to the GPR imaging
techniques, among which the back-projection (BP) algorithm is the most
significant and commonly used GPR imaging method in NDT industry [3]. Back
projection is based on diffraction stacking that determines the location of
the targets. On the other hand, the current practice of GPR data collection
still requires human inspectors to pre-mark a grid map on the ground, and push
the GPR cart along the straight lines [4]; or mount the GPR device on a track
to make the measurements [3]. Due to the above limitations, the current GPR
data collection methods lack the motion freedom and the accurate metric
positioning is difficult to obtain.

GPR scanning data fused with metric information are widely researched in
recent years as well. M. Pereira [5] introduced a new GPR system integration
with Augmented Reality (AR) based positioning available on Google smartphones.
However, the limitation in this work is that the positioning system (i.e.,
Google project Tango) provided by the Google is out of service now. GPR users
cannot further obtain the benefits from the system they proposed. Similarly,
H. Li [6] introduced an automatic subsurface pipeline mapping and 3D
reconstruction by fusing a GPR and a Camera. They proposed a robotic
underground pipeline mapping model to facilitate GPR-based 3D reconstruction.
Their major contribution is that they used the visual Simultaneous
Localization and Mapping (V-SLAM), J-linkage and maximum likelihood method to
estimate the radii and locations of all pipelines. However, the authors still
need to pre-mark the grid map on the surface and manually push the GPR cart
along the grid lines to take GPR measurements.

In this paper, we propose a novel 3D GPR imaging method that consists of three
modules. A _robotics data collection_ module to provide a free motion pattern
for GPR scanning and tag metric positioning information with GPR measurements;
a _background noise removal_ module to clean the noisy data in the GPR B-scan
images; and a _DielectricNet_ to estimate the dielectric property of
subsurface media.

##  II Methodology

The system architecture is illustrated in Fig.1, which consists of the robotic
data collection module and the GPR data analysis software. The Omni-
directional robot carries a GPR antenna at the bottom of the chassis to detect
and map underground objects and an RGB-D tracking camera to obtain the
accurate 6-DOF pose [7] in real time. By tagging the GPR measurements with
accurate position information in a synchronized way, it enables the robot to
scan the ground surface in arbitrary and irregular trajectories and
facilitates high-resolution 3D GPR imaging. The robot is able to perform a
non-linear free motion that will revolutionize how GPR data is collected,
interpreted, and visualized. The GPR data analysis software includes the DNN-
based GPR noise removal module and the dielectric prediction module. For each
input B-scan image, the noise removal module directly decodes the input image
into hyperbolic features. The dielectric prediction module takes the
segmentation masks from the noise removal module and pools the dielectric
information from a Convolutional Recurrent Neural Network (CRNN).

Figure 1: The system architecture consists of a robotic GPR data collection
module and a DNN-based GPR data analysis software which consists of noise
removal module and dielectric prediction module.

###  II-A GPR 3D Metric Imaging with Free Motion Pattern

To create a 3D GPR imaging, the BP algorithm is implemented in this section.
The essence of the A-scan represents the amplitude of EM energy, while the BP
is a process of aggregation that converts different amplitude of energy into a
_semi-sphere_ format at different time. As illustrated in Fig.2, the brighter
semi-sphere indicates the higher amplitude part in A-scan. Furthermore, the
radius of each semi-sphere in BP image indicates the depth between the ground
and the object, which is depicted by Equ.1[6]:

$\displaystyle{\forall}A{{}_{q}^{k}}\in B{{}_{k}},\
(x-a){{}^{2}}+(y-b){{}^{2}}=(a{{}_{t}}*t){{}^{2}},\ y<0$ (1)

where $a,b$ represents the specific position of each A-scan measurement in a
concrete slab. $A{{}_{q}^{k}}=\\{a{{}_{t}}|t=1,...,n_{q}\\}$ represents the
$q$-th A-scan measurement in $k$-th B-scan data, while $t$ and $a{{}_{t}}$
indicate the traveling time and amplitude of A-scan signal respectively, then
$n{{}_{q}}$ means the total samples in a A-scan measurements. Meanwhile, we
also have $B{{}_{k}}=\\{A{{}_{q}^{k}}|q=1,...,n{{}_{k}}\\}$ that represents
the $k$-th B-scan consisting of $n{{}_{k}}$ A-scans. By implementing the BP
algorithm on a B-scan data, the intersection of multiple back-projected A-scan
signals with the highest energy level is the possible target location.

Figure 2:  BP algorithm converts the A-scan raw data into a set of semi-
spheres.

Free Motion Based Data Collection:  The current practice for GPR data
collection requires a human inspector to mark the grid map, push the GPR
device, carefully follow the straight lines in X-Y directions, and use the
wheel encoder to trigger GPR sampling. These GPR measurements are collected at
equal spacing and stored slice by slice along X or Y directions. The current
GPR scanning method heavily limits the freedom of motion and manually records
these linear motion trajectories for GPR data post-processing. It is time-
consuming and tedious to mark the grid map and perform the GPR date collection
covering a large area. We propose an automatic GPR data collection and 3D
metric GPR imaging method that combines robot control and vision-based
positioning with GPR signal processing to locate and map subsurface targets.
Instead of relying on a linear wheel encoder, we use an RGB-D tracking camera
to estimate the position and orientation in real time [8, 9]. As shown in
Fig.1, we used an Omni-directional robot carrying both a GPR antenna and RGB-D
camera that can move in any direction without spinning and thus provide a non-
linear motion for the GPR scanning [7].

By taking advantage of this non-linear free-motion pattern, the GPR data is
automatically and continuously collected and tagged with position and
orientation [X, Y, Theta] at each measurement. Fig.3 illustrates how GPR
B-scan data is collected along a zig-zag pattern, and it doesn’t require the
intervention of the human inspector.

Figure 3: The B-scan profile tagged with metric positioning information.

###  II-B DNN-based GPR Data Preprocessing

Due to the GPR background noise and the uncertainty of the material
dielectric, it is almost impossible to reconstruct the underground objects
with an accurate metric. To resolve this issue, we propose two models, which
are the GPR object segmentation model and the dielectric prediction model, to
facilitate accurate subsurface object reconstruction and mapping. The DNN-
based GPR data analytics are illustrated in Fig.1, where we use a U-Net [10]
to perform subsurface object segmentation and a DielectricNet to predict the
dielectric of the subsurface material.

####  II-B1 GPR Background Noise Removal

GPR works by sending an EM wave into the ground. EM wave attenuates as it
travels in the medium and reflects when it encounters a change in material
dielectric. GPR antenna would thus record the strength of each reflection as
well as the time it takes to return to the receiver. Therefore, when GPR
surveys over an underground object, some of the GPR energy pulse reflects when
they hit the subsurface object, some energy continues to penetrate through the
object until they become too weak to get back. However, those weak reflections
would also generate hyperbolic features in the B-scan image, which could
affect the GPR migration result of the targets. We call those weak responses
as GPR Background Noise (see in Fig.1). Hence, it is important for us not only
to focus on enhancing and sharping B-scan features but also removing those
background noise.

Specifically, we firstly get the hyperbolic masks from the input raw B-scan
images by taking advantage of a segmentation model (i.e., UNet). Then, we
filter the raw B-scan data with mask B-scan features so that the filtered data
only keeps the strongest target reflection signal.

####  II-B2 GPR Targets Depth Prediction

The dielectric constant is crucial for the GPR migration process since it
determines the depth of each target shown in a B-scan image. As shown in
Equ.3, the EM wave which is emitted from the GPR antenna is supposed to be the
speed of light in a vacuum. However, the dielectric ${\mathcal{D}}$ of
subsurface materials would make a difference to the traveling speed, which
further impacts the depth prediction of the target.

$\displaystyle v$ $\displaystyle=\frac{C}{\sqrt{\mathcal{D}}}$ (2)
$\displaystyle{D}_{tar}$ $\displaystyle=\frac{{T}_{tr}*v}{2}$ (3)

where ${D}_{tar}$ denotes the depth of subsurface targets while ${T}_{tr}$ is
the two-way travel time of GPR EM signal.

Thus, for any underground object, if the dielectric of the surrounding
material is high, the B-scan hyperbolic feature would be shallower than the
same target with a lower value of the environment dielectric. On account of
this property and inspired by [11], we introduced a CRNN model in this letter
(i.e., DielectricNet) to estimate the dielectric, $d_{t(i)},i\in\\{0,1,...\\}$
for each target.

Given a GPR scan in media with an unknown dielectric, an informative and
discriminative feature representation plays a significant role in dielectric
estimation. The feature representation should preserve the dielectric property
of the underground media. Moreover, it should consider the spatial
distribution of all hyperbolic features to enable better dielectric
prediction. We take advantage of Recurrent Neural Network (RNN), which is able
to take a sequence of signals, either spatially continuous or temporally
continuous, to estimate the dielectric. The $ResNet34$ [12] is implemented as
the encoder and an RNN model as the decoder, where the GPR images are fed into
a series of convolutional layer to obtain the feature map that contains
encoded dielectric information [13].

In our implementation, the input noise-free B-scan images are resized to
$224\times 224$ while the $ResNet34$ architecture is changed to work in a
fully convolutional manner by converting fc6 to fully convolutional layers.
Then, a bidirectional long short-term memory (Bi-LSTM) decoder extracts
dielectric property from the CNN encoded feature descriptors, where it
sequentially outputs their corresponding confidence scores for the predicted
dielectric value. This design helps the model to reason the dielectric
property.

Loss Design: Once the dielectric property is predicted, we optimize the model
by using a Connectionist Temporal Categorical (CTC) loss.

$\centering\sum_{i}P(\mathcal{D}\mid\mathbf{x})=\sum_{i}P(\mathcal{D}\mid\pi_{i})P(\pi_{i}\mid
x)\@add@centering$ (4)

where ${\mathcal{D}}$ is the ground truth of the dielectric property,
$\mathbf{x}$ is the input sequence that comes from the RNN model, and
$\pi_{i}$ denotes each possible value of the dielectric. The depth of the
subsurface target can be further calculated according to Equ.3.

##  III Experimental Study

To demonstrate the effectiveness of the proposed 3D metric GPR imaging method,
we conducted both simulation and experimental study. The synthetic data were
generated by gprMax [14] while the field data were collected on a concrete
slab at CCNY Robotics Lab Testing Pit. We also compare the effectiveness
between our method and other approaches, and discuss the effectiveness of the
DielectricNet model.

To create the dataset for experimental purposes, we have collected two kinds
of data: 1) the field data collected on CCNY Testing Pit, which contains $120$
B-scan data; 2) synthetic data we proposed in [15], and it contains $628$
B-scan data. Note that all the B-scan images are taken as the input for noise
cancellation and dielectric prediction task and then used to verify the
proposed method.

(a) The ground truth of the synthetic slab generated by gprMax.

(b) The ground truth of the field CCNY Robotics Lab concrete slab.

(c) The raw B-scan of the synthetic slab.

(d) The raw B-scan of the field slab.

(e) Filtered B-scan of the synthetic slab.

(f) Filtered B-scan of the field slab.

(g) The migration result with the synthetic GPR data by proposed 3D metric
method.

(h) The migration result with the field GPR data by proposed 3D metric method.

(i) The migration result with the synthetic GPR data by the conventional
method[4].

(j) The migration result with the field GPR data by the conventional
method[4].

(k) The migration result with the synthetic GPR data by the method proposed in
[9].

(l) The migration result with the field GPR data by the method proposed in
[9].

Figure 4: Effectiveness and noise comparison of proposed method on both
synthetic and field GPR data, all the 3D GPR imaging results are shown with a
front view. Note that in subfigure $(b)$, the black bounding box demonstrates
the field data collection area.

###  III-A Comparison Study

As illustrate in Fig.4, both synthetic and field GPR data have been used to
validate the effectiveness of our method. Specifically, Fig.4$(a)$ shows a
synthetic slab which is $1.3m$ meters long and $0.25$ meters deep. In
addition, Fig.4$(b)$ shows the ground truth of the field concrete slab, note
that the black bounding box showing in $(b)$ demonstrates the field GPR data
collection area, which is $1.55$ meters long and $0.2$ meters deep. Fig.4$(c)$
to $(f)$ shows the raw B-scan data and noise removed B-scan data respectively.
Fig.4$(g)$ and $(h)$ shows the proposed 3D metric GPR imaging results.

To further validate the proposed method, we compare our approach with the
conventional BP method [4] and our previous DNN-based migration method
proposed in [9], where different DNN architecture was used to remove the noise
in B-scan data.

Fig.4$(g)$-$(l)$ demonstrate GPR imaging results generated by the above
methods respectively. Specifically, $(i)$ and $(j)$ show that the conventional
BP results contain too much noisy data while in subfigure $(k)$ and $(l)$, the
noise is removed and the result is sharper using our old DNN-based migration
method. The best 3D GPR imaging results are shown in subfigure $(g)$ and $(h)$
using our new 3D metric migration method.

Table.II provides quantitative evaluation of these three methods in terms of
the GPR image noise level and similarity effectiveness, which are evaluated
with four different metrics: Euclidean Distance, Mean Square Error, Signal-to-
Noise-Ratio and Structural Similarity Index on both synthetic dataset and
field dataset. For Euclidean Distance and Mean Square Error, the lower the
value, the better performance is. For Signal-to-Noise-Ratio and Structural
Similarity Index, the larger the value, the better effectiveness it
represents. Our new DNN-based method outperformed other methods.

TABLE I: Effectiveness Comparison between proposed method and others with multiple metrics | E_distance | MSE | SNR | SSMI
---|---|---|---|---
Ours w/ synthetic dataset | 2.23e+04 | 284.774 | 16.982 | 0.987
[9] w/ synthetic dataset | 4.05e+04 | 351.455 | 16.068 | 0.976
[4] w/ synthetic dataset | 9.08e+04 | 453.541 | 14.961 | 0.983
Ours w/ field dataset | 1.39e+03 | 397.384 | 15.810 | 0.980
[9] w/ field dataset | 1.92e+03 | 485.132 | 14.943 | 0.971
[4] w/ field dataset | 3.40e+03 | 983.156 | 11.876 | 0.938

The reason is that a segmentation model could capture the shape of the
hyperbolic feature, which contains all the useful information in B-scan data.
In contrast, the information in the bounding box region generated by the
detection-oriented model would contain some noise data besides the useful
B-scan data.

###  III-B Why CRNN model is used for dielectric prediction?

The CRNN model has widely used for the scene text recognition tasks in
computer vision community [11]. The input to the CRNN model is the image that
contains text, while the output is the text sequence extracted from the image.
This end-to-end method is proven to be effective and inspired us to consider
that the hyperbolic features contained in GPR B-scan images could also be
analyzed in a similar way by the CRNN model. The hyperbolic features are
normally determined by several factors including the dielectric property of
surrounding media, the dimension, material, and depth of the buried utility
pipelines. As long as the dataset contains enough objects with different
combinations of the above properties, the CRNN model shall be able to extract
the dielectric value of the surrounding media with the various values of other
factors. In our dataset, the dimension, material, and depth of the targets are
illustrated in Fig.4 (a) and (b). Since the dielectric value and depth of
hyperbolic features are correlated, we can use a CRNN model to encode this
property as an abstract sequence, and further decode and estimate this
abstract sequence as a specific dielectric value. Specifically, the CNN
encoder we implement is ResNet34 and the RNN decoder is Bi-LSTM. In addition,
different methods, which are a CRNN model with VGG16 encoder and Bi-LSTM
decoder and a single RseNet34 network[9] respectively, are illustrated by
comparing their prediction accuracy and running time, since the loss function
in each method is not the same. As shown in Table II, our CRNN model achieves
the highest dielectric prediction accuracy and at least $24.6\%$ faster in
comparison with other baseline methods.

TABLE II: Performance Comparison between proposed method and others | Ours | VGG16 + Bi-LSTM | ResNet34
---|---|---|---
Acc. | 98.29 | 97.56 | 98.05
Time(sec.) | 0.0049 | 0.0065 | 0.0075

##  IV Conclusion

This paper introduces a DNN-based 3D metric GPR imaging method, which can
enhance the GPR imaging performance by taking the following steps. 1)
Introducing the robotic GPR data collection that provides free motion pattern
for GPR scanning and tags the position information with GPR measurements. 2)
Using a DNN-based segmentation model to remove the background noise from raw
GPR B-scan images. 3) Using DielectricNet to estimate the dielectric value of
the underground environment. Experimental results show that our proposed
method performs better in comparison with other approaches. Our method makes
the GPR data collection significantly easier by enabling the automatic scan of
the flat surface in arbitrary trajectories with minimal human intervention. It
eliminates the time, hassle, and cost to laying out grid lines on flat terrain
and reducing the hassle to closely follow the grid lines and the note-taken
time to record the linear motion trajectory in X-Y directions. Our 3D metric
GPR imaging software processes the GPR data tagged with pose information to
reveal subsurface objects with sharper images and make inspection results easy
to understand.

## References

  * [1] U. Pe’er and J. G. Dy, “Automated target detection for geophysical applications,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 55, no. 3, pp. 1563–1572, 2016.
  * [2] T. Thitimakorn, N. Kampananon, N. Jongjaiwanichkit, and S. Kupongsak, “Subsurface void detection under the road surface using ground penetrating radar (gpr), a case study in the bangkok metropolitan area, thailand,” _International Journal of Geo-Engineering_ , vol. 7, no. 1, pp. 1–9, 2016\.
  * [3] S. Demirci, E. Yigit, I. H. Eskidemir, and C. Ozdemir, “Ground penetrating radar imaging of water leaks from buried pipes based on back-projection method,” _Ndt & E International_, vol. 47, pp. 35–42, 2012.
  * [4] X. Xie, J. Zhai, and B. Zhou, “Back-fill grouting quality evaluation of the shield tunnel using ground penetrating radar with bi-frequency back projection method,” _Automation in Construction_ , vol. 121, p. 103435.
  * [5] M. Pereira, D. Burns, D. Orfeo, R. Farrel, D. Hutson, and T. Xia, “New gpr system integration with augmented reality based positioning,” in _Proceedings of the 2018 on Great Lakes Symposium on VLSI_ , 2018, pp. 341–346.
  * [6] H. Li, C. Chou, L. Fan, B. Li, D. Wang, and D. Song, “Toward automatic subsurface pipeline mapping by fusing a ground-penetrating radar and a camera,” _IEEE Transactions on Automation Science and Engineering_ , vol. 17, no. 2, pp. 722–734, 2019.
  * [7] J. Feng, L. Yang, E. Hoxha, S. Sotnikov, D. Sanakov, and J. Xiao, “Gpr-based model reconstruction system for underground utilities using gprnet,” _arXiv preprint arXiv:2011.02635_ , 2020.
  * [8] L. Yang, I. Dryanovski, R. G. Valenti, G. Wolberg, and J. Xiao, “Rgb-d camera calibration and trajectory estimation for indoor mapping,” _Autonomous Robots_ , vol. 44, no. 8, pp. 1485–1503, 2020.
  * [9] J. Feng, L. Yang, H. Wang, Y. Song, and J. Xiao, “Gpr-based subsurface object detection and reconstruction using random motion and depthnet,” in _2020 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2020, pp. 7035–7041.
  * [10] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in _International Conference on Medical image computing and computer-assisted intervention_. Springer, 2015, pp. 234–241.
  * [11] B. Shi, X. Bai, and C. Yao, “An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 39, no. 11, pp. 2298–2304, 2017.
  * [12] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [13] X. Rong, C. Yi, and Y. Tian, “Unambiguous text localization and retrieval for cluttered scenes,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2017, pp. 5494–5502.
  * [14] C. Warren, A. Giannopoulos, and I. Giannakis, “gprmax: Open source software to simulate electromagnetic wave propagation for ground penetrating radar,” _Computer Physics Communications_ , vol. 209, pp. 163–170, 2016.
  * [15] J. Feng, L. Yang, H. Wang, Y. Tian, and J. Xiao, “Subsurface pipes detection using dnn-based back projection on gpr data,” in _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)_ , January 2021, pp. 266–275.