# Semantics-aware LiDAR-Only Pseudo Point Cloud Generation
for 3D Object Detection

Tiago Cortinhal1,∗, Idriss Gouigah1,∗, and Eren Erdal Aksoy1 *These authors
contributed equally to this work.1Halmstad University, School of Information
Technology, Center for Applied Intelligent Systems Research, Halmstad, Sweden
{tiago.cortinhal<EMAIL_ADDRESS>work was funded by the European
Union (grant no. 101069576).Views and opinions expressed are however those of
the author(s) only and do not necessarily reflect those of the European Union
or the European Climate, Infrastructure and Environment Executive Agency
(CINEA). Neither the European Union nor the granting authority can be held
responsible for them.Special thanks to Abu Mohammed Raisuddin for his time
helping with this paper.

###### Abstract

Although LiDAR sensors are crucial for autonomous systems due to providing
precise depth information, they struggle with capturing fine object details,
especially at a distance, due to sparse and non-uniform data. Recent advances
introduced pseudo-LiDAR, i.e., synthetic dense point clouds, using additional
modalities such as cameras to enhance 3D object detection. We present a novel
LiDAR-only framework that augments raw scans with denser pseudo point clouds
by solely relying on LiDAR sensors and scene semantics, omitting the need for
cameras. Our framework first utilizes a segmentation model to extract scene
semantics from raw point clouds, and then employs a multi-modal domain
translator to generate synthetic image segments and depth cues without real
cameras. This yields a dense pseudo point cloud enriched with semantic
information. We also introduce a new semantically guided projection method,
which enhances detection performance by retaining only relevant pseudo points.
We applied our framework to different advanced 3D object detection methods and
reported up to $2.9\%$ performance upgrade. We also obtained comparable
results on the KITTI 3D object detection test set, in contrast to other state-
of-the-art LiDAR-only detectors.

##  I INTRODUCTION

Recent works on LiDAR-based perception show that the LiDAR modality plays a
pivotal role in enabling autonomous systems to understand complex
environments. While LiDARs excel in capturing precise depth information and
generating accurate point cloud data in a wide field of view, these active
sensors may encounter limitations in capturing fine-grained object details,
particularly for distant or unreflective surfaces. This is mainly because the
rendered LiDAR data is sparse, unstructured, and follows a non-uniform
sampling.

LiDAR-only 3D object detection techniques [1, 2, 3, 4, 5, 6, 7, 8]
conventionally rely on such raw sparse LiDAR data to perceive the
surroundings. Recent scientific investigations [9, 10, 11, 12] reveal the
potential of pseudo-LiDAR, a novel technique that leverages mono and stereo
imaging data to generate synthetic point cloud representations. These studies
heavily rely on the fusion of pseudo-LiDAR data with the original LiDAR scans.
Such a fusion of both data sources indeed augments the information available
to the 3D object detection system. This is mainly because, unlike the original
sparse LiDAR point clouds, the pseudo point clouds are relatively denser,
enhancing the overall richness of the data. Notably, the synthetic data from
pseudo-LiDAR fills the gaps left by traditional LiDAR scans. By capitalizing
on the complementary strengths of both data sources, the integrated approaches
demonstrate substantial potential in enhancing the object detection.

With this motivation, we introduce a novel modular framework that augments raw
LiDAR scans with denser pseudo point cloud data. Our proposed framework
differs from all other relevant works [9, 10, 11, 12] in that it solely relies
on the LiDAR sensor and the scene semantics without incorporating any
additional modality such as mono or stereo cameras. Furthermore, our framework
is unique since the final output is semantically segmented dense point clouds
with 3D bounding boxes for each detected object in the scene as shown in Fig.
LABEL:fig:scene.

As the first step, our framework employs an off-the-shelf segmentation model
(e.g., SalsaNext [13]) to extract the scene semantics of the raw and sparse
full-scan LiDAR point clouds. Next, a state-of-the-art multi-modal domain
translator, named TITAN-Next [14], is triggered to translate LiDAR segments
into expected synthetic image segments and depth cues, without requiring any
real camera data. A dense pseudo point cloud enriched with semantic
information is then directly rendered from these estimated synthetic camera
depth cues.

The higher the density of pseudo points, the greater the level of object
details in the scene, but also the larger the noise in the point cloud and the
longer the computation time. The inclusion of all new pseudo points may lead
to an excessive number of irrelevant and redundant points, potentially
overwhelming, for instance, the object detection system and hindering its
efficiency [12]. Therefore, in the context of point cloud augmentation,
analysis of this factor is of utmost importance as the amount of density
introduces challenges to the downstream tasks. To address this issue, we
introduce a novel Semantically Guided Projection (SGP) technique to select the
most important foreground points in the pseudo point cloud. This approach
selectively retains points only from these classes that are highly relevant to
the object detection task (such as pedestrians, vehicles, or cyclists), thus,
significantly reducing the computational burden and improving the overall
detection performance. This proposed targeted projection not only reduces the
data volume but also enhances the discriminative power of the pseudo point
cloud for object detection.

We applied our framework to different advanced 3D object detection methods
(e.g., PointPillars [15], SECOND [16], and Voxel-RCNN [17]) and obtained a
certain performance upgrade with little to no modification to the detection
model. These experiments convey that our framework is agnostic to object
detector architectures, however, works with higher performance in the case of
having detectors specifically designed for pseudo point cloud data, such as
VirConv [12]. We also obtained comparable results on the KITTI test set in
contrast to other state-of-the-art LiDAR-only detectors.

In a nutshell, our contributions are manifold:

  * •

We propose a modular LiDAR-only pseudo point cloud generation framework
without requiring any additional modality, such as cameras.

  * •

We introduce a new projection scheme, SGD, to reduce the density of the pseudo
point cloud by selecting the points with the most relevant semantic
information.

  * •

Our framework returns not only dense point clouds but also semantic segments
and 3D object bounding boxes.

  * •

We conduct extensive experiments on different advanced 3D object detection
models and show the impact of our synthesized pseudo point cloud data.

Figure 2:  Our proposed modular framework has three blocks, each of which is
depicted by a unique background color. In the red box, a raw 3D LiDAR point
cloud $p$ is first projected onto the 2D range image plane $p^{\prime}$ to be
segmented by SalsaNext [13], predicting $p^{\prime}_{s}$. The green box
highlights the TITAN-Next [14] generative model, conditioned on the
concatenated $p^{\prime}$ and $p^{\prime}_{s}$ to generate a synthetic segment
map $\widehat{y}_{s}$ and a synthetic depth map $\widehat{y}_{d}$ in the
expected front-view camera space, without requiring any camera information.
Finally, as shown in the blue box, these generated segment and depth maps
($\widehat{y}_{s}$ and $\widehat{y}_{d}$) are synthesized to render a labeled
pseudo point cloud by our new Semantically Guided Projection method. The
segmented original LiDAR point cloud ($p^{\prime}_{s}$) is then concatenated
with the pseudo counterpart to obtain the final semantically segmented dense
point cloud, which is further fed to the subsequent 3D object detector.

##  II Related Work

Several studies dedicated to 3D object detection employ point cloud
augmentations, however, they are mostly multimodal. In such methods,
information coming from other sensors (e.g., RGB cameras) is fused at the
early stage to enhance the raw LiDAR point clouds. These early fusion methods
can be divided into two categories: Point painting [18, 19, 20, 21] and pseudo
point cloud generation [9, 10, 12, 22, 23].

#### Point Painting

PointPainting [18] is a pioneering method that uses camera images to _paint_
3D LiDAR point clouds. PointPainting effectively leverages the high-resolution
and semantic details from cameras to improve object localization and
classification in dense and sparse LiDAR data. This way, the model achieves
superior performance in detecting objects with fine-grained details and can
handle occlusions, making it well-suited for complex urban driving scenarios.
Following methods [19, 20] extend the idea by painting the point cloud using
more relevant features.

#### Pseudo point cloud generation

These methods such as MVP [22] leverage the camera information by generating a
depth map from an image (either by disparity map or using a neural network),
which is then projected into the LiDAR space. The goal is to reduce the
sparsity of the LiDAR data by increasing the overall number of points. Sparse
Fuse Dense [23] decorates the generated pseudo point cloud with RGB values
from the camera and uses grid-wise attentive fusion to merge features coming
from the pseudo point cloud with that of the raw point cloud data. More
recently, VirConv [12] introduced a model specifically designed to tackle the
drawbacks of pseudo point clouds, such as the longtail and misalignment
issues. With noise-resistant convolution and stochastic voxel discard, VirConv
[12] manages to reach a high level of performance in the car detection
challenge of the KITTI dataset.

Although these methods in both mainstream categories help improve the
performance of 3D detection, they require an additional modality such as RGB
cameras, and come with the cost of high computation time. Moreover, to the
best of our knowledge, these two categories have so far been studied
separately. Our approach differs from all these methods as it couples both
point painting and pseudo point cloud generation approaches, while we solely
rely on the LiDAR sensor and employ the semantic information to select the
most valuable pseudo points, reducing the processing cost.

##  III Method

As illustrated in Fig. 2, we proposed a framework involving three individual
modules. The blue box represents the main technical contribution of this work,
whereas the red and green boxes involve other trained state-of-the-art
networks.

###  III-A Pipeline

As shown in the red box in Fig. 2, the proposed pipeline starts with the
semantic segmentation of the raw and sparse full-scan LiDAR point cloud. Given
a point cloud $p\in\mathcal{R}^{N\times 4}$, where $N$ is the number of points
with $x$, $y$, $z$ point coordinates and $i$ intensity values, we first employ
an off-the-shelf semantic segmentation model (e.g., SalsaNext [13]) to extract
the scene semantics. In this case, the point cloud $p$ is first projected onto
the 2D range image plane $p^{\prime}$ to be segmented by SalsaNext [13],
predicting a semantic segment map $p^{\prime}_{s}\in\mathcal{R}^{N}$. Please
note that any other segmentation model can be employed here instead of
SalsaNext [13].

At the next step (highlighted in the green box), the concatenated $p^{\prime}$
and $p^{\prime}_{s}$ are fed to a generative domain translation module, named
TITAN-Next [14], to generate a synthetic segmentation map $\widehat{y}_{s}$
and a synthetic depth map $\widehat{y}_{d}$ in the expected front-view camera
space, without requiring a real camera. TITAN-Next is a multi-modal domain
translation network [24] and maps the segmented LiDAR range view projection
onto the expected camera image space. Thus, TITAN-Next synthesizes semantic
segments and the corresponding depth information in this estimated camera
space even though no real camera is available. We here again note that since
TITAN-Next is not forming the main contribution of this framework, it can be
replaced by any other cross-domain mapping method working in a multi-modal
setup.

Finally, as depicted in the blue box in Fig. 2, these generated segment and
depth maps ($\widehat{y}_{s}$ and $\widehat{y}_{d}$) are synthesized to render
a labeled pseudo point cloud by our new Semantically Guided Projection method,
which is detailed next.

###  III-B Semantically Guided Projection (SGP)

Methods such as [9] generate a pseudo point cloud from depth estimation and
feed it as input to the detector entirely. However, the density of the
generated point cloud makes the computation time prohibitive for real-time
applications. Methods such as MVP [22] reduce the size of the point cloud by
leveraging 2D detection results, such that a point cloud is created only in
the regions of interest yielded by a 2D detector. However, the 2D detection
results are redundant together with the 3D detection and cannot be leveraged
in other downstream tasks. Instead, we propose to leverage the semantic
segmentation information in the LiDAR space to reduce the size of the pseudo
point cloud, while still being useful for other relevant subsequent downstream
tasks such as free space detection, in addition to object detection.

Semantic Guided Projection (SGP) is a very simple yet efficient method. SPG
translates the rendered semantic segmentation maps ($\widehat{y}_{s}$) and
depth estimation ($\widehat{y}_{d}$) into 3D dense pseudo point cloud data and
filters out noisy points as illustrated in Fig. 3.

First, SPG associates $\widehat{y}_{s}$ and $\widehat{y}_{d}$ pixel to pixel
since both maps are aligned. Here, SPG only selects depth values associated
with pertinent object classes for object detection tasks, such as cars,
pedestrians, or cyclists. This vastly reduces the density of the generated
pseudo point cloud. Next, by utilizing the available calibration information,
SGP conducts a transformation that projects the selected depth estimations
from the camera’s intrinsic frame of reference to the LiDAR’s extrinsic frame
of reference. The correspondence between the depth points and the 3D spatial
points can be mathematically described as follows:

$\tilde{y}=(u\times z,v\times z,z,1)~{}~{},$ (1)
$T^{velo}_{cam}=(T^{cam}_{velo})^{-1}~{}~{},$ (2)
$(T^{cam}_{velo})\times\tilde{y}=\tilde{x}~{}~{},$ (3)

where $T^{cam}_{velo}$ is the projection matrix from 3D LiDAR space to 2D
camera $(u,v)$ coordinate space, and $\tilde{y}$ is the homogenous coordinates
with $z$ being the predicted depth.

Figure 3: Semantically Guided Projection. The synthetic segment (top-left) and
depth (bottom-left) maps are pixel-to-pixel associated only for the relevant
semantic classes. In this example, the dense pseudo point cloud shown on the
right is generated only for the segmented car object. Note that red points are
from the original LiDAR point cloud, whereas gray points represent the
generated pseudo points.

As comprehensively described in [12, 25], pseudo point clouds introduce
inherent challenges compared to real counterparts, primarily due to inaccurate
depth estimation around object boundaries, leading to misalignment and long
tails, particularly for near object edges. While specialized methods such as
VirConv [12] address these concerns, not all techniques (including
PointPillars [1]) are optimized for pseudo point clouds, resulting in noise
and hindering the performance of the off-the-shelf object detectors.

To tackle this issue, we enrich SGP with a cleaning strategy rooted in the
analysis of original sparse LiDAR data. The core rationale here is that
objects in the real world are expected to yield corresponding LiDAR data
points in their vicinity. Given that LiDAR scans provide a relatively coarse
representation of the scene, we utilize this insight to identify outliers
within the pseudo point cloud. Specifically, if a pseudo point lacks nearby
real LiDAR points, it is indicative of an absence of a corresponding physical
object in the real world. Consequently, we establish a volumetric region
around each pseudo point and eliminate those lacking real points within this
specific volume. This methodology ensures the selection of only those pseudo
points that closely correspond to objects, effectively mitigating the
challenges associated with long tail and ghost measurements caused by border
effects in semantic segmentation and depth prediction. Fig. 4 shows a sample
scene before and after applying our pseudo point cloud cleaning method.

###  III-C 3D Object Detection

Finally, the segmented original LiDAR point cloud ($p^{\prime}_{s}$) is then
concatenated with the cleaned pseudo counterpart to obtain the final
semantically segmented dense point cloud, which is fed to the subsequent 3D
object detector as shown in the blue box in Fig. 2.

We here note that our method aims at augmenting the point cloud to improve the
detection results while bringing little to no modification to the detection
model. However, after extensive investigation, it has become clear that to be
processed correctly and yield better results, a pseudo point cloud needs to be
processed by a dedicated model. For instance, VirConv [12] constructed from
Voxel-RCNN [17] introduces a specific virtual sparse convolution to deal with
the challenges coming with the pseudo point clouds. VirConv [12] makes use of
a set of new modules called Noise Resistant Convolution (NRConv) and
Stochastic Voxel Discard (StVD) to improve the detection results by a large
margin. Thus, we perform experiments with both generic object detectors (e.g.,
PointPillars [1]) and object detectors designed for pseudo point clouds (e.g.,
VirConv [12]).

##  IV Datasets & Results

We here present our results on the KITTI datasets and compare with other
state-of-the-art methods in the literature.

###  IV-A The KITTI dataset

The KITTI 3D Object Detection dataset [26] is a widely used benchmark for
autonomous driving. It includes 7,481 annotated training frames and 7,518 test
frames captured in real-world urban environments. The dataset comprises
various sensor data modalities, including high-resolution RGB images, 3D point
clouds from LiDAR sensors, and calibration data for sensor alignment. It
offers precise 3D bounding box annotations for objects like cars, pedestrians,
and cyclists in each frame.

Figure 4: On the right, we show the effects of applying our cleaning to the
pseudo point cloud. Notice how the pseudo points (in gray) appear behind the
person on the left are properly removed, and we are left with pseudo points in
proximity to real points corresponding to the segmented person.

| Car | Pedestrian | Cyclists | Overall
---|---|---|---|---
Easy | Med | Hard | Easy | Med | Hard | Easy | Med | Hard | Easy | Med | Hard
PointPillars [1] | 87.5 | 78.7 | 75.7 | 57.2 | 50.9 | 46.3 | 82.5 | 62.0 | 58.6 | 75.7 | 63.9 | 60.2
Augmented PointPillars (Ours) | 88.3 | 79.5 | 76.5 | 58.9 | 52.6 | 47.9 | 82.7 | 63.1 | 59.7 | 76.6 | 65.1 | 61.4
Delta | +0.8 | +0.8 | +0.8 | +1.7 | +1.7 | +1.6 | +0.2 | +1.1 | +1.1 | +0.9 | +1.2 | +1.2
SECOND [16] | 88.1 | 78.2 | 73.2 | 60.0 | 52.8 | 47.3 | 75.8 | 61.1 | 57.5 | 74.6 | 64.0 | 59.4
Augmented SECOND (Ours) | 88.7 | 78.7 | 75.1 | 63.1 | 54.7 | 48.4 | 79.8 | 62.5 | 59.3 | 77.2 | 65.3 | 60.9
Delta | +0.6 | +0.5 | +1.9 | +3.1 | +1.9 | +1.1 | +4 | +1.4 | +1.8 | +2.6 | +1.3 | +1.5
Voxel-RCNN [17] | 92.0 | 84.9 | 82.6 | - | - | - | - | - | - | - | - | -
Augmented Voxel-RCNN (Ours) | 92.8 | 85.7 | 83.3 | - | - | - | - | - | - | - | - | -
Delta | +0.8 | +0.8 | +0.7 | - | - | - | - | - | - | - | - | -
Augmented Voxel-RCNN (Ours) + NRConv | 92.6 | 87.8 | 85.4 | - | - | - | - | - | - | - | - | -
Delta | +0.6 | +2.9 | +2.8 | - | - | - | - | - | - | - | - | -

TABLE I: Effects of our augmentation on different state-of-the-art LiDAR-based
object detectors on the KITTI validation set. NRConv boosts the performance
scores of our method by a large margin in the car detection task. Note that
since VirConv [12] is particularly trained only for detecting cars, the other
two classes (pedestrians and cyclists) are omitted.

Method | Modality | Car 3D AP (R40) | Car BEV AP (R40) | Time
---|---|---|---|---
|  | Easy | Mod. | Hard | Easy | Mod. | Hard |
PV-RCNN [27] | LiDAR | 90.25 | 81.43 | 76.82 | 94.98 | 90.65 | 86.14 | 80
Voxel-RCNN [17] | LiDAR | 90.90 | 81.62 | 77.06 | 94.85 | 88.83 | 86.13 | 40
CT3D [3] | LiDAR | 87.83 | 81.77 | 77.16 | 92.36 | 88.83 | 84.07 | 70
SE-SSD [28] | LiDAR | 91.49 | 82.54 | 77.15 | 95.68 | 91.84 | 86.72 | 30
BtcDet [29] | LiDAR | 90.64 | 82.86 | 78.09 | 92.81 | 89.34 | 84.55 | 90
CasA [30] | LiDAR | 91.58 | 83.06 | 80.08 | 95.19 | 91.54 | 86.82 | 86
Graph-Po [31] | LiDAR | 91.79 | 83.18 | 77.98 | 95.79 | 92.12 | 87.11 | 60
3ONet [32] | LiDAR | 92.03 | 85.47 | 78.64 | 95.87 | 90.07 | 85.09 | 60
Aug-VirConv (Ours) | LiDAR | 90.53 | 83.84 | 79.10 | 94.52 | 91.00 | 88.08 | 92

TABLE II: Quantitative Results on the KITTI test set. For the sake of
fairness, we show LIDAR-only methods. The best results are marked in bold, and
the second-best results are coloured red.

| Modality | Car 3D
---|---|---
| Easy | Med | Hard
PointPillars [15] | LIDAR | 88.28 | 79.32 | 76.72
SECOND [16] | LIDAR | 88.11 | 78.15 | 73.23
PV-RCNN [33] | LIDAR | 91.46 | 84.12 | 82.22
Voxel-RCNN [17] | LIDAR | 92.38 | 85.29 | 82.86
SE-SSD [28] | LIDAR | 93.19 | 86.12 | 83.31
BtcDet [29] | LIDAR | 93.15 | 86.28 | 83.86
3ONet [32] | LIDAR | 94.24 | 87.32 | 84.17
Aug-VirConv (Ours) | LIDAR | 92.62 | 87.76 | 85.34

TABLE III: Results on the KITTI validation set for the LiDAR-only method for
the average precision with 40 recall thresholds.

###  IV-B Quantitative Results

Table I shows the obtained quantitative results for three different models
(PointPillars [1], SECOND [16], and Voxel-RCNN [17]) trained with and without
our augmentation framework on the KITTI validation set. For each model, we
report the overall precision together with the individual mean average
precision (mAP) scores for the car, pedestrian, and cyclists classes across
all difficulty strata (easy, medium, and hard).

This table conveys that all these networks benefit from the dense pseudo-point
clouds generated by our framework. Each of these detectors showcases a
performance boost. For instance, in the case of overall medium difficulty, we
obtain $+1.2\%$ and $+1.3\%$ performance upgrade for PointPillars [1] and
SECOND [16], respectively. Note that since VirConv [12] is particularly
trained only for detecting cars, the other two classes (pedestrian and
cyclists) are omitted in Table I. When we particularly focus on the car medium
scores, we observe that although there are slight improvements for the
PointPillars [1] and SECOND [16], the performance upgrade on car detection
substantially increases up to $+2.9\%$ in the case of introducing Noise
Resistant Convolution (NRConv) in VirConv [12] to the Augmented Voxel-RCNN
(Ours). This clearly shows that models specifically designed to address the
noise of pseudo point clouds (e.g., VirConv [12]) benefit more from our
proposed technique than any other model.

We evaluate our method on the KITTI test set and compare it with the state-of-
the-art models on the leaderboard. Table II reports obtained results for the
Car 3D and BEV detection benchmarks. In this experiment, we augment VirConv
[12] with our pseudo-LiDAR data and obtain comparable results with the other
LiDAR-only detectors. We see that Aug-VirConv (Ours) sets a new state-of-the-
art result ($88.08\%$) in the Car BEV Hard difficulty, whereas it achieves the
second-best score ($83.84\%$) for the Car 3D Medium difficulty.

We also compare the performances of the LiDAR-only methods on the KITTI
validation set. The results in Table III show that our method is the best in
medium and hard difficulty levels. The performance drop of our approach in the
test set shows that the generalization power of our framework needs to be
improved. Our method is limited by the performances of TiTAN-Next [14] and
will benefit from improvements in the multi-modal domain translation research
area.

|  StVD [12] |  SGP (Ours) |  3D AP
---|---|---|---
LiDAR points (Voxel-RCNN [17]) |  No |  No |  84.9
Early Fusion |  Yes
No |  No
Yes |  85.6
85.7
Late Fusion |  Yes
No |  No
Yes |  86.8
87.8
TABLE IV: Ablation study on the KITTI validation set for discarding pseudo
points on the car-only detection. The goal is to compare StVD [12] with SGP.

Figure 5: Study of the input discard rate in our framework. We observe that
after applying SGP, dropping more points decreased detection scores.

###  IV-C Ablation study

#### Influence of the projection method

VirConv [12] generates a pseudo point cloud and then employs Stochastic Voxel
Discard (StVD) to drop about $80\%$ of the point cloud at every pass. Our
Semantic Guided Projection (SGP) method already selects only the points with
relevant semantic information. To validate the utility of our pseudo point
discard method, we repeat the 3D car detection experiment while switching
between StVD and SGP methods. The results are reported in Table IV. Using
Voxel-RCNN only on the LiDAR point cloud yields a performance of 84.9 mAP.
When we adopt an early fusion approach, that is we concatenate the pseudo
point cloud with the real LiDAR, StVD and SGP yield comparable result, with
ours being better by 0.1 mAP. When we adopt the late fusion approach of
VirConv-T, however, we can see that our method outperforms the stochastic
approach of VirConv. Late fusion with StVD brings a 1.9 mAP improvement to the
baseline, while our method brings a 2.9 mAP improvement for the medium
difficulty of the car detection problem.

#### Input discard influence

The pipeline of VirConv [12] sets a discard of $80\%$ of the voxels at the
input of the model. Our method already drops more than $80\%$ of the points
while applying our projection scheme SGP, thus, having an input discard is
less relevant in our setting. Nevertheless, we investigate whether having
additional discard could improve the performances by serving as a data
augmentation method. The results are plotted in Figure 5. We can see that the
best result is already achieved when there is no discard. This proves that the
projection scheme SGP removes redundant points and there is no need for an
additional discard.

#### Multimodal data

Our framework follows a LiDAR-only fashion while making use of a multi-modal
domain translator (i.e., TITAN-Next [14]) to create a pseudo point cloud from
synthetic depth and semantic segmentation images. This pipeline can also work
in a multimodal setting by replacing the synthetic images with real images
obtained from a real camera. For this purpose, we use the images from the
KITTI dataset and generate the depth and semantic segmentation maps using the
off-the-shelf models MIDAS [34] and SD-Net [35], respectively. Next, we employ
SGP to generate the pseudo point cloud. The results are reported in Table V.
VirConv [12] adds the real RGB information to the pseudo point cloud while
generating the point cloud. To have a fair comparison, we train the VirConv
model without RGB values and compare the results with this version. We observe
that for the multimodal setting, SGP and StVD have comparable performances,
with slightly better results for StVD [12]. One reason for that may be that
our method cumulates the error from both the depth prediction and the semantic
segmentation, while VirConv has depth prediction error, only. We believe that
advances in semantic segmentation will bridge the performance gap between
these two methods. Moreover, compared to VirConv [12] our method has the
benefit of being non-stochastic. StVD randomly drops $80\%$ of the pseudo
points at the input stage, which for a single frame may result in dropping the
most important points and a loss in performances. However, our method is
deterministic since we select the points based on semantic segmentation data.
Thus, for the same frame, we always get the same input point cloud. This means
that the performance is more stable and the reasons for failure are easier to
trace in real world applications.

| Mod | Car 3D | Car BEV
---|---|---|---
| Easy | Med | Hard | Easy | Med | Hard
VirConv-T∗ [12] | LC | 92.8 | 88.3 | 87.8 | 96.4 | 93.4 | 91.3
Augmented VirConv-T∗ (ours) | LC | 95.0 | 88.0 | 86.7 | 95.5 | 93.2 | 91.1

TABLE V: Results obtained when training in a multimodal setting. VirConv-T∗
[12] represents the model trained without the RGB values. For a fair
comparison, real camera segment and depth maps are added to our augmentation
framework.

##  V Discussion and Conclusion

In this work, we introduce a novel semantics-aware LiDAR-only pseudo-point
cloud generation for 3D Object Detection. We leverage the scene semantic and
depth information coming from the multi-modal domain translation module TITAN-
Next without requiring any additional sensor modality, such as cameras. The
final output of our framework is a denser LiDAR point cloud with semantic
segments and 3D bounding boxes for some specific objects.

To the best of our knowledge, this proposed framework is the first of its kind
that returns augmented and semantically segmented LiDAR scans without camera
sensors.

Reported experimental results showed that our framework is agnostic to object
detector architectures, however, works with higher performance in the case of
having detectors specifically designed for pseudo point cloud data, such as
VirConv [12].

The main limitation of the proposed framework is the computation time (see
Table II) needed to create the synthetic depth and semantic segmentation maps.
Since each module forward passes is quite fast, an efficient pipeline that
reduces dead time can mitigate this issue. For example, TITAN-Next could
process the next frame while the detection module is processing the current
point cloud.

## References

  * [1] A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for object detection from point clouds,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2019.
  * [2] J. Mao, M. Niu, H. Bai, X. Liang, H. Xu, and C. Xu, “Pyramid r-cnn: Towards better performance and adaptability for 3d object detection,” _ICCV_ , 2021\.
  * [3] H. Sheng, S. Cai, Y. Liu, B. Deng, J. Huang, X.-S. Hua, and M.-J. Zhao, “Improving 3d object detection with channel-wise transformer,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , October 2021, pp. 2743–2752.
  * [4] S. Shi, X. Wang, and H. Li, “Pointrcnn: 3d object proposal generation and detection from point cloud,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2019.
  * [5] S. Shi, Z. Wang, J. Shi, X. Wang, and H. Li, “From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 43, no. 8, pp. 2647–2664, 2021.
  * [6] Z. Yang, Y. Sun, S. Liu, and J. Jia, “3dssd: Point-based 3d single stage object detector,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2020.
  * [7] Z. Yang, Y. Sun, S. Liu, X. Shen, and J. Jia, “Std: Sparse-to-dense 3d object detector for point cloud,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , October 2019.
  * [8] D. Zhou, J. Fang, X. Song, C. Guan, J. Yin, Y. Dai, and R. Yang, “Iou loss for 2d/3d object detection,” in _2019 International Conference on 3D Vision (3DV)_ , 2019, pp. 85–94.
  * [9] Y. Wang, W.-L. Chao, D. Garg, B. Hariharan, M. Campbell, and K. Weinberger, “Pseudo-lidar from visual depth estimation: Bridging the gap in 3d object detection for autonomous driving,” in _CVPR_ , 2019.
  * [10] Y. You, Y. Wang, W.-L. Chao, D. Garg, G. Pleiss, B. Hariharan, M. Campbell, and K. Q. Weinberger, “Pseudo-lidar++: Accurate depth for 3d object detection in autonomous driving,” in _ICLR_ , 2020.
  * [11] X. Ma, S. Liu, Z. Xia, H. Zhang, X. Zeng, and W. Ouyang, “Rethinking pseudo-lidar representation,” in _Computer Vision – ECCV 2020_ , A. Vedaldi, H. Bischof, T. Brox, and J.-M. Frahm, Eds. Cham: Springer International Publishing, 2020, pp. 311–327.
  * [12] H. Wu, C. Wen, S. Shi, and C. Wang, “Virtual sparse convolution for multimodal 3d object detection,” in _CVPR_ , 2023.
  * [13] T. Cortinhal, G. Tzelepis, and E. Erdal Aksoy, “Salsanext: Fast, uncertainty-aware semantic segmentation of lidar point clouds,” in _Advances in Visual Computing_ , G. Bebis, Z. Yin, E. Kim, J. Bender, K. Subr, B. C. Kwon, J. Zhao, D. Kalkofen, and G. Baciu, Eds. Cham: Springer International Publishing, 2020, pp. 207–222.
  * [14] T. Cortinhal and E. Erdal Aksoy, “Depth- and Semantics-aware Multi-modal Domain Translation: Generating 3D Panoramic Color Images from LiDAR Point Clouds,” _arXiv e-prints_ , p. arXiv:2302.07661, Feb. 2023.
  * [15] A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for object detection from point clouds,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2019, pp. 12 697–12 705.
  * [16] Y. Yan, Y. Mao, and B. Li, “Second: Sparsely embedded convolutional detection,” _Sensors_ , vol. 18, no. 10, p. 3337, 2018.
  * [17] J. Deng, S. Shi, P. Li, W. Zhou, Y. Zhang, and H. Li, “Voxel r-cnn: Towards high performance voxel-based 3d object detection,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 35, no. 2, 2021, pp. 1201–1209.
  * [18] S. Vora, A. H. Lang, B. Helou, and O. Beijbom, “Pointpainting: Sequential fusion for 3d object detection,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 4604–4612.
  * [19] S. Xu, D. Zhou, J. Fang, J. Yin, Z. Bin, and L. Zhang, “Fusionpainting: Multimodal fusion with adaptive attention for 3d object detection,” in _2021 IEEE International Intelligent Transportation Systems Conference (ITSC)_. IEEE, 2021, pp. 3047–3054.
  * [20] C. Wang, C. Ma, M. Zhu, and X. Yang, “Pointaugmenting: Cross-modal augmentation for 3d object detection,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 11 794–11 803.
  * [21] T. Huang, Z. Liu, X. Chen, and X. Bai, “Epnet: Enhancing point features with image semantics for 3d object detection,” in _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XV 16_. Springer, 2020, pp. 35–52.
  * [22] T. Yin, X. Zhou, and P. Krähenbühl, “Multimodal virtual point 3d detection,” _Advances in Neural Information Processing Systems_ , vol. 34, pp. 16 494–16 507, 2021.
  * [23] X. Wu, L. Peng, H. Yang, L. Xie, C. Huang, C. Deng, H. Liu, and D. Cai, “Sparse fuse dense: Towards high quality 3d detection with depth completion,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 5418–5427.
  * [24] T. Cortinhal, F. Kurnaz, and E. E. Aksoy, “Semantics-aware multi-modal domain translation: From lidar point clouds to panoramic color images,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops_ , October 2021, pp. 3032–3048.
  * [25] P. Yin, J. Qian, Y. Cao, D. Held, and H. Choset, “Fusionmapping: Learning depth prediction with monocular images and 2d laser scans,” _arXiv preprint arXiv:1912.00096_ , 2019.
  * [26] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the kitti vision benchmark suite,” in _2012 IEEE Conference on Computer Vision and Pattern Recognition_ , 2012, pp. 3354–3361.
  * [27] S. Shi, C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li, “Pv-rcnn: Point-voxel feature set abstraction for 3d object detection,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2020.
  * [28] W. Zheng, W. Tang, L. Jiang, and C.-W. Fu, “Se-ssd: Self-ensembling single-stage object detector from point cloud,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2021, pp. 14 494–14 503.
  * [29] Q. Xu, Y. Zhong, and U. Neumann, “Behind the curtain: Learning occluded shapes for 3d object detection,” _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 36, no. 3, pp. 2893–2901, Jun. 2022. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/20194
  * [30] H. Wu, J. Deng, C. Wen, X. Li, C. Wang, and J. Li, “Casa: A cascade attention network for 3-d object detection from lidar point clouds,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 60, pp. 1–11, 2022.
  * [31] H. Yang, Z. Liu, X. Wu, W. Wang, W. Qian, X. He, and D. Cai, “Graph r-cnn: Towards accurate 3d object detection with semantic-decorated local graph,” in _Computer Vision – ECCV 2022_ , S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, and T. Hassner, Eds. Cham: Springer Nature Switzerland, 2022, pp. 662–679.
  * [32] H. A. Hoang and M. Yoo, “3onet: 3d detector for occluded object under obstructed conditions,” _IEEE Sensors Journal_ , 2023.
  * [33] S. Shi, C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li, “Pv-rcnn: Point-voxel feature set abstraction for 3d object detection,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 10 529–10 538.
  * [34] R. Ranftl, K. Lasinger, D. Hafner, K. Schindler, and V. Koltun, “Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 44, no. 3, 2022.
  * [35] M. Ochs, A. Kretz, and R. Mester, “SDNet: Semantically Guided Depth Estimation Network,” _arXiv e-prints_ , p. arXiv:1907.10659, Jul. 2019.


: Improving Reliability with Bit Voting in Synchronous Transmission-based IoT Networks

Industrial Internet of Things (IIoT) networks must meet strict reliability, latency, and low energy consumption requirements. However, traditional low-power wireless protocols are ineffective in finding a sweet spot for balancing these performance metrics. Recently, network flooding protocols based on Synchronous Transmissions (STX) have been proposed for better performance in reliability-critical IIoT, where simultaneous transmissions are possible without packet collisions. STX-based protocols can offer a competitive edge over routing-based protocols, particularly in dependability. However, they notably suffer from the beating effect, a physical layer phenomenon that results in sinusoidal interference across a packet and, consequently, packet loss.
Thus, we introduce , an error correction scheme that can handle errors caused by beating effects. Importantly, we utilize transmission redundancy already inherent within STX protocols so do not incur additional on-air overhead. Through simulation, we demonstrate can provide a 40% increase in reliability. We subsequently implement on  devices and perform extensive experiments. The results confirm that improves reliability by 25-28% for BLE 5 PHYs and 8% for IEEE 802.15.4; thus, it can complement existing error correction schemes.

§ INTRODUCTION

Traditionally, wireless network communication protocols, such as Carrier Sense Multiple Access (CSMA) or Time Division Multiple Access (TDMA), have been designed assuming that packet collisions are inherently destructive and should be avoided as much as possible. Methods such as carrier sensing, handshaking, and transmission scheduling have been utilized to avoid such collisions. However, after Ferrari et al. [1] showed that by tightly synchronizing transmissions, the packet collisions were not inherently damaging in certain physical-layer standards, there has been a considerable body of research into network flooding protocols based on Synchronous Transmissions (STX) [2] (also referred to in literature as Concurrent Transmissions). Such protocols can significantly reduce latency and power consumption as various complexities and overhead of MAC and routing can be eliminated.

In theory, given perfect temporal and frequency synchronization between transmitters, signals overlap constructively and increase the probability of a packet's successful reception at its destination. However, in real-world scenarios, the carrier frequency of the radios of transmitting devices are typically offset due to imperfections in their crystal oscillators [3]. This relative Carrier Frequency Offset (CFO) between devices causes beating, a sinusoidal waveform of both constructive and destructive interference. The destructive interference period in beating weakens the signal and can create errors in transmissions, negatively affecting reliability.

STX-based networking protocols typically use re-transmissions [1, 4, 5], channel hopping [5, 6], network coding [6, 7, 8], or a combination of these, to increase reliability. These methods exploit the constructive interference or capture effect, which is susceptible to packet loss when the number of transmitters increases. There are methods such as Forward Error-Correction (FEC) or interleaving at the physical layer, but their impact on reliability in STX protocols is yet to be explored [2]. Baddeley et al. [3] point out that errors occurring due to beating can be discreet or burst depending on network conditions. However, FEC or Direct Sequence Spread Spectrum (DSSS) cannot handle burst errors where contiguous bits are corrupted beyond a certain point, as indicated in [9]. Consequently, coded BLE 5 physical layers and are vulnerable to decoding errors under certain beating conditions.

Harnessing repetitions in STX protocols Though the authors of [3] show that protocols would benefit from an error-correction mechanism capable of handling both burst and discrete errors, no such solution has yet been proposed. This paper addresses this gap and proposes , a novel packet-level error correction scheme that leverages the multiple repetitions inherent in commonly used STX protocols to combat beating-based packet loss and improve reliability. In the event of an error, attempts to create a correct packet out of previously received error packets. Unlike traditional error correction schemes, is specifically targeted at STX-based protocols, where errors due to beating are likely to be different for each (re)transmission of a particular packet.

Our contributions By extending an open-source STX simulation [3], we firstly explore how bit-voting can improve reliability across BLE 5 and IEEE 802.15.4 physical layers. We subsequently implement on the  platform within the Open Synchronous-Flooding (OSF) framework [10] and conduct an extensive evaluation in a local and the D-cube public testbed [11]. We compare STX reliability with and without across the nRF's BLE 5 and IEEE 802.15.4 physical layers. We show that using under strong wide and narrow beating conditions increases reliability by 25-28% and 8% on the BLE 5 and IEEE 802.15.4 PHYs, respectively. The complete implementation package of is available as open source[<https://github.com/PINetDalhousie/STX_Vote>] for reproducibility and extension. To the best of our knowledge, is the first realistic approach to enhancing STX reliability without incurring overhead.

Paper outline We provide detailed background information on the beating effect in Sect. <ref>, and explore the wider issue of reliability in STX protocols in Sect. <ref>. In Sect. <ref>, we cover design and implementation details on the bit-voting technique behind , while simulation and experimental results are provided in Sect. <ref>. Finally, we conclude the paper in Sect. <ref>.

§ BACKGROUND AND MOTIVATION

Beating effect when frequency offset transmissions overlap.

Types of beating [3].

The beating effect is a phenomenon that occurs when signals from non-coherent transmitters with slightly different carrier frequencies overlap in the air [9]. This results in a sinusoidal waveform of alternating periods of constructive and destructive interference. Although devices transmit at the same carrier frequency, imperfections in their crystal oscillators lead carrier frequencies to deviate from the intended level and give rise to a realtive carrier frequency offset (CFO) between the transmitted waveforms (Fig. <ref>). Bits falling in destructive valleys tend to get corrupted, causing errors in the packet. Baddeley et al. [3] showed that the relative CFO between devices, the time for which the packet is in the wireless transmitting medium, and the power difference between synchronous transmissions give rise to four basic types of beating patterns as shown in Fig. <ref>.

* Wide and Strong: Occurs when the CFO between the devices and the power difference between signals is small. Depending on the packet air time, destructive valleys in the waveform can last longer during packet transmission, possibly resulting in burst errors.

* Wide and Weak:  Occurs when the CFO between the devices is small, and the power difference between signals is significant but less than the capture threshold. The destructive valleys in the waveform are not as intense as wide and strong, thus generating burst errors. Power differences larger than the capture threshold may lead to the capture effect.

* Narrow and Strong: Occurs when the CFO between devices is large, and the power difference between signals is small. The narrow destructive valleys in the beating waveform are more likely to cause discrete errors across the packet.

* Narrow and Weak: Occurs when the CFO between devices is small, and the power difference between signals is significant. The destructive valleys in the beating waveform are less intense compared to narrow and strong beating and will result in discrete errors.

Importantly, different physical layers behave differently depending on how the beating manifests. Extensive experimentation in [3] concludes that the uncoded BLE 5 PHYs can perform better in wide beating scenarios due to the fact that fast transmission rates and multiple re-transmissions increase the probability of one of the transmissions occurring during a constructive beating peak. However, they suffer during narrow beating as the constructive beating peaks are narrow, and they do not have any error correction mechanism that can help mitigate narrow beating.

On the other hand, coded BLE 5 PHYs perform better during narrow beating because the forward error-correction (FCE) mechanism used in BLE 5 coded PHYs (125K, 500K) and the Direct Sequence Spread Spectrum (DSSS) used in IEEE 802.15.4 can correct burst errors up to a certain extent [3]. For example, in the case of IEEE 802.15.4, if the width of the error is narrower than the DSSS symbol length, then correction is possible [9]. To handle burst errors caused by wide beating using these correction mechanisms, one might suggest increasing the redundancy in FEC or the DSSS symbol length. However, the number of bits affected by beating cannot be quantified as it occurs randomly. Also, the error handling steps can increase the decoding time at the receiver and the coding overhead at the sender.

Finally, the redundancy added by convolution codes in the coded BLE 5 PHYs might get corrupted in the packets themselves, leading to unsuccessful error correction attempts.

Capture effect, where a signal of significantly higher power than other signals can be correctly demodulated, is another way to mitigate beating, as suggested in [3]. However, for the capture effect to work, the power of the dominant transmitter must be higher than the sum of other simultaneous transmissions for a packet reception to be possible at the receiver [12], which can negatively affect the scalability and simplicity advantages of using STX-based protocols.

Overall, the STX-based protocols are complex and cannot scale to mitigate the beating effect. On the other hand, FEC and DSSS can add communication overhead that can lead to synchronization drift, thus degrading the reliability. The proposed fills the above gaps by introducing an error correction mechanism that does not add overhead while successfully mitigating the beating effect both in discrete and burst errors. Also, the can complement the existing solutions and improve their reliability.

§ RELATED WORK

Channel Hopping There are several works that use channel hopping as a method to increase network resilience. Lim et al. [5] employed channel hopping for every communication slot to avoid any possible interference that a previous transmission slot might have encountered. The authors of  [13] focused on improving resiliency of the CRYSTAL protocol using channel hopping and noise detection. The Transmission and Acknowledgment (TA) pairs of the protocol transmit on different channels to overcome the limitation of the longer awake time of the noise detection method. RedFixHop [4] is another protocol that uses channel hopping. Every node in all of these protocols either performs channel assessment for adaptive channel-hopping or determines a fixed channel-hopping sequence among nodes.
Beating can still occur under channel hopping because of the Relative Frequency Offset (RFO) between the transmitting devices, leading to packet corruption. Thus, these solutions need an error correction mechanism while considering the beating effect.

Network Coding Network coding employs mathematical algorithms to combine multiple message units from different nodes into a single transmission towards the receiver.In STX-based networks,it leverages the capture effect to enhance reception probability and reliability by giving multiple possibilities for the message encoded in different simultaneous transmissions to reach the destination. Splash [14] uses XOR coding, Ripple [15] uses Reed-Solomon coding focused on one-to-many communication, while Mixer [7] use Random Linear Network coding (RLNC) targeting many-to-many communication. However, these solutions do not consider the beating effect and can suffer from the same limitations as in channel hopping. For more works on STX, the reader is directed to [2].

Re-transmissions Ferrari et al. [1] (use Rx-Tx... pattern) and RobustFlooding [5] (use Rx-Tx-Tx... pattern) use aggressive re-transmission of packets to exploit the possibility that at least one of the transmissions will reach the destination. All nodes using these communication protocols listen to all packets being transmitted in the network and follow their transmission patterns, which increases the probability of successful reception using either constructive interference or the capture effect. Because of the beating effect if one of the nodes does not receive the correct transmission in a slot, then it may suffer from synchronization drift and impact the entire flooding. Thus, we need an error correction scheme.

Packet Combining This error correction mechanism relies on incorrect packet receptions: when there are enough received error packets, packet merging is performed to find the correct packet. The authors of [16] perform XOR operations on the incorrect packets to find the error positions and then employ a brute-force method to predict the bit value and use a CRC to check which bit values make up the correct packet. RSSI (Received Signal Strength Indicator) based packet combining [17] uses the symbol with the highest power among the received incorrect packets to form a new packet and uses error checks to verify that the packet created is correct and then transmits it. Finally, the authors of [18] make a database of corrected packets based on the error positions and then use it to estimate the possible correct packet.
The packet combining with FEC is useful to mitigate the beating effects due to its periodic nature [16]. However, packet combining is not deployed in synchronous flooding to tackle errors generated due to beatings. This work fills that gap and proposes .

§ ERROR CORRECTION THROUGH BIT-VOTING

As introduced in Sect. <ref>, STX-based communications can suffer from transmission errors (beating errors) due to the discrepancy between oscillators of a communicating pair. However, existing BLE 5 and IEEE 802.15.4 physical layers cannot handle such errors at receivers [3].

workflow at a receiver.

The phase of the beats occurring can be different in different receptions of the same packet; thus, a specific contiguous block of bits may corrupt in one reception but receive correctly in another. Based on this observation, votes on the values of the received bits. If the majority votes happen on the correct portion of a packet, then it can successfully correct the packet. Specifically, works at the receiver, as shown in Fig. <ref>, by maintaining an array of signed integers that represent votes on the value of each bit in received error packets, where these error packets contain the same payload across multiple communication rounds. This voting array is used to construct the intended packet and validate it against a special CRC appended by the sender.

§.§ Voting Process

A reception is considered incorrect if the CRC check fails, which the receiver's hardware performs. The receiver maintains votes on the value of each bit among multiple incorrect receptions of the same packet. The voting array is reset every time a new unique packet is received (including a correct one).

The voting operation of at a receiver.

As shown in Fig. <ref>, if the bit value at index zero in a received error packet is 1, the integer value at index zero in the voting array is incremented. The integer value is decremented if the bit value is 0. Thus, if a node receives several error packets with the same payload in a row, the votes will accumulate and determine each bit’s likeliness of being 1 or 0 based on the sign of the integer, which is illustrated in Section <ref>.

§.§ Error Correction

When a ROF (Robust Flooding) [5] communication round ends without correct reception, a receiver performs the error correction. Specifically, the sender calculates a 2-byte CRC on each packet and appends it at the end. If the receiver receives two or more incorrect packets, it uses the corresponding voting array to construct the correct packet and its CRC. In particular, if the voted value at an index is positive, the corresponding bit value of the packet is set to 1; otherwise, it is 0.

As shown in Fig. <ref>, a CRC is calculated from this constructed packet (ignoring the last 2 bytes of the packet, which is the constructed CRC). If this calculated CRC is identical to the constructed CRC, the packet reception is considered successful based on successful packet recovery. Otherwise, the packet is not recovered, and the receiver waits for the following reception, which can lead to further voting on the packet or receiving a correct reception. In the case of correct reception, all error correction activities are suspended, i.e., the receiver does not perform error correction. Otherwise, the receiver initiates voting and follows the same correction procedure at the end of the round in the case of errors. Voting arrays and relevant counters are reset based on the packet ID for every new packet.

§ EVALUATION

We first assess the performance of STX over IEEE802.15.4 and BLE physical layers and measure the corresponding Packet Error Rate (PER) in a simulator to highlight the effect of different types of beating. Next, we show how can reduce the beating errors in STX-based communications. Finally, we conduct a series of experiments measuring the Packet Delivery Ratio (PDR) on real platforms to show the efficiency and deployment feasibility of in complementing existing error correction mechanisms.

§.§ Simulation

Wide (RFO = 500Hz - 2000Hz)

Narrow (RFO = 10KHz - 40KHz)

Wide (RFO = 500Hz - 2000Hz)

Narrow (RFO = 10KHz - 40KHz)

The average PER for strong and weak beatings.

We adapt the simulator[<https://github.com/ADEscobar/ct-simulator>] [3] to evaluate the packet error rate of with two synchronous transmitters over the four BLE and the IEEE 802.15.4 PHYs across different beating frequencies. We assume an additive white Gaussian noise and no synchronization errors while simulating different CFOs between two transmitters. Different CFOs result in wide and narrow beating frequencies, while the transmission power decides weak and strong beating. We generate a random packet stream with eight samples per symbol following [12], where two synchronous transmissions suffer from amplitude distortion if their identical transmissions overlap in the air. Finally, we implement the proposed bit voting, error detection, and error correction mechanism of with a validation process by comparing the transmitted packets with the reconstructed ones.

Discussion on results.
Fig. <ref> shows the average PER over different beating frequencies showcasing wide and narrow beating with a 25dBm Signal-to-Noise ratio. These beatings can be strong or weak. We will first present the results for the strong one, which is shown in Fig. <ref> and Fig. <ref>. Uncoded PHYs (BLE 1M and BLE 2M) offer an increase in PER as the beating frequency increases. In the case of narrow beating, PER increases as seen in Fig. <ref> as packet transmission spans one or more destructive valleys, resulting in frequent errors. Uncoded PHYs cannot handle this error due to the lack of an error correction mechanism despite having high transmission rates. However, we observe a significant decrease in PER from 67.88% to 21.39% for BLE 1M and from 63.57% to 18.78% for BLE 2M  when is enabled, confirming its effectiveness over strong narrow beating. In the case of wide beating, i.e., when an RFO between synchronous transmitters is not high, the PER is not as high as compared to narrow beating as seen in Fig. <ref> because their fast transmission rates increase the probability of one of the transmissions occurring during a constructive beating peak, however they require for the transmissions lying in the beating valley while experiencing strong wide beating which is why we can see a 15% decrease in PER for BLE 1M and 18% decrease in PER for BLE 2M.

Coded PHYs (BLE 125K, BLE00K and IEEE 802.15.4) are equipped with error correction mechanisms. For instance, FEC for BLE 125K and BLE00K, while IEEE802.15.4 uses DSSS. The role of these mechanisms under beating is evident from Fig. <ref>. These mechanisms can handle narrow beating, giving similar performance to , which suggests that the coding used by BLE 125K, BLE00K and IEEE802.15.4 can effectively correct the shorter error bursts introduced in the packet due to narrow beating. However, the trend is reversed in wide beatings as seen in Fig. <ref>, which may stem from their distortions that spread across multiple symbols and require a robust error correction mechanism like . It reduces the errors by 15% for BLE 125K, by 23% for IEEE and by 63% for BLE00K.

Also, we conclude that weak beating does not significantly impact PER, as seen in Fig. <ref> and Fig. <ref>. In that case, signal distortion due to the higher transmission power of a signal compared to other concurrent ones is not strong enough to corrupt a bit. Thus, uncoded PHYs do not require any error correction mechanism in the presence of weak beating. Using their existing error correction mechanisms, coded PHYs can handle such signal distortion due to weak beating.

§.§ Lab Experiments

Setup.  We use 3  devices that support all BLE PHYs and IEEE802.15.4 and place them on a desk at a random distance to create a single-hop communication among the nodes, assuming no synchronization errors. These nodes perform specific tasks according to the assigned roles, e.g., a source generates packets of length 255 bytes for BLE and 125 bytes for , a forwarder (also acts as another synchronous transmitter), and a destination node. We use a transmission power of -40dBm. The chosen parameters allow us to observe beating manifestation and identify the types of beating occurring. The source generates 100 packets for each PHY to measure the average Packet Delivery Ratio (PDR).

Beating patterns in the physical layers over local experiments.

Beating encountered in D-cube experiments.

Discussion on results. Fig. <ref> presents the average PDR with and without using . We observe that improves the overall PDR for all PHYs. For instance, in uncoded PHYs, BLE 1M and BLE 2M obtained a performance gain of approximately 40%  and 55%, respectively. BLE 2M suffers from the beating effect the most due to its high transmission rate, the lack of an error correction mechanism, and the presence of narrow beating as seen in Figs. <ref> to <ref>. successfully detects and corrects its corrupted packets at receivers. Specifically, due to its high transmission rates and narrow beating, BLE 2M suffers from discrete errors that corrupt a small portion of a packet. Also, due to the periodic nature of the beats, this corruption manifests in different parts of a corrupted packet for each transmission. Thus, voting on multiple such packets results in successful corrections in BLE 2M. Similarly, BLE 1M gains performance by adopting , where the degree of improvement is proportional to its transmission rates.

On the other hand, coded PHYs BLE 125K, BLE 500K, and IEEE802.15.4 offer approximately 50%, 20%, and 20%, respectively, improvement with enabled. These PHYs suffer from burst errors generated by wide beating (Figs. <ref> to <ref>) with BLE 125K is the worst one. This behaviour is due to its lowest transmission rates and ineffective error corrections using FEC. However, incorporating BLE 125K can successfully correct the receptions that its underlying FEC missed. Similarly, complements the FEC of BLE 500K and DSSS of IEEE802.15.4 with a performance improvement. All the PHYs can increase their PDR, but uncoded PHYs can significantly increase their PDR compared to coded PHYs using .

The average PDR for local and D-cube experiments.

§.§ Experiment on D-cube Testbed

Setup. Over-the-air experiments were performed on the D-cube public testbed [11], which is equipped with  devices that support all BLE PHYs and IEEE 802.15.4. Three single-hop nodes were used in the experiments: a source node that generated and sent packets with randomized payloads of 246 Bytes when was enabled (so that the CRC generated by and packet header could be added, resulting in the packet size of 255 bytes), a destination node, and a forwarder node that acted as another synchronous transmitter.

Since multiple node combinations are possible in D-cube, we conducted an exploratory experimentation drive to find the node combinations that showed beating that was destructive enough for the physical layers to encounter errors from which their respective error correction schemes could not recover. Thus, could correct errors and increase the reliability of the network.

For this experimentation drive, we selected ten random three-node combinations with roles as used in the previous experimentation and ran tests using the transmission powers -16dBm, -20dBm, and -40dBm. In each case, the source transmitted 200 packets of length 255 bytes for the BLE physical layers and of size 128 bytes for IEEE802.15.4. We present the results for -20dBm due to the space constraints (the trend is similar in other cases).

Discussion on results.
Fig. <ref> shows a significant improvement of almost 20% in PDR for BLE 1M, but no improvement for BLE 2M. The beating pattern of 1M is shown in Figs. <ref>, which is narrow; however, the beating happens to be wide as there is no distinct envelope seen in the case of BLE 2M as shown in Figs. <ref>. Thus, with its fast transmission rate, BLE 2M can fit a transmission in constructive beating crest [3], which is not the case in BLE 1M.

Similarly, even though coded BLE PHYs are equipped with the FEC mechanism, the contiguous bursts of error in the transmission caused by wide beating as seen in Figs. <ref> and <ref>  and low power was high enough to cause their built-in error-correction schemes to fail for some of the transmissions. With , BLE PHYs 125K and 500K offer around 28% and 25% increase in PDR, respectively, thus proving that provides robustness against wide beating. Even IEEE 802.15.4 experiences wide beating, as seen in Figs. <ref>. Though its DSSS error correction scheme can handle most errors, the beating width exceeded the DSSS coding depth, which can cause its error correction scheme to fail. helps when DSSS fails and increases the PDR by $\sim$8%.

§ CONCLUSION

In this paper we have presented , a novel error correction scheme specifically designed to address beating-induced errors in synchronous transmissions, an issue that has long been an acute challenge within the community. Through simulation and experimentation we have demonstrated that increases reliability by capitalizing on the inherent repetitions present in STX-based protocols and using majority voting to determine the intended value of each bit and therefore reconstruct the correct packet at the receiver. Moreover, we have shown that is applicable across different beating scenarios where devices experience `narrow' or `wide' beating dependent on the underlying physical layer [3].

Specifically, we show that uncoded BLE PHYs exhibit improved reliability by up to $\sim$20% on the D-Cube testbed environment, while we additionally find that improves reliability on coded PHYs by up to $\sim$28%.

These promising results indicate that the bit-voting technique in can successfully be used to mitigate beating-induced errors in STX-based protocols. As this work has only considered a single hop, and many STX protocols are flooding based, future work should examine the effectiveness of this technique in multi-hop scenarios as well as its scalability when considering a larger number of transmitters.

[1]
F. Ferrari et al., “Efficient network flooding and time
synchronization with glossy,” in Proc. of the IPSN Conf., 2011.

[2]
M. Zimmerling et al., “Synchronous Transmissions in Low-Power
Wireless: A Survey of Communication Protocols and Network Services,”
ACM Computing Surveys (CSUR), vol. 53, no. 6, 2020.

[3]
M. Baddeley et al., “Understanding Concurrent Transmissions,”
ACM TOSN, 2023.

[4]
Escobar et al., “RedFixHop with Channel Hopping: Reliable
Ultra-Low-Latency Network Flooding,” in Proc. of the DCSI Conf.,

[5]
Lim et al., “Competition: Robust Flooding using Back-to-Back
Synchronous Transmissions with Channel-Hopping.” in Proc. of the
$14^{th}$ EWSN Conf., 2017.

[6]
X. Ma et al., “Harmony: Saving Concurrent Transmissions from Harsh RF
Interference,” in Proc. of the 2020 INFOCOM Conf., 2020.

[7]
C. Herrmann et al., “Mixer: Efficient Many-to-All Broadcast in Dynamic
Wireless Mesh Networks,” in Proc. of the EWSN Conf., 2018.

[8]
M. Mohammad et al., “Codecast: Supporting Data Driven In-Network
Processing for Low-Power Wireless Sensor Networks,” in Proc. of the
$17^{th}$ IPSN Conf., 2018.

[9]
C.-H. Liao et al., “Revisiting the So-Called Constructive Interference
in Concurrent Transmission,” in Proc of the LCN Conf., 2016.

[10]
M. Baddeley et al., “OSF: An Open-Source Framework for Synchronous
Flooding over Multiple Physical Layers,” in Proc. of the $19^{th}$
EWSN Conf., Oct. 2022.

[11]
M. Schuß et al., “Moving beyond competitions: Extending D-Cube to
seamlessly benchmark low-power wireless systems,” in Proc. of the
$1^{st}$ CPSBench Workshop, 2018.

[12]
A. D. Escobar Molero, “Using Concurrent Transmissions to Improve the
Reliability and Latency of Low-Power Wireless Mesh Networks,” Ph.D.
dissertation, RWTH Aachen University, 2020.

[13]
T. Istomin et al., “Interference-Resilient Ultra-Low Power Aperiodic
Data Collection,” in Proc. of the $17^{th}$ IPSN Conf., 2018.

[14]
M. Doddavenkatappa et al., “Splash: Fast Data Dissemination with
Constructive Interference in Wireless Sensor Networks,” in Proc of
the $10^{th}$ NSDI Conf., 2013.

[15]
D. Yuan et al., “Ripple: High-Throughput, Reliable and
Energy-Efficient Network Flooding in Wireless Sensor Networks,” in
Prof. of the WoWMoM Conf., 2015.

[16]
Dubois-Ferriere et al., “Packet Combining in Sensor Networks,” in
Proc. of the $3^{rd}$ SenSys Conf., 2005.

[17]
T. Sakdejayont et al., “Beating the Beat: RSSI-Based Packet Combining
in Concurrent Transmission Sensor Networks,” in Proc. of the
$14^{th}$ SenSys Conf., 2016.

[18]
S. Kothapalli et al., “Extended Packet Combining Schemes for Error
Control in Wireless Networks,” in Proc. of the $2^{nd}$ AISP Conf.,