# SLP-Net:An efficient lightweight network for segmentation of skin lesions

Bo Yang  Hong Peng  Chenggang Guo  Xiaohui Luo<EMAIL_ADDRESS>Jun Wang
Xianzhong Long

###### Abstract

Prompt treatment for melanoma is crucial. To assist physicians in identifying
lesion areas precisely in a quick manner, we propose a novel skin lesion
segmentation technique namely SLP-Net, an ultra-lightweight segmentation
network based on the spiking neural P(SNP) systems type mechanism. Most
existing convolutional neural networks achieve high segmentation accuracy
while neglecting the high hardware cost. SLP-Net, on the contrary, has a very
small number of parameters and a high computation speed. We design a
lightweight multi-scale feature extractor without the usual encoder-decoder
structure. Rather than a decoder, a feature adaptation module is designed to
replace it and implement multi-scale information decoding. Experiments at the
ISIC2018 challenge demonstrate that the proposed model has the highest Acc and
DSC among the state-of-the-art methods, while experiments on the PH2 dataset
also demonstrate a favorable generalization ability. Finally, we compare the
computational complexity as well as the computational speed of the models in
experiments, where SLP-Net has the highest overall superiority.

###### keywords:

Convolutional neural network , Spiking neural P systems , Skin lesion
segmentation , Multi-scale feature , Lightweight network

††journal: Elseviermytitlenotemytitlenotefootnotetext: We have submitted this
paper to Elsevier in March 2023 and it may be published later on.

[inst1]organization=School of Computer and Software
Engineering,addressline=Xihua University, city=Chengdu, postcode=610039,
state=Sichuan, country=China [inst2]organization=School of Electrical
Engineering and Electronic Information,addressline=Xihua University,
city=Chengdu, postcode=610039, state=Sichuan, country=China
[inst3]organization=School of Computer Science,addressline=Nanjing University
of Posts and Telecommunications, city=Nanjing, postcode=210023,
state=Jiangshu, country=China

##  1 Introduction

Melanoma is a common skin disease which, if left to develop, can develop into
skin cancer and its mortality rate is quite high. Dermoscopy can be used for
early lesion diagnosis and is essential for diagnosing melanoma, moreover
reducing the mortality rate[1]. However, diagnosing dermoscopic images can be
time-consuming and tedious for physicians[2]. In contrast, the segmentation of
skin lesions with a computer is considerably more efficient. Automatic
segmentation of skin lesions is considered a crucial step in the computer-
aided diagnosis(CAD) of melanoma. Automatic segmentation of skin lesions in
CAD is highly desirable and will greatly facilitate the dermatologist and
improve the accuracy of the analysis. However, most of these systems are
highly dependent on the segmentation of skin lesions to identify areas of
lesions with distinct boundaries from dermoscopic images. Therefore, it is
necessary to obtain accurate lesion segmentation results for dermoscopic image
analysis and dermatological diagnosis.

Automated segmentation of skin lesions has been a topic of research for many
years, and early techniques often relied on the assumption that the border of
the lesion was the most essential feature for distinguishing it from the
surrounding skin background. However, accurately segmenting skin lesions from
healthy skin at a pixel level remains a challenging task for researchers. To
address this challenge, recent research has turned to deep convolutional
neural networks (DCNNs) for automated segmentation of skin lesions. DCNNs have
been extensively used in medical image analysis and have achieved considerable
success in various applications, including segmentation, classification, and
detection of skin lesions. DCNNs have several advantages over traditional
image processing techniques. They can learn features automatically from raw
data and have the ability to capture complex spatial dependencies, which is
particularly useful in the context of skin lesion segmentation. Moreover,
DCNNs have been shown to be effective in handling noisy and irregular data,
making them well-suited for the noisy and heterogeneous images often
encountered in dermatology.

Despite the promising results of DCNN in skin lesion segmentation, there are
still some challenges that need to be addressed. Firstly, deep learning
algorithms tend to perform poorly when the number of samples is too small,
which is a common problem in medical image analysis where datasets are often
very limited. This makes it difficult to train accurate models for skin lesion
segmentation. Another challenge is related to the ability of convolutional
neural networks to capture information at multiple scales. Skin lesion
boundaries can be highly irregular, and some images may also contain
interfering factors such as hair and scale ruler as shown in Fig. 1. These
factors can make it difficult for DCNNs to accurately segment the skin lesion
from the surrounding tissue. DCNNs must be able to capture subtle differences
in texture and color that distinguish skin lesions from normal skin tissue
while avoiding interference from other factors. This requires the development
of new segmentation techniques that can effectively extract the relevant
features from skin lesion images. Besides, to the best of our knowledge, few
literatures have been made to build convolutional neural networks with
different neural network mechanisms. Finally, most existing segmentation
convolutional neural networks such as FCN[3], U-Net[4], Deeplabv3+[5] still
have a large number of parameters, which leads to longer training times and
larger hardware overheads. This limits their practical application,
particularly in settings where rapid diagnosis is crucial. There is a need to
develop more efficient and lightweight segmentation models that can provide
accurate results while minimizing computational costs.

In this paper, we propose a novel convolutional neural network for skin lesion
image segmentation, which is based on the spiking neural P(SNP) systems
convolution mechanism. Specifically, SLP-Net is comprised of three main
components, which include the SNP-type lightweight pyramid (SLP), SNP-type
feature self-adaption skip connection (SFA), and SNP-type downsampling (SDS).
We connect SLP in series with SDS, as shown in Fig. 3, to form the encoder of
the network. And SFA is utilized as a decoder for each SLP separately at
different resolutions. SLP is a multi-scale feature extractor, which is mainly
composed of multiple sets of asymmetric convolutions with different dilated
rates. As it is lightweight, it is placed in different resolution sizes for
feature processing. SLP-Net is an asymmetric structure in which we discard the
decoder in the network. Using a connection similar to that of ResNet[6], the
proposed SFA can then effectively solve the problem about the mismatch between
the encoder and the final output feature map size at all levels. SDS enables
downsampling of the feature maps, thereby reducing the spatial dimensions of
the data and improving computational efficiency. We conduct a comprehensive
evaluation of our proposed model using two publicly available datasets
ISIC2018[7] and PH2[8]. We compare the performance of our model with several
state-of-the-art methods in the field. The experimental results indicate that
our model outperforms most existing methods in terms of accuracy while
exhibiting the superior running speed and the smallest number of parameters.

Our main contributions are as follows:

  1. 1)

A novel network architecture SLP-Net is proposed, which is based on the SNP-
type convolution mechanism. It has only 0.2M parameters, which is merely one
hundred and fiftieth of the parameters of the classical convolutional neural
network U-Net.

  2. 2)

We generalize the formulation of ConvSNP from a single channel convolutional
layer to multiple channels and propose Multi-channel ConvSNP(MSConvSNP). Based
on this, we propose SLP, SFA, and SDS components. They serve as feature
extractors, feature adaptive jump connection methods, and downsampling,
respectively, and can be ported to other networks or construct new networks
with promising generality.

  3. 3)

To assess the potential of SLP-Net, we conducted a series of rigorous
experiments, evaluating its performance on the publicly available datasets
ISIC2018 and PH2. SLP-Net not only exhibited faster computational speed but
also achieved superior segmentation performance compared to most state-of-the-
art methods.

Figure 1: Images from the ISIC2018, Three common dataset segmentation
challenges are demonstrated, namely, hair interference, scale interference,
and lesion boundary blurring irregularities.

##  2 Related work

###  2.1 Skin lesion segmentation

Skin lesion image segmentation methods can be classified into two major
categories. The first category includes classical unsupervised algorithms that
rely on techniques such as edge extraction and threshold segmentation. The
second category consists of modern deep learning algorithms that are based on
convolutional neural networks.

Unsupervised algorithms constitute a family of image-processing techniques
that heavily rely on the extraction and recognition of low-level features that
are intrinsic to the image. Remarkably, these techniques are often
computationally inexpensive and don’t require excessive GPU resources to
operate. For example, histogram thresholding segmentation can be used to
differentiate between lesioned areas in skin images and healthy ones by
leveraging differences in color distribution[9, 10, 11]. Lin et al. [12]
contrasted the U-Net and C-means clustering approaches, where it was confirmed
that the U-Net network segmentation results were significantly higher than the
clustering approach, though the clustering approach was simple to implement
and much faster to execute. An active contour model and a boundary-driven
density-based clustering algorithm were also proposed by Mete et al.[13]. To
achieve faster segmentation of skin lesion areas, Peruch et al. [14] proposed
a skin lesion image segmentation technique that integrates dimensionality
reduction, color feature extraction, and post-processing. The classical
methods rely on feature extraction, yielding recognition accuracy that is
typically much lower than that of modern deep learning networks. This is
particularly challenging when dealing with fuzzy and irregularly shaped image
boundaries.

In recent years, deep learning has emerged as the dominant paradigm for image
segmentation, detection, and recognition in computer vision. The advent of
powerful GPU computing technology has facilitated the development of a
plethora of deep learning convolutional neural networks (CNNs) specifically
designed for medical image processing. Among these, several innovations have
emerged to address specific challenges in medical image segmentation. The
fully convolutional network (FCN) architecture was one of the first CNNs to
achieve state-of-the-art results in semantic segmentation. Instead of fully
connected layers, FCN employs convolutional layers to enable pixel-level
semantic segmentation, greatly improving accuracy over traditional methods.
The DeepLab[5] architecture builds on this idea by introducing a dilated
convolution operation that avoids information loss from pooling layers, and
subsequently uses a conditional random field (CRF) algorithm to further
optimize the segmentation process. Another influential architecture is U-Net,
which symmetrizes the network structure and introduces skip connections to
fuse information across different levels of semantic abstraction. U-Net has
proven particularly effective in the field of medical imaging and has served
as a baseline network for numerous subsequent modifications aimed at improving
its segmentation ability in specific medical areas. For instance, Oktay et
al.[15] integrated the attention gating mechanism into U-Net. Tang et al.[16]
proposed a U-Net architecture with separable convolutional blocks, then
addresses the overfitting problem using a random weight averaging strategy.
Tang et al.[17] proposed a boundary with multi-stage U-Nets and weighted
Jaccard distance loss to automatically detect lesions. Wu et al.[18] combined
Inception-like convolutional blocks, recursive convolutional blocks and
expanded convolutional layers to propose a modified U-Net model for
segmentation of skin lesions in dermoscopic images. Benefiting from the advent
of migration learning, many researchers have used models pre-trained on large
datasets as feature extractors to design networks to apply them to the medical
image areas. Wu et al.[19] combined the transformer structure with the U-Net
encoder-decoder architecture to propose a dual-encoder segmentation network.
To achieve better segmentation, Abraham et al.[20] combine soft attention
gates and U-Net, with a focal loss function based on the Tversky index to
solve the problem of data imbalance in medical image segmentation. To improve
the performance of U-Net in various segmentation tasks, Jha et al.[21]
proposed DoubleU-Net with Atrous Spatial Pyramid Pooling (ASPP), which
consists of two U-Net, one of which uses a pre-trained model VGG-19 as an
encoder. Zhang et al.[22] proposed a deeply supervised multi-scale network
that aggregates shallow and deep information through the utilization of the
lateral output layers of the network. A multiscale connectivity block was also
devised to handle the size variations of various cancers. Bi et al.[23]
proposed a hyper-fusion network to continuously optimize the pathological
feature acquisition with user input features based on the separation of
features from lesion images as well as user input. He et al.[24] constructed a
segmentation network consisting entirely of a transformer as a feature
extractor. Although all the above methods obtained promising segmentation
results, it was at the expense of a large amount of computational power in
exchange. Furthermore, due to the presence of network pooling layers, the
problem of detail and edge information loss caused by repeated downsampling is
not effectively addressed.

##  3 Method

###  3.1 SNP-type convolution mechanism

In the past several years, SNP system and its variants have been widely
applied to image processing[25, 26, 27, 28, 29, 30] and time series
prediction[31, 32] as one of the classical models for membrane computing[33].
Nonlinear spiking neural P(NSNP)[34] systems is a variant of SNP systems. NSNP
systems derive a deep learning model, ConvSNP[35], which has been shown to be
effective when applied to networks such as ResNet, VGG, etc. because of its
different working mechanism from traditional deep neural networks. Fig. 2
shows the classical neuron and the SNP-type neuron. (a) and (b) denote
conventional convolutional neurons and ConvSNP neurons, respectively. We can
observe that the two types of neurons differ in their working mechanisms,
mainly in the order of activation and weighted summation. For the traditional
convolution neuron, let $X=\left[x_{1},x_{2},...,x_{s}\right]^{T}\in R^{s}$
denotes an input signal. As it passes through a neuron, it is multiplied and
summed with the corresponding weight
$W=\left[w_{1},w_{2},...,w_{s}\right]^{T}\in R^{s}$, finally biased with b as
well as activated by an activation function into an output signal y. However,
the signal $X$ of the ConvSNP neuron perform the activation operation before
the weighted summation with $W$ to obtain output $Y$. The derivation of the
formula for ConvSNP is explained very explicitly in the literature[35], please
refer to it for details. The mathematical formula for the conventional
convolutional neuron and the ConvSNP neuron are expressed as Eq. 1 and Eq. 2,
respectively.

$Y=f\left(WX\right)=f\left(w_{1}x_{1}+w_{2}x_{2}+...+w_{s}x_{s}+b\right)$ (1)
$Y=Wf\left(X\right)=w_{1}f(x_{1})+w_{2}f(x_{2})+...+w_{s}f(x_{s})+b$ (2)

where $f(\cdot)$ denotes the activation function. In extant networks, ReLU
activation function is generally used. $b\in R$ indicates the bias. Further,
we consider the activated input signal X having an r-channel subsignal x and
set a set of weights that can be multiplied with it. The working mechanism of
SNP convoluted cells can be rewritten as follows:

$Y=Wf\left(X\right)=\sum_{i=1}^{r}\varpi_{1}^{i}f(x_{1})+\sum_{i=1}^{r}\varpi_{2}^{i}f(x_{2})+...+\sum_{i=1}^{r}\varpi_{s}^{i}f(x_{s})+b$
(3)

where, for any element $w$ in $W$,
$w=\varpi^{1}+\varpi^{2}+...+\varpi^{r},(\varpi^{1},\varpi^{2},...,\varpi^{r})\in
R^{r}$. The extended SNP convolutional neurons are designed based on this
mechanism, we name it Multi-channel ConvSNP(MSConvSNP) which is shown in Fig.
2. MSConvSNP provides more scaled weight values for the input signal $X$, and
it can better fit more complex data. In backpropagation, for $X$, each
$\varpi$ is used individually for the solution of the gradient, instead of as
a whole to find the partial derivative of loss as in Eq. 2. Furthermore, for
the identical MSConvSNP, the weights of each scale share the same bias $b$.
Therefore, only one gradient of $b$ needs to be calculated when
backpropagating the gradient.

Figure 2: Three neurons with different working mechanisms

###  3.2 SLP-Net architecture

The proposed SLP-Net is shown in Fig. 3. The network consists of one
initblock, three SNP-type downsampling(SDS) modules, three SNP-type
lightweight(SLP) modules, two SNP-type feature self-adaptation(SFA) modules,
and a upsampling module(US). All SNP-type modules are built in the
convolutional form of MSCovSNP.

Figure 3: The architecture of the proposed SLP-Net.

The initblock is a three-layer block of successive convolutional layers that
converts the original three channels into sixteen channels. We use a
downsampling module similar to [36], which concatenates a 3*3 convolution
kernel and a 2*2 max-pooling, both with a step size of two. As shown in Fig.
4, the feature map with channel number C is downsampled by maximum pooling and
convolution respectively, and the information of both downsampling operations
can be utilized by concatenation. This design effectively supplements the
information loss due to the pooling-only layer. By augmenting the input with
additional channels of information, SLP assumes the role of a feature
extractor, capable of extracting intricate details with precision. The
upsampling module mainly upsamples the size of the feature map extracted by
SLP3 directly eight times to the output resolution size. We arrange three SLP
modules in the network to allow the model to capture information from feature
maps of different resolution sizes, then add the output results together by
SFA module in a ResNet manner. In the network architecture as shown in Fig. 3,
the original image is integrated into each corresponding SDS through a process
of continuous scaling. The original image is downsampled to one-half, one-
quarter, and one-eighth of its original size, and subsequently concatenated
with the corresponding resolution size of SDS. This supplementation of the
original image information to different scale encoder requires only a small
amount of computation. In the next two subsections, we will introduce the
proposed SLP module and SFA module in detail.

Figure 4: The proposed SDS module. The input and output channels for each
layer are in parentheses. $C$ denotes concatenate.

###  3.3 The proposed SLP module

The proposed SLP module is supported by three technologies, namely, depthwise
separable convolution, asymmetric convolution, and dilated convolution. In
this subsection, we will briefly introduce each of the three technologies, and
then gather the advantages of the three technologies to introduce the proposed
SLP module.

Depth-separable convolution has been used in two prestigious models,
Xception[37] and MobileNet[38], which are two significant achievements of
Google’s team from the same period. Depthwise separable convolution can be
divided into two types: Depthwise Convolution and Pointwise Convolution. Among
them, depthwise convolution is a variant of group convolution. The number of
convolution kernel channels of depthwise convolution is equal to the number of
groups, that is, each input channel uses a convolution kernel separately. The
standard convolution is that multiple channels share multiple convolution
kernels. Furthermore, pointwise convolution can reduce dimension. Since the
use of depthwise convolution has no correlation between the output map from
each channel, pointwise convolution as a 1 * 1 convolution kernel can
effectively increase the nonlinear relationship between the outputs of
different output channels.

Asymmetric convolution is the factorization of a standard two-dimensional
convolution kernel into two one-dimensional convolution kernels. In other
words, an n*1 convolution kernel and a 1*n convolution kernel can be combined
to form an n*n convolution kernel. This mechanism can be described in Eq. 4.

$\sum_{i=-M}^{M}\sum_{j=-N}^{N}\\!W(i,j)I(\alpha\\!-\\!i,\beta\\!-\\!j)\\!=\\!\sum_{i=-M}^{M}\\!W_{\alpha}(i)[\sum_{j=-N}^{N}W_{\beta}(j)I(\alpha\\!-\\!i,\beta\\!-\\!j)]$
(4)

where $I$ denotes a 2D image, $W$ denotes a 2D convolution kernel,
$W_{\alpha}$ denotes a 1D convolution kernel in the $\alpha$ dimension, and
$W_{\beta}$ denotes a 1D convolution kernel in the $\beta$ dimension. Both $I$
and $W$ are represented in the calculation by two-dimensional matrices,
$W_{\alpha}$ and $W_{\beta}$ are represented as two vectors respectively. When
the 2D convolution kernel is 3*3, the decomposed convolution kernels are 1*3
and 3*1, resulting in a 33% reduction in computational overhead from 9 to 6
parameters, though the difference in accuracy exhibited by the model is
small[38].

Dilated convolution is a distinct form of convolution in which the receptive
field is increased by adding zeros between two consecutive parameters in the
convolution kernel, without increasing the number of parameters. The dilated
convolution pyramid module is introduced in the Deeplab family[5, 39, 40] to
capture information at multiple scales, which means that information is
extracted from the feature map using dilated convolution with different
dilation rates.

Inspired by the atrous spatial pyramid polling(ASPP) from the Deeplab family,
we design a multi-scale feature extractor SNP-type lightweight(SLP) module,
which is much more lightweight than the ASPP. The benefit of the lightweight
is that we can use more SLP modules in the network, allowing it to perform
feature extraction on different resolutions. As shown in Fig. 5, the SLP
consists of two SNP-type neurons, the SNP-type depthwise convolution neuron
and the SNP-type pointwise convolution neuron, both of which are indicated by
dashed boxes. SNP-type depthwise convolution neuron contains four sets of
convolution layers and an activation function. Each set of convolution layers
is composed of 3*1 and 1*3 depthwise convolutions, whose dilated rates are 0,
4, 8, and 16, respectively. Each set of convolutions is an asymmetric
convolution resulting from the deformation of a normal 3*3 convolution. The
number of input channels of SLP is equal to the number of output channels. Let
the number of input channels be 2N. When the feature map is convolved by the
3*1 convolution layer, the number of channels will be reduced to half of the
number of 2N. The final number of output channels will be the same as the
input channels after the 1*3 convolution layer. Finally, the results of
several convolutions with different expansion rates are summed to obtain the
feature map of 2N channels. The input image is eventually up-dimensioned to a
2N-channel feature map by a 1*1 SNP-type point convolution neuron. The SNP-
type depthwise convolution neuron uses the computational mechanism of
MSConvSNP, which means that the 4 groups of convolutions share a bias b and an
input feature map.

Figure 5: The proposed SLP module. DW Conv and PW Conv denote depthwise
convolution and pointwise convolution, respectively. $f(\cdot)$ denotes
activation function. $r$ and $d$ denote the expansion rate and the number of
zeros to be filled between two neighboring parameters in the convolution
kernel, respectively.

The SNP-type pointwise convolution neuron serves a purpose by encoding the
multiscale information received from the SNP-type depthwise convolution
neuron. Its primary objective is to uncover and capture the intricate inter-
relationships that exist between individual channels. The SNP-type pointwise
convolution neuron and the SNP-type depthwise convolution neuron are embedded
in the network in a residual-connected manner.

###  3.4 The proposed SFA module

To compensate for the loss of information caused by the feature maps after
successive downsampling, we adopt skip connections like U-Net[4]. However, as
can be seen in Fig. 3, the proposed network does not use a symmetrical
structure similar to U-Net. We only use an 8x upsampling in the decoder stage.
Therefore, when building the network, we are not able to directly concatenate
the information from the encoder stage SLP to the final upsampling module.
Based on the issues above, we propose the SFA module. As shown in Fig. 6, the
SFA module is essentially an SNP-type convolution block. It should be noted
that the 3*3 convolution and 1*1 convolution used together here are similar in
form to the residuals. The 3*3 and 1*1 convolutions are used to adaptively
encode the low-level semantic information captured by the encoder, allowing it
to choose different sizes of convolution on its own. Let
$X^{SFA}=\left[x^{SFA}_{1},x^{SFA}_{2},...,x^{SFA}_{2N}\right]^{T}\in R^{2N}$
be the input feature map with 2N channels, whose channel number turns to N
channels after two kinds of convolution. Then both feature maps are
concatenated, finally yielding the output feature map
$Y^{SFA}\left[Y^{SFA}_{1},Y^{SFA}_{2},...,Y^{SFA}_{2N},\right]^{T}\in R^{2N}$
with 2N channels. The working principle of SFA can be summarised in Eq. 5 and
Eq. 6.

$Y^{SFA}=\psi^{T}_{3}(\sigma_{\lambda}(X^{SFA}))\oplus\psi^{T}_{1}(\sigma_{\lambda}(X^{SFA}))$
(5) $\sigma_{\lambda}(X^{SFA})=max(0,X^{SFA})+\lambda\times min(0,X^{SFA})$
(6)

where $\sigma_{\lambda}$ denotes PReLU activation funtion, and $\lambda$
represents the slope on the negative axis of PReLU. $\psi^{T}_{3}$ and
$\psi^{T}_{1}$ denote convolutions, consisting of a set of 3*3 matrices and a
set of 1*1 matrices, respectively. $\oplus$ denotes the concatenate operation.
The SFA module is set up after SLP1 and SLP2. Finally, by up-sampling, we can
concatenate the feature maps from various resolutions to obtain multi-scale
information.

Figure 6: The proposed SFA module. DS denotes upsampling, $f(\cdot)$ denotes
activation function. C denotes concatenate.

##  4 Experiments

###  4.1 Datasets

In this paper, SLP-Net is implemented and evaluated on two publicly accessible
datasets.

The 2018 International Skin Imaging Collaboration(ISIC) skin lesion
segmentation challenge dataset[7] has a total of 2594 RGB images with non-
uniform resolution size. We take 2074 images from the dataset as the training
set and the remaining 520 as the test set.

PH2 dataset[8] contains 200 RGB dermoscopic images, 40 of which are melanoma
and 160 benign nevus.

Both datasets have the official ground truth provided.

###  4.2 Implementation

The experiments are based on the PyTorch deep learning framework. The batch
size is set to 20 and the training epoch is 50. We employ Adam optimization
with a learning rate of 1e-3, and a weight decay is 1e-4. Since the data set
is sufficient in number, we do not increase the number of entities in the
dataset. All images are scaled to 224*224 pixels and then fed into the
network. It is noteworthy that, in the experiments, we have incorporated both
rotated and flipped data enhancements into the dataset at the time of loading
data, rather than expanding the number of entities in the dataset directly
before network training. As a result, the images utilized during each epoch
training round are unique, thereby enabling the network to capture and analyze
the dataset from multiple perspectives while improving the accuracy and
precision of the system. Also because of the data diversity it allows the
network to generalize better and not overfit. To mitigate the effects of
randomness and to make a fair comparison with other methods, the methods
listed in the table were all trained four times and averaged in the same
environment to allow for a fair comparison.

###  4.3 Evaluation metrics

To evaluate the performance of lesion segmentation,we adopt metrics
recommended by the ISIC,which includes: Accuracy(Acc), Sensitivity(Sens),
Specificity(Spec), Jaccard(JI) and Dice coefficient(DSC).

$Acc=\frac{TP+TN}{TP+TN+FP+FN}$ (7) $Sens=\frac{Tp}{TP+FN}$ (8)
$Spec=\frac{TN}{TN+FP}$ (9) $JI=\frac{TP}{TP+FP+FN}$ (10) $DSC=\frac{2\times
TP}{2\times TP+FP+FN}$ (11)

Where TP, TN, FP and FN are true positive, true negative, false positive and
false negative respectively. All evaluation metrics have a value between 0 and
1. The closer the value is to 1, the better the segmentation result, and vice
versa.

###  4.4 Result

We compare the proposed method with eight state-of-the-art methods on ISIC2018
dataset and PH2 dataset including FCN[3], SegNet[41], U-Net[4], AttU-Net[15],
EDANet[42], PSPNet[43], Deeplabv3+[40], and FAT-Net[19]. PSPNet, Deeplabv3+,
and FAN-Net have pre-trained models, while all other methods do not employ
pre-trained models. Since the proposed SLP-Net is a lightweight architecture,
in comparison with PSPNet and Deeplabv3+, two methods that also have multi-
scale information extraction capabilities, we would like to use a relatively
lightweight pre-trained network as their backbone to compare with our method.
PSPNet and Deeplabv3+ both adopt Mobile-Net[38], which was pre-trained on
Visual Object Classes Challenge 2012 (VOC2012), as their feature extractor.
FAN-Net employs ResNet-34[6], pre-trained on ImageNet2012dataset, as the
feature extraction model. We conduct the experiments according to the dataset
partitioning approach introduced in Sec. 4.1. It is worth noting that we do
not split the PH2 dataset into train and validation sets due to the small
number of images in the PH2 dataset; Instead, we directly train the results
derived from the 2074 images in the ISIC2018 dataset and use them for testing
the PH2 dataset. The generalization ability of our model can also be checked
with such an approach.

###  4.5 Results on the ISIC2018 dataset.

Tab. 1 visualizes the metric performance of the proposed model SLP-Net and
other advanced methods on the ISIC2018 dataset. The proposed SLP-Net achieved
93.87%, 89.30%, 95.36%, 80.61%, and 88.21% in Acc, Sens, Spec, JI, and DSC
metrics respectively. Among them, Acc and DSC reach the highest. U-Net, the
most popular network for medical segmentation, falls behind us in terms of all
metrics, except Spec. A most serious concern is that U-Net can not effectively
detect images like the one illustrated in Fig. 7(c) where the color contrast
between the lesion area and the background skin is relatively small. In other
words, it has a low sensitivity. Moreover, the same problem exists in AttU-
Net, a U-Net succession that has a similar structure to it. One obvious fact
here is that the pre-trained networks all appear to reach higher DSC and JI
metrics. However, SLP-Net, as a completely unpre-trained model, can achieve
the highest DSC and the second highest JI after the first place. Deeplabv3+
achieves the highest score on Sens with 91.44%. We fall behind him with 2.14%.
Nonetheless, Sens as a model sensitivity is required to be balanced with high
DSC, otherwise, that would be erroneous, which is particularly evident in Fig.
7(b),(c),(g). In these three images, most of the models segments relatively
small regions, whereas Deeplabv3+ incorrectly segment much larger regions. In
Fig. 7(b), almost all of the area is segmented by Deeplabv3+ as a lesion area.

The proposed SLP-Net can effectively detect not only the complex edges of
areas of small lesions but also identify larger lesion areas, as reflected in
Fig. 8. It is clear from row (a) that U-Net, AttU-Net, EDANet, and FAT-Net all
identify non-lesioned areas as lesions with varying degrees of accuracy. We
box out the misidentified areas with a green circle. Instead, due to the
absence of a multi-scale feature extractor, U-Net misidentifies the scale
table in the image(b) as well. Although this problem has been improved on
AttU-Net, there is still redundant segmentation. (c) is also a very
representative image, and many networks have difficulty identifying the lesion
area. However, with the proposed SLP-Net, the lesion areas can be identified
more accurately.

Figure 7: Segmentation results for different models on the ISIC2018 datasets.
The first and second columns show the original images and the groud truth,
respectively. The remaining four columns show, from left to right, the outputs
of U-Net, AttU-Net, Deeplabv3+, EDANet, FAT-Net and the proposed SLP-Net.
Figure 8: Comparison of segmentation details in the ISIC2018 dataset. In each
figure, The parts outlined by the red and blue lines are the results of ground
truth and model segmentation respectively. The area circled by the green line
indicates the wrong segmentation. Figure 9: Segmentation results for different
models on the PH2 datasets. In each figure, The parts outlined by the red and
blue lines are the results of ground truth and model segmentation
respectively.

| Acc | Sens | Spec | JI | DSC
---|---|---|---|---|---
FCN8s | 92.88 $\pm$ 0.27 | 87.19 $\pm$ 1.68 | 95.46 $\pm$ 0.95 | 78.00 $\pm$ 0.36 | 86.10 $\pm$ 0.24
SegNet | 92.11 $\pm$ 0.98 | 87.35 $\pm$ 1.76 | 94.63 $\pm$ 1.61 | 76.32 $\pm$ 2.52 | 85.08 $\pm$ 1.98
U-Net | 92.99$\pm$ 0.29 | 87.08$\pm$ 1.81 | 95.97$\pm$ 0.39 | 78.03$\pm$ 0.42 | 86.20$\pm$ 0.42
AttU-Net | 92.79$\pm$ 0.29 | 87.27$\pm$ 0.34 | 95.59$\pm$ 0.71 | 77.88$\pm$ 0.35 | 86.14$\pm$ 0.14
EDANet | 93.31$\pm$ 0.16 | 88.67$\pm$ 2.92 | 95.64$\pm$ 0.58 | 78.87$\pm$ 0.31 | 86.96$\pm$ 0.16
PSPNet(pretrained) | 93.51$\pm$ 0.12 | 89.85$\pm$ 0.96 | 93.12$\pm$ 0.47 | 79.64$\pm$ 0.24 | 87.63$\pm$ 0.14
Deeplabv3+(pretrained) | 93.67$\pm$ 0.28 | 91.44$\pm$ 0.98 | 92.72$\pm$ 1.15 | 80.66$\pm$ 0.28 | 88.17$\pm$ 0.18
FAT-Net(pretrained) | 92.66$\pm$ 1.12 | 88.80$\pm$ 2.49 | 93.29$\pm$ 3.67 | 78.17$\pm$ 1.31 | 86.45$\pm$ 1.06
SLP-Net(Proposed) | 93.87$\pm$ 0.14 | 89.30$\pm$ 1.21 | 95.36$\pm$ 0.82 | 80.61$\pm$ 0.41 | 88.21$\pm$ 0.28

Table 1: Results on the ISIC2018 datasets

| Acc | Sens | Spec | JI | DSC
---|---|---|---|---|---
FCN8s | 92.88 $\pm$ 0.27 | 89.25 $\pm$ 2.86 | 93.34 $\pm$ 1.56 | 76.58 $\pm$ 1.60 | 85.64 $\pm$ 1.08
SegNet | 89.34 $\pm$ 0.75 | 89.91 $\pm$ 1.77 | 92.27 $\pm$ 1.62 | 73.36 $\pm$ 1.70 | 83.56 $\pm$ 1.30
U-Net | 90.48$\pm$ 1.29 | 88.54$\pm$ 3.85 | 94.70$\pm$ 0.90 | 76.84$\pm$ 1.95 | 85.67$\pm$ 1.74
AttU-Net | 90.48$\pm$ 1.16 | 90.67$\pm$ 0.37 | 93.67$\pm$ 1.61 | 76.62$\pm$ 3.44 | 85.78$\pm$ 2.23
EDANet | 91.11$\pm$ 0.30 | 93.75$\pm$ 1.47 | 91.18$\pm$ 1.15 | 77.45$\pm$ 0.82 | 86.45$\pm$ 0.58
PSPNet(pretrained) | 92.21$\pm$ 0.58 | 95.10$\pm$ 0.63 | 91.67$\pm$ 0.36 | 80.26$\pm$ 1.22 | 88.50$\pm$ 0.87
Deeplabv3+(pretrained) | 92.91$\pm$ 0.10 | 96.00$\pm$ 0.89 | 91.55$\pm$ 1.36 | 81.33$\pm$ 0.40 | 89.19$\pm$ 0.35
FAT-Net(pretrained) | 92.25$\pm$ 0.13 | 93.70$\pm$ 0.13 | 93.16$\pm$ 0.13 | 81.03$\pm$ 0.25 | 88.86$\pm$ 0.17
SLP-Net(Proposed) | 91.62$\pm$ 0.60 | 92.16$\pm$ 1.06 | 93.14$\pm$ 1.69 | 79.17$\pm$ 1.78 | 87.20$\pm$ 1.25

Table 2: Results on the PH2 datasets

###  4.6 Results on the PH2 dataset.

To further validate the generalization of the model, we apply the model, pre-
trained by ISIC2018, straightforwardly to the PH2 dataset for testing. The
performance of the metrics on this single dataset is not satisfactory. As
shown in Table 2, the results of the proposed method fall behind the class of
pre-trained models like Deeplabv3+, PSPNet, and FAN-Net in terms of metrics.
However, while the pre-trained models inherently has stronger generalization,
SLP-Net also has advantages that the pre-trained model does not have, which
are less training time, lower hardware consumption, and faster running speed.
This is explained in detail in Sec. 4.7 Then there are comparisons with the
models unpre-trained, where the proposed model achieves the highest DSC and JI
of the models compared in the table.

In the PH2 test, U-Net, AttU-Net, and FAT-Net all show excessive under-
segmentation when processing image Fig. 9(a), in contrast to EDANet, which
showed over-segmentation. Deeplabv3+ has been shown to over-segment lesion
areas on the ISIC2018 dataset, with the same effect on PH2, which shows in
both Fig. 9(f)(g). In contrast, SLP-Net shows excellent stability in the
segmentation of both small lesion areas and large lesion areas without
excessive under-segmentation or over-segmentation.

| GFLOPs | Params | FPS | Params-size
---|---|---|---|---
FCN | 15.45 | 15.12M | 155 | 57.67MB
SegNet | 30.07 | 29.44M | 108 | 112.32MB
U-Net | 41.90 | 31.04M | 47 | 118.40MB
AttU-Net | 51.01 | 34.88M | 38 | 135.27MB
EDA-Net | 0.85 | 0.68M | 90 | 2.60MB
PSPNet(pretrained) | 0.58 | 2.41M | 115 | 9.20MB
Deeplabv3+(pretrained) | 5.05 | 5.81M | 101 | 22.18MB
FAT-Net(pretrained) | 30.51 | 28.76M | 36 | 123.30MB
SLP-Net | 2.30 | 0.20M | 190 | 0.75MB

Table 3: Comparison of the number of parameters and computational complexity
of each model.

###  4.7 Computational complexity

All of the experiments are implemented on a Tesla P100 GPU card with Ubuntu
18.04. Table 3 shows a comprehensive comparison of the computational
complexity of all the experimented models. As can be seen from the table, the
proposed model achieves better values in all metrics except GFLOPs. The
highest FPS value of the proposed network is 190 while the U-Net has only 47.
The processing speed of our model is about 4 times its processing speed in the
same computational environment. Models that require pre-training, Deeplabv3+,
PSPNet, and FAT-Net, inherently require a high pre-training cost. Due to a
large number of parameters, the segmentation results depend on the
effectiveness of the pre-training. Fewer parameters mean that the model can be
implemented significantly easier so that it can be deployed and applied to
real-time processing areas like mobile and embedded devices.

##  5 Conclusion

In this paper, we propose MSConvSNP by generalizing the formula of ConvSNP and
propose a novel SNP-type convolutional neural network SLP-Net. The
architecture of SLP-Net consists of two major parts, SLP and SFA, both of
which are modules inspired by the SNP-type convolution mechanism. The SNP
mechanism allows better results with significantly fewer parameters. As a
lightweight multi-scale feature extractor, SLP stands out as a solution to
complex skin lesion segmentation problems. The relatively small number of
parameters allows us easily apply it to feature maps of various resolution
sizes, thus facilitating information extraction at different resolutions. The
SLP learns more global features, facilitating the model to distinguish between
skin lesion areas of different sizes, as well as providing accurate
recognition of lesion images that do not vary significantly in contrast.
Furthermore, instead of the popular encoder-decoder structure, we abandon the
decoder and use SFA for adaptive feature recovery. It takes a form of residual
concatenation that allows adaptive recovery of boundary information, adding
more high-level information to the image. Extensive experiments are done with
the ISIC2018 challenge and demonstrate that better segmentation performance
can be achieved with less computation cost. Better generalization of SLP-Net
is also shown on the PH2 dataset compared to unpre-trained state-of-the-art
networks. In our future work, we will further promote SNP and continue to
explore the application of SNP in attention mechanisms.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or
personal relationships that could have appeared to influence the work reported
in this paper.

## Data availability

Data will be made available on request.

## Acknowledgement

## References

  * [1] R. L. Siegel, K. D. Miller, N. S. Wagle, A. Jemal, Cancer statistics, 2023, CA: a Cancer Journal for Clinicians 73 (1) (2023) 17–48.
  * [2] H. W. Rogers, M. A. Weinstock, S. R. Feldman, B. M. Coldiron, Incidence estimate of nonmelanoma skin cancer (keratinocyte carcinomas) in the us population, 2012, JAMA Dermatology 151 (10) (2015) 1081–1086.
  * [3] J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3431–3440.
  * [4] O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for biomedical image segmentation, in: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, Springer, 2015, pp. 234–241.
  * [5] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, A. L. Yuille, Semantic image segmentation with deep convolutional nets and fully connected crfs, arXiv preprint arXiv:1412.7062 (2014).
  * [6] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.
  * [7] N. C. Codella, D. Gutman, M. E. Celebi, B. Helba, M. A. Marchetti, S. W. Dusza, A. Kalloo, K. Liopyris, N. Mishra, H. Kittler, et al., Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic), in: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), IEEE, 2018, pp. 168–172.
  * [8] T. Mendonça, P. M. Ferreira, J. S. Marques, A. R. Marcal, J. Rozeira, Ph 2-a dermoscopic image database for research and benchmarking, in: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, 2013, pp. 5437–5440.
  * [9] D. D. Gomez, C. Butakoff, B. K. Ersboll, W. Stoecker, Independent histogram pursuit for segmentation of skin lesions, IEEE Transactions on Biomedical Engineering 55 (1) (2007) 157–161.
  * [10] M. E. Yueksel, M. Borlu, Accurate segmentation of dermoscopic images by image thresholding based on type-2 fuzzy logic, IEEE Transactions on Fuzzy Systems 17 (4) (2009) 976–982.
  * [11] K. Møllersen, H. M. Kirchesch, T. G. Schopf, F. Godtliebsen, Unsupervised segmentation for digital dermoscopic images, Skin Research and Technology 16 (4) (2010) 401–407.
  * [12] B. S. Lin, K. Michael, S. Kalra, H. R. Tizhoosh, Skin lesion segmentation: U-nets versus clustering, in: 2017 IEEE Symposium Series on Computational Intelligence (SSCI), IEEE, 2017, pp. 1–7.
  * [13] M. Mete, N. M. Sirakov, Lesion detection in demoscopy images with novel density-based and active contour approaches, in: BMC bioinformatics, Vol. 11, BioMed Central, 2010, pp. 1–13.
  * [14] F. Peruch, F. Bogo, M. Bonazza, V.-M. Cappelleri, E. Peserico, Simpler, faster, more accurate melanocytic lesion segmentation through meds, IEEE Transactions on Biomedical Engineering 61 (2) (2013) 557–565.
  * [15] O. Oktay, J. Schlemper, L. L. Folgoc, M. Lee, M. Heinrich, K. Misawa, K. Mori, S. McDonagh, N. Y. Hammerla, B. Kainz, et al., Attention u-net: Learning where to look for the pancreas, arXiv preprint arXiv:1804.03999 (2018).
  * [16] P. Tang, Q. Liang, X. Yan, S. Xiang, W. Sun, D. Zhang, G. Coppola, Efficient skin lesion segmentation using separable-unet with stochastic weight averaging, Computer Methods and Programs in Biomedicine 178 (2019) 289–301.
  * [17] Y. Tang, F. Yang, S. Yuan, et al., A multi-stage framework with context information fusion structure for skin lesion segmentation, in: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), IEEE, 2019, pp. 1407–1410.
  * [18] J. Wu, E. Z. Chen, R. Rong, X. Li, D. Xu, H. Jiang, Skin lesion segmentation with c-unet, in: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, 2019, pp. 2785–2788.
  * [19] H. Wu, S. Chen, G. Chen, W. Wang, B. Lei, Z. Wen, Fat-net: Feature adaptive transformers for automated skin lesion segmentation, Medical Image Analysis 76 (2022) 102327.
  * [20] N. Abraham, N. M. Khan, A novel focal tversky loss function with improved attention u-net for lesion segmentation, in: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), IEEE, 2019, pp. 683–687.
  * [21] D. Jha, M. A. Riegler, D. Johansen, P. Halvorsen, H. D. Johansen, Doubleu-net: A deep convolutional neural network for medical image segmentation, in: 2020 IEEE 33rd International Symposium on Computer-based Medical Systems (CBMS), IEEE, 2020, pp. 558–564.
  * [22] G. Zhang, X. Shen, S. Chen, L. Liang, Y. Luo, J. Yu, J. Lu, Dsm: A deep supervised multi-scale network learning for skin cancer segmentation, IEEE Access 7 (2019) 140936–140945.
  * [23] L. Bi, M. Fulham, J. Kim, Hyper-fusion network for semi-automatic segmentation of skin lesions, Medical Image Analysis 76 (2022) 102334.
  * [24] X. He, E.-L. Tan, H. Bi, X. Zhang, S. Zhao, B. Lei, Fully transformer network for skin lesion analysis, Medical Image Analysis 77 (2022) 102357.
  * [25] D. Díaz-Pernil, M. A. Gutiérrez-Naranjo, H. Peng, Membrane computing and image processing: a short survey, Journal of Membrane Computing 1 (2019) 58–73.
  * [26] B. Li, H. Peng, J. Wang, X. Huang, Multi-focus image fusion based on dynamic threshold neural p systems and surfacelet transform, Knowledge-Based Systems 196 (2020) 105794.
  * [27] B. Li, H. Peng, X. Luo, J. Wang, X. Song, M. J. Pérez-Jiménez, A. Riscos-Núñez, Medical image fusion method based on coupled neural p systems in nonsubsampled shearlet transform domain, International Journal of Neural Systems 31 (01) (2021) 2050050.
  * [28] B. Li, H. Peng, J. Wang, A novel fusion method based on dynamic threshold neural p systems and nonsubsampled contourlet transform for multi-modality medical images, Signal Processing 178 (2021) 107793.
  * [29] H. Peng, B. Li, Q. Yang, J. Wang, Multi-focus image fusion approach based on cnp systems in nsct domain, Computer Vision and Image Understanding 210 (2021) 103228.
  * [30] S. Mi, L. Zhang, H. Peng, J. Wang, Medical image fusion based on dtnp systems and laplacian pyramid, Journal of Membrane Computing 3 (2021) 284–295.
  * [31] Q. Liu, L. Long, H. Peng, J. Wang, Q. Yang, X. Song, A. Riscos-Núñez, M. J. Pérez-Jiménez, Gated spiking neural p systems for time series forecasting, IEEE Transactions on Neural Networks and Learning Systems (2021).
  * [32] Q. Liu, L. Long, Q. Yang, H. Peng, J. Wang, X. Luo, Lstm-snp: A long short-term memory model inspired from spiking neural p systems, Knowledge-Based Systems 235 (2022) 107656.
  * [33] G. Paun, Membrane computing, Scholarpedia 5 (1) (2010) 9259.
  * [34] H. Peng, Z. Lv, B. Li, X. Luo, J. Wang, X. Song, T. Wang, M. J. Pérez-Jiménez, A. Riscos-Núñez, Nonlinear spiking neural p systems, International Journal of Neural Systems 30 (10) (2020) 2050008.
  * [35] S. Zhao, L. Zhang, Z. Liu, H. Peng, J. Wang, Convsnp: A deep learning model embedded with snp-like neurons, Journal of Membrane Computing 4 (1) (2022) 87–95.
  * [36] A. Paszke, A. Chaurasia, S. Kim, E. Culurciello, Enet: A deep neural network architecture for real-time semantic segmentation, arXiv preprint arXiv:1606.02147 (2016).
  * [37] F. Chollet, Xception: Deep learning with depthwise separable convolutions, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1251–1258.
  * [38] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, H. Adam, Mobilenets: Efficient convolutional neural networks for mobile vision applications, arXiv preprint arXiv:1704.04861 (2017).
  * [39] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, A. L. Yuille, Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs, IEEE Transactions on Pattern Analysis and Machine Intelligence 40 (4) (2017) 834–848.
  * [40] L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, H. Adam, Encoder-decoder with atrous separable convolution for semantic image segmentation, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 801–818.
  * [41] V. Badrinarayanan, A. Kendall, R. Cipolla, Segnet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence 39 (12) (2017) 2481–2495.
  * [42] S.-Y. Lo, H.-M. Hang, S.-W. Chan, J.-J. Lin, Efficient dense modules of asymmetric convolution for real-time semantic segmentation, in: Proceedings of the ACM Multimedia Asia, 2019, pp. 1–6.
  * [43] H. Zhao, J. Shi, X. Qi, X. Wang, J. Jia, Pyramid scene parsing network, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 2881–2890.


position=current page.east, angle=90, nodeanchor=south, vshift=1.6mm,
opacity=1, scale=3, contents=Accepted version of the published paper with DOI:
10.1109/EuCNC/6GSummit58263.2023.10188332

# Impact of Array Configuration on Head-Mounted Display Performance at mmWave
Bands

Alexander Marinšek∗, Xuesong Cai†, Lieven De Strycker∗, Fredrik Tufvesson†,
Liesbet Van der Perre∗ ∗ESAT-WaveCore, KU Leuven, Ghent, Belgium
†Department of Electrical and Information Technology, Lund University, Lund,
Sweden
{alexander.marinsek<EMAIL_ADDRESS>
###### Abstract

Immersing a user in life-like extended reality (XR) scenery using a head-
mounted display (HMD) with a constrained form factor and hardware complexity
requires remote rendering on a nearby edge server or computer. Millimeter-wave
(mmWave) communication technology can provide sufficient data rate for
wireless XR content transmission. However, mmWave channels exhibit severe
sparsity in the angular domain. This means that distributed antenna arrays are
required to cover a larger angular area and to combat outage during HMD
rotation. At the same time, one would prefer fewer antenna elements/arrays for
a lower complexity system. Therefore, it is important to evaluate the trade-
off between the number of antenna arrays and the achievable performance to
find a proper practical solution. This work presents indoor 28 GHz mmWave
channel measurement data, collected during HMD mobility, and studies the
dominant eigenmode (DE) gain. DE gain is a significant factor in understanding
system performance since mmWave channel sparsity and eigenmode imbalance often
results in provisioning the majority of the available power to the DE.
Moreover, it provides the upper performance bounds for widely-adopted analog
beamformers. We propose 3 performance metrics – gain trade-off, gain
volatility, and minimum service trade-off – for evaluating the performance of
a multi-array HMD and apply the metrics to indoor 28 GHz channel measurement
data. Evaluation results indicate, that 3 arrays provide stable temporal
channel gain. Adding a 4th array further increases channel capacity, while any
additional arrays do not significantly increase physical layer performance.

###### Index Terms:

Extended reality, wireless, millimeter-wave, antenna configuration, channel
measurements

##  I Introduction

An extended reality (XR) head-mounted display (HMD) aims to immerse the user
in a virtual environment or intertwine digital objects with the user’s
physical surroundings. An HMD primarily focuses on deceiving the human visual
system by means of high-resolution video data, requiring powerful graphics
processing unit (GPU) rendering hardware. To avoid burdening the HMD with
additional processing components and batteries, rendering can take place at
either an edge server [1] or a nearby computer, requiring transport medium
data rates in excess of $1\text{\,}\mathrm{G}\mathrm{b}\mathrm{p}\mathrm{s}$
and more than $10\text{\,}\mathrm{G}\mathrm{b}\mathrm{p}\mathrm{s}$ for
streaming unencoded video. HMDs, therefore, often rely on a tethered
connection to the rendering machine. However, a cable will hinder user
mobility and compromise XR content fidelity – the reason for needing to
transfer video data in the first place. Wireless communication technologies
5G, 6G, and IEEE 802.11ad/ay can supply peak multi-Gbps data rates by
leveraging the ample bandwidth availability in the millimeter-wave (mmWave)
and THz spectrum [2, 3, 4]. Unfortunately, mmWave channels are known for their
angular sparsity, i.e., having few distinct multipath components (MPCs) [5,
6]. Hence, an HMD, equipped with widely-adopted directional patch antennas
(with a field of view less than $<90^{\circ}$ [7]) should feature several
antenna arrays, placed along the HMD’s outside perimeter, to provide
sufficient angular coverage during rotation. In addition to misalignment,
mmWave MPCs are highly prone to blockage by other users and oneself [8, 9].
Adding antenna arrays can also mitigate the adverse effects of blockage since
the broader angular coverage allows the HMD to receive other MPCs
(reflections, unaffected by blockage).

MmWave reception quality has recently been studied in the context of 5G NR
mobile networks. Handheld devices are subject to usage in adverse
circumstances – movement, orientation flipping, shadowing, and physical
contact between its antennas and the user – that can disturb the mmWave data
link. Walking $15\text{\,}\mathrm{m}$ from one end of a corridor to its far
end will cause a degradation of around $1520\text{\,}\mathrm{dB}$, with
additional $\pm$$15\text{\,}\mathrm{dB}$ fluctuations depending on the
specific user [10]. Rotating the smartphone from vertical to horizontal
orientation can further cause a two-fold change in the achieved performance,
e.g., [11] notes a decrease in IEEE 802.11ad data rate by half upon
orientation change. User body shadowing can effectively attenuate
$2025\text{\,}\mathrm{dB}$ in various handheld use cases [12]. A palm, holding
the smartphone, will absorb up to $15\text{\,}\mathrm{dB}$ signal strength,
whereas a mere finger interacting with the handheld device can already disturb
the mmWave data link gain by $3\text{\,}\mathrm{dB}$ [13]. All these issues
can plague XR HMDs mmWave communications systems. Previous studies concerning
mmWave channel measurements for XR HMDs have demonstrated, that high data
rates are achievable in cluttered indoor scenarios even with limited mmWave
link bandwidth [14], and that a device featuring a single phase-shifted mmWave
array will result in inappropriately volatile channel conditions for XR video
content streaming [15]. However, horn antennas and first-generation consumer
off-the-shelf (COTS) equipment, used in prior art, offer limited insight into
how future HMDs, employing mmWave technology might perform. In the work at
hand, we utilize a multi-antenna-array mmWave channel sounder to assess
channel conditions during small-scale HMD movement due to head rotation, as
well as large-scale fading due to line of sight (LoS) obstruction. We pursue
the answers to: 1) What are the important physical layer performance metrics
for configuring multiple mmWave antenna arrays on an HMD? 2) What is the
minimal number of antenna arrays that still allows the HMD to reap the
benefits of high-bandwidth mmWave communications? Performance is evaluated
based on the channel’s dominant eigenmode (DE) gain. The reason for this is,
that mmWave channels are sparse, and any additional eigenmodes, beyond the
first, are of much lower quality. Hence, often the best strategy for
applications with high data rate and somewhat looser reliability requirements,
compared to vehicular and industrial applications, is allocating most if not
all power to the dominant eigenmode. Leveraging on a dedicated measurement
campaign, our contributions are:

  * •

MmWave channel DE transmission performance is evaluated for a multi-array HMD
during rotation.

  * •

Physical layer performance metrics, relevant for antenna array configuration
on an HMD, are proposed.

  * •

The profitability of using a rear headband on the HMD as an antenna array
mounting point is assessed.

Section II describes the experiment setup. The post-processing procedures and
proposed performance metrics are introduced in Section III and used in Section
IV to evaluate $28\text{\,}\mathrm{GHz}$ channel measurements. Lastly, Section
V summarizes the work and outlines future research directions.

##  II Experiment setup

###  II-A Measurement equipment

We collect channel impulse response (CIR) data using a switched array
multiple-input multiple-output (MIMO) channel sounder [16], custom-built by
Lund University and SONY. The sounder operates at $28\text{\,}\mathrm{GHz}$
with a $768\text{\,}\mathrm{MHz}$ bandwidth ($1.3\text{\,}\mathrm{ns}$ time-
delay resolution) and employs a Zadoff-Chu waveform. A top-down view of the
user equipment (UE) is illustrated in Fig. 1. It consists of 8 planar patch
antenna arrays, referred to as panels, offset by
$45\text{\,}\mathrm{\SIUnitSymbolDegree}$ in azimuth and featuring
$4\text{\,}$$\times$$4\text{\,}$ antennas each. Fig. 1 shows the studied
forward-facing 1–7 panel configurations using the outer octagons. These are
used throughout the work at hand, since we envision that future XR HMDs will
prioritize both ergonomics and aesthetics; hence, they will not feature a rear
headband by default. The inner octagons show the backward-facing
configurations, evaluated only in Section IV-D. Conversely, the access point
(AP) features a single $16\text{\,}$$\times$$4\text{\,}$ planar array. All
antenna elements are dual-polarized, resulting in a
$256\text{\,}$$\times$$128\text{\,}$ channel matrix. Sampling is done at
$18.3\text{\,}\mathrm{\SIUnitSymbolMicro s}$ per antenna element combination,
which includes averaging over 4 sounding sequences to reduce measurement
noise. The CIR snapshot sampling rate is $1\text{\,}\mathrm{Hz}$ to allow
additional time for memory writing. Rubidium clocks and additional preambles
in the sounding waveform are used for accurate synchronization. Table I
summarizes the system parameters, while we refer the reader to
[tataria_275-295_2021] for further details on the channel sounder. Fig. 2(b)
includes a picture of the UE and AP. Note, that although we consider
relatively large UE antenna arrays (approx.
$2.5\text{\,}$$\times$$2.5\text{\,}\mathrm{cm}$), the learnings presented in
this work are also applicable to smaller arrays with less elements since we
study the trade-offs between a full 8-panel setup and an HMD 111UE refers
strictly to the receiving octagonal antenna array structure, while HMD is used
in a broader context that emphasizes the studied use case. with less antenna
panels.

TABLE I: System parameters. Parameter |  Value
---|---
Carrier frequency |  $28\text{\,}\mathrm{GHz}$
Bandwidth |  $768\text{\,}\mathrm{MHz}$
Time-delay resolution |  $1.3\text{\,}\mathrm{ns}$
Max. observable time-delay |  $2.7\text{\,}\mathrm{\SIUnitSymbolMicro s}$
Element combination sampling time |  $18.3\text{\,}\mathrm{\SIUnitSymbolMicro s}$
CIR snapshot sampling frequency |  $1\text{\,}\mathrm{Hz}$
Single-position measurement duration |  $33\text{\,}\mathrm{s}$
|
AP array size |  4$\times$16
UE array size |  4$\times$4
Number of UE arrays (panels) |  8
CIR snapshot size |  $256\times 128\times 2048$

1VIVIIIIIIVIIIVIIVIBackward-facing configuration (used in Section
IV-D)Forward-facing configuration (used throughout the
paper)2VIVIIIIIIVIIIVIIVI3VIVIIIIIIVIIIVIIVI4VIVIIIIIIVIIIVIIVISymmetry
line5VIVIIIIIIVIIIVIIVI6VIVIIIIIIVIIIVIIVI7VIVIIIIIIVIIIVIIVIVIVIIIIIIVIIIVIIVI

Figure 1: UE top-down view, showing the forward- and backward-facing 1–7 panel
configurations on the HMD.

###  II-B Measurement environment

Measurements were carried out at 11 discrete random positions in a conference
room, depicted in Fig. 2. The initial UE orientation is randomly selected for
each position and is shown by the red line at each UE position in Fig. 2(a).
This orientation serves as the starting point for the mobility pattern,
described in Section II-C. Channel sounding is repeated at each position for
non-line-of-sight (NLoS) conditions, where a fiberglass water-filled human
phantom [8] represents a person obstructing the LoS. Since the phantom was
initially designed for $60\text{\,}\mathrm{GHz}$ experiments, we first
characterized it against 6 volunteers at $28\text{\,}\mathrm{GHz}$. The
difference in attenuation between the phantom and the human subjects ranges
from $-0.6\text{\,}$ to $2.1\text{\,}\mathrm{dB}$, hence, the phantom is
suited for $28\text{\,}\mathrm{GHz}$ experiments. The environment remains
static during CIR sampling, except for UE mobility, detailed in the upcoming
subsection.

P0P1P2P3P4P5P6P7P8P9P10AP1UE0UE1UE2UE3UE4UE5UE6UE7UE8UE9UE10Figure 2b
standpoint$\times$PhantomInitial orientation

(a) Floor plan. Brown, cyan, gray, and magenta mark tables and the door,
windows, metal whiteboards, and a projector screen, respectively. The room
size is $6\times$$9.15\text{\,}\mathrm{m}$.

HMD6PhantomAPWhiteboardDoor

(b) Setup for UE6, NLoS. Superimposed: mean difference between the phantom’s
attenuation and that of 6 human subjects (in $\text{\,}\mathrm{dB}$).

Figure 2: Conference room measurement environment.

###  II-C Mobility pattern

A UE mobility pattern, outlined in Fig. 3, has been designed to mimic
different possible HMD rotations during usage. Initially, the UE is standing
upright with panel VII oriented in the direction shown in Fig. 2(a). Recall
from Fig. 1 that panel VII is oriented in the HMD looking direction. A
$\Delta\alpha=30^{\circ}$ leftward (positive) yaw rotation is executed first,
followed by pitching $\Delta\beta=30^{\circ}$ towards the ground, and
concluded by a second $\Delta\alpha=30^{\circ}$ leftward rotation in yaw,
while pitched downwards. All rotations are of extrinsic $xyz$ Euler type.
Movement is executed by manually rotating the tripod’s head, on which the UE
is mounted. Antenna velocity is kept below
$1\text{\,}\mathrm{cm}\text{\,}{\mathrm{s}}^{-1}$, thus, less than one carrier
wavelength per second. Considering the $25\text{\,}\mathrm{cm}$ distance
(marked on the left side of Fig. 3) between the center of rotation and the
octagonal UE’s center of mass – representing the offset between an HMD and the
human cervical vertebrae [17] – the $30$-$30$-$30^{\circ}$ pattern requires
$3$-$15$-$15$ $\text{\,}\mathrm{s}$ or $33\text{\,}\mathrm{s}$ in total. We
refer to the resulting 33 CIR matrices as one measurement. Note, that the
second and third rotation displaces the UE’s center of mass by approximately
$13\text{\,}$ and $6.5\text{\,}\mathrm{cm}$, correspondingly, which make it
lean beyond the phantom during parts of NLoS measurements.

Figure 3: UE mobility pattern in extrinsic Euler rotations.

##  III Post-processing procedures

In this work, we evaluate the DE gain, i.e., the gain of single-spatial-stream
transmission with channel information at both AP and UE, per orthogonal
frequency-division multiplexing (OFDM) subcarrier, for different UE antenna
panel configurations. As previously mentioned, the combination of high data
rate and loose reliability requirements, combined with channel sparsity, in
practice results in the majority of power being allocated to the DE. For
example, when applying waterfilling [18]. Furthermore, when evaluated over the
entire bandwidth, the DE transmission scheme’s gain provides both the upper
gain bound for analog beamformers, often found in mmWave COTS devices, and the
average gain of a joint spectral-spatial OFDM -MIMO precoder, commonly found
in lower-frequency cellular and IEEE 802.11 networks. Note that fully-digital
MIMO and hybrid beamforming can in practice reach higher channel capacity in
the absence of a dominant signal component, while the latter also manages to
maintain relatively high energy efficiency [19]. Both fully-digital MIMO and
hybrid beamforming are beyond the scope of this work. For clarity, Table II
lists the relevant indices in this section.

TABLE II: Index overview. Index |  Meaning |  Index |  Meaning
---|---|---|---
p |  Number of UE panels |  i |  Channel snapshot index
u |  UE position |  k |  Subcarrier index
s |  LoS/NLoS scenario |  |

###  III-A De-noising and dominant eigenvalue extraction

We first de-noise the CIR to minimize the amount of noise that propagates into
the frequency domain upon Fourier transformation (sum over CIR samples),
enabling us to observe lower DE gain, e.g., for a single array configuration
in NLoS conditions, without it sinking below the noise floor. Let
$\bm{H}(\tau)$ denote the 2-dimensional MIMO CIR at delay $\tau$ and let
$\lambda_{1}(\tau)\geq...\geq\lambda_{n}(\tau)$ be the squared singular values
of $\bm{H}(\tau)$ (eigenvalues of $\bm{H}\bm{H}^{H}$). We apply the de-noising
procedure outlined in Algorithm 1. First, we select $\lambda_{1}(\tau)$ values
at time delays far exceeding any reasonable or observed MPC delays, i.e.,
$\tau\in(1.35,2.7)$ $\text{\,}\mathrm{\SIUnitSymbolMicro s}$ (second half of
the CIR). The result is a vector of the largest eigenvalues of 1024 noisy
square matrices, denoted as $\bm{\lambda}_{\text{noise}}$. The values have a
Tracy-Widom $\mathcal{T}\mathcal{W}_{2}$ distribution [20], and, due to the
limited CIR dimensions, we use the 95th percentile as the noise threshold,
instead of applying, for example, the Marčenko-Pastur law. Next, we apply the
eigenvalue threshold $\Lambda$ for setting noisy CIR samples to 0. Finally,
all CIR entries exceeding the $\tau_{\text{max}}=105$ $\text{\,}\mathrm{ns}$
delay spread, corresponding to $31.5\text{\,}\mathrm{m}$ or the longest path
traveled by a 2nd order reflection, are suppressed to 0. This leaves us with
at most 80 non-zero CIR samples. We estimate the potential gain loss by
considering the worst-case scenario, where 64 of the 80 windowed samples
($105\text{\,}\mathrm{ns}$) represent at least 2nd order reflections with a
gain just below the threshold $\Lambda$, and that $18\text{\,}\mathrm{dB}$
separates the gain of the LoS and the reflected MPCs– $8\text{\,}\mathrm{dB}$
for path loss and $2\text{\,}$$\times$$5\text{\,}\mathrm{dB}$ for reflections.
Then the calculated DE gain after de-noising would be $3\text{\,}\mathrm{dB}$
too low. Contrarily, removing 64/80 noisy CIR samples from the summation in
the Fourier transform allows us to study $18\text{\,}\mathrm{dB}$ smaller
eigenmodes in the frequency domain.

Algorithm 1 CIR de-noising

1:procedure De-noise($\bm{H}$)

2: $\bm{\lambda}_{\text{noise}}=\lambda_{1}(\tau),\text{ for
}\tau\in(1.35,2.7)$ $\text{\,}\mathrm{\SIUnitSymbolMicro s}$ $\triangleright$
Evaluate noise

3: $\Lambda=P_{95}(\bm{\lambda}_{\text{noise}})$ $\triangleright$ Get
threshold

4: $\bm{H}(\tau)\leftarrow\bm{0},\text{ where
}\bm{\lambda}_{\text{noise}}<\Lambda$ $\triangleright$ Below threshold

5: $\bm{H}(\tau)\leftarrow\bm{0},\text{ for }\tau>\tau_{\text{max}}$
$\triangleright$ Out of delay spread

6:end procedure

We convert each CIR to its channel transfer function (CTF) using the fast
Fourier transform (FFT), before extracting the dominant eigenmode gain per
subcarrier. We use a 2048-point FFT for convenience since the CIR has 2048
time-delay samples. All further mention of eigenvalues/modes refers explicitly
to the frequency domain. We drop the index 1 in view of brevity and denote the
$k$-th tone’s dominant eigenmode as $\lambda[u,s,i,k]$, where $k\in[0,2047]$
($27.61628.384\text{\,}\mathrm{GHz}$), while $u$, $s$, and $i$ represent the
UE position, scenario (LoS/NLoS), and channel snapshot index, correspondingly
(see Table II).

###  III-B Performance metrics

This section proposes processing procedures for evaluating the performance of
a 1–7 panel HMD, based on $\lambda[u,s,i,k]$, where $u,s,i,k$ are listed in
Table II.

####  III-B1 Gain trade-off

The first metric is the DE gain trade-off between a full 8-panel HMD and a
p-panel HMD. It allows us to identify whether changing the panel count results
in merely a gain difference proportional to beamforming gain or if there are
other driving forces behind the difference, such as diversity gain. The trade-
off is derived as follows:

$\overline{\Delta\lambda_{p}}[u,s,i]=\frac{1}{K}\sum_{k=0}^{K-1}\frac{\lambda_{p}[u,s,i,k]}{\lambda_{8}[u,s,i,k]},$
(1)

where $\lambda_{p}[u,s,i,k]$ and $\lambda_{8}[u,s,i,k]$ are the DEs of a
p-panel and an 8-panel HMD. An average over 2048 subcarriers for each channel
snapshot constitutes the final value.

####  III-B2 Gain volatility

An important performance metric for XR applications is wireless link
stability, since it determines how much video stream buffer is required, how
often the resolution is adapted, and whether there is a need for radio access
technology switching. Channel gain spread and persistence are evaluated
through the gain’s standard deviation and autocorrelation, respectively. The
latter is derived according to:

$r_{p}[u,s]=\frac{\sum\limits_{i=0}^{I-2}(\overline{\lambda_{p}}[u,s,i]\\!-\\!\overline{\overline{\lambda_{p}}}[u,s])(\overline{\lambda_{p}}[u,s,i\\!+\\!1]\\!-\\!\overline{\overline{\lambda_{p}}}[u,s])}{\sum\limits_{i=0}^{I-1}(\overline{\lambda_{p}}[u,s,i]-\overline{\overline{\lambda_{p}}}[u,s])^{2}},$
(2)

where the number of snapshots per measurement is $I=33$,
$\overline{\lambda_{p}}[u,s,i]$ is the average DE over all subcarriers, and
$\overline{\overline{\lambda_{p}}}[u,s]$ represents its mean value. The
evaluation is carried out independently at each measurement position and for
LoS/NLoS scenarios. A high standard deviation will indicate a large DE gain
spread during the course of one measurement, while a low autocorrelation is
the result of frequent changes in gain over time. Observing both high standard
deviation and low autocorrelation, in combination with low average gain, would
signal a potentially unstable XR service since there are large variations in
the already adverse channel conditions.

####  III-B3 Minimal service and capacity trade-off

We obtain the minimal service trade-off by comparing the 3rd percentile DE
gain for a p-panel configuration against that of an 8-panel HMD. We use the
3rd percentile based on the $97\text{\,}\mathrm{\char 37\relax}$ link
reliability constraint, corresponding to the largest allowed error ratio for
satisfactory encoded video quality [21]. The metric shows how much channel
capacity is lost due to employing less panels in the most adverse conditions,
without considering $3\text{\,}\mathrm{\char 37\relax}$ of the lowest recorded
gains. Assuming $\text{\,}\mathrm{S}\mathrm{N}\mathrm{R}$${}_{p}>>1$, the
minimal service trade-off is calculated as follows [18]:

$\Delta
C_{p,97}\;\approx\;\left|\log_{2}\left(\frac{P_{3}(\bm{\overline{\lambda_{p}}})}{P_{3}(\bm{\overline{\lambda_{8}}})}\right)\right|,$
(3)

where $\bm{\overline{\lambda_{p}}}$ represents a vector of the DEs at each
position, scenario, and snapshot ($u$, $s$, $i$), averaged over all 2048
subcarriers ($k$). Furthermore, we derive the channel capacity trade-off for
each individual measurement, which allows us to evaluate how much capacity is
wasted due to employing less than 8 panels. It is calculated according to:

$\overline{\Delta
C_{p}}[u,s,i]\;\approx\;\left|\frac{1}{K}\sum_{k=0}^{K-1}\log_{2}\left(\frac{\lambda_{p}[u,s,i,k]}{\lambda_{8}[u,s,i,k]}\right)\right|,$
(4)

where $\overline{\Delta C_{p}}$ represents the mean channel capacity trade-off
over all 2048 subcarriers.

###  III-C Profit from a rear headband

Lastly, we evaluate the benefit of fitting a rear headband to the HMD and
placing an antenna panel on it by assessing the overall gain difference
between the default, forward-facing, panel configuration and its rotated
counterpart. The latter always employs the backward-facing panel III, wheres
the former excludes it (see Fig. 1). We use a similar approach to Equation 1
and evaluate the rear headband benefits using:

$\overline{\Delta\lambda_{p}^{rh}}[u,s,i]=\frac{1}{K}\sum_{k=0}^{K-1}\frac{\lambda_{p,\text{back}}[u,s,i,k]}{\lambda_{p,\text{front}}[u,s,i,k]},$
(5)

where $\lambda_{p,\text{front}}[u,s,i,k]$ and
$\lambda_{p,\text{back}}[u,s,i,k]$ represent the forward- and backward-facing
p-panel configuration DE gains.

##  IV Results

###  IV-A Gain trade-off and impact of mobility

Fig. 4 plots the DE gain trade-off, dependent on the number of antenna panels.
The median gain trade-off, across all channel snapshots (full lines),
increases logarithmically with the number of employed panels; however, the
observed gain only starts conforming with the beamforming gain for an HMD with
5 or more panels. This leads us to two conclusions 1) the beamforming gain for
1–4 panels is overestimated since not all elements on an 8-panel HMD are
illuminated, and 2) 4 panels capture the available diversity gain (sufficient
azimuth coverage) and adding any additional panels results in merely
beamforming gain. We also observe a reduction in the spread of the
distributions as the number of employed panels grows, with the most noticeable
improvements from 1 to 2 and from 2 to 3 panels, while 4 panels offer a
marginal improvement over 3 panels. Finally, a 4-panel HMD follows the gain
trend of an 8-panel HMD the most accurately due to its good symmetry (see Fig.
1), i.e., an evenly distributed azimuth gain.

1234567Number of panels
[p]-18.0-16.0-14.0-12.0-10.0-8.0-6.0-4.0-2.00.0$\displaystyle\overline{\Delta\lambda_{p}}\;($\text{\,}\mathrm{dB}$)$Relative
beamforming gain 10 log $\displaystyle\left(\frac{p}{8}\right)\;\;\;$1)
$\displaystyle\Delta\alpha=30^{\circ}$2)
$\displaystyle\Delta\beta=30^{\circ}$3)
$\displaystyle\Delta\alpha=30^{\circ}$Combined

Figure 4: Gain ratio per mobility pattern (dotted) and combined (full line).
Median values are depicted in orange.

The dotted distributions in Fig. 4 show that all, except a 2-panel HMD,
experience somewhat better performance during the initial
$\Delta\alpha=30^{\circ}$ rotation, when the UE is standing upright. This is
most visible for a single-panel HMD. We assume the carpeted floor and numerous
table and chair legs result in poor MPC reflection, that the forward-pitched
UE could receive. A 2-panel HMD exhibits favorable performance in the final
part of the mobility. This is due to the NLoS experiments, where the second
$\Delta\alpha=30^{\circ}$ rotation, at some point, directly exposes one of the
two panels to the LoS that bypasses the phantom.

###  IV-B Gain volatility

The gain’s standard deviation ($\delta_{p}$) and autocorrelation ($r_{p}$) are
depicted in Fig. 5. The LoS and NLoS scenarios are separated by star/cross
markers, while marker size shows the average gain for each of the 22
measurements. A single panel yields high gain volatility, since adjacent
measurements (less than 1 cm apart) may exhibit a high difference in gain as
the panel moves or rotates from LoS to NLoS or vice-versa. Gain spread remains
high for a 2-panel HMD, however, gain perturbations are gradual over time
(high autocorrelation). This benefits the HMD since resolution adaptations and
handovers are less frequent. An HMD with 3 or more panels shows less
pronounced gain spread and higher mean gain. The low autocorrelation points
originate from more adverse circumstances, where 3 or 4 panels receive the
strongest MPC from a large angle, relative to the individual radiation
patterns. For example, take a forward-pitched 4-panel HMD at positions UE4 or
UE9, where the LoS angle of arrival strides around
$\pm$$45\text{\,}\mathrm{\SIUnitSymbolDegree}$ or right in-between two panels.
Yet, this is a less severe problem since fluctuations around a high mean gain
are less likely to cause service outage. Conversely, a 2-panel HMD would have
one of its panels well-exposed in this case, while it could undergo longer
faded periods in other scenarios. Hence, the high autocorrelation and standard
deviation. Gain volatility does not significantly change for 5–8 panels.

0.00.51.01.52.0$\displaystyle\delta_{1}\;($\text{\,}\mathrm{dB}$)$0255075100$\displaystyle
r_{1}\;(\%)$

(a) 1-panel configuration.

0.00.51.01.52.0$\displaystyle\delta_{2}\;($\text{\,}\mathrm{dB}$)$0255075100$\displaystyle
r_{2}\;(\%)$ LoS NLoS

(b) 2-panel configuration.

0.00.51.01.52.0$\displaystyle\delta_{3}\;($\text{\,}\mathrm{dB}$)$0255075100$\displaystyle
r_{3}\;(\%)$ Mean values Mean values

(c) 3-panel configuration.

0.00.51.01.52.0$\displaystyle\delta_{4}\;($\text{\,}\mathrm{dB}$)$0255075100$\displaystyle
r_{4}\;(\%)$ Mean gain ($\text{\,}\mathrm{dB}$)-50 -45 -40 -35

(d) 4-panel configuration.

Figure 5: DE gain standard deviation ($\delta$) and autocorrelation ($r$).
Least to most prosperous regions (approx.): red, orange, and green (darkest to
brightest in grayscale). Marker size shows the mean gain during a measurement.

###  IV-C Minimal service and channel capacity trade-off

Table III shows that the reduction in the minimal attainable service level is
most pronounced when moving from 3 to 2 and from 2 to 1 panel/-s, which
results in $0.58\text{\,}$ and
$1.35\text{\,}\mathrm{bit}\text{/}\mathrm{s}\text{/}\mathrm{Hz}$ lower channel
capacity, respectively. The noticeably smaller differences between other
configurations show no clear trend.

TABLE III: Minimal service trade-off ($97\text{\,}\mathrm{\char 37\relax}$ reliability).  Number of panels | 1 | 2 | 3 | 4 | 5 | 6 | 7
---|---|---|---|---|---|---|---
$\Delta C_{p,97}$ [$\text{\,}\mathrm{bit}\text{/}\mathrm{s}\text{/}\mathrm{Hz}$] | 3.07 | 1.72 | 1.14 | 0.86 | 0.71 | 0.47 | 0.31
$\Delta C_{p+1,97}-\Delta C_{p,97}$ | 1.35 | 0.58 | 0.28 | 0.15 | 0.24 | 0.16 | 0.31

Fig. 6 confirms the poor performance for either 2 panels or a single panel
configuration, which can suffer in excess of $3\text{\,}$ and
$5\text{\,}\mathrm{bit}\text{/}\mathrm{s}\text{/}\mathrm{Hz}$ lower channel
capacity than an 8-panel HMD, correspondingly. However, we also notice that a
3-panel HMD exhibits a more slanted CDF than the 4-panel HMD, in addition to
achieving $0.7\text{\,}\mathrm{bit}\text{/}\mathrm{s}\text{/}\mathrm{Hz}$
lower capacity in the worst case. A 4-panel HMD offers the most deterministic
performance, since its CDF has the steepest slope, best matching 8-panel
performance. We can conclude, that, contrary to the previous two performance
metrics, the 4-panel configuration has a slight edge over the 3-panel HMD
since the performance gap starts to increase already when decreasing from 4 to
3 panels.

0.00.51.01.52.02.53.03.54.04.55.0$\displaystyle\overline{\Delta
C_{p}}\;($\text{\,}\mathrm{bit}\text{\,}{\mathrm{s}}^{-1}\text{\,}{\mathrm{Hz}}^{-1}$)$0.00.10.20.30.40.50.60.70.80.91.0CDF
($\displaystyle\overline{\Delta C_{p}}$)1 panel2 panels3 panels4 panels5
panels6 panels7 panels8 panels

Figure 6: Channel capacity trade-off.

###  IV-D Profit from a rear headband

Fig. 7 shows that, across all available snapshots, a backward-facing single-
panel setup yields higher mean gain, which means that orienting antennas
towards the ceiling results in higher gain than pointing them towards the
floor, since the HMD spends the majority of time tilted downwards. As noted in
Section IV-A, the reason for the lower gain might be a carpeted and cluttered
floor. A 2-panel setup also mildly prioritizes the backward-facing
configuration, however, we associate this with our mobility pattern design,
where the backward-facing configuration prospers when pitching towards or away
from the AP ($15\text{\,}$ of the total $33\text{\,}\mathrm{s}$). Conversely,
a 3-panel HMD performs better in the forward-facing configuration, where 2
panels are placed diagonally at the back and only one panel at the front,
thus, emphasizing ceiling-bound MPCs. The remaining panel configurations do
not favor a forward- or a backward facing configuration, so we only include
the representative 4-panel HMD results.

-12-10-8-6-4-2024681012$\displaystyle\overline{\Delta\lambda_{p}^{rh}}\;($\text{\,}\mathrm{dB}$)$0.00.10.20.30.40.50.6PDF ($\displaystyle\overline{\Delta\lambda_{p}^{rh}}$)1 panel 2 panels 3 panels 4 panels Mean valuePrioritize backward-facingPrioritize forward-facing

Figure 7: PDF of the gain ratio between the backward- and forward-facing panel
configuration.

##  V Conclusion

The work at hand evaluates the performance of multiple mmWave antenna array
configurations an XR HMD, based on indoor channel sounding data. We introduce
physical layer performance metrics, relevant for selecting the optimal mmWave
array configuration on an HMD, and use them to evaluate the performance of an
HMD employing 1–7 antenna panels. Results demonstrate that fitting an HMD with
a single panel will offer poor performance in all aspects. Using an additional
panel improves temporal gain stability (few abrupt changes), yet, a 2-panel
HMD still struggles in providing sufficient gain during rotation. At least 3
panels are needed to offer a stable and consistent gain, while including a 4th
panel further improves the minimal achievable service quality and channel
capacity by a limited amount. Any additional patch antenna arrays will offer
minute performance improvements at the cost of greater HMD complexity and
higher energy consumption. In future work, we aim to closely investigate the
effects of blockage and derive the number of antenna arrays that are required
for maximally benefiting from the available MPCs and mitigating the effects of
blockage. Moreover, future work should evaluate the most prosperous array
configurations (2–4 panels) from the perspective of multi-stream MIMO
transmission to provide accurate upper performance bounds and evaluate the
role of the number of antennas (in addition to arrays) on small-scale fading.
Looking from another perspective, the results also support the advocated trend
for 6G networks: distributed infrastructure will benefit reliability and
robustness. Thus, the potential of lowering HMD complexity by relying on a
distributed antenna infrastructure is a worthy subject for future research.

## Acknowledgement

Thanks to Meifang Zhu, Gilles Callebaut, and the volunteers at Lund Uni.

This work has received funding from the EU’s Horizon 2020 and Horizon Europe
programmes under grant agreements No. 861222 (MINTS) and 101096302 (6G
Tandem).

The work is also partially supported by the Horizon Europe Framework Programme
under the Marie Skłodowska-Curie grant agreement No.,101059091, the Swedish
Research Council (Grant No. 2022-04691), the strategic research area ELLIIT,
Excellence Center at Linköping — Lund in Information Technology, and Ericsson.

## References

  * [1] F. Firouzi et al., “The Convergence and Interplay of Edge, Fog, and Cloud in the AI-Driven Internet of Things (IoT),” Information Systems, vol. 107, p. 101840, 2022.
  * [2] M. Shafi et al., “5G: A Tutorial Overview of Standards, Trials, Challenges, Deployment, and Practice,” IEEE JSAC, vol. 35, pp. 1201–1221, June 2017.
  * [3] P. Zhou et al., “IEEE 802.11ay-Based mmWave WLANs: Design Challenges and Solutions,” IEEE Communications Surveys & Tutorials, vol. 20, no. 3, pp. 1654–1681, 2018.
  * [4] X. Cai et al., “Toward 6G with Terahertz Communications: Understanding the Propagation Channels,” IEEE Commun. Mag., to appear, 2023.
  * [5] X. Cai et al., “Dynamic Channel Modeling for Indoor Millimeter-Wave Propagation Channels Based on Measurements,” IEEE Trans. Commun., vol. 68, pp. 5878–5891, Sept. 2020.
  * [6] T. Choi et al., “Measurement Based Directional Modeling of Dynamic Human Body Shadowing at 28 GHz,” in 2018 GLOBECOM, (Abu Dhabi, United Arab Emirates), pp. 1–6, IEEE, Dec. 2018.
  * [7] R. Hoque et al., “Distinguishing Performance of 60-GHz Microstrip Patch Antenna for Different Dielectric Materials,” in 2014 ICEEICT, (Dhaka, Bangladesh), pp. 1–4, IEEE, Apr. 2014.
  * [8] C. Gustafson and F. Tufvesson, “Characterization of 60 GHz Shadowing by Human Bodies and Simple Phantoms,” in 2012 EUCAP, (Prague, Czech Republic), pp. 473–477, IEEE, Mar. 2012.
  * [9] L. Vähä-Savo et al., “Empirical Evaluation of a 28 GHz Antenna Array on a 5G Mobile Phone Using a Body Phantom,” IEEE Trans. Antennas Propag., 2020.
  * [10] J. Hejselbak et al., “Measured 21.5 GHz Indoor Channels With User-Held Handset Antenna Array,” IEEE Trans. Antennas Propag., vol. 65, pp. 6574–6583, Dec. 2017.
  * [11] S. Aggarwal et al., “A First Look at 802.11ad Performance on a Smartphone,” in Proc. ACM mmNets, pp. 13–18, Oct. 2019.
  * [12] K. Zhao et al., “User Body Effect on Phased Array in User Equipment for the 5G mmWave Communication System,” IEEE Antennas Wireless Propag. Lett., vol. 16, pp. 864–867, 2017.
  * [13] B. Xu et al., “Radiation Performance Analysis of 28 GHz Antennas Integrated in 5G Mobile Terminal Housing,” IEEE Access, vol. 6, pp. 48088–48101, 2018.
  * [14] R. Gomes et al., “A mmWave Solution to Provide Wireless Augmented Reality in Classrooms,” in 2018 ISWCS, (Lisbon), pp. 1–6, IEEE, Aug. 2018.
  * [15] J. Struye et al., “Opportunities and Challenges for Virtual Reality Streaming over Millimeter-Wave: An Experimental Analysis,” in 2022 NoF, (Ghent, Belgium), pp. 1–5, IEEE, Oct. 2022\.
  * [16] X. Cai et al., “A Switched Array Sounder for Dynamic Millimeter-Wave Channel Characterization: Design, Implementation and Measurements,” IEEE Trans. Antennas Propag., submitted, 2023.
  * [17] M. Kunin et al., “Rotation Axes of the Head During Positioning, Head Shaking, and Locomotion,” Journal of Neurophysiology, vol. 98, pp. 3095–3108, Nov. 2007.
  * [18] A. F. Molisch, Wireless Communications, p. 398.  Wiley Publishing, New Jersey, US, 3rd ed., 2022.
  * [19] X. Qi et al., “Energy-Efficient Power Allocation in Multi-User mmWave-NOMA Systems With Finite Resolution Analog Precoding,” IEEE Trans. Veh. Commun, vol. 71, no. 4, pp. 3750–3759, 2022.
  * [20] M. Chiani, “Distribution of the Largest Eigenvalue for Real Wishart and Gaussian Random Matrices and a Simple Approximation for the Tracy-Widom Distribution,” CoRR, vol. abs/1209.3394, 2012.
  * [21] J. Nightingale et al., “Subjective Evaluation of the Effects of Packet Loss on HEVC Encoded Video Streams,” in 2013 ICCE, (Berlin, Germany), pp. 358–359, IEEE, Sept. 2013.