(cvpr) Package cvpr Warning: Single column document - CVPR requires papers to
have two-column layout. Please load document class ‘article’ with ‘twocolumn’
option (cvpr) Package cvpr Warning: Package ‘hyperref’ is not loaded, but
highly recommended for camera-ready version

# Invisible-to-Visible: Privacy-Aware Human Segmentation using Airborne
Ultrasound via Collaborative Learning Probabilistic U-Net

Risako Tanigawa<EMAIL_ADDRESS>Technology Division,
Panasonic Holdings Corporation  Yasunori Ishii<EMAIL_ADDRESS>Technology Division, Panasonic Holdings Corporation  Kazuki Kozuka
<EMAIL_ADDRESS>Technology Division, Panasonic Holdings
Corporation  Takayoshi Yamashita<EMAIL_ADDRESS>Machine Perception
and Robotics Group, Chubu University

###### Abstract

Color images are easy to understand visually and can acquire a great deal of
information, such as color and texture. They are highly and widely used in
tasks such as segmentation. On the other hand, in indoor person segmentation,
it is necessary to collect person data considering privacy. We propose a new
task for human segmentation from invisible information, especially airborne
ultrasound. We first convert ultrasound waves to reflected ultrasound
directional images (ultrasound images) to perform segmentation from invisible
information. Although ultrasound images can roughly identify a person’s
location, the detailed shape is ambiguous. To address this problem, we propose
a collaborative learning probabilistic U-Net that uses ultrasound and
segmentation images simultaneously during training, closing the probabilistic
distributions between ultrasound and segmentation images by comparing the
parameters of the latent spaces. In inference, only ultrasound images can be
used to obtain segmentation results. As a result of performance verification,
the proposed method could estimate human segmentations more accurately than
conventional probabilistic U-Net and other variational autoencoder models.

##  1 Introduction

Segmentation has attracted wide attention because of its applications, such as
medical image diagnoses, robotics, and action recognition [18, 5, 31, 24, 26,
34]. A camera is a well-developed sensor for segmentation tasks. Although
camera-based segmentation has been widely investigated and achieved high
precision, camera images do not preserve privacy for human segmentation.

Audio signals is a privacy-preserved segmentation [28, 12, 16, 20, 27, 33].
Predicting depth maps and segmentations from audio signals have been proposed
[28, 12]. These methods can generate images from invisible physical
information and can be used to recognize human actions by analyzing
segmentation images. Although these methods can visualize sounding objects,
detecting non-sounding objects are difficult. From the human recognition
standpoint, people who do not make sounds, such as not talking or walking,
cannot be detected.

Airborne ultrasound echoes could be used to detect non-sounding people. There
are methods for detecting the surrounding information by analyzing ultrasound
echoes [17, 4, 21, 11, 8, 22, 32]. Although these methods can estimate the
position of objects, methods that specialize in human recognition and estimate
human segmentation have not been well investigated. If human segmentation can
be estimated from echoes, it would be possible to detect non-sounding people
and it can be used to estimate human action.

Figure 1: Concept of our work. The top row shows the setting of a sensing
environment. The bottom row illustrates the concept of our method.

In this study, we propose a method for estimating human segmentation from
airborne ultrasound detected by multichannel microphones using a neural
network. The concept of our work is illustrated in Figure 1. In the sensing
section, an ultrasonic transducer at 62 kHz resonance, and 16 channels of
microelectromechanical system (MEMS) microphone array are used. The ultrasound
emitted by the ultrasonic transducer is reflected by a human body and then
captured by microphones. The reflected ultrasound directional images
(hereinafter, ultrasound images) can be obtained by analyzing the differences
between multichannel ultrasound signals. Because the ultrasound image
represents the intensity of the reflected wave at each pixel, it has ambiguous
shapes that are far from humans. Therefore, we introduce a deep neural network
to obtain human segmentation images from ultrasound images. We use
probabilistic models for the task because the ultrasound images have different
edge positions from segmentation images. Kohl et al. [15] proposed a
probabilistic U-Net that combines variational autoencoder (VAE) [14] and U-Net
[25]. During the training phase, the probabilistic U-Net learns a network to
match latent spaces of input and the input/ground truth by comparing the
latent spaces with Kullback-Leibler divergence (KLD). A latent vector, sampled
from the latent space of input/ground truth, combined with the last activation
map of a U-Net. The spatial positions of the ultrasound and segmentation
images are similar and the edges are different. Hence, the latent
distributions of ultrasound and segmentation images are roughly close because
of shrinking the spatial dimensions by convolutions. Thus, reducing the
distance between latent distributions with precisely estimating the edges is
difficult. Therefore, we propose a collaborative learning probabilistic U-Net
(CLPU-Net), which uses mean squared error (MSE) instead of KLD to minimize the
distance between latent distributions. Experiments showed that human
segmentation images could be generated from ultrasound images. To the best of
our knowledge, this is the first work to estimate human segmentation images
from airborne ultrasounds.

The main contributions to this work are as follows:

  * •

A human body sensing system using an airborne ultrasound and a microphone
array for privacy-aware human segmentation.

  * •

An architecture of CLPU-Net that learns a network with matching latent
variables from segmentation and ultrasound images.

  * •

We showed that despite a simple mean-squared error, the distance between
latent variables can be shortened by expressing the distribution of high-
dimensional latent space with mean and variance.

##  2 Related work

###  2.1 Privacy-preserved human segmentation

The camera-based methods have been well investigated and have achieved high
accuracy. However, privacy concerns should be considered for applications such
as home surveillance. Cameraless human segmentation methods have also been
investigated to address privacy issues. Wang et al. [29] proposed a method for
estimating human segmentation images, joint heatmaps, and part affinity fields
using Wi-Fi signals. They used three transmitting-receiving antenna pairs and
thirty electromagnetic frequencies with five sequential samples. The channel
state information [9] was analyzed to input networks. The networks comprise
upsampling blocks, residual convolutional blocks, U-Net, and downsampling
blocks. Although this method achieved privacy-friendly fine-grained person
perception with Wi-Fi antennas and routers, the Wi-Fi signals are highly
affected by the surrounding environment due to the multipath effect.

Alonso et al. [1] proposed an event camera-based semantic segmentation method.
The event camera detects pixel information when the brightness changes, such
as when the subject moves. They do not capture personal information more
clearly than the RGB cameras because event cameras only capture changes in
intensities on a pixel-by-pixel basis. In [1], event information from event
cameras was formed as $6$-channel images. The first two channels were
histograms of positive and negative events, whereas the other four channels
were the mean and standard deviation of normalized timestamps at each pixel
for the positive and negative events. They showed that an Xception-based
encoder–decoder architecture could learn semantic segmentation from the
$6$-channel information. Although this method achieves semantic segmentation
from privacy-preserved event cameras, it is difficult to detect people who are
not moving.

Irie et al. [12] proposed a method that generates segmentation images from
sounds. They recorded sounds using $4$-channel microphone arrays. They used
Mel frequency cepstral coefficients and angular spectrum from sounds for
estimating segmentation images. This method can estimate human and
environmental objects only from sounds. However, estimating segmentation
images for non-sounding people is difficult in principle because this method
analyzes the sound emitted from objects.

###  2.2 Airborne ultrasonic sensing

Airborne ultrasound could be used to detect non-sounding people. The positions
of people can be detected by analyzing ultrasound echos reflected by them.
Although the multipath effect influences detection, it is less effective than
radio waves because the propagation speed of ultrasound is slower than that of
radio waves.

Airborne ultrasonic sensors have been used to detect distance in various
industries, such as the automobile [30] and manufacturing industries [7, 3,
13]. Ultrasonic sensors emit short pulses at regular intervals. The
ultrasounds are reflected if there are objects in the propagation path. The
distance of objects can be determined by analyzing the time differences
between emitted and reflected sounds [2].

When sounds are captured using microphone arrays, the directions of objects
can be detected in addition to their distances [19]. Sound localization using
beamforming algorithms has been developed. A delay-and-sum (DAS) method is a
common beamforming algorithm [23]. The DAS method can estimate the direction
of sound sources by adding array microphone signals delayed by a given amount
of time. By using ultrasonic transducers and microphone arrays, the positions
of objects can also be estimated using the DAS with regarding a reflected
position as a sound source. Although this method can detect the position of
objects, it is difficult to obtain the actual shape of objects from echoes of
a single pulse.

Hwang et al. [11] performed three-dimensional shape detection using wideband
ultrasound and neural networks. Although the analysis of multiple frequencies
can precisely detect the positions and shapes of objects, the sound system for
emitting wideband frequency becomes large, and the number of data increases
because of the high sampling rate required to sense wideband ultrasound.
Therefore, we consider obtaining segmentation images from the positional
information on reflected objects analyzed by narrowband frequency ultrasound
using a neural network.

##  3 Ultrasound sensing in the proposed method

First, we describe the hardware setup of our ultrasound sensing system. Then,
we describe the preprocessing for converting ultrasound waves to ultrasound
images.

###  3.1 Ultrasound sensing system

The hardware setup is shown in Figure 2. The transmitter comprised a function
generator and an ultrasonic transducer at 62 kHz resonance. The ultrasonic
transducer was driven with burst waves of $20$ cycles and $50\,\rm{ms}$
intervals at $62\,\rm{kHz}$ using the function generator. The receiver
comprised a MEMS microphone array, an analog-to-digital converter, a field-
programmable gate array (FPGA), and a PC. A $4\times 4$ grid MEMS microphone
array, whose microphones were mounted on a $30\,\rm{mm^{2}}$ substrate at
$3.25\,\rm{mm}$ intervals, was used. The analog signals captured by the
microphones were converted to digital signals and imported into the PC through
the FPGA. The distance between the microphone array and ultrasonic transducer
was set to $30\,\rm{mm}$.

Figure 2: Hardware setup of ultrasonic wave transmitter and receiver system.
Figure 3: Diagram of preprocessing of ultrasound signals. The multiple
ultrasound signals are filtered and then divided into single direct-reflected
wave pairs. The upsampled reflected waves are extracted and DAS beamformed,
and an ultrasound image is created.

###  3.2 Data preprocessing

The diagram of the data preprocessing is shown in Figure 3. First, a band-pass
filter with a center frequency of $62\,\rm{kHz}$ and bandwidth of
$10\,\rm{kHz}$ was used for signals captured by the $16$ microphones. The
filtered ultrasound signals were divided into blocks including direct and
reflected waves. Then, we upsampled the ultrasound waves four times at each
block to improve the accuracy of direction estimation. Following that, we
produced ultrasound images from reflected ultrasounds via DAS beamforming. The
beamformed signal $y$ can be defined as

$y(t)=\sum_{m=1}^{M}x_{m}(t-\Delta_{m}),$ (1)

where $t$ is the time, $M$ is the number of microphones, $x_{m}$ is the signal
received by the $m$-th microphone, and $\Delta_{m}$ is the time delay for the
$m$-th microphone, which is determined by the speed of sound and distance
between the $m$-th microphone and observing points. We set the observing
points in the range of $\theta=-45$–$45$ degrees in the azimuthal direction
and $\phi=-60$–$60$ degrees in the polar direction. Then, we calculated
beamformed signals and obtained the reflected directional heat maps. To reduce
the noise from reflected waves from objects other than people, we calculated
ultrasound heat maps $H_{\rm{us}}$ by subtracting a reference map
$H_{\rm{ref}}$ from reflected directional heat maps $H$ as

$H_{\rm{us}}(i,j)=H(i,j)-kH_{\rm{ref}}(i,j),$ (2)

where $(i,j)$ is the pixel of the heat maps, and $k$ is the coefficient, which
is determined by

$k=\frac{H(i_{\rm{max}},j_{\rm{max}})}{H_{\rm{ref}}(i_{\rm{max}},j_{\rm{max}})},$
(3)

where $(i_{\rm{max}},j_{\rm{max}})$ is the index of the maximum pixel of
$H_{\rm{ref}}$. Notably, the reference map was calculated using the data
without people, and $H_{\rm{us}}$ was normalized as

$X_{\rm{us}}(i,j)=\begin{cases}0,&H_{\rm{us}}(i,j)<0\\\
\frac{H_{\rm{us}}(i,j)}{\max(H_{\rm{us}})},&H_{\rm{us}}(i,j)\geq 0\end{cases}$
(4)

when it was converted to ultrasound images $X_{\rm{us}}$.

Examples of segmentation and ultrasound images are shown in Figure 4. The
segmentation images in the top row were annotated from RGB images. The
ultrasound images, which represent the intensity of the reflected ultrasound
at each pixel, had ambiguous shapes at the positions corresponding to a
person. Furthermore, there were artifacts in the region without any people.

Figure 4: Example of segmentation and ultrasound images. The top row is the
segmentation images genera by RGB images and the bottom row is the ultrasound
images.

##  4 Human segmentation via ultrasound

We first describe an overview of the proposed network. Second, we briefly
explain probabilistic U-Net, which is the basis of the proposed method.
Finally, we describe the proposed network in detail.

###  4.1 Overview

As shown in Fig. 4, the ultrasound images do not have the shape of a person,
and their edges are very different from those of the segmentation images. When
there is a lot of ambiguity in the input image or when the difference between
input and output is large, it is better to use the probabilistic method than
the deterministic method. Thus, we consider performing human segmentation
based on probabilistic models. Kohl et al. [15] proposed a probabilistic U-Net
combining VAE [14] with U-Net. This method learns prior and posterior
networks, which output the latent spaces of input and input/segmentation
images, respectively, to be close. The latent variables sampled from the
latent space of the posterior network are added to the last layer of U-Net.
This method deals with large discrepancies between the input and output images
by sampling latent variables from the latent space output by the prior network
during the inference phase. However, it is difficult to handle with the
existing prior network because the difference in appearance between the
ultrasound and segmentation images is huge. Thus, we propose collaborative
learning probabilistic U-net based on probabilistic U-Net. Details of the
proposed method are described below.

Figure 5: Network architecture. (a) is the conventional probabilistic U-Net
and (b) is the proposed CLPU-Net.

###  4.2 Probabilistic U-Net [15]

In this section, we explain the probabilistic U-Net. The architecture of the
probabilistic U-Net is shown in Figure 5(a). During the inference phase, the
input image $X_{\rm{in}}$ is input into the prior network and U-Net. The prior
network, which is parameterized by $\omega$, encodes the input image and
outputs the parameters of the mean $\mu_{\rm{prior}}(X_{\rm{in}};\omega)$ and
variance $\sigma_{\rm{prior}}(X_{\rm{in}};\omega)$. Then the data $z$ was
randomly sampled from the prior distribution $P$ as

$z\sim
P(\cdot|X_{\rm{in}})=\mathcal{N}\bigl{(}\mu_{\rm{prior}}(X_{\rm{in}};\omega),\rm{diag}(\sigma_{\rm{prior}}(X_{\rm{in}};\omega))\bigr{)},$
(5)

and broadcasted to the N-channel feature map to match the shape to the
segmentation image. The feature maps are concatenated at the last layer of the
U-Net, and the output image is estimated by convoluting the feature maps from
the prior network and U-Net.

During the training phase, the posterior network is added to encode the
information about the segmentation image. The posterior network parameterized
by $\nu$ encodes the data that contains the input image $X_{\rm{in}}$ and
segmentation image $X_{\rm{seg}}$. The parameters of the mean
$\mu_{\rm{post}}(X_{\rm{in}},X_{\rm{seg}};\nu)$ and variance
$\sigma_{\rm{post}}(X_{\rm{in}},X_{\rm{seg}};\nu)$ of the posterior
distribution $Q$ are obtained using the posterior network. Then, the data $z$
is sampled from the posterior distribution $Q$ as

$z\sim
Q(\cdot|X_{\rm{in}},X_{\rm{seg}})=\mathcal{N}\bigl{(}\mu_{\rm{post}}(X_{\rm{in}},X_{\rm{seg}};\nu),\rm{diag}(\sigma_{\rm{post}}(X_{\rm{in}},X_{\rm{seg}};\nu))\bigr{)},$
(6)

and combined to the feature map at the last layer of the U-Net. The loss
function consists of two terms. The first term is cross entropy to minimize
the error between the estimated image and ground truth. The second term is KLD
to close the distribution between the prior distribution $P$ and posterior
distribution $Q$. The losses are combined as a weighted sum,

$L(X_{\rm{seg}},X_{\rm{in}})=\mathbb{E}_{z\sim
Q(\cdot|X_{\rm{seg}},X_{\rm{in}})}\left[-\log
P(X_{\rm{seg}}|X_{\rm{out}}(X_{\rm{in}},z))\right]\\\ +\beta
D_{\rm{KL}}(Q(z|X_{\rm{seg}},X_{\rm{in}})\parallel P(z|X_{\rm{in}})),$

where $X_{\rm{out}}(\cdot)$ outputs the estimated segmentation image and
$\beta$ is the weight parameter.

###  4.3 CLPU-Net

Although the appearance of ultrasound and ground truth segmentation images
differ significantly at the edges, the appearance of the other parts,
particularly positions, is relatively similar. Therefore, it is important to
learn latent space by focusing on the difference in edges. In probabilistic
U-Net, prior distribution $P$ and posterior distribution $Q$ are penalized
using KLD. Reducing the distance between the distributions to estimate edges
with high accuracy by comparing the distributions obtained after the spatial
dimensions decreased at the prior/posterior network. Because the ultrasound
and segmentation images are roughly matched other than edges. Thus, we propose
a method to use mean squared error (MSE) of means and variances instead of
KLD.

The errors between the standard normal distribution calculated by KLD and MSE
are shown in Figure 6. The MSE loss was calculated as

$\rm{MSE}=(\mu-\mu_{0})^{2}+(\sigma-\sigma_{0})^{2},$ (7)

where $\mu_{0}=0$ and $\sigma_{0}=1$ are the mean and variance of the standard
normal distribution. Since the value around the error of 0 for the MSE loss
changes more rapidly than that of the KLD (Fig. 6), the MSE loss is more
sensitive than the KLD. Therefore, converging to an optimal result by KLD is
difficult because of the small gradient of loss landscape, since the
ultrasound and segmentation images are roughly matched other than edges. Thus,
we propose a method that uses MSE loss of the means and variances to penalize
the differences of the latent spaces. Additionally, KLD regularization is
added to the posterior distribution to approach a standard normal
distribution.

Figure 6: Comparison of errors with standard normal distribution by KLD and
MSE. The left figure shows the errors calculated by KLD, and the right figure
shows the errors calculated by MSE.

The proposed network is illustrated in Figure 5(b). The comparison method
between the prior and posterior distributions at the training phase
distinguishes the probabilistic U-Net. The loss function $L$ of the proposed
method is as follows:

$L=\alpha L_{\rm{VAE}}+(1-\alpha)L_{\rm{MSE}},$ (8)

where $\alpha$ is the weights adjusting the scale. The first term
$L_{\rm{VAE}}$ is

$L_{\rm{VAE}}=\mathbb{E}_{z\sim Q(\cdot|X_{\rm{seg}},X_{\rm{us}})}\left[-\log
P(X_{\rm{seg}}|X_{\rm{out}}(X_{\rm{us}},z))\right]\\\ +\beta
D_{\rm{KL}}(P_{0}(z)\parallel Q(z|X_{\rm{seg}},X_{\rm{us}})),$

where $X_{\rm{us}}$ is the ultrasound image and $P_{0}(z)$ is the standard
normal distribution. The second term $L_{\rm{MSE}}$ is

$L_{\rm{MSE}}=\frac{1}{N}\sum_{n=1}^{N}(\mu_{\rm{prior}\it{,n}}-\mu_{\rm{post}\it{,n}})^{2}+\frac{1}{N}\sum_{n=1}^{N}(\sigma_{\rm{prior}\it{,n}}-\sigma_{\rm{post}\it{,n}})^{2},$
(9)

where $N$ is the dimension of the latent vector.

##  5 Experiments

In this section, we describe the dataset configuration, experimental setup,
and evaluation results.

Table 1: Number of images at each condition. Condition number | 1 | 2 | 3 | 4 | 5 | 6 | total
---|---|---|---|---|---|---|---
Number of images |  7,792 |  7,768 |  7,634 |  7,727 |  7,777 |  7,796 |  46,494

###  5.1 Experimental setup

####  5.1.1 Datasets

We created a dataset111 This data acquisition experiment was judged by our
institution to be not the subject of the examination by the Institutional
Review Board. Participants in data acquisition have given their written
consent.  because no datasets have been previously used airborne ultrasound to
detect humans. For $10\,\rm{s}$, we captured the ultrasounds at
$192\,\rm{kHz}$ sampling from $16$ channel microphones and videos at $30$
frames per second (fps) from the RGB camera (a built-in camera of Let’s Note,
CF-SV7, Panasonic), which was located $35\,\rm{mm}$ under the microphone
array. The resolution was $180\times 120$, and the videos were used for
creating segmentation images for the training phase. The data were extracted
at $10$ fps because the time interval of the ultrasound generation was $20$
bursts per second and the frame rate of the video was $30$ fps. We
automatically produced segmentation images using Mask R-CNN [10]. We used the
dataset that people, who were located from $1$ to $3\,\rm{m}$ away from the
sensing devices, performed motions such as standing, sitting, walking, and
running. Examples of the segmentation images were shown in Figure 7. These are
the images for standing, sitting, walking, running, respectively. There were
six participants and they performed in four different rooms. The rooms are
shown in Figure 8. Rooms 1 and 3 were surrounded by relatively acoustically
reflective walls and rooms 2 and 4 were surrounded by relatively acoustically
absorbent materials. The number of images at each condition is shown in Table
1. The condition numbers $1$ to $6$ indicate the participants’ number. Images
were captured about 7,700 images at each condition, the total number was
46,494.

Figure 7: Segmentation images. The motions are standing, sitting, walking, and running in order from the left. Figure 8: Rooms used for data acquisition. Confidential areas are blurred. Table 2: IoU, accuracy, precision, recall, F1-socre of VAE, Joint-VAE, Probabilistic U-Net, and CLPU-Net. Each metric was averaged for all six patterns of the k-fold cross-validation. Model | IoU |  Accuracy |  Precision |  Recall |  F1-score
---|---|---|---|---|---
VAE |  0.265 | 0.880 | 0.351 | 0.526 | 0.403
Joint-VAE | 0.278 | 0.889 | 0.376 | 0.507 | 0.392
Probabilistic U-Net | 0.329 | 0.912 | 0.485 | 0.490 | 0.456
CLPU-Net | 0.388 | 0.921 | 0.536 | 0.546 | 0.519

####  5.1.2 Evaluation

We evaluated the proposed method using k-fold cross-validation. To confirm the
robustness of the unknown person data, the dataset was divided based on the
person, which corresponds to the conditions listed in Tab. 1. Therefore, $k$
was set to six, and all six patterns were trained and evaluated, for example
learning at condition number 2 to 6 and evaluating at condition number 1. The
performance of the model was evaluated using an intersection-over-union (IoU),
accuracy, precision, recall, and F1-score.

####  5.1.3 Implementation details

The prior and posterior networks consisted of four blocks of layers. A block
consisted of three pairs of a 2-dimensional convolution layer and ReLU and an
average pooling layer. The output channels of the four blocks were
$32,64,128,192$ in order. All input images were resized to $128\times 128$
pixels. The dimension of means $\mu_{\rm{prior}},\mu_{\rm{post}}$ and
variances $\sigma_{\rm{prior}},\sigma_{\rm{post}}$ was set to $20$. The batch
size was $16$ and the initial learning rate was $0.0001$. We used an Adam
optimizer with $\beta_{1}=0.9,\beta_{2}=0.999$ in training. $\alpha$ was
$0.0001$ and $\beta$ was $0.3$, respectively, which are determined by a grid
search.

###  5.2 Experimental results

We first describe the performance of the proposed CLPU-Net and compare it with
other methods. Then, we described the comparison of the accuracy by distances
between the sensor and person. Then, we explain the comparison of the accuracy
by rooms. Finally, quantitative results are described.

####  5.2.1 Performance of CLPU-Net

We first evaluated the performance of the proposed CLPU-Net. Then, we compared
our model with probabilistic U-Net, VAE, Joint-VAE [6] to confirm the validity
of learning with ultrasound and segmentation images. The VAE and Joint-VAE
were trained using segmentation images and inferred using ultrasound images.
The CLPU-Net and probabilistic U-Net were trained using ultrasound and
segmentation images and inferred using ultrasound images. Table 2 illustrates
the qualitative results. Each metric was averaged for all six patterns of the
k-fold cross-validation. The proposed CLPU-Net was marked highest in all
metrics of the four models.

Figure 9: Histograms of IoU of CLPU-Net and conventional methods. The
frequency distributions of the data with distances between $1$ m and less than
$2$ m, and the data of distances between $2$ m and less than $3$ m were
displayed.

####  5.2.2 Comparison by distances between sensor and person

The amplitude of the reflected ultrasound decreases as the distance between
the sensors and person increases. Therefore, we investigated the robustness of
the distance between the sensor and a person. Figure 9 shows the IoU
histogram, which categorizes the dataset into two types; distances between $1$
m and less than $2$ m and distances between $2$ m and less than $3$ m. The
horizontal axis represents the IoU values, and the vertical axis represents
the frequency, which means the number of images at each bin. The bin width was
set to $0.01$. The blue bins were the data of distances between $1$ m and less
than $2$ m, and the orange bins were the data of distances between $2$ m and
less than $3$ m. Although the frequencies differ due to the difference in the
number of images for the conditions, the trends in these frequency
distributions were not very different. Therefore, the accuracy is not affected
by the distance within a range of $1$ to $3$ m. We verified that robustness is
sufficient in the range of use because we assume an application will be used
at home.

Figure 10: IoUs of rooms $1$, $2$, $3$, and $4$ of CLPU-Net and conventional
methods. The semi-transparent square represents the IoU of an estimated image,
and the circle with a black line represents the averaged IoU. The blue, green,
orange, and brown represent the IoUs of rooms $1$, $2$, $3$, and $4$,
respectively. Figure 11: Quantitative results. The top row is ultrasound
images, the second to fourth rows are estimated images by conventional
methods, the fifth row is estimated images by CLPU-Net, and the bottom row is
ground truth.

####  5.2.3 Comparison by rooms

There are various acoustic absorption characteristics in rooms assuming an
application will be used at home. Therefore, we investigated the robustness
against the room differences. The IoU for each room is illustrated in Figure
10. The IoU of all estimated images were plotted as semi-transparent squares,
and the averaged IoU was plotted as a circle with a black line. The blue,
green, orange, and brown represent the IoU of rooms $1$, $2$, $3$, and $4$,
respectively. The accuracies of rooms $2$ and $4$, which have a relatively
high acoustic absorption rate, tend to be higher and the accuracies of rooms
$1$ and $3$, which have a relatively low acoustic absorption rate, tend to be
lower. The decrease in accuracy in reverberant rooms seems to be the effect of
higher-order reflected and environmental ultrasounds. The accuracies can be
improved in reverberant rooms by changing the analysis section of ultrasound
waves at preprocessing according to the distance between the sensor and
person.

####  5.2.4 Quantitative result

The quantitative result is shown in Figure 11. In the VAE and Joint-VAE, the
shapes are not properly estimated and there are no people in some estimated
images. In these methods, the information on ultrasound images was excluded at
the training phase, and they cannot estimate segmentation images from
ultrasound images. Therefore, using the information in the segmentation images
at the training phase affects the estimation. Although the probabilistic U-Net
can capture the shapes better than those of the VAE and Joint-VAE, the
estimation fails if the input and segmentation images have a large
discrepancy. Alternatively, the proposed method has been estimated under such
conditions. Some images from CLPU-Net, such as $1$st, $5$th, $8$th, and $12$th
columns from the right, are closer to the ground truth. In contrast, those
images from probabilistic U-Net tend to swell or shrink.

##  6 Conclusions

We proposed privacy-aware human segmentation from airborne ultrasound using
CLPU-Net. Our method used the MSE of the means and variances, which are the
output of the prior and posterior networks, instead of KLD for comparison of
the latent spaces of the prior and posterior networks in probabilistic U-Net.
This enables optimization suitable for the ultrasound image obtained by our
proposed device. This method can be used to detect human actions in situations
where privacy is required, such as home surveillance because the
sound/segmentation images cannot be reconstructed to RGB images. In the
future, we will increase various data and improve the ultrasound image
generation process to improve the accuracy in unknown environments.

## References

  * [1] Inigo Alonso and Ana C. Murillo.  EV-SegNet: Semantic segmentation for event-based cameras.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1624–1633, 2019.
  * [2] Alessio Carullo and Marco Parvis.  An ultrasonic sensor for distance measurement in automotive applications.  IEEE Sensors Journal, 1(2):143–147, 2001.
  * [3] D.E. Chimenti.  Review of air-coupled ultrasonic materials characterization.  Ultrasonics, 54(7):1804–1816, 2014.
  * [4] Amit Das, Ivan Tashev, and Shoaib Mohammed.  Ultrasound based gesture recognition.  In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 406–410, 2017.
  * [5] G.N. Desouza and A.C. Kak.  Vision for mobile robot navigation: a survey.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(2):237–267, 2002.
  * [6] Emilien Dupont.  Learning disentangled joint continuous and discrete representations.  In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems (NeurIPS), volume 31. Curran Associates, Inc., 2018.
  * [7] Yiming Fang, Lujun Lin, Hailin Feng, Zhixiong Lu, and Grant W. Emms.  Review of the use of air-coupled ultrasonic technologies for nondestructive testing of wood and wood products.  Computers and Electronics in Agriculture, 137:79–87, 2017.
  * [8] Biying Fu, Jakob Karolus, Tobias Grosse-Puppendahl, Jonathan Hermann, and Arjan Kuijper.  Opportunities for activity recognition using ultrasound doppler sensing on unmodified mobile phones.  In Proceedings of the 2nd international Workshop on Sensor-based Activity Recognition and Interaction, pages 1–10, 2015.
  * [9] Daniel Halperin, Wenjun Hu, Anmol Sheth, and David Wetherall.  Tool release: Gathering 802.11n traces with channel state information.  SIGCOMM Computer Communication Review, 41(1):53, 2011.
  * [10] Kaiming He, Georgia Gkioxari, Piotr Dollar, and Ross Girshick.  Mask R-CNN.  In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 2961–2969, 2017.
  * [11] Gunpil Hwang, Seohyeon Kim, and Hyeon-Min Bae.  Bat-G net: Bat-inspired high-resolution 3D image reconstruction using ultrasonic echoes.  In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems (NeurIPS), volume 32. Curran Associates, Inc., 2019.
  * [12] Go Irie, Mirela Ostrek, Haochen Wang, Hirokazu Kameoka, Akisato Kimura, Takahito Kawanishi, and Kunio Kashino.  Seeing through sounds: Predicting visual semantic segmentation results from multichannel audio signals.  In International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 3961–3964, 2019.
  * [13] R. Kažys, A. Demčenko, E. Žukauskas, and L. Mažeika.  Air-coupled ultrasonic investigation of multi-layered composite materials.  Ultrasonics, 44:e819–e822, 2006.
  * [14] Diederik P Kingma and Max Welling.  Auto-encoding variational bayes.  arXiv preprint arXiv:1312.6114, 2014.
  * [15] Simon Kohl, Bernardino Romera-Paredes, Clemens Meyer, Jeffrey De Fauw, Joseph R. Ledsam, Klaus Maier-Hein, S. M. Ali Eslami, Danilo Jimenez Rezende, and Olaf Ronneberger.  A probabilistic U-Net for segmentation of ambiguous images.  In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems (NeurIPS), volume 31. Curran Associates, Inc., 2018.
  * [16] Gierad Laput, Karan Ahuja, Mayank Goel, and Chris Harrison.  Ubicoustics: Plug-and-play acoustic activity recognition.  In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, pages 213–224, 2018.
  * [17] Manni Liu, Linsong Cheng, Kun Qian, Jiliang Wang, Jin Wang, and Yunhao Liu.  Indoor acoustic localization: A survey.  Human-centric Computing and Information Sciences, 10, 2020.
  * [18] Shervin Minaee, Yuri Boykov, Fatih Porikli, Antonio Plaza, Nasser Kehtarnavaz, and Demetri Terzopoulos.  Image Segmentation Using Deep Learning: A Survey.  arXiv e-prints, page arXiv:2001.05566, 2020.
  * [19] Marco Moebus and Abdelhak M. Zoubir.  Three-dimensional ultrasound imaging in air using a 2D array on a fixed platform.  In 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP ’07, volume 2, pages II–961–II–964, 2007.
  * [20] Bruna Salles Moreira, Angelo Perkusich, and Saulo OD Luiz.  An acoustic sensing gesture recognition system design based on a hidden markov model.  Sensors, 20(17), 2020.
  * [21] Thomas S Murray, Daniel R Mendat, Kayode A Sanni, Philippe O Pouliquen, and Andreas G Andreou.  Bio-inspired human action recognition with a micro-Doppler sonar system.  IEEE Access, 6:28388–28403, 2017.
  * [22] Kranti Kumar Parida, Siddharth Srivastava, and Gaurav Sharma.  Beyond image to depth: Improving depth prediction using echoes.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8268–8277, 2021.
  * [23] Vincent Perrot, Maxime Polichetti, François Varray, and Damien Garcia.  So you think you can DAS? A viewpoint on delay-and-sum beamforming.  Ultrasonics, 111:106309, 2021.
  * [24] Intisar Rizwan I Haque and Jeremiah Neubert.  Deep learning approaches to biomedical image segmentation.  Informatics in Medicine Unlocked, 18:100297, 2020.
  * [25] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-Net: Convolutional networks for biomedical image segmentation.  In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi, editors, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, pages 234–241, Cham, 2015. Springer International Publishing.
  * [26] Feng Shi, Jun Wang, Jun Shi, Ziyan Wu, Qian Wang, Zhenyu Tang, Kelei He, Yinghuan Shi, and Dinggang Shen.  Review of artificial intelligence techniques in imaging data acquisition, segmentation, and diagnosis for covid-19.  IEEE Reviews in Biomedical Engineering, 14:4–15, 2021.
  * [27] Jae Mun Sim, Yonnim Lee, and Ohbyung Kwon.  Acoustic sensor based recognition of human activity in everyday life for smart home services.  International Journal of Distributed Sensor Networks, 11(9), 2015\.
  * [28] Arun Balajee Vasudevan, Dengxin Dai, and Luc Van Gool.  Semantic object prediction and spatial sound super-resolution with binaural sounds.  In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, Computer Vision – ECCV 2020, pages 638–655, Cham, 2020. Springer International Publishing.
  * [29] Fei Wang, Sanping Zhou, Stanislav Panev, Jinsong Han, and Dong Huang.  Person-in-WiFi: Fine-grained person perception using WiFi.  In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 5451–5460, 2019.
  * [30] W. Wang, Y. Song, J. Zhang, and H. Deng.  Automatic parking of vehicles: A review of literatures.  International Journal of Automotive Technology, 15(6):967–978, 2014\.
  * [31] Linlin Xia, Jiashuo Cui, Ran Shen, Xun Xu, Yiping Gao, and Xinying Li.  A survey of image semantics-based visual simultaneous localization and mapping: Application-oriented solutions to autonomous navigation of mobile robots.  International Journal of Advanced Robotic Systems, 17(3):1729881420919185, 2020.
  * [32] Yadong Xie, Fan Li, Yue Wu, and Yu Wang.  HearFit: Fitness monitoring on smart speakers via active acoustic sensing.  In IEEE INFOCOM 2021-IEEE Conference on Computer Communications, pages 1–10. IEEE, 2021.
  * [33] Koji Yatani and Khai N Truong.  Bodyscope: A wearable acoustic sensor for activity recognition.  In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, pages 341–350, 2012.
  * [34] Hong-Bo Zhang, Yi-Xiang Zhang, Bineng Zhong, Qing Lei, Lijie Yang, Ji-Xiang Du, and Duan-Sheng Chen.  A comprehensive survey of vision-based human action recognition methods.  Sensors, 19(5), 2019.


# Asymptotic Stability and Strict Passivity of Port-Hamiltonian Descriptor
Systems
via State Feedback

Delin Chu111Department of Mathematics, National University of Singapore,
Singapore 119076. Email<EMAIL_ADDRESS>Volker Mehrmann222Institute of
Mathematics, TU Berlin, Straße des 17. Juni 136, 10623 Berlin, Germany. Email:
<EMAIL_ADDRESS>
###### Abstract

While port-Hamiltonian descriptor systems are known to be stable and passive,
they may not be asymptotically stable or strictly passive. Necessary and
sufficient conditions are presented when these properties as well as the
regularity and the index one property can be achieved via state feedback while
preserving the port-Hamiltonian structure.

Keywords: Port-Hamiltonian descriptor system, proportional state feedback,
regularity, index reduction, asymptotic stability, strict passivity.

AMS subject classification: 93B05, 93B40, 93B52, 65F35

##  1 Introduction

In this paper we consider linear time-invariant port-Hamiltonian (pH)
descriptor systems of the form

$\displaystyle E{\dot{x}}$ $\displaystyle=$ $\displaystyle(J-R)x+(G-P)u,$ (1)
$\displaystyle y$ $\displaystyle=$ $\displaystyle(G+P)^{T}x+(S+N)u,$

where $E,J,R\in{\mathbb{R}}^{n\times n}$, $G,P\in{\mathbb{R}}^{n\times m}$,
$D=S+N\in{\mathbb{R}}^{m\times m}$ ($S=S^{T}=\frac{1}{2}(D+D^{T})$,
$N=-N^{T}=\frac{1}{2}(D-D^{T})$), and the functions $x,y,u$ are the state,
output, and input of the system. The coefficient matrices satisfy

$E=E^{T}\geq 0,\ \left[\begin{array}[]{cc}J&G\\\
-G^{T}&N\end{array}\right]=-\left[\begin{array}[]{cc}J&G\\\
-G^{T}&N\end{array}\right]^{T},$ $W=W^{T}=\left[\begin{array}[]{cc}R&P\\\
P^{T}&S\end{array}\right]\geq 0,$ (2)

where for a matrix $M$, $M\geq 0$ $(M>0)$ denotes that $M$ is positive
semidefinite (definite) and ${\mathbb{R}}^{n\times m}$ denotes the real
$n\times m$ matrices. As usual in system theory, we analyze the stability
properties of the system (1) via the eigenstructure of the associated
_dissipative Hamiltonian matrix pencil_ $sE-(J-R)$.

The class of pH descriptor systems represents an ideal modeling framework for
control systems in almost all real world physical systems, see [2, 7, 11, 13,
12, 14, 15, 16] for a large number of applications and a detailed discussion
of the many important properties. A characteristic property of systems of the
form (1) is the availability of a _storage function (Hamiltonian)_ which is
given by $\mathcal{H}(x)=\frac{1}{2}x^{T}Ex\geq 0$ and that the following
power balance equation holds, as well as the resulting dissipation inequality,
see e.g. [11].

######  Theorem 1.

Consider a pH descriptor system of the form (1). Then for any input function
$u$, the _power balance equation_

$\frac{d}{dt}\mathcal{H}(x=-\begin{bmatrix}x\\\
u\end{bmatrix}^{T}\left[\begin{array}[]{cc}R&P\\\
P^{T}&S\end{array}\right]\begin{bmatrix}x\\\ u\end{bmatrix}+y^{T}u$ (3)

holds along any solution $x$. In particular, the _dissipation inequality_

$\mathcal{H}(x(t_{2}))-\mathcal{H}(x(t_{1}))\leq\int_{t_{1}}^{t_{2}}(y(\tau)^{T}u(\tau))\
d\tau$ (4)

holds.

The dissipation inequality (4) immediately implies that the system is
_passive_ , see [3, 17], but since the inequality in (4) may not be strict,
the system may not be _strictly passive_.

Similarly, it is well known, see e.g. [9], that pH descriptor systems of the
form (1) are _stable_ , in the sense that all finite eigenvalues of $sE-(J-R)$
are in the closed left half plane and those on the imaginary axis are semi-
simple. But since purely imaginary eigenvalues may occur, pH descriptor
systems are not necessarily _asymptotically stable_.

Furthermore, it is shown in [9] that the _index_ of a pH descriptor system,
i.e. the size of the largest Jordan block associated with the eigenvalue
$\infty$ of $sE_{(}J-R)$, can be at most $2$, and it is shown in [10] that the
_singularity_ of the pencil $sE-(J-R)$, i.e. that $\det(sE-(J-R))$ is
identically $0$, is characterized by a common nullspace of the coefficients
$E,J,R$.

It is well known [8] that neither (asymptotic) stability nor (strict)
passivity and also neither the regularity and the index of a descriptor system
are invariant under feedback. On the contrary, typically feedback is used to
make an unstable or non-passive system asymptotically stable, respectively
stritcly passive and to make a descriptor system regular and of index at most
one, [5].

The goal of this paper, is therefore, to study the question whether a pH
descriptor system can be made regular, of index at most one, and furthermore
strictly passive, respectively asymptotically stable, by applying a state
feedback $u=Fx+v$ with $F\in{\mathbb{R}}^{m\times n}$ to (1) while preserving
the pH structure.

Applying such a state feedback, the closed-loop system has the form

$\displaystyle E\dot{x}$ $\displaystyle=$
$\displaystyle[J-R+(G-P)F]x+(G-P)v=(\tilde{J}-\tilde{R})x+(\tilde{G}-\tilde{P})v,$
(5) $\displaystyle y$ $\displaystyle=$
$\displaystyle(G^{T}+P^{T}+(S+N)F)x+(S+N)u=(\tilde{G}+\tilde{P})^{T}x+(S+N)v,$

where

$\displaystyle\tilde{J}$ $\displaystyle=$ $\displaystyle J+{1\over
2}[(G-P)F-F^{T}(G-P)^{T}],$ $\displaystyle\tilde{R}$ $\displaystyle=$
$\displaystyle R-{1\over 2}[(G-P)F+F^{T}(G-P)^{T}],$ $\displaystyle\tilde{G}$
$\displaystyle=$ $\displaystyle G+{1\over 2}F^{T}(S+N)^{T},\
\tilde{P}=P+{1\over 2}F^{T}(S+N)^{T}.$

We present a complete characterization for the following two questions.

  * Question 1: When does there exist a state feedback matrix $F$ such that the closed-loop system (5) associated with (1) is port-Hamiltonian, regular, of index at most one, and asymptotically stable.

  * Question 2: When does there exists a state feedback matrix $F$ such that the closed-loop system (5) associated with (1) is port-Hamiltonian, regular, of index at most one, and strictly passive, i.e., the dissipation matrix of the closed loop system satisfies

$\tilde{W}=\tilde{W}^{T}=\left[\begin{array}[]{cc}\tilde{R}&\tilde{P}\\\
\tilde{P}^{T}&S\end{array}\right]>0.$ (6)

The paper is organized as follows. In Section 2 we present some preliminary
results. In Section 3 we present the answers to the two questions and we
finish with some conclusions and open problems in Section 4.

##  2 Preliminaries

In this section we introduce some notation and present some preliminary
results.

We denote the open left (right) half complex plane by
${\mathbb{C}}_{-}({\mathbb{C}}_{+})$ and the imaginary axis by
${\mathbb{C}}_{0}$. A column orthogonal matrix with its columns spanning the
right nullspace of a matrix $M$ is denoted by ${\cal S}_{\infty}(M)$ and with
its columns spanning the range space of $M$ by ${\cal R}(M)$. The Moore-
Penrose inverse of a matrix $M$ is denoted by $M^{(+)}$ and the unique
positive definite _square root_ of a matrix $M=M^{T}>0$ by $M^{\frac{1}{2}}$.

We have the following elementary lemmas.

######  Lemma 2.

[4] Consider $E,J,R\in{\mathbb{R}}^{n,n}$ with $E=E^{T}\geq 0$, $J=-J^{H}$,
$R=R^{T}\geq 0$, partitioned in blocks as

$E=\hbox{}\;\vbox{\kern 26.29883pt\hbox{$\kern
50.91896pt\kern-8.75002pt\left[\kern-50.91896pt\vbox{\kern-26.29883pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&n_{1}&n_{2}\crcr\kern 2.0pt\cr n_{1}$\hfil\kern
2.0pt\kern 8.75002pt&E_{11}&E_{12}\cr n_{2}$\hfil\kern 2.0pt\kern
8.75002pt&E_{12}^{T}&E_{22}\cr\cr}}\kern-12.0pt}\,\right]$}},\quad
R=\hbox{}\;\vbox{\kern 23.99663pt\hbox{$\kern
45.87483pt\kern-8.75002pt\left[\kern-45.87483pt\vbox{\kern-23.99663pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&n_{1}&n_{2}\crcr\kern 2.0pt\cr n_{1}$\hfil\kern
2.0pt\kern 8.75002pt&R_{11}&0\cr n_{2}$\hfil\kern 2.0pt\kern
8.75002pt&0&0\cr\cr}}\kern-12.0pt}\,\right]$}},\quad J=\hbox{}\;\vbox{\kern
26.29883pt\hbox{$\kern
53.57172pt\kern-8.75002pt\left[\kern-53.57172pt\vbox{\kern-26.29883pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&n_{1}&n_{2}\crcr\kern 2.0pt\cr n_{1}$\hfil\kern
2.0pt\kern 8.75002pt&J_{11}&J_{12}\cr n_{2}$\hfil\kern 2.0pt\kern
8.75002pt&-J_{12}^{T}&J_{22}\cr\cr}}\kern-12.0pt}\,\right]$}},$

where $R_{11}>0$. Then the following statements hold:

(i) The matrix pencil $sE-(J-R)$ has all its finite eigenvalues in the open
left half complex plane if and only if

$\mathop{\rm
rank}\nolimits\left[\begin{array}[]{cc}sE_{12}^{T}+J_{12}^{T}&sE_{22}-J_{22}\end{array}\right]=n_{2}\
\mbox{{f}or all }\ s\in{\mathbb{C}}_{0}.$ (7)

(ii) the matrix $J-R$ is nonsingular if and only if

$\mathop{\rm
rank}\nolimits\left[\begin{array}[]{cc}-J_{12}^{T}&J_{22}\end{array}\right]=n_{2}.$

A simple characterization when a pH descriptor system can be made
asymptotically stable via state feedback is given by the following lemma.

######  Lemma 3.

Consider $E,J,R\in{\mathbb{R}}^{n\times n}$ and $B\in{\mathbb{R}}^{n\times
m}$, with $E=E^{T}\geq 0$, $R=R^{T}\geq 0$ and $J=-J^{T}$, and suppose that

$\mathop{\rm
rank}\nolimits\left[\begin{array}[]{cc}sE-(J-R)&B\end{array}\right]=n\
\mbox{{f}or all }\ s\in{\mathbb{C}}_{0}.$ (8)

Let $F\in{\mathbb{R}}^{n\times m}$ be such that $\tilde{R}:=R-{1\over
2}(BF+F^{T}B^{T})\geq 0$ and

$\mathop{\rm rank}\nolimits\tilde{R}=\mathop{\rm
rank}\nolimits\left[\begin{array}[]{cc}R&B\end{array}\right].$ (9)

Then the pencil $sE-(J-R+BF)$ has all its eigenvalues in ${\mathbb{C}}_{-}$.

###### Proof.

By first performing an echelon form of $B$ and then compressing $R$ in the
part associated with the left nullspace of $B$, there exist nonsingular
matrices $X$ and $Y$ such that

$XRX^{T}=\hbox{}\;\vbox{\kern 32.63438pt\hbox{$\kern
64.6772pt\kern-8.75002pt\left[\kern-64.6772pt\vbox{\kern-32.63438pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&n_{1}&n_{2}&n_{3}\crcr\kern 2.0pt\cr n_{1}$\hfil\kern
2.0pt\kern 8.75002pt&R_{11}&0&0\cr n_{2}$\hfil\kern 2.0pt\kern
8.75002pt&0&I&0\cr n_{3}$\hfil\kern 2.0pt\kern
8.75002pt&0&0&0\cr\cr}}\kern-12.0pt}\,\right]$}},\quad
XBY^{T}=\hbox{}\;\vbox{\kern 32.24551pt\hbox{$\kern
57.96483pt\kern-8.75002pt\left[\kern-57.96483pt\vbox{\kern-32.24551pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&n_{1}&m-n_{1}\crcr\kern 2.0pt\cr n_{1}$\hfil\kern
2.0pt\kern 8.75002pt&I&0\cr n_{2}$\hfil\kern 2.0pt\kern 8.75002pt&0&0\cr
n_{3}$\hfil\kern 2.0pt\kern 8.75002pt&0&0\cr\cr}}\kern-12.0pt}\,\right]$}}.$

Note that (due to the congruence transformation) $R_{11}$ is still positive
semidefinite. Transforming and partitioning the other matrices accordingly as

$XEX^{T}=\hbox{}\;\vbox{\kern 36.84991pt\hbox{$\kern
74.4773pt\kern-8.75002pt\left[\kern-74.4773pt\vbox{\kern-36.84991pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&n_{1}&n_{2}&n_{3}\crcr\kern 2.0pt\cr n_{1}$\hfil\kern
2.0pt\kern 8.75002pt&E_{11}&E_{12}&E_{13}\cr n_{2}$\hfil\kern 2.0pt\kern
8.75002pt&E_{12}^{T}&E_{22}&E_{23}\cr n_{3}$\hfil\kern 2.0pt\kern
8.75002pt&E_{13}^{T}&E_{23}^{T}&E_{33}\cr\cr}}\kern-12.0pt}\,\right]$}},\quad
XJX^{T}=\hbox{}\;\vbox{\kern 36.84991pt\hbox{$\kern
81.23418pt\kern-8.75002pt\left[\kern-81.23418pt\vbox{\kern-36.84991pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&n_{1}&n_{2}&n_{3}\crcr\kern 2.0pt\cr n_{1}$\hfil\kern
2.0pt\kern 8.75002pt&J_{11}&J_{12}&J_{13}\cr n_{2}$\hfil\kern 2.0pt\kern
8.75002pt&-J_{12}^{T}&J_{22}&J_{23}\cr n_{3}$\hfil\kern 2.0pt\kern
8.75002pt&-J_{13}^{T}&-J_{23}^{T}&J_{33}\cr\cr}}\kern-12.0pt}\,\right]$}},$

and

$Y^{-T}FX^{T}=\hbox{}\;\vbox{\kern 24.3855pt\hbox{$\kern
90.61853pt\kern-8.75002pt\left[\kern-90.61853pt\vbox{\kern-24.3855pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&n_{1}&n_{2}&n_{3}\crcr\kern 2.0pt\cr n_{1}$\hfil\kern
2.0pt\kern 8.75002pt&F_{11}&F_{12}&F_{13}\cr m-n_{1}$\hfil\kern 2.0pt\kern
8.75002pt&F_{21}&F_{22}&F_{23}\cr\cr}}\kern-12.0pt}\,\right]$}},$

it follows from condition (9) that $F_{13}=0$ and

$\left[\begin{array}[]{cc}R_{11}-{1\over 2}(F_{11}+F_{11}^{T})&-{1\over
2}F_{12}\\\ -{1\over 2}F_{12}^{T}&I\end{array}\right]>0.$

Then

$X\tilde{J}X^{T}:=X(J+{1\over
2}(BF-F^{T}B^{T}))X^{T}=\left[\begin{array}[]{ccc}J_{11}+{1\over
2}(F_{11}-F_{11}^{T})&J_{12}+{1\over 2}F_{12}&J_{13}\\\ -J_{12}^{T}-{1\over
2}F_{12}^{T}&J_{22}&J_{23}\\\
-J_{13}^{T}&-J_{23}^{T}&J_{33}\end{array}\right],$

and condition (8) implies that

$\mathop{\rm
rank}\nolimits\left[\begin{array}[]{ccc}sE_{13}^{T}+J_{13}^{T}&sE_{23}^{T}+J_{23}^{T}&sE_{33}-J_{33}\end{array}\right]=n_{3}\
\mbox{{f}or all }\ s\in{\mathbb{C}}_{0}.$

Therefore, using Lemma 2 (i) we have that the pencil $sXEX^{T}-X(J-R+BF)X^{T}$
and therefore also the equivalent pencil $sE-(J-R+BF)$ has all its finite
eigenvalues in ${\mathbb{C}}_{-}$.

To analyze when a system is regular and of index at most one we will use the
following Lemma, see e.g. [5, 6] for an analogous result for general
descriptor systems.

######  Lemma 4.

Given $E,J,R\in{\mathbb{R}}^{n\times n}$, and $B\in{\mathbb{R}}^{n\times m}$
with $E=E^{T}\geq 0$, $R=R^{T}\geq 0$ and $J=-J^{T}$. Suppose that

$\mathop{\rm rank}\nolimits\left[\begin{array}[]{ccc}E&(J-R){\cal
S}_{\infty}(E)&B\end{array}\right]=n,$ (10)

and let $F\in{\mathbb{R}}^{m\times n}$ be such that $\tilde{R}:=R-{1\over
2}(BF+F^{T}B^{T})\geq 0$ and

$\mathop{\rm rank}\nolimits(\tilde{R})=\mathop{\rm
rank}\nolimits\left[\begin{array}[]{cc}R&B\end{array}\right].$ (11)

Then the pencil $sE-(J-R+BF)$ is regular and of index at most one.

###### Proof.

By first compressing the matrix $E$ and then the matrix $R$ in the nullspace
of $E$, using the semidefiniteness of $E$ and $R$, there exists a nonsingular
matrix $X$ such that

$XEX^{T}=\hbox{}\;\vbox{\kern 32.24551pt\hbox{$\kern
64.96536pt\kern-8.75002pt\left[\kern-64.96536pt\vbox{\kern-32.24551pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&n_{1}&n_{2}&n_{3}\crcr\kern 2.0pt\cr n_{1}$\hfil\kern
2.0pt\kern 8.75002pt&E_{11}&0&0\cr n_{2}$\hfil\kern 2.0pt\kern
8.75002pt&0&0&0\cr n_{3}$\hfil\kern 2.0pt\kern
8.75002pt&0&0&0\cr\cr}}\kern-12.0pt}\,\right]$}},\quad
X\tilde{R}X^{T}=\hbox{}\;\vbox{\kern 33.41222pt\hbox{$\kern
64.91585pt\kern-8.75002pt\left[\kern-64.91585pt\vbox{\kern-33.41222pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&n_{1}&n_{2}&n_{3}\crcr\kern 2.0pt\cr n_{1}$\hfil\kern
2.0pt\kern 8.75002pt&\tilde{R}_{11}&0&0\cr n_{2}$\hfil\kern 2.0pt\kern
8.75002pt&0&\tilde{R}_{22}&0\cr n_{3}$\hfil\kern 2.0pt\kern
8.75002pt&0&0&0\cr\cr}}\kern-12.0pt}\,\right]$}},$

where $E_{11}$ and $\tilde{R}_{22}$ are positive definite. Set, with analogous
partitioning,

$XB=\hbox{}\;\vbox{\kern 26.91327pt\hbox{$\kern
24.68912pt\kern-8.75002pt\left[\kern-24.68912pt\vbox{\kern-26.91327pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&\crcr\kern 2.0pt\cr n_{1}$\hfil\kern 2.0pt\kern
8.75002pt&B_{1}\cr n_{2}$\hfil\kern 2.0pt\kern 8.75002pt&B_{2}\cr
n_{3}$\hfil\kern 2.0pt\kern 8.75002pt&B_{3}\cr\cr}}\kern-12.0pt}\,\right]$}},\
XRX^{T}=\hbox{}\;\vbox{\kern 36.84991pt\hbox{$\kern
73.61281pt\kern-8.75002pt\left[\kern-73.61281pt\vbox{\kern-36.84991pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&n_{1}&n_{2}&n_{3}\crcr\kern 2.0pt\cr n_{1}$\hfil\kern
2.0pt\kern 8.75002pt&R_{11}&R_{12}&R_{13}\cr n_{2}$\hfil\kern 2.0pt\kern
8.75002pt&R_{12}^{T}&R_{22}&R_{23}\cr n_{3}$\hfil\kern 2.0pt\kern
8.75002pt&R_{13}^{T}&R_{23}^{T}&R_{33}\cr\cr}}\kern-12.0pt}\,\right]$}},\
XJX^{T}=\hbox{}\;\vbox{\kern 36.84991pt\hbox{$\kern
81.23418pt\kern-8.75002pt\left[\kern-81.23418pt\vbox{\kern-36.84991pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&n_{1}&n_{2}&n_{3}\crcr\kern 2.0pt\cr n_{1}$\hfil\kern
2.0pt\kern 8.75002pt&J_{11}&J_{12}&J_{13}\cr n_{2}$\hfil\kern 2.0pt\kern
8.75002pt&-J_{12}^{T}&J_{22}&J_{23}\cr n_{3}$\hfil\kern 2.0pt\kern
8.75002pt&-J_{13}^{T}&-J_{23}^{T}&J_{33}\cr\cr}}\kern-12.0pt}\,\right]$}}.$

Condition (11) then implies that $B_{3}=0$. Therefore, also $R_{33}=0$. Since
$R\geq 0$ and $\tilde{R}\geq 0$, it follows that

$R_{13}=0,\ R_{23}=0,\ B_{1}F_{3}=0,\ B_{2}F_{3}=0.$

Transforming and partitioning $\tilde{J}=J+{1\over 2}(BF-F^{T}B^{T})$ as

$X\tilde{J}X^{T}=\hbox{}\;\vbox{\kern 37.62775pt\hbox{$\kern
81.23418pt\kern-8.75002pt\left[\kern-81.23418pt\vbox{\kern-37.62775pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&n_{1}&n_{2}&n_{3}\crcr\kern 2.0pt\cr n_{1}$\hfil\kern
2.0pt\kern 8.75002pt&\tilde{J}_{11}&\tilde{J}_{12}&J_{13}\cr n_{2}$\hfil\kern
2.0pt\kern 8.75002pt&-\tilde{J}_{12}^{T}&\tilde{J}_{22}&J_{23}\cr
n_{3}$\hfil\kern 2.0pt\kern
8.75002pt&-J_{13}^{T}&-J_{23}^{T}&J_{33}\cr\cr}}\kern-12.0pt}\,\right]$}},$

using the condition (10) it follows that

$\mathop{\rm
rank}\nolimits\left[\begin{array}[]{cc}-J_{23}^{T}&J_{33}\end{array}\right]=n_{3}.$

Hence, using Lemma 2 (ii), we have that

$\left[\begin{array}[]{cc}\tilde{J}_{22}-\tilde{R}_{22}&J_{23}\\\
-J_{23}^{T}&J_{33}\end{array}\right]$

is nonsingular and thus the pencil $sXEX^{T}-(X(J-R+BF)X^{T})$ is regular and
of index at most one, or equivalently, the pencil $sE-(J-R+BF)$ is regular and
of index at most one.

Using these preliminary lemmas, in the next section we present our main
results.

##  3 Main Results

In this section we present the answers to the two questions when a pH
descriptor system can be made regular, index at most one and asymptotically
stable as well as strictly passive. We present proofs that can be implemented
in the construction of the desired feedback and we begin with Question 1.

######  Theorem 5.

Consider a pH descriptor system of the form (1). There exists a matrix
$F\in{\mathbb{R}}^{m\times n}$ such that closed loop system (5) is port-
Hamiltonian,and the pencil $sE-(\tilde{J}-\tilde{R})$ is regular, of index at
most one, and has all its finite eigenvalues in ${\mathbb{C}}_{-}$ if and only
if the following two conditions hold:

$\displaystyle\mathop{\rm
rank}\nolimits\left[\begin{array}[]{ccc}sE-(J-R)&(G-P)(S+N)^{(+)}{\cal
R}(S)&(G-P){\cal S}_{\infty}(S+N)\end{array}\right]=n$ (13)
$\displaystyle\hskip 170.71652pt\mbox{{f}or all}\ s\in{\mathbb{C}}_{0},$ (14)

and

$\mathop{\rm rank}\nolimits\left[\begin{array}[]{cccc}E&(J-R){\cal
S}_{\infty}(E)&(G-P)(S+N)^{(+)}{\cal R}(S)&(G-P){\cal
S}_{\infty}(S+N)\end{array}\right]=n.$ (15)

###### Proof.

System (1) is port-Hamiltonian, so, (2) holds. It follows from $S\geq 0$, see
e.g. [1], that there exists an orthogonal matrix $U$ such that

$U^{T}(S+N)U=\hbox{}\;\vbox{\kern 34.54771pt\hbox{$\kern
82.02966pt\kern-8.75002pt\left[\kern-82.02966pt\vbox{\kern-34.54771pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&m_{1}&m_{2}&m_{3}\crcr\kern 2.0pt\cr m_{1}$\hfil\kern
2.0pt\kern 8.75002pt&D_{11}&D_{12}&0\cr m_{2}$\hfil\kern 2.0pt\kern
8.75002pt&-D_{12}^{T}&D_{22}&0\cr m_{3}$\hfil\kern 2.0pt\kern
8.75002pt&0&0&0\cr\cr}}\kern-12.0pt}\,\right]$}},\quad
U^{T}SU=\hbox{}\;\vbox{\kern 32.24551pt\hbox{$\kern
72.04872pt\kern-8.75002pt\left[\kern-72.04872pt\vbox{\kern-32.24551pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&m_{1}&m_{2}&m_{3}\crcr\kern 2.0pt\cr m_{1}$\hfil\kern
2.0pt\kern 8.75002pt&S_{11}&0&0\cr m_{2}$\hfil\kern 2.0pt\kern
8.75002pt&0&0&0\cr m_{3}$\hfil\kern 2.0pt\kern
8.75002pt&0&0&0\cr\cr}}\kern-12.0pt}\,\right]$}},$ (16)

where

$D_{22}^{T}=-D_{22},\quad\mathop{\rm
rank}\nolimits\left[\begin{array}[]{cc}D_{11}&D_{12}\\\
-D_{12}^{T}&D_{22}\end{array}\right]=m_{1}+m_{2},\quad\mathop{\rm
rank}\nolimits(S_{11})=m_{1},\quad S_{11}>0.$ (17)

Partitioning the resulting matrices as

$\displaystyle U=\hbox{}\;\vbox{\kern 15.74774pt\hbox{$\kern
59.7404pt\kern-8.75002pt\left[\kern-59.7404pt\vbox{\kern-15.74774pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&m_{1}&m_{2}&m_{3}\crcr\kern 2.0pt\cr$\hfil\kern
2.0pt\kern 8.75002pt&U_{1}&U_{2}&U_{3}\cr\cr}}\kern-12.0pt}\,\right]$}},\
PU=\hbox{}\;\vbox{\kern 15.74774pt\hbox{$\kern
59.7404pt\kern-8.75002pt\left[\kern-59.7404pt\vbox{\kern-15.74774pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&m_{1}&m_{2}&m_{3}\crcr\kern 2.0pt\cr$\hfil\kern
2.0pt\kern 8.75002pt&P_{1}&P_{2}&P_{3}\cr\cr}}\kern-12.0pt}\,\right]$}},$
$\displaystyle(G-P)U\left[\begin{array}[]{ccc}D_{11}&D_{12}&0\\\
-D_{12}^{T}&D_{22}&0\\\ 0&0&I\end{array}\right]^{-1}=\hbox{}\;\vbox{\kern
15.74774pt\hbox{$\kern
59.7404pt\kern-8.75002pt\left[\kern-59.7404pt\vbox{\kern-15.74774pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&m_{1}&m_{2}&m_{3}\crcr\kern 2.0pt\cr$\hfil\kern
2.0pt\kern 8.75002pt&B_{1}&B_{2}&B_{3}\cr\cr}}\kern-12.0pt}\,\right]$}},$

we have

$\displaystyle U_{3}$ $\displaystyle=$ $\displaystyle{\cal
S}_{\infty}(S+N),\quad U_{1}={\cal R}(S),$ $\displaystyle B_{1}$
$\displaystyle=$ $\displaystyle(G-P)(S+N)^{(+)}U_{1}=(G-P)(S+N)^{(+)}{\cal
R}(S),$ $\displaystyle B_{3}$ $\displaystyle=$
$\displaystyle(G-P)U_{3}=(G-P){\cal S}_{\infty}(S+N).$

Moreover, by (2) we have $P_{2}=0$ and $P_{3}=0$.

Partition also a transformed feedback matrix $F\in{\mathbb{R}}^{m\times n}$ as

$\left[\begin{array}[]{ccc}D_{11}&D_{12}&0\\\ -D_{12}^{T}&D_{22}&0\\\
0&0&I\end{array}\right]U^{T}F=\hbox{}\;\vbox{\kern 26.91327pt\hbox{$\kern
27.19955pt\kern-8.75002pt\left[\kern-27.19955pt\vbox{\kern-26.91327pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&\crcr\kern 2.0pt\cr m_{1}$\hfil\kern 2.0pt\kern
8.75002pt&F_{1}\cr m_{2}$\hfil\kern 2.0pt\kern 8.75002pt&F_{2}\cr
m_{3}$\hfil\kern 2.0pt\kern 8.75002pt&F_{3}\cr\cr}}\kern-12.0pt}\,\right]$}}.$

We first show the necessity of conditions (14) and (15). If the closed-loop
system (5) is a pH descriptor system, then

$\left[\begin{array}[]{cc}\tilde{R}&\tilde{P}\\\
\tilde{P}^{T}&S\end{array}\right]=\left[\begin{array}[]{cc}R-{1\over
2}[(G-P)F+F^{T}(G-P)^{T}]&P+{1\over 2}F^{T}(S+N)^{T}\\\ P^{T}+{1\over
2}(S+N)F&S\end{array}\right]\geq 0$

which in the partitioned form implies that

$\left[\begin{array}[]{cccc}R-{1\over
2}[B_{1}F_{1}+B_{2}F_{2}+B_{3}F_{3}+(B_{1}F_{1}+B_{2}F_{2}+B_{3}F_{3})^{T}]&P_{1}+{1\over
2}F_{1}^{T}&{1\over 2}F_{2}^{T}&0\\\ P_{1}^{T}+{1\over
2}F_{1}&S_{11}&0&0\cr{1\over 2}F_{2}&0&0&0\cr 0&0&0&0\end{array}\right]\geq
0.$

Thus $F_{2}=0$ and

$\left[\begin{array}[]{cc}R-{1\over
2}[B_{1}F_{1}+B_{3}F_{3}+(B_{1}F_{1}+B_{3}F_{3})^{T}]&P_{1}+{1\over
2}F_{1}^{T}\\\ P_{1}^{T}+{1\over 2}F_{1}&S_{11}\end{array}\right]\geq 0.$

It follows that

$(G-P)F=\left[\begin{array}[]{cc}B_{1}&B_{3}\end{array}\right]\left[\begin{array}[]{cc}F_{1}\\\
F_{3}\end{array}\right].$

Since the pencil $sE-(J-R+(G-P)F)$ is regular and of index at most one, and
also has all finite eigenvalues in ${\mathbb{C}}_{-}$, it follows that

$\mathop{\rm
rank}\nolimits\left[\begin{array}[]{ccc}sE-(J-R)&B_{1}&B_{3}\end{array}\right]=n\
\mbox{{f}or all}\ s\in{\mathbb{C}}_{0},$ (20)

and

$\mathop{\rm rank}\nolimits\left[\begin{array}[]{cccc}E&(J-R){\cal
S}_{\infty}(E)&B_{1}&B_{3}\end{array}\right]=n,$ (21)

i.e., the conditions (14) and (15) are satisfied.

To show the sufficiency, assume that conditions (14) and (15) hold, or
equivalently that the conditions (20) and (21) hold. Since

$\left[\begin{array}[]{cc}R&P\\\ P^{T}&S\end{array}\right]\geq 0,$

also

$\left[\begin{array}[]{cc}I&0\\\
0&U\end{array}\right]^{T}\left[\begin{array}[]{cc}R&P\\\
P^{T}&S\end{array}\right]\left[\begin{array}[]{cc}I&0\\\
0&U\end{array}\right]\geq 0,$

which gives

$\left[\begin{array}[]{cc}R&P_{1}\\\ P_{1}^{T}&S_{11}\end{array}\right]\geq
0,$

and thus

$\displaystyle\left[\begin{array}[]{cc}R+B_{1}P_{1}^{T}+P_{1}B_{1}^{T}+B_{1}S_{11}B_{1}^{T}&P_{1}+B_{1}S_{11}\\\
P_{1}^{T}+S_{11}B_{1}^{T}&S_{11}\end{array}\right]$ $\displaystyle=$
$\displaystyle\left[\begin{array}[]{cc}I&B_{1}\\\
0&I\end{array}\right]\left[\begin{array}[]{cc}R&P_{1}\\\
P_{1}^{T}&S_{11}\end{array}\right]\left[\begin{array}[]{cc}I&B_{1}\\\
0&I\end{array}\right]^{T}\geq 0.$

We obtain

$R+B_{1}P_{1}^{T}+P_{1}B_{1}^{T}+B_{1}S_{11}B_{1}^{T}\geq 0,$ (24)

and can construct a feedback matrix via

$F_{2}=0,\quad F_{1}=-2(B_{1}S_{11}+P_{1})^{T}.$

Then

$\displaystyle\tilde{R}$ $\displaystyle=$ $\displaystyle R-{1\over
2}(G-P)F-{1\over 2}F^{T}(G-P)^{T}$ $\displaystyle=$ $\displaystyle R-{1\over
2}B_{3}F_{3}-{1\over
2}F_{3}^{T}B_{3}^{T}+(B_{1}P_{1}^{T}+P_{1}B_{1}^{T}+2B_{1}S_{11}B_{1}^{T}).$

Since $R\geq 0$, we can determine a nonsingular matrix $Z$ and orthogonal
matrices $V_{1}$ and $V_{3}$ such that

$\displaystyle ZB_{3}V_{3}$ $\displaystyle=$
$\displaystyle\hbox{}\;\vbox{\kern 41.1944pt\hbox{$\kern
58.61206pt\kern-8.75002pt\left[\kern-58.61206pt\vbox{\kern-41.1944pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&\mu_{1}&m_{3}-\mu_{1}\crcr\kern
2.0pt\cr\mu_{1}$\hfil\kern 2.0pt\kern 8.75002pt&I&0\cr\mu_{2}$\hfil\kern
2.0pt\kern 8.75002pt&0&0\cr\mu_{3}$\hfil\kern 2.0pt\kern
8.75002pt&0&0\cr\mu_{4}$\hfil\kern 2.0pt\kern
8.75002pt&0&0\cr\cr}}\kern-12.0pt}\,\right]$}},\quad
ZB_{1}S_{11}^{1/2}V_{1}=\hbox{}\;\vbox{\kern 41.58327pt\hbox{$\kern
58.61206pt\kern-8.75002pt\left[\kern-58.61206pt\vbox{\kern-41.58327pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&\mu_{2}&m_{1}-\mu_{2}\crcr\kern
2.0pt\cr\mu_{1}$\hfil\kern 2.0pt\kern 8.75002pt&0&B_{12}\cr\mu_{2}$\hfil\kern
2.0pt\kern 8.75002pt&I&0\cr\mu_{3}$\hfil\kern 2.0pt\kern
8.75002pt&0&0\cr\mu_{4}$\hfil\kern 2.0pt\kern
8.75002pt&0&0\cr\cr}}\kern-12.0pt}\,\right]$}},$ $\displaystyle ZRZ^{T}$
$\displaystyle=$ $\displaystyle\hbox{}\;\vbox{\kern 43.88547pt\hbox{$\kern
88.01677pt\kern-8.75002pt\left[\kern-88.01677pt\vbox{\kern-43.88547pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&\mu_{1}&\mu_{2}&\mu_{3}&\mu_{4}\crcr\kern
2.0pt\cr\mu_{1}$\hfil\kern 2.0pt\kern
8.75002pt&R_{11}&R_{12}&0&0\cr\mu_{2}$\hfil\kern 2.0pt\kern
8.75002pt&R_{12}^{T}&R_{22}&0&0\cr\mu_{3}$\hfil\kern 2.0pt\kern
8.75002pt&0&0&I&0\cr\mu_{4}$\hfil\kern 2.0pt\kern
8.75002pt&0&0&0&0\cr\cr}}\kern-12.0pt}\,\right]$}},$

where due to the congruence transformation

$\left[\begin{array}[]{cc}R_{11}&R_{12}\\\
R_{12}^{T}&R_{22}\end{array}\right]\geq 0$

Partition the transformed part $F_{3}$ of the feedback matrix as

$V_{3}^{T}F_{3}Z^{T}=\hbox{}\;\vbox{\kern 24.41663pt\hbox{$\kern
114.63898pt\kern-8.75002pt\left[\kern-114.63898pt\vbox{\kern-24.41663pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&\mu_{1}&\mu_{2}&\mu_{3}&\mu_{4}\crcr\kern
2.0pt\cr\mu_{1}$\hfil\kern 2.0pt\kern 8.75002pt&F_{31}&F_{32}&F_{33}&F_{34}\cr
m_{3}-\mu_{1}$\hfil\kern 2.0pt\kern
8.75002pt&0&0&0&0\cr\cr}}\kern-12.0pt}\,\right]$}},$

with $F_{31}$, $F_{32}$, $F_{33}$ and $F_{34}$ to be determined and set

$V_{1}^{T}S_{11}^{-1/2}P_{1}^{T}Z^{T}=\hbox{}\;\vbox{\kern
25.19441pt\hbox{$\kern
114.59729pt\kern-8.75002pt\left[\kern-114.59729pt\vbox{\kern-25.19441pt\vbox{\halign{$#$\hfil\kern
2\p<EMAIL_ADDRESS>2.0pt\kern 8.75002pt&\mu_{1}&\mu_{2}&\mu_{3}&\mu_{4}\crcr\kern
2.0pt\cr\mu_{2}$\hfil\kern 2.0pt\kern 8.75002pt&P_{11}&P_{12}&P_{13}&P_{14}\cr
m_{1}-\mu_{2}$\hfil\kern 2.0pt\kern
8.75002pt&\hat{P}_{11}&\hat{P}_{12}&\hat{P}_{13}&\hat{P}_{14}\cr\cr}}\kern-12.0pt}\,\right]$}}.$

Then it follows from (24) that

$\displaystyle Z(R+B_{1}P_{1}^{T}+P_{1}B_{1}^{T}+B_{1}S_{11}B_{1}^{T})Z^{T}$
$\displaystyle=$
$\displaystyle\left[\begin{array}[]{cccc}R_{11}+B_{12}\hat{P}_{11}+\hat{P}_{11}^{T}B_{12}^{T}+B_{12}B_{12}^{T}&R_{12}+B_{12}\hat{P}_{12}+P_{11}^{T}&B_{12}\hat{P}_{13}&B_{12}\hat{P}_{14}\\\
R_{12}^{T}+\hat{P}_{12}^{T}B_{12}^{T}+P_{11}&R_{22}+P_{12}+P_{12}^{T}+I&P_{13}&P_{14}\\\
\hat{P}_{13}^{T}B_{12}^{T}&P_{13}^{T}&I&0\\\
\hat{P}_{14}^{T}B_{12}^{T}&P_{14}^{T}&0&0\end{array}\right]$
$\displaystyle\geq$ $\displaystyle 0,$

which implies that

$B_{12}\hat{P}_{14}=0,\quad P_{14}=0,$

and

$\left[\begin{array}[]{cc}R_{22}+P_{12}+P_{12}^{T}+I&P_{13}\\\
P_{13}^{T}&I\end{array}\right]\geq 0,$

consequently,

$R_{22}+P_{12}+P_{12}^{T}+I-P_{13}P_{13}^{T}\geq 0,$

and

$R_{22}+P_{12}+P_{12}^{T}+2I-P_{13}P_{13}^{T}=(R_{22}+P_{12}+P_{12}^{T}+I-P_{13}P_{13}^{T})+I>0.$
(28)

and furthermore,

$\left[\begin{array}[]{cc}R_{22}+P_{12}+P_{12}^{T}+2I&P_{13}\\\
P_{13}^{T}&I\end{array}\right]>0,\quad
rank\left[\begin{array}[]{cc}R_{22}+P_{12}+P_{12}^{T}+2I&P_{13}\\\
P_{13}^{T}&I\end{array}\right]=\mu_{2}+\mu_{3}.$ (29)

A simple calculation yields that

$\displaystyle Z\tilde{R}Z^{T}$ $\displaystyle=$
$\displaystyle\left[\begin{array}[]{cccc}R_{11}+B_{12}\hat{P}_{11}+\hat{P}_{11}^{T}B_{12}^{T}+2B_{12}B_{12}^{T}&R_{12}+B_{12}\hat{P}_{12}+P_{11}^{T}&B_{12}\hat{P}_{13}&0\\\
R_{12}^{T}+\hat{P}_{12}^{T}B_{12}^{T}+P_{11}&R_{22}+P_{12}+P_{12}^{T}+2I&P_{13}&0\\\
\hat{P}_{13}^{T}B_{12}^{T}&P_{13}^{T}&I&0\\\ 0&0&0&0\end{array}\right]$
$\displaystyle-{1\over
2}\left[\begin{array}[]{cccc}F_{31}+F_{31}^{T}&F_{32}&F_{33}&F_{34}\\\
F_{32}^{T}&0&0&0\\\ F_{33}^{T}&0&0&0\\\ F_{34}^{T}&0&0&0\end{array}\right],$

and we can choose

$F_{32}=2(R_{12}+B_{12}\hat{P}_{12}+P_{11}^{T}),\quad
F_{33}=2B_{12}\hat{P}_{13},\quad F_{34}=0,$

and any $F_{31}$ such that

$R_{11}-{1\over
2}(F_{31}+F_{31}^{T})+B_{12}\hat{P}_{11}+\hat{P}_{11}^{T}B_{12}^{T}+B_{12}B_{12}^{T}>0.$

We then have

$\left[\begin{array}[]{cc}Z&0\\\
0&I\end{array}\right]\left[\begin{array}[]{cccc}I&B_{1}&0&0\\\ 0&I&0&0\\\
0&0&I&0\\\ 0&0&0&I\end{array}\right]\left[\begin{array}[]{cc}I&0\\\
0&U^{T}\end{array}\right]\left[\begin{array}[]{cc}\tilde{R}&\tilde{P}\\\
\tilde{P}^{T}&S\end{array}\right](\left[\begin{array}[]{cc}Z&0\\\
0&I\end{array}\right]\left[\begin{array}[]{cccc}I&B_{1}&0&0\\\ 0&I&0&0\\\
0&0&I&0\\\ 0&0&0&I\end{array}\right]\left[\begin{array}[]{cc}I&0\\\
0&U^{T}\end{array}\right])^{T}$ $=\left[\begin{array}[]{ccccccc}R_{11}-{1\over
2}(F_{31}+F_{31}^{T})+B_{12}\hat{P}_{11}+\hat{P}_{11}^{T}B_{12}^{T}+B_{12}B_{12}^{T}&0&0&0&0&0&0\\\
0&R_{22}+P_{12}+P_{12}^{T}+I&P_{13}&0&0&0&0\\\ 0&P_{13}^{T}&I&0&0&0&0\\\
0&0&0&0&0&0&0\\\ 0&0&0&0&S_{11}&0&0\\\ 0&0&0&0&0&0&0\\\
0&0&0&0&0&0&0\end{array}\right]$ $\geq 0,$

and thus

$\left[\begin{array}[]{cc}\tilde{R}&\tilde{P}\\\
\tilde{P}^{T}&S\end{array}\right]\geq 0,$

i.e. the closed-loop system (5) is a pH descriptor system. Moreover,

$\displaystyle Z\tilde{R}Z^{T}$ $\displaystyle=$ $\displaystyle Z(R-{1\over
2}\left[\begin{array}[]{cc}B_{1}&B_{3}\end{array}\right]\left[\begin{array}[]{c}F_{1}\\\
F_{3}\end{array}\right]-{1\over
2}(\left[\begin{array}[]{cc}B_{1}&B_{3}\end{array}\right]\left[\begin{array}[]{c}F_{1}\\\
F_{3}\end{array}\right])^{T}Z^{T}$ $\displaystyle=$
$\displaystyle\left[\begin{array}[]{cccc}R_{11}-{1\over
2}(F_{31}+F_{31}^{T})+B_{12}\hat{P}_{11}+\hat{P}_{11}^{T}B_{12}^{T}+2B_{12}B_{12}^{T}&0&0&0\\\
0&R_{22}+P_{12}+P_{12}^{T}+2I&P_{13}&0\\\ 0&P_{13}^{T}&I&0\\\
0&0&0&0\end{array}\right]$ $\displaystyle\geq$ $\displaystyle 0,$

and

$\displaystyle\mathop{\rm rank}\nolimits(\tilde{R})$ $\displaystyle=$
$\displaystyle\mathop{\rm rank}\nolimits(R_{11}-{1\over
2}(F_{31}+F_{31}^{T})+B_{12}\hat{P}_{11}+\hat{P}_{11}^{T}B_{12}^{T}+2B_{12}B_{12}^{T})$
$\displaystyle+\mathop{\rm
rank}\nolimits\left[\begin{array}[]{cc}R_{22}+P_{12}+P_{12}^{T}+2I&P_{13}\\\
P_{13}^{T}&I\end{array}\right]$ $\displaystyle=$
$\displaystyle\mu_{1}+\mu_{2}+\mu_{3}$ $\displaystyle=$
$\displaystyle\mathop{\rm
rank}\nolimits\left[\begin{array}[]{ccc}R&B_{1}&B_{3}\end{array}\right].$

Hence, it follows from Lemmas 3 and 4 that the pencil $sE-(J-R+(G-P)F)$ is
regular, of index at most one, and has all eigenvalues in the open left half
plane.

Question 2. is answered in the following theorem.

######  Theorem 6.

There exists a matrix $F$ such that the closed-loop system (5) is strictly
passive if and only if $S>0$ and

$R+{1\over 2}(G-P)(S+N)^{-1}(G+P)^{T}+{1\over 2}(G+P)(S+N)^{-T}(G-P)^{T}>0.$
(36)

###### Proof.

By setting $B=G-P$, $D=S+N$, we have the following list of equivalences.

$\displaystyle\left[\begin{array}[]{cc}\tilde{R}&\tilde{P}\\\
\tilde{P}^{T}&S\end{array}\right]>0$ $\displaystyle\Leftrightarrow$
$\displaystyle\left[\begin{array}[]{cc}R-{1\over 2}BF-{1\over
2}F^{T}B^{T}&P+{1\over 2}F^{T}D^{T}\\\ P^{T}+{1\over
2}DF&S\end{array}\right]>0$ $\displaystyle\Leftrightarrow$
$\displaystyle\left[\begin{array}[]{cc}I&BD^{-1}\\\
0&I\end{array}\right]\left[\begin{array}[]{cc}R-{1\over 2}BF-{1\over
2}F^{T}B^{T}&P+{1\over 2}F^{T}D^{T}\\\ P^{T}+{1\over
2}DF&S\end{array}\right]\left[\begin{array}[]{cc}I&BD^{-1}\\\
0&I\end{array}\right]^{T}>0$ $\displaystyle\Leftrightarrow$
$\displaystyle\left[\begin{array}[]{cc}R+BD^{-1}P^{T}+PD^{-T}B^{T}+BD^{-1}SD^{-T}B^{T}&P+BD^{-1}S+{1\over
2}F^{T}D^{T}\\\ P^{T}+SD^{-T}B^{T}+{1\over 2}DF&S\end{array}\right]>0$
$\displaystyle\Leftrightarrow$ $\displaystyle S>0,\mbox{{a}nd }\
R+BD^{-1}P^{T}+PD^{-T}B^{T}+BD^{-1}SD^{-T}B^{T}>0,$

i.e., the condition (36) holds (since $S={1\over 2}(D+D^{T})$). Moreover,
under the condition (36),

$F=-2(S+N)^{-1}[P^{T}+S(S+N)^{-T}B^{T}]=-(S+N)^{-1}(G+P)^{T}-(S+N)^{-T}(G-P)^{T}$

is a feedback matrix that achieves strict passivity of the closed loop system.

##  4 Conclusions and Future Work

We have presented a complete characterization when a pH descriptor system (1)
can be made asymptotically stable and strictly passive, as well as regular and
of index at most one via state feedback and preserving the pH structur.

An interesting question that remains to be answered is to use the discussed
state feedback to obtain a representation that maximizes the distance to
instability, respectively non-passivity. For both questions only partial
solutions are available, see [12] and the references therein.

Similar questions can also be studied when output feedback $u=Fy+v$ is used
instead of state feedback. A complete characterization when a pH descriptor
system (1) with $D=0$ can be made regular, of index at most one and
asymptotically stable by output feedback while preserving the pH structure has
been presented in [4], while the question how to achieve a regular, index at
most one and asymptotically stable or strictly passive system via output
feedback for a pH descriptor system (1) with $D\not=0$ is still an open
problem.

## Acknowledgment

D. C. was supported by a Humboldt Fellowship at TU Berlin (Germany). V. M. has
been supported by Deutsche Forschungsgemeinschaft (DFG) through the SPP1984
“Hybrid and multimodal energy systems” Project: _Distributed Dynamic Security
Control_ and by the DFG Research Center Math+, Project No. 390685689.
_Advanced Modeling, Simulation, and Optimization of Large Scale Multi-Energy
Systems_.

## References

  * [1] F. Achleitner, A. Arnold, and E.A. Carlen.  The hypocoercivity index for the short and large time behavior of ODEs.  J. Differential Equations, 371:83–115, 2023.
  * [2] C. Beattie, V. Mehrmann, H. Xu, and H. Zwart.  Port-Hamiltonian descriptor systems.  Math. Control Signals Systems, 30(17):1–27, 2018.
  * [3] K. Cherifi, H. Gernandt, and D. Hinsen.  The difference between port-Hamiltonian, passive and positive real descriptor systems.  Math. Control Signals Systems, pages 1–32, 2023.
  * [4] D. Chu and V. Mehrmann.  Stabilization of linear port-Hamiltonian descriptor systems via output feedback.  arXiv preprint arXiv:2403.18967, 2024.
  * [5] D. Chu, V. Mehrmann, and N. K. Nichols.  Minimum norm regularization of descriptor systems by mixed output feedback.  Linear Algebra Appl., 296(1–3):39–77, 1999.
  * [6] L. Dai.  Singular Control Systems, volume 118 of Lecture Notes in Control and Inform. Sci. Springer-Verlag, Berlin, Heidelberg, 1989.
  * [7] B. Jacob and H. Zwart.  Linear port-Hamiltonian systems on infinite-dimensional spaces, volume 223 of Operator Theory: Advances and Applications.  Birkhäuser, Basel, 2012.
  * [8] T. Kailath.  Linear Systems, volume 156.  Prentice-Hall Englewood Cliffs, NJ, 1980.
  * [9] C. Mehl, V. Mehrmann, and M. Wojtylak.  Linear algebra properties of dissipative Hamiltonian descriptor systems.  SIAM J. Matrix Anal. Appl., 39(3):1489–1519, 2018.
  * [10] C. Mehl, V. Mehrmann, and M. Wojtylak.  Distance problems for dissipative Hamiltonian systems and related matrix polynomials.  Linear Algebra Appl., 623:335–366, 2021.
  * [11] V. Mehrmann and R. Morandin.  Structure-preserving discretization for port-Hamiltonian descriptor systems.  In 58th IEEE Conference on Decision and Control (CDC), Nice, France, pages 6863–6868, 2019.
  * [12] V. Mehrmann and B. Unger.  Control of port-Hamiltonian differential-algebraic systems and applications.  Acta Numerica, pages 395–515, 2023.
  * [13] V. Mehrmann and A.J. van der Schaft.  Differential-algebraic systems with dissipative Hamiltonian structure.  Math. Control Signals Systems, pages 1–44, 2023.
  * [14] R. Morandin.  Modeling and Numerical Treatment of Port-Hamiltonian Descriptor Systems.  PhD thesis, TU Berlin, 2024.
  * [15] A. van der Schaft and D. Jeltsema.  Port-Hamiltonian systems theory: An introductory overview.  Foundations and Trends in Systems and Control, 1(2-3):173–378, 2014\.
  * [16] A.J. van der Schaft and V. Mehrmann.  Linear port-Hamiltonian DAE systems revisited.  Systems Control Lett., 177:105564, 2023.
  * [17] J. C. Willems.  Least squares stationary optimal control and the algebraic Riccati equation.  IEEE Trans. Automat. Control, 16(6):621–634, 1971.