# Zero-Shot Transfer Learning for Structural Health Monitoring using
Generative Adversarial Networks and Spectral Mapping

Mohammad Hesam Soleimani-Babakamali  soleimanibabakamali@gmail,com Roksana
Soleimani-Babakamali<EMAIL_ADDRESS>Kourosh Nasrollahzadeh
<EMAIL_ADDRESS>Onur Avci<EMAIL_ADDRESS>Serkan Kiranyaz
<EMAIL_ADDRESS>Ertugrul Taciroglu<EMAIL_ADDRESS>Department of Civil
and Environmental Engineering, University of California, Los Angeles, CA, USA
Department of Computer Science, University of Vienna, Vienna, Austria
Department of Civil Engineering, K. N. Toosi University of Technology, Tehran,
Iran  Department of Civil and Environmental Engineering, West Virginia
University, WV, USA  Department of Electrical Engineering, Qatar University,
Qatar

###### Abstract

Gathering properly labeled, adequately rich, and case-specific data for
successfully training a purely data-driven or hybrid model for structural
health monitoring (SHM) applications is a challenging task. We posit that a
Transfer Learning (TL) method that utilizes available data in any relevant
source domain and directly applies to the target domain through domain
adaptation can provide substantial remedies to address this issue.
Accordingly, we present a novel TL method that differentiates between the
source’s no-damage and damage cases and utilizes a domain adaptation (DA)
technique. The DA module transfers the accumulated knowledge in contrasting
no-damage and damage cases in the source domain to the target domain, given
only the target’s no-damage case. High-dimensional features allow employing
signal processing domain knowledge to devise a generalizable DA approach. The
Generative Adversarial Network (GAN) architecture is adopted for learning
since its optimization process accommodates high-dimensional inputs in a zero-
shot setting. At the same time, its training objective conforms seamlessly
with the case of no-damage and damage data in SHM since its discriminator
network differentiates between real (no damage) and fake (possibly unseen
damage) data. An extensive set of experimental results demonstrates the
method’s success in transferring knowledge on differences between no-damage
and damage cases across three strongly heterogeneous independent target
structures. The area under the Receiver Operating Characteristics curves (Area
Under the Curve - AUC) is used to evaluate the differentiation between no-
damage and damage cases in the target domain, reaching values as high as 0.95.
With no-damage and damage cases discerned from each other, zero-shot
structural damage detection is carried out. The mean $F_{1}$ scores for all
damages in the three independent datasets are 0.978, 0.992, and 0.975. The
success of the proposed TL approach is expected to pave the way for further
improvements in the accuracy and generalizability of data-driven SHM
applications.

###### keywords:

Transfer Learning; SHM; Domain Adaptation; Zero-shot Learning; Generative
Adversarial Networks; Structural Damage Detection.

††journal: Journal of Mechanical Systems and Signal Processing

##  1 Introduction

Dense wireless networks featuring inexpensive sensor units are now providing
massive amounts of data [1]. Such sensor networks can be utilized for
structural health monitoring (SHM) of civil infrastructure [2, 3]—–more
specifically, for detecting, localizing, and quantifying structural damage.
Moreover, recent developments in big data science and computing hardware are
rendering the analyses of massive data routinely accessible [4]. The missing
piece of the puzzle for pervasive SHM at large scales is learning and
transferring information across structures or structure populations. The
inherent diversity of civil infrastructure (_e.g._ , bridges, buildings,
tunnels, dams, and power plants) require versatile, robust, and generalizable
approaches.

SHM algorithms—operating on multi-modal sensor data such as accelerations,
strains, temperature, etc.—can achieve pattern recognition and distinguish
between no-damage and damage structural cases [5]. Accordingly, SHM
procedures, including Structural Damage Detection (SDD), often make use of
Machine Learning (ML) techniques with various supervision levels (supervised,
semi-supervised, or unsupervised). In classical methods— _e.g._ , parametric
approaches that track modal features [6, 7, 8, 9]—various distance metrics
across signals [10, 11], as well as statistical or heuristic ad hoc
representative values of signals [12, 13] are used for training ML or
statistical models. Even though such features can be extracted globally from
various systems, prior studies revealed that the sensitivity of those
parameters to damage cases is not scalable across different systems [14].

With the emergence of deep neural networks and breakthroughs in their
training— _e.g._ , ADAM optimizer [15], rectified linear-unit activation
layers [16], and back-propagation [17]—customized regression-based data-driven
or model-based SHM approaches are constantly losing ground to deep neural
networks [18]. Deep neural networks are able to extract damage-sensitive
features automatically through representation learning [17] from high-
dimensional input data and simultaneously perform damage detection [14, 19,
20, 21]. However, these techniques have certain critical limitations: ($i$)
they need labeled data for each new (unseen) structure, ($ii$) even if
properly labeled data exists, one must train a new network for every
structure, and even when this is possible, and ($iii$) such supervised methods
could arguably contradict the ultimate goal of SHM, which is to detect unseen
anomalies when/if they take place in the future. If these challenges can be
addressed properly, the resulting methods would be generalizable and pave the
way for the next generation of SHM tools that can be applied to entire
populations of structures.

There appear to be two major directions to address the need for labeled
information for a new structure. The first direction involves using generative
models, and the second involves developing Transfer Learning (TL) mechanisms.
Generative models can generate SHM data with considerable variation [22].
Generative Adversarial Networks (GANs) have recently been used for data
augmentation in SHM applications [23] and to alleviate the data imbalance
issues [24, 25]. Through data generation, Soleimani-Babakamali et al. proposed
an unsupervised SHM technique generating samples from the incoming data stream
to address the need for comprehensive prior information [22, 26]. In
comparison, TL can transfer the accumulated knowledge of discerning between
no-damage and damage structural states across different systems (_i.e._ ,
“source” and “target ” domains) through either model updating (_i.e._ , model
transfer) or domain adaptation (_i.e._ , feature transfer). Fine-tuning is an
instance of model transfer in which the trained source model parameters are
updated via the target domain data and objectives (_e.g._ , classification).
Several studies reported its application in SHM, most of which are image-based
[27, 28, 29, 30]. The image-based fine-tuning owes its success to the
availability of massive image datasets, such as ImageNet [31], which takes
advantage of pre-trained DL models on such datasets. Yet, labeled data must be
available in the target domain to adapt the pre-trained networks through fine-
tuning. Despite the usefulness of images for different damage types (_i.e._ ,
mechanical faults), managing such large datasets and unseen damages are the
major complications of that method.

In contrast, with the domain adaptation (DA) scheme to transfer the target
features to the source domain, the network trained over the source domain
applies to the target structure without further model training. Thus,
regardless of the potential of generative models, “zero-shot” (_i.e._ , online
damage detection without witnessing damage) DA-based TL is well-aligned with
the requirements and constraints of large-scale SHM. In large-scale SHM, SHM
tools are applied to populations of diverse infrastructure in a city, for
instance, where gathering prior (no-damage and damage cases) data for every
single entity is not practical. It can void the need for labeled data in a new
(target) structure by using the knowledge gained from comprehensive source
domain data while covering the second intricacy via a domain-knowledge-based
DA method. Additionally, a TL method can contain generative models and benefit
from their associated capabilities.

Transfer Learning for SHM has received increasing interest in the past two
years, with several notable attempts appearing in the open literature. A
recent TL study by Wang and Xia [32] explored a re-weighting method where no
labeled data is required from the target space using adversarial domain
adoption. In this approach, the TL takes place through re-weighting and
discarding the source data cases that do not share considerable information
with the target domain. Yet, there are two challenges with this approach.
First, the target database should contain damage cases, even though it is
unlabeled. Second, training a neural network with source and target domain
data is required in one of the quadruple stages of that approach.

Population-based TL, published in four parts, including knowledge transfer
across homogeneous (_e.g._ , the population of wind turbines) [33], and
heterogeneous population of structures [34, 35, 36], is another notable study
in the SHM-TL area. This method is capable of TL for the target dataset
without using its labeled data. An abstract representation of structures is
formed for heterogeneous systems to determine whether the information is
transferable. Then communities of structures that can communicate information
are formed. Later, the knowledge (_i.e._ , the detector trained on the source
class) is transferred to identify anomalies in the unlabeled target class.
This approach entails two constraints. First, it employs highly compressed
representations of structural states; second, TL only occurs in systems within
the same community. Summarizing the entire system to an abstract
representation is an instance of dimensionality reduction (DR). DR and how it
impacts the detection of unseen anomalies were studied in detail by Soleimani
Babakamali [37]. Results indicated that DR-induced loss of information could
mask damages since, in zero-shot SHM, data dimensions through which future
damages might manifest themselves can get removed.

To overcome the obstacles mentioned above, this study offers a novel TL
approach for transferring the binary knowledge of no-damage/damage conditions
across different structures for zero-shot damage detection. The proposed
approach has the following characteristics: First, data features are selected
as full-spectrum Fast Fourier Transform (FFT) amplitudes of input signals—
_i.e._ , no damage-sensitive feature extraction is carried out, which helps
generalizability. Second, the method transfers knowledge through domains
without altering the neural network trained to differentiate between the
source’s no-damage and damage cases. Accordingly, the novel and significant
contributions of this study can be summarized as follows:

  * 1.

Utilizing full-spectrum FFT amplitudes as features allow the TL to take place
across strongly heterogeneous, real-world structures—herein, the “Z24 bridge,”
“Yellow Frame,” and “Qatar University Grandstand Simulator.” These structures
are of different types and sizes and exhibit distinct damage types. Moreover,
data collected from each structure were collected by different research
groups.

  * 2.

The proposed TL method operates on the target no-damage case data only—that
is, the “baseline” data that comes in as the SHM tool starts to work. This
attribute enables large-scale SHM.

  * 3.

DA is performed by reordering and multiplying the spectral data of the target
structure to have the same shape as the spectral data of the source structure.
This simple arithmetic domain adaptation transforms the target data so that
the model trained to differentiate between the source’s no-damage and damage
cases applies to the target domain.

##  2 Datasets

In this section, three benchmark datasets that are used to evaluate the
proposed TL framework are introduced. All of them contain vibration data,
including ($i$) a concrete box girder bridge (Z24 bridge), ($ii$) a scaled
steel frame ambient excitation (Yellow Frame), and ($iii$) a scaled stadium
model excited in a laboratory setting (Qatar University Grandstand Simulator).

###  2.1 The Yellow Frame

The benchmark “Yellow Frame” has been the source for various control and SHM
datasets. This one-third scaled, three-story steel frame located at the
University of British Columbia is equipped with modular structural elements,
including masses, braces, beams, and columns, to synthesize various structural
states [38] (Fig. 1(a)). The SHM-based Yellow Frame dataset studied herein
considers “brace removal” as damage. The configuration of sensors is shown in
Fig. 1(b). Accordingly, data cases are explained in Table 1. This table
includes the fundamental frequency (_i.e._ , first-mode frequency) of data
cases obtained via the ARTeMIS software [39] with the Enhanced Frequency
Domain Decomposition method [40]. This dataset has a sampling rate of 1000 Hz.

(a)

(b)

Figure 1: Yellow Frame: (a) structure (b) brace and sensor configuration.
Table 1: Yellow Frame dataset summary; brace labels are according to Fig.
1(b).

Damage class |  Removed brace ID |  F.F (Hz)* |  Damage class |  Removed brace ID |  F.F (Hz)
---|---|---|---|---|---
DC0 | None |  7.62 | DC11 |  DC10 + (17, 19, 25, 27)(I)** |  6.77
DC1 | 2,4(II)*** |  5.79 | DC12 |  DC7 + (1, 3, 17, 18)(I) |  6.40
DC2 | DC1 + (18, 20)(II) |  4.76 | DC13 | (10, 12)(II) |  6.41
DC3 |  DC2+ (1, 3, 17, 19)(II) |  4.88 | DC14 |  DC13 + 21(II), 23(I) |  6.52
DC4 | DC1 + (17, 19)(II) |  5.38 | DC15 | (21, 23)(II) |  6.87
DC5 | DC1 + (18, 20)(I) |  5.40 | DC16 |  (7-8, 21, 22)(I) |  6.98
DC6 | 2(II) |  7.19 | DC17 |  (5, 6, 7, 8, 21, 24)(I) |  6.58
DC7 | (2 , 4)(I) |  6.85 | DC18 |  DC17 + (7, 8, 21, 22)(I) |  6.04
DC8 | (25, 27)(I) |  7.29 | DC19 |  DC18 + (5, 6, 23, 24)(I) |  4.92
DC9 |  (29, 31, 8, 6)(I) |  6.90 | DC20 |  (6, 8)(II), (21, 22, 23, 24)(I) |  5.40
DC10 |  (21, 23 ,29 , 31)(I) |  7.18 |  |  |

  * 1.
  * *

Fundamental mode frequency

  * **

One brace is removed

  * ***

Both braces are removed

###  2.2 The Z24 Bridge

Before its demolition, Z24 Bridge overpassed the A1 highway between Bern and
Zürich in Switzerland (Fig. 2(a)). Two benchmark datasets from the Z24 Bridge
exist, one from long-term observations captured over a year, and the other
covers a series of damage states during the last month before its demolition.
Long-term monitoring was primarily utilized for zero-shot anomaly detection
(_i.e._ , no-damage and damage cases) in the literature [41, 6, 42]. These
prior studies used comprehensive long-term data to build a baseline under
various environmental and operational conditions [43]. That baseline is used
for anomaly detection on the last portion of damage case data. The short-term
dataset includes 17 data cases from various severity of seven distinct damage
cases. This study utilizes the short-term dataset, with one class from each of
the seven damage cases. Z24 dataset summary is reported in Table 2. This
dataset has a sampling rate of 100 Hz.

(a) Z24 geometry [44].

(b) Z24 sensor setups (adopted from [44]).

Figure 2: Z24 bridge.

Z24 instrumentation includes several setups (Fig. 2(b)). Each setup includes
multiple sensors and data channels (_i.e._ , vertical, horizontal, and
transverse channels). We filtered out faulty data channels to ensure damages
are being identified from the structure’s vibrations rather than faults in the
sensors’ readings (_e.g._ , channels with trending, dead, or square wave
signals). After inspecting all data channels in the eight data cases above
(_i.e._ , one no-damage and seven damage cases), 15 channels are selected from
various sensor setups across the bridge (Fig. 2(b)), reported in Table 3.

Table 2: Z24 dataset summary. Data class |  Description [45]
---|---
DC0 | First reference measurement (Reference class)
DC1 | Lowering of the pier, 80 mm
DC2 | Lifting of the pier, the tilt of the foundation
DC3 | Spalling of concrete at soffit, $24m^{2}$
DC4 | Landslide of 1 m at abutment
DC5 | Failure of the concrete hinge
DC6 | Failure of 4 anchor heads
DC7 | Rupture of 6 out of 16 tendons
Table 3: Selected channels from Z24 benchmark [44]. Setup | Channel | Setup |  |  Channel |
---|---|---|---|---|---
Setups 1, 7 | 299V | Setup 4 | 100V | 101V | 103V
Setup 2 | 199V | Setup 5 | 99V | 101V | 299V
Setups 3, 9 | 300V | Setup 6 | 101V | 299V | 300V
Setup 8 | 101V |  |  |  |

###  2.3 The Qatar University Grandstand Simulator

Qatar University Grandstand Simulator (QUGS) dataset aims at benchmarking
stadium models for SHM and damage detection purposes. The structure is an
inclined steel frame representing a classical stadium seating layout in grid
form (Fig. 3). QUGS instrumentation and details are made available by Avci et
al. [46]. This dataset contains one no-damage (DC0) and thirty damage cases
(DC1-DC30) induced by bolt-loosening at the beam to girder connections [47,
48]. At each joint, one permanent accelerometer is dedicated to collect data
(Fig. 3) with a sampling frequency of 1000 Hz.

Figure 3: QUGS structure [47]. Figure 4: QUGS joint detailing [49].

##  3 Methodology

The overview and application of the proposed TL technique are shown in Fig. 5.
It consists of three modules, including ($i$) a source-no-damage identifier
that differentiate between the source domain’s no-damage and damage cases,
($ii$) a DA module that transforms the target domain data features obtained
through the feature extraction routine (Section 3.1), so that the source-no-
damage identifier applies to the target domain, and ($iii$) a threshold tuning
method for zero-shot structural damage detection (SDD). The majority of the
article describes and assesses the first two modules. A threshold tuning
method is introduced in Section 4, and zero-shot SDD is carried out in the
results section.

Figure 5: Overview of the proposed TL method.

###  3.1 Feature Extraction

The source-no-damage identifier is designed to work with the full-spectrum FFT
amplitudes of input signals without performing any damage-sensitive feature
selection. There are two reasons for this: ($i$) to avoid the sensitivity loss
to unseen damages [37], and ($ii$) to accommodate knowledge transfer across
various systems since the full-spectrum captures a broadband representation of
systems’ dynamic characteristics. Each structure’s sensor (vibration) data is
considered a distinct data channel where the number of data channels varies
among the structures. Performing the data acquisition from the $N$ data
channels of a particular structure, each data channel’s feature extraction is
carried out as follows: First, each channel data is segmented into the
analysis windows of length $W$. We perform $W$-points Fast Fourier Transform
(FFT), and the first $W/2$ power spectral coefficients are divided over their
mean value and are used for analysis. GANs, which contain the source-no-damage
identifiers, have shown impressive generative capabilities over bounded data—
_e.g._ , image data (between 0-255 at each color-channel) [50] and text data
(between -1 and 1 via Transformers [51]). Considering the vibration data, the
application of GAN and how bounding GAN inputs impact its generative power
(_i.e._ , the better the data generation, the better discriminator
distinguishes between real and fake data) was studied by Soleimani- Babakamali
et al. [25]. The bounding value of 10— _i.e._ , rounding normalized amplitudes
greater than ten to ten—for features to vary between 0 to 10, was suggested in
that study. We apply the same standard rounding normalized FFT amplitudes over
ten to ten. The feature vector is labeled as $F_{i}$ for the channel $i$. The
concatenation of all $N$-channel features constitutes the final feature vector
for each analysis window, $F$ ($F\in\mathbb{R}^{(N*\frac{W}{2}\times 1)}$).
$F$ is naturally high-dimensional feature vector considering typical $W$ and
$N$ values in SHM problems. For instance, each $F$ has 15000 dimensions for
$N$ = 15 and $W=2000$.

###  3.2 source-no-damage Identifier Module

Generative Adversarial Networks’ [52] optimization process aligns perfectly
with the SHM; the GAN models are optimized by training a Discriminator
($\mathcal{D}$) network to distinguish between “real,” _i.e._ , inputs of
$\mathcal{D}$, and “fake” data generated by the Generator ($\mathcal{G}$)
network. When the real data is sampled from the no-damage case of the source
structure, $\mathcal{D}$ will learn to differentiate the no-damage case from
any other data—including the damage case data. Here the basic assumption is
that the no-damage case data patterns are sufficiently different from the
damage case. Therefore, $\mathcal{D}$ can detect any data of the source
structure damage case with proper training. Using the damage case data as the
validation set, during the training, the $\mathcal{D}$ model, which gives the
highest detection accuracy over the damage case of the source structure, can
then be selected as the source-no-damage identifier network. It is worth
mentioning that in online SHM, the current state of structures equals the no-
damage state, as sought damages occur in the future.

In the GAN training process, $\mathcal{G}$ samples from a “latent space”,
denoted as $P_{z}$, generating fake data. A schematic of GAN’s training with
features $F$ is shown in Fig. 6. The GAN’s training is shown on that image
with the Yellow Frame data features with $W=1000$ and $N=10$, leading to 5000
spectral lines in $F$ (Section 3.1). In that image, $\ell_{\mathcal{D}}$ and
$\ell_{\mathcal{G}}$ are loss functions that optimize $\mathcal{G}$, and
$\mathcal{D}$, respectively. $\ell_{\mathcal{G}}$ penalizes $\mathcal{G}$ if
$\mathcal{G}$-generated fake data are correctly identified by $\mathcal{D}$ as
fake; otherwise, $\ell_{\mathcal{D}}$ penalizes $\mathcal{D}$. The impact of
$\ell_{\mathcal{G}}$ shows itself with $\mathcal{G}$ implicitly understanding
the training data distribution on the course of training, which led to
generating more real-resembling data. As the training goes on with more
realistic generations, $\mathcal{D}$ better understands the patterns in the
no-damage case to differentiate it from dissimilar data. Those loss functions
resemble a zero-sum game, in which the gain of one participant (_e.g._ ,
$\mathcal{D}$) is the loss of the other (_i.e._ , $\mathcal{G}$). The losses
are expected to reach an equilibrium state, following the Nash equilibrium
[53]. There exist various loss formulations for GAN; The following formulae
are selected from the seminal GAN paper [52] with Sigmoid activation on
$\mathcal{D}$’s output layer:

$\ell_{\mathcal{D}}=-\mathbb{E}_{x\sim
P_{x}}\left[\log\mathcal{D}(x)\right]-\mathbb{E}_{z\sim
P_{z}}\left[\log\left(1-\mathcal{D}(\mathcal{G}(z))\right)\right]$ (1)
$\ell_{\mathcal{G}}=-\mathbb{E}_{z\sim
P_{z}}\left[\log\mathcal{D}\left(\mathcal{G}(z)\right)\right],$ (2)

where $x$ is a data instance from the training set, herein the no-damage case,
and $z$ is the random latent data drawn by $\mathcal{G}$ from the latent space
$P_{z}$.

Figure 6: GAN’s training schema; $\ell_{\mathcal{D}}$ and $\ell_{\mathcal{G}}$
denote training losses for $\mathcal{D}$ and $\mathcal{G}$, respectively.
Sample GAN’s generation at iteration 511 shows the progress of $\mathcal{G}$
in generating more real-resembling fake data as the training continues.

####  3.2.1 GAN Architecture

The Long Short-Term Memory (LSTM), a recurrent neural network, was reported to
decipher FFT-amplitude-based features efficiently and with high
generalizability in SHM queries [25, 26]. Therefore, the LSTM-based detector
model is used in this study, as shown in Fig. 7(a), consisting of one layer of
LSTM at each branch. The channel-based architecture scales $\mathcal{D}$ to
structures with different numbers of sensors. The Generator $\mathcal{G}$
architecture is outlined in Fig. 7(b) as a Multi-Layer Perceptron (MLP)
network. $\mathcal{G}$ samples from $P_{z}$ with 100 dimensions, followed by
layers, up-scaling the latent sample to have the same dimensions as $F$
($\in\mathbb{R}^{\frac{W}{2}*N\times 1}$). In case $W/2*N\leq 3750$, which
occurs experimenting with different $W$ and $N$ values, $\mathcal{G}$ will
have the first three layers with an output layer size of $W/2*N$.

(a) LSTM-based $\mathcal{D}$ architecture.

(b) MLP-based $\mathcal{G}$ architecture.

Figure 7: Proposed GAN architecture.

With the Sigmoid activation function, the output neuron hypothetically outputs
0 for the most probable fake data and 1 for the most probable real data. From
that output, we formulate an anomaly detection score, $S$, as:

$s_{x}=-\textrm{log}(\mathcal{D}(x)),$ (3)

where $s_{x}$ denotes an instance of $S$ for an arbitrary signal $x$. The
range of $S$ is between 0 for the most probable real data to $+\infty$ for the
most probable fake data.

####  3.2.2 GAN Training

For training, we choose the ADAM optimizer [15] with a suggested learning rate
of $1e-4$ and the first and second moment’s decay rate of $0.9$ and $0.999$ to
train GANs. GANs are mainly used for their generative capability, in which the
equilibrium state between $\ell_{\mathcal{D}}$ and $\ell_{\mathcal{G}}$
signals the end of training. In the case of SHM, though, the trained
$\mathcal{D}$ that offers the best discrimination between the source’s no-
damage and damage cases is sought. The main assumption of TL is that both no-
damage and damage cases are known for the source domain. During training
various $\mathcal{D}$-$\mathcal{G}$ models will emerge per iteration (Fig. 6).
Different GAN training loss patterns from which $\mathcal{D}$ is drawn— _e.g._
, equilibrium or overfitted $\mathcal{D}$—might offer different sensitivity to
damages [26]. We train GAN until saturation and select the $\mathcal{D}$ that
offers the best performance according to the area under the Receiver Operating
Characteristics (ROC) curves (Area Under the Curve - AUC)— _i.e._ , highest
true alarms and lowest false alarms over the source data. In ROC, the damage
detection threshold is set at different values. The true alarm ratio is the
Recall rate of the damage cases, the ratio of damage case data identified by
the detector with $S$ over the threshold. The false alarm ratio is 1 – Recall
of the no-damage case. Since the curve is constructed with all possible values
of damage detection threshold $T$, the area under the curve (_i.e._ , AUC)
offers a threshold-bias-free estimation of how well $\mathcal{D}$ contrasts
those cases. $\textrm{AUC}=1$ indicates the perfect distinction between no-
damage and damage cases’ detection scores. In contrast, an $\textrm{AUC}=0.5$
indicates the random guess (_i.e._ , values below 0.5 are worse than a random
guess: the model understood the data incorrectly). An instance of building the
ROC curve from the detection scores, $S$, is shown in Fig. 8 for a GAN trained
on the Yellow Frame no-damage case with $W=1000$, and $N=10$.

Figure 8: Constructing the ROC curve from the detection scores; a Yellow frame
example.

The GAN training process, and the loss functions (Eqs. 1 and 2), yield the $S$
of the no-damage case toward zero and the damage cases toward higher $S$
scores. A sample GAN’s training result with 100000 epochs is shown in Fig. 9
for the Yellow Frame structure with $W=1000$ and $N=10$. A close inspection of
the figure will reveal the fact that $\mathcal{D}$ reaches high AUC values
discriminating no-damage from damage cases fairly fast, around the epoch 100.
PyTorch [54] is used to carry out this study; thus, an $S$ of 40 (_i.e._ ,
$10^{-40}$) is the smallest PyTorch logarithmic Float precision. Consequently,
as a rule of thumb, the source-no-damage identifier is selected as follows;
the detector with the highest AUC without $S$ scores equals or above 40
(_i.e._ , float precision). Following the above criteria, Epoch 21475 is
chosen for the Yellow Frames source-no-damage identifier. This source-no-
damage identifier is labeled as YS10_1000 (_i.e._ , Yellow Frame source with
$N=10$ and $W=1000$); other source-no-damage identifiers are labeled
accordingly throughout this study.

Figure 9: Evolution of $\mathcal{D}$’s differentiation between no-damage and
damage cases during GAN’s training; Yellow Frame dataset with $W=1000$ and
$N=10$.

###  3.3 Domain Adaptation

Domain-knowledge-based DA are more generalizable than their DL-based
counterparts; the reason behind the current surge of physics-guided Machine
Learning, specifically for mechanical problems [55, 56]. In this study, and
with the FFT-based features $F$, we set the frequency power spectrum as the
grounds of the domain-knowledge-based DA method. Instead of direct
manipulation of the target $F$, the digital signal processing (DSP) domain
knowledge is used herein. The signal’s power spectrum offers the relative
magnitudes of frequency components that make up the signal upon combination.
To estimate the power spectrum, the signal is windowed to smaller and
sometimes overlapping frames (_e.g._ , Welche’s method [57]). Herein, the
source and target signals are framed with windows of size $W$ (Section 3.1).
The mean-square of normalized features of the source domain is taken as an
estimation of the source domain power spectrum—“source spectrum” (SS).
Accordingly, the target power spectrum— ”target spectrum” (ST)—is created
utilizing the target domain no-damage case data. A spectral line mapping,
$\mathcal{M}$, is formed by transforming the target spectrum to the source
spectrum. This transformation serves the core DA operation needed to use the
source-no-damage identifier over the target vibration data.

A sample source spectrum for Yellow Frame with $W=1000$ and $N=10$ is shown
for its first data channel, $SS_{C1}$, in Fig. 10(a). A target spectrum for
the QUGS’s first data channel with $W=1000$ and $N=10$, $TS_{C1}$, is shown in
Fig. 10(a). The two spectra are different, depicting strongly heterogeneous
systems with different vibrational content. In that regard, the following
channel-wise mapping is proposed. First, the spectral line positions of both
source and target domain no-damage cases (_i.e._ , from 1 to $W/2$) are sorted
according to their amplitudes (_i.e._ , argsort), named $Arg_{S}$ and
$Arg_{T}$, respectively. The sorting order (_i.e._ , ascending or descending)
should be the same for both domains. The argsort operator defines the mapping
(_i.e._ , reordering of the target frequency spectrum), in which target
spectral lines at positions $Arg_{T_{i}}$ are moved to the positions
$Arg_{S{i}}$. The procedure is recapped in Algorithm 1. $\mathcal{M}$ is
pictorially shown with target QUGS and source Yellow Frame with $W=1000$ for
the first data channel in Fig. 10(b). As shown in the figures, with
reordering, source and target power spectra align but with different
amplitudes. Therefore, a post-reordering scaling multiplier, $C_{S-T}$
(Algorithm 2), is applied to them. The correction of amplitudes pursuing the
reordering is necessary since spectral lines’ relative amplitudes demonstrate
their contribution to the structure’s vibrational content. With reordering,
target spectral lines are ordered the same as the source domain regarding
their relative amplitudes. The argsort and scaling multiplier values describe
the mapping $\mathcal{M}$. $\mathcal{M}$ is obtained with the source and
target domain’s no-damage cases and is applied to any incoming data $F$,
transforming it into $F^{*}$ (Algorithm 2). In brief, if the target (new) data
has no damages, the mapping is expected to create a spectral line similar to
the no-damage case in the source domain, and thus, the detector can easily
classify it as the real (no damage); otherwise, as fake (damage case). A
sample $F$ and $F^{*}$ pairs are shown in Figs. 10(c), and 10(d).

(a) Source spectrum: Yellow Frame first channel, $W=1000$.

(b) Target spectrum: QUGS first channel, $W=1000$.

(c) A sample $F$ from the QUGS first channel; $W=1000$ $N=10$.

(d) $F^{*}=\mathcal{M}(F)$ with $F$ shown in Fig.10(c).

Figure 10: Instance of DA application.

Input: Channel i’s source spectrum $SS_{C_{i}}$, Channel i’s target spectrum
$TS_{C_{i}}$

Output: Mapping $\mathcal{M}_{i}$

$Arg_{S}\leftarrow argsort(SS_{C_{i}})$

$Arg_{T}\leftarrow argsort(TS_{C_{i}})$

$C_{ST}\leftarrow\sqrt{\frac{SS_{C{i}}(Arg_{S})}{TS_{C_{i}}(Arg_{T})}}$

$\mathcal{M}_{i}\leftarrow Arg_{S},Arg_{T},C_{ST}$

Algorithm 1 DA: obtaining channel $i$’s spectral line mapping,
$\mathcal{M}_{i}$

Input: Channel $i$’s mapping $\mathcal{M}_{i}$, Channel $i$’s incoming data
features $F_{i}$

Output: $F^{*}_{i}$

$Arg_{S},Arg_{T},C_{ST}\leftarrow\mathcal{M}_{i}$

${F_{i}}^{*}(Arg_{S})=F(Arg_{T})*C_{ST}$

Algorithm 2 DA: transforming incoming data features.

##  4 Experimental Setup

This study decouples the differentiation between no-damage and damage cases
from threshold tuning since devising a tuning standard is a source of bias.
That bias is the reason for using control charts, like ROC curves, to obtain a
bias-free estimation of how well a detector model performs. We argue that
damage detection will become a plain procedure if the no-damage and damage
cases are well-separated. Given the reasoning above, the performance
assessment of the proposed TL method is divided into three parts. First, the
performance of the source-no-damage identifier on the source domain. Given the
GAN and signals’ full-spectrum FFT amplitudes, the source-no-damage identifier
is expected to distinguish between no-damage and damage cases in the source
domain. The second assessment is regarding the devised DA method: whether the
target domain features can be mapped to the source domain in such a way that
the source-no-damage identifier discerns between the target’s no-damage and
unseen damage cases. Third, a threshold tuning method is introduced to perform
and evaluate such zero-shot detection accuracy of the proposed approach.
Details of those analyses are given in this section, and results are provided
in the Results section.

Regarding the source-no-damage identifier performance on the source domain,
the first evaluation will assess whether $\mathcal{D}$ can reach a high AUC
score over the source data. Then to evaluate the DA method ($i$) the proposed
spectral mapping $\mathcal{M}$ is accomplished over the target’s no-damage
case, and ($ii$) $\mathcal{M}$ transforms the (unseen) target data $F$, to
$F^{*}$ (Section 3.3). Different combinations of no-damage/damage cases for
each target dataset are formed for evaluation. For instance, for QUGS, there
are 30 damage cases (DC1-DC30); each represents the damage in one of the 30
joints. Thus, 30 no-damage/damage scenarios are formed to test the proposed
approach. We use 10% of the available no-damage case data to perform the DA,
and the rest is used for testing. Suppose the DC1 damage case is sought to be
identified within target QUGS through the source: Yellow Frame; the process is
shown in Fig. 11. DA is performed by taking the first 10% of each channel’s
data (the first channel is shown as an example) to define each channel’s
$\mathcal{M}$. The formed mapping transforms incoming data features $F$ into
$F^{*}$. Inputting $F^{*}$ to the source-no-damage identifier results in $S$
shown in Fig. 11’s lower left image. Without DA, inputting target $F$ to the
source-no-damage identifier produces $S$ on the lower right image. Obviously,
without DA, the source-no-damage detector $\mathcal{D}$ cannot distinguish
between the no-damage and damage cases’ target data (Fig. 12)— _i.e._ ,
observes both as novelties regarding its source domain—outputting
significantly high $S$ for them.

The threshold tuning method is outlined in Algorithm 3. Previously, 10% of the
no-damage case data were used for DA. Since dataset sizes are known for
evaluating the accuracy, the next 40% data from the no-damage case is used to
tune the anomaly detection threshold (Algorithm 3). First, given
$\mathcal{M}$. obtained through DA over the 10% of no-damage case data,
features $F$ are transformed to $F^{*}$ for all channels (Alg 2). Then, 40% of
the target no-damage case data scores are used for tuning. The Normal
distribution is fit to that chunk’s detection scores. Mean, $\mu$, and
standard deviation, std, of the distribution are estimated, and $T$ is
assigned as $\mu+3\textrm{std}$. Yet, abiding by the GAN’s training paradigm,
$T$ takes the maximum of $\mu+3\textrm{std}$ and $-log10(0.5)=0.3$, which is
the $S$ of GAN’s sigmoid activation for the uncertain case of fake and real
data. Finally, precision, recall, and F1 scores evaluate the target damage
detection performance.

Figure 11: $S$ given by the source-no-damage identifier with and without DA;
$W=2000$, $N=15$. Figure 12: Evaluating DA through ROC curves given Fig. 11.

Input: $\mathcal{M}$, $F$ from target’s no-damage case.

Output: Damage detection threshold $T$

$F^{*}=\mathcal{M}(F)$

mean, std =Normfit($-\log 10(\mathcal{D}(F^{*})))$ Fitting a Gaussian
distribution to the $S$ of $F^{*}$

$T=max(\mu+3*\textrm{std},-log10(0.5))$

Algorithm 3 Threshold tuning for zero-shot damage detection.

$N$ and $W$ are the only TL parameters; accordingly, TL scenarios reported in
Table 4 are designed to perform the above evaluations across the three
datasets. As outlined in this table, the target and source domain data should
be derived with the same $W$ and $N$; parameters which are readily adoptable
by users. Regarding that table, a total of 57 (_i.e._ , Yellow Frame: 20,
QUGS: 30, and Z24: 7) damage cases exist. Thus, 57 combinations of no-
damage/damage cases are available for evaluation. Given three $W$ and $N$
values and two source structures for each target structure, $57*3*3*3=1026$
damage identification/detection instance exists with hundreds of data points
(_e.g._ , Fig. 11). The Z24 dataset has the lowest number of data samples
among the others, given its shorter data acquisition time and sampling
frequency of 100 Hz (1000 Hz for the other datasets). Accordingly, 10% of the
Z24 no-damage case is allocated to perform the DA consisting of only 3 data
points given $W=2000$. Accordingly, 50% of the no-damage case data is given to
the DA algorithm. Algorithm 3 remains the same regarding the SDD, allocating
the other 40% of the Z24 no-damage case for threshold tuning. The remaining
10% no-damage and damage cases remain for testing true and false alarms.
Concerning the different sampling frequencies, we argue that the mapping is
possible. The reason is that the relative position of spectral lines and not
their actual frequency is the input of the $\mathcal{D}$ (_i.e._ , feature
extraction in Section 3.1). Z24 with a sampling frequency of 100 HZ or the
Yellow frame with a sampling frequency of 1000 Hz, both span the same
dimensions in $F$.

Table 4: TL scenarios between Z24, Yellow Frame, and QUGS datasets.

Transfer label | Source | Target |  $W$ |  Source sensors |  Target sensors
---|---|---|---|---|---
Y2Q15_2000 | Yellow F. | QUGS | 2000 | 15: All channels in Fig. 1 | 15: Odd channels in Fig. 3
Y2Q15_1000 | Yellow F. | QUGS | 1000 | 15: All channels in Fig. 1 | 15: Odd channels in Fig. 3
Y2Q15_500 | Yellow F. | QUGS | 500 | 15: All channels in Fig. 1 | 15: Odd channels in Fig. 3
Y2Q10_2000 | Yellow F. | QUGS | 2000 | 10: Channels 1,3,5,6,8,9,10,12,13,15 (Fig. 1) | 10: Channels 1,3,5,9,13,17,21,25,27,29 (Fig. 3)
Y2Q10_1000 | Yellow F. | QUGS | 1000 | 10: Channels 1,3,5,6,8,9,10,12,13,15 (Fig. 1) | 10: Channels 1,3,5,9,13,17,21,25,27,29 (Fig. 3)
Y2Q10_500 | Yellow F. | QUGS | 500 | 10: Channels 1,3,5,6,8,9,10,12,13,15 (Fig. 1) | 10: Channels 1,3,5,9,13,17,21,25,27,29 (Fig. 3)
Y2Q6_2000 | Yellow F. | QUGS | 2000 | 6: Channels 1,3,6,10,12,14 (Fig. 1) | 6: 1,5,13,17,25,29 (Fig. 3)
Y2Q6_1000 | Yellow F. | QUGS | 1000 | 6: Channels 1,3,6,10,12,14 (Fig. 1) | 6: 1,5,13,17,25,29 (Fig. 3)
Y2Q6_500 | Yellow F. | QUGS | 500 | 6: Channels 1,3,6,10,12,14 (Fig. 1) | 6: 1,5,13,17,25,29 (Fig. 3)
Y2Z15_2000 | Yellow F. | Z24 | 2000 | 15: All channels in Fig. 1 | 15: All channels in Table 3
Y2Z15_1000 | Yellow F. | Z24 | 1000 | 15: All channels in Fig. 1 | 15: All channels in Table 3
Y2Z15_500 | Yellow F. | Z24 | 500 | 15: All channels in Fig. 1 | 15: All channels in Table 3
Y2Z10_2000 | Yellow F. | Z24 | 2000 | 10: Channels 1,3,5,6,8,9,10,12,13,15 (Fig. 1) | 10: Setups 4,5, 6-channel 1 + the rest (Table 3)
Y2Z10_1000 | Yellow F. | Z24 | 1000 | 10: Channels 1,3,5,6,8,9,10,12,13,15 (Fig. 1) | 10: Setups 4,5, 6-channel 1 + the rest (Table 3)
Y2Z10_500 | Yellow F. | Z24 | 500 | 10: Channels 1,3,5,6,8,9,10,12,13,15 (Fig. 1) | 10: Setups 4,5, 6-channel 1 + the rest (Table 3)
Y2Z6_2000 | Yellow F. | Z24 | 2000 | 6: Channels 1,3,6,10,12,14 (Fig. 1) | 6: Setups 3,4,6,7-channel 1, setup 5-channels 1, 2 (Table 3)
Y2Z6_1000 | Yellow F. | Z24 | 1000 | 6: Channels 1,3,6,10,12,14 (Fig. 1) | 6: Setups 3,4,6,7-channel 1, setup 5-channels 1, 2 (Table 3)
Y2Z6_500 | Yellow F. | Z24 | 500 | 6: Channels 1,3,6,10,12,14 (Fig. 1) | 6: Setups 3,4,6,7-channel 1, setup 5-channels 1, 2 (Table 3)
Q2Z15_2000 | QUGS | Z24 | 2000 | 15: 15: Odd channels in Fig. 3 | 15: All channels in Table 3
Q2Z15_1000 | QUGS | Z24 | 1000 | 15: 15: Odd channels in Fig. 3 | 15: All channels in Table 3
Q2Z15_500 | QUGS | Z24 | 500 | 15: 15: Odd channels in Fig. 3 | 15: All channels in Table 3
Q2Z10_2000 | QUGS | Z24 | 2000 | 10: Channels 1,3,5,9,13,17,21,25,27,29 (Fig. 3) | 10: Setups 4,5, 6-channel 1 + the rest (Table 3)
Q2Z10_1000 | QUGS | Z24 | 1000 | 10: Channels 1,3,5,9,13,17,21,25,27,29 (Fig. 3) | 10: Setups 4,5, 6-channel 1 + the rest (Table 3)
Q2Z10_500 | QUGS | Z24 | 500 | 10: Channels 1,3,5,9,13,17,21,25,27,29 (Fig. 3) | 10: Setups 4,5, 6-channel 1 + the rest (Table 3)
Q2Z6_2000 | QUGS | Z24 | 2000 | 6: 1,5,13,17,25,29 (Fig. 3) | 6: Setups 3,4,6,7-channel 1, setup 5-channels 1, 2 (Table 3)
Q2Z6_1000 | QUGS | Z24 | 1000 | 6: 1,5,13,17,25,29 (Fig. 3) | 6: Setups 3,4,6,7-channel 1, setup 5-channels 1, 2 (Table 3)
Q2Z6_500 | QUGS | Z24 | 500 | 6: 1,5,13,17,25,29 (Fig. 3) | 6: Setups 3,4,6,7-channel 1, setup 5-channels 1, 2 (Table 3)

  * *

TL scenarios are reciprocal— _e.g._ , Y2Q15_2000 means that Q2YS15_2000 is
also investigated.

##  5 Experimental Results

Three sets of evaluations are reported in this section: ($i$) the source-no-
damage intensifier’s performance over the source data, ($ii$) DA assessment on
the target domain, and ($iii$) the zero-shot damage detection performance. The
initial step is training the $\mathcal{D}$ models each for one of the TL
scenarios (Table 4). A mean AUC of 0.998 is obtained across all datasets
(Section 3.2.2). Accordingly, the first element of the TL model is deemed
practical where each $\mathcal{D}$ model can differentiate between the
source’s no-damage and damage cases given various $W$ and $N$ values. The rest
of this section concerns DA evaluation and performing the zero-shot damage
detection.

###  5.1 Target: The Yellow Frame

The $\mathcal{D}$ models trained on QUGS and Z24 are evaluated over the Yellow
Frame as the target per Section 3.3. ROC curves for 6 of the TL scenarios
outlined in Table 4 are shown in Fig. 13. Each figure embodies 20 ROC curves—
_i.e._ , 20 damage cases in Yellow Frame— with the mean AUC values reported.
With $W=2000$ and $N=15$ (Fig. 13(a)), almost all ROC curves are ideal, with a
mean AUC of 0.9984. DC6 and DC8 show a lower AUC than the others. Regarding
parameters $W$ and $N$, lowering those values reduces not all but several
damage cases’ AUCs. With the lowest $W$ and $N$, DC6 and DC8 AUCs drop to
values as low as 0.7 with QUGS as the source. Z24 shows better overall
performance on the Yellow Frame. Results suggest that the TL performs better
with more sensors (_i.e._ , higher $N$), which is trivial as more sensors
increase the chance of capturing damage-related data. Yet, the notable
observation is that with only six out of the initial 15 sensors ($W=2000$,
Fig. 13(b)), 18 out of 20 damages are distinguished from the no-damage case
almost certainly (_i.e._ , the AUC is near 1) with Z24 as the source
structure.

Regarding the impact of $W$ on the TL’s performance, a lower $W$ tends to
lower the AUC, which is expected. With a lower $W$, the spectral resolution of
both FFTs and power spectrums drops. For instance, impacts of lowering $N$ and
$W$ are shown for several $W$ and $N$ cases in Fig. 14 for the Yellow Frame
$\mathcal{D}$ model tested on itself (_i.e._ , no TL). A lower $W$ in a
constant $N$ deteriorates the detection scores, increasing false alarms and
reducing AUC in return. DC8 and DC6 have the closest $S$ (Fig. 14) and the
closest fundamental frequency to the no-damage case (Table 1) and are the
cases that undergo the most AUC reduction with lower $W$ and $N$ (Fig. 14(a)
to Fig. 14(c)). Yet, given the whole spectrum of AUC values for all TL cases
of Yellow Frame (Figs. 15), and with full sensors, the TL method thus shows
acceptable robustness to $W$. TL-based anomaly detection scores for the Yellow
Frame dataset are shown in Fig. 14(d). It depicts how well DA transferred the
differentiation power of the QUGS detector between no-damage and damage cases
data to the target Yellow Frame, with DA only applied over 10% of its no-
damage data.

(a) Q2YS15_2000; ROC of 20 damages.

(b) Z2YS6_2000; ROC of 20 damages.

(c) Z2YS10_1000; ROC of 20 damages.

(d) Q2YS6_1000; ROC of 20 damages.

(e) Q2YS15_500; ROC of 20 damages.

(f) Z2YS6_500; ROC of 20 damages.

Figure 13: TL performance; Yellow Frame as the target domain.

(a) YS15_2000 performance on Yellow Frame (no TL).

(b) YS10_1000 performance on Yellow Frame (no TL).

(c) YS6_500 performance on Yellow Frame (no TL).

(d) Q2YS10_1000 performance on Yellow Frame with all no-damage case data given
for the DA.

Figure 14: Impact of $W$ and $N$ on source-no-damage identifiers; Yellow
Frame.

(a) Source QUGS.

(b) Source Z24.

Figure 15: TL performance; AUC values for all Yellow Frame TL scenarios.

###  5.2 Target: The QUGS

TL for the target structure QUGS with 30 damage cases is performed with the
source structures Yellow Frame and Z24. ROC curves for 6 of the TL scenarios
in Table 4 are shown in Fig. 16. As with the Yellow Frame TL, $W=2000$ and
$N=15$ provide a seamless TL performance, with a mean AUC of 0.9975, in which
DC29, DC30, and DC2 show the least performance. Further lowering $W$ and $N$,
the AUC scores of those cases are reduced to 0.7. Since Yellow Frame has 15
sensors, only the odd sensors (15 sensors) are used in TL for the QUGS (Table
4 and Fig. 3). Thus, no sensors are present near damages DC2 and DC30, a
probable reason for the lower AUCs. As with the Yellow Frame, DC3, DC31, and
DC30 have patterns similar to the no-damage cases. Lowering $N$ and $W$ values
makes them less distinctive from the no-damage case. Inspecting the
differentiation performance between QUGS damage and no-damage cases without
DA, DC2, DC29, and DC30 have the most similar detection scores with the no-
damage case, as shown in Fig. 17(a) ($W=20000$ and $N=15$). Further reduction
on $W$ and $N$ reduces their differences, and the AUC values eventually drop
even more. The same trend can be seen on the TL of QUGS with Yellow Frame
$W=2000$ and $N=15$ (Fig. 17(b)). It still offers good discrimination between
no-damage and all damage cases, including DC2 and DC30. The spectra of all
experiments’ AUC values are shown in Figs. 18(a) and 18(a), with the source
Yellow Frame and Z24, respectively. With $N=15$, which are half of QUGS
sensors, and with different $W$s, 27 damage cases are seamlessly
differentiated from the no-damage case with AUC scores close to 1. Yet, for
the cases DC2, DC29, and DC30, lower AUC scores are observed with lower $W$
and $N$. Overall, for the QUGS dataset, the proposed DA method shows
satisfactory robustness to $W$, specifically if more sensors are used.

(a) Y2QS15_2000; ROC of 30 damages.

(b) Z2QS6_2000; ROC of 20 damages.

(c) Z2QS10_1000; ROC of 20 damages.

(d) Y2QS6_1000; ROC of 20 damages.

(e) Y2QS15_500; ROC of 20 damages.

(f) Z2QS6_500; ROC of 20 damages.

Figure 16: TL performance; QUGS as the target domain.

(a) QS15_2000 performance on QUGS (no TL).

(b) Y2QS15_1000 performance on QUGS, given the whole no-damage data as DA
input.

Figure 17: Impact of $W$ and $N$ on source-no-damage identifiers; QUGS target.

(a) Source Yellow Frame.

(b) Source Z24.

Figure 18: TL performance; AUC values for all QUGS TL scenarios.

###  5.3 Target : The Z24

TL for the target structure Z24 with seven damage cases is performed with the
source structures Yellow Frame and QUGS. ROC curves for 6 of the TL scenarios
in Table 4 are shown in Fig. 19. A notable point in Z24 is that it retains
higher AUCs with lower $N$ and $W$. Z24 damages are large-scale (and more
realistic) structural damages compared to the QUGS and Yellow Frame, and thus
lower $W$, and $N$ values do not alter the AUC as much as in those datasets.
Accordingly, DA-fed source-no-damage identifier achieves a high discriminative
potential between all no-damage and damage cases, even with $W=500$ and $N=6$.
All scenarios’ AUCs are presented in Fig. 20, showing the DA robustness to
$W$.

(a) Y2ZS15_2000; ROC of seven damages.

(b) Q2ZS6_2000; ROC of seven damages.

(c) Q2ZS10_1000; ROC of seven damages.

(d) Y2ZS6_1000; ROC of seven damages.

(e) Y2ZS15_500; ROC of seven damages.

(f) Q2ZS6_500; ROC of seven damages.

Figure 19: TL performance; Z24 as the target domain.

(a) Source Yellow Frame.

(b) Source QUGS.

Figure 20: TL performance; AUC values for all Z24 TL scenarios.

###  5.4 TL-based Zero-shot SDD Results

This sub-section demonstrates the proposed threshold tuning algorithm’s
performance in zero-shot SDD. Zero-shot SDD is carried out using each
dataset’s highest $N$ (Table 4) and $W=2000$ for all damage cases in three
target structures: QUGS, Yellow Frame, and Z24. Damage detection results for
target structure Yellow Frame are shown in Figs 21 and 22 when the QUGS and
Z24 are the source structures. For the QUGS structure as the target, SDD
results are shown in Figs 23 and 24 when the Yellow Frame and Z24 are the
source structures. Finally, for the Z24 as the target and Yellow Frame and
QUGS as source structures, SDD results are shown in Figs 25 and 26,
respectively.

(a) Detection scores.

(b) Precision, Recall and F1 values for all cases.

Figure 21: Zero-shot SDD with source QUGS and target Yellow Frame; $W=2000$,
$N=15$.

(a) Detection scores.

(b) Precision, Recall and F1 values for all cases.

Figure 22: Zero-shot SDD with source Z24 and target Yellow Frame; $W=2000$,
$N=15$.

(a) Detection scores.

(b) Precision, Recall and F1 values for all cases.

Figure 23: Zero-shot SDD with source Yellow Frame and target QUGS; $W=2000$,
$N=15$.

The results indicate that with only two possible source structures and dozens
of distinct and complicated damage cases, almost all damages in the target
domains are detected with a high $F1$ score. Another important observation is
that $S$ values have a wide range of distributions, indicating that the
proposed DA and the trained $\mathcal{D}$ models not only distinguish no-
damage and damage cases but also discern between damage cases. With the Yellow
frame as the target, Z24 contrasts between damage cases better (_i.e._ ,
different ranges of $S$) than QUGS, while with the QUGS as the target, the
Yellow frame offers a better contrast compared to Z24. The differences between
the contributions of each source structure to each target structure can be the
basis for future multi-view ensemble studies in more advanced DA applications
for SHM and SDD. Such multi-view approaches tend to lower false alarms in
cases such as Yellow Frame SDD with Z24 as the source, where a non-optimal SDD
threshold provokes false alarms while damage and no-damage cases are
distinguished from each other.

(a) Detection scores.

(b) Precision, Recall and F1 values for all cases.

Figure 24: Zero-shot SDD with source Z24 Frame and target QUGS; $W=2000$,
$N=15$.

(a) Detection scores.

(b) Precision, Recall and F1 values for all cases.

Figure 25: Zero-shot SDD with source Yellow Frame and target Z24; $W=2000$,
$N=15$.

(a) Detection scores.

(b) Precision, Recall and F1 values for all cases.

Figure 26: Zero-shot SDD with source QUGS and target Z24; $W=2000$, $N=15$.

##  6 Conclusions

Deep-Learning (DL)-aided Structural Health Monitoring (SHM) has already
reached high-performance levels on individual structures, ranging from
laboratory-based setups to real-world infrastructure under varying
environmental and operational effects. However, these DL-based SHM methods
offer only case-dependent solutions, hindering SHM applications on populations
of diverse infrastructure (large-scale SHM) where no damage cases are yet
present. Difficulties include the need for labeled data, case-dependent
dimensionality reduction, and method re-training for each new structure. Case
dependency is, thus, the major obstacle to establishing large-scale SHM
tools/methods operating over diverse infrastructure with no comprehensive
prior data. Transfer Learning (TL)—which accepts full-spectrum FFT amplitudes
as features and is well-aligned with the zero-shot learning paradigm—is a
viable solution to address the aforementioned challenges. This study presented
a novel and highly effective TL approach that addresses the aforementioned
limitations and shortcomings. A dedicated detector $\mathcal{D}$ model of GAN
was trained only in the source structure(s), which accomplished highly
accurate damage detection on target domains using a domain adaptation (DA)
technique that was also introduced in the present study.

The proposed approach was rigorously evaluated on three benchmark datasets:
Z24 (seven damage and one no-damage cases), Yellow Frame (20 damage and one
no-damage case), and Qatar University Grandstand Simulator - QUGS (30 damage
and one no-damage cases). Results revealed that the accumulated knowledge on
distinguishing no-damage cases from damage cases is transferred seamlessly
across those datasets. The area under the Receiver Operating Characteristics
curves (Area Under the Curve - AUC) is used for assessing the quality of
differentiation between the no-damage and damage case while avoiding any
threshold. The achieved high AUC values (mean AUC of 0.9986 across all
datasets using all data channels) allow for selecting a simple Structural
Damage Detection (SDD) threshold tuning. For the datasets studied in this
paper, 10% of the no-damage case data is used for spectral mapping and the
other 40% for threshold tuning. A Gaussian distribution is fitted to the
observed data with the mean plus three times the standard deviation set as the
threshold. The proposed zero-shot SDD approach achieved mean precision,
recall, and $F1$ scores that ranged between 0.96 to 0.99 for all target
structures.

## Acknowledgments

The authors would like to thank Dr. Carlos Ventura and Dr. Alexander Mendler
for providing the Yellow Frame dataset and Dr. Giacomo Bernagozzi for
providing its modal information. Resources and codes to reproduce results in
Sections 3.2.2 and 5 will be made publicly available on GitHub upon
publication at https://github.com/Hesam-92-19/Transfer_Learning_SHM, which
will enable users to interact with the repository using their datasets.

## Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to this
article’s research, authorship, and/or publication.

## References

  * [1] R. A. Ramadan, Efficient intrusion detection algorithms for smart cities-based wireless sensing technologies, Journal of Sensor and Actuator Networks 9 (3) (2020).
URL https://www.mdpi.com/2224-2708/9/3/39

  * [2] R. Mohammadi Ghazi, J. G. Chen, O. Büyüköztürk, Pairwise graphical models for structural health monitoring with dense sensor arrays, Mechanical Systems and Signal Processing 93 (2017) 578–592.  doi:https://doi.org/10.1016/j.ymssp.2017.02.026.
URL https://www.sciencedirect.com/science/article/pii/S0888327017300936

  * [3] A. Entezami, H. Shariatmadar, S. Mariani, Fast unsupervised learning methods for structural health monitoring with large vibration data from dense sensor networks, Structural Health Monitoring 19 (6) (2020) 1685–1710.  arXiv:https://doi.org/10.1177/1475921719894186, doi:10.1177/1475921719894186.
URL https://doi.org/10.1177/1475921719894186

  * [4] S. E. Bibri, J. Krogstie, Ict of the new wave of computing for sustainable urban forms: Their big data and context-aware augmented typologies and design concepts, Sustainable Cities and Society 32 (2017) 449–474.  doi:https://doi.org/10.1016/j.scs.2017.04.012.
URL https://www.sciencedirect.com/science/article/pii/S2210670716302475

  * [5] K. Worden, C. R. Farrar, G. Manson, G. Park, The fundamental axioms of structural health monitoring, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 463 (April) (2007) 1639–1664.  doi:https://doi.org/10.1098/rspa.2007.1834.
  * [6] W. Soo Lon Wah, Y.-T. Chen, J. S. Owen, A regression-based damage detection method for structures subjected to changing environmental and operational conditions, Engineering Structures 228 (2021) 111462.  doi:https://doi.org/10.1016/j.engstruct.2020.111462.
  * [7] R. de Almeida Cardoso, A. Cury, F. Barbosa, Automated real-time damage detection strategy using raw dynamic measurements, Engineering Structures 196 (2019) 109364.  doi:https://doi.org/10.1016/j.engstruct.2019.109364.
  * [8] C. A. Perez-Ramirez, J. P. Amezquita-Sanchez, H. Adeli, M. Valtierra-Rodriguez, R. d. J. Romero-Troncoso, A. Dominguez-Gonzalez, R. A. Osornio-Rios, Time-frequency techniques for modal parameters identification of civil structures from acquired dynamic signals, Journal of Vibroengineering 18 (5) (2016) 3164–3185.
URL http://https://doi.org/10.21595/jve.2016.17220

  * [9] R. Langone, E. Reynders, S. Mehrkanoon, J. A. Suykens, Automated structural health monitoring based on adaptive kernel spectral clustering, Mechanical Systems and Signal Processing 90 (2017) 64–78.  doi:https://doi.org/10.1016/j.ymssp.2016.12.002.
URL https://www.sciencedirect.com/science/article/pii/S0888327016305131

  * [10] J. S. Lee, H. Min Kim, S. Il Kim, H. Min Lee, Evaluation of structural integrity of railway bridge using acceleration data and semi-supervised learning approach, Engineering Structures 239 (2021) 112330.  doi:https://doi.org/10.1016/j.engstruct.2021.112330.
URL https://www.sciencedirect.com/science/article/pii/S0141029621004806

  * [11] S. Hoell, P. Omenzetter, Optimal selection of autoregressive model coefficients for early damage detectability with an application to wind turbine blades, Mechanical Systems and Signal Processing 70-71 (2016) 557–577.  doi:https://doi.org/10.1016/j.ymssp.2015.09.007.
URL https://www.sciencedirect.com/science/article/pii/S0888327015004021

  * [12] J. P. Amezquita-Sanchez, H. Adeli, Nonlinear measurements for feature extraction in structural health monitoring, Scientia Iranica 26 (6) (2019) 3051–3059.
  * [13] J. P. Amezquita-Sanchez, H. Adeli, Synchrosqueezed wavelet transform-fractality model for locating, detecting, and quantifying damage in smart highrise building structures, Smart Materials and Structures 24 (6) (2015) 065034.
URL https://doi.org/10.1088/0964-1726/24/6/065034

  * [14] O. Abdeljaber, O. Avci, M. S. Kiranyaz, B. Boashash, H. Sodano, D. J. Inman, 1-d cnns for structural damage detection: Verification on a structural health monitoring benchmark data, Neurocomputing 275 (2018) 1308–1317.  doi:https://doi.org/10.1016/j.neucom.2017.09.069.
URL https://www.sciencedirect.com/science/article/pii/S0925231217315886

  * [15] D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014).  doi:https://doi.org/10.48550/arXiv.1412.6980.
  * [16] A. F. Agarap, Deep learning using rectified linear units (relu), CoRR abs/1803.08375 (2018).  arXiv:1803.08375.
URL http://arxiv.org/abs/1803.08375

  * [17] D. E. Rumelhart, G. E. Hinton, R. J. Williams, Learning representations by back-propagating errors, nature 323 (6088) (1986) 533–536.
URL https://doi.org/10.1038/323533a0

  * [18] O. Avci, O. Abdeljaber, S. Kiranyaz, An overview of deep learning methods used in vibration-based damage detection in civil engineering, in: K. Grimmelsman (Ed.), Dynamics of Civil Structures, Volume 2, Springer International Publishing, Cham, 2022, pp. 93–98.
  * [19] M. Azimi, G. Pekcan, Structural health monitoring using extremely compressed data through deep learning, Computer-Aided Civil and Infrastructure Engineering 35 (6) (2020) 597–614.  doi:https://doi.org/10.1111/mice.12517.
URL https://onlinelibrary.wiley.com/doi/abs/10.1111/mice.12517

  * [20] A. Reddy, V. Indragandhi, L. Ravi, V. Subramaniyaswamy, Detection of cracks and damage in wind turbine blades using artificial intelligence-based image analytics, Measurement 147 (2019) 106823.  doi:https://doi.org/10.1016/j.measurement.2019.07.051.
URL https://www.sciencedirect.com/science/article/pii/S0263224119306803

  * [21] X. Jian, H. Zhong, Y. Xia, L. Sun, Faulty data detection and classification for bridge structural health monitoring via statistical and deep-learning approach, Structural Control and Health Monitoring 28 (11) (2021) e2824.  arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/stc.2824, doi:https://doi.org/10.1002/stc.2824.
URL https://onlinelibrary.wiley.com/doi/abs/10.1002/stc.2824

  * [22] M. H. Soleimani-Babakamali, R. Sepasdar, K. Nasrollahzadeh, R. Sarlo, A system reliability approach to real-time unsupervised structural health monitoring without prior information, Mechanical Systems and Signal Processing 171 (2022) 108913.  doi:https://doi.org/10.1016/j.ymssp.2022.108913.
  * [23] F. Luleci, F. N. Catbas, O. Avci, Generative adversarial networks for labeled acceleration data augmentation for structural damage detection, Journal of Civil Structural Health Monitoring (2022) 1–18doi:https://doi.org/10.1007/s13349-022-00627-8.
URL https://doi.org/10.1007/s13349-022-00627-8

  * [24] M. Zareapoor, P. Shamsolmoali, J. Yang, Oversampling adversarial network for class-imbalanced fault diagnosis, Mechanical Systems and Signal Processing 149 (2021) 107175.  doi:https://doi.org/10.1016/j.ymssp.2020.107175.
  * [25] M. H. Soleimani-Babakamali, R. Soleimani-Babakamali, R. Sarlo, A general framework for supervised structural health monitoring and sensor output validation mitigating data imbalance with generative adversarial networks-generated high-dimensional features, Structural Health Monitoring 21 (3) (2022) 1167–1182.  doi:https://doi.org/10.1177/14759217211025488.
  * [26] M. H. Soleimani-Babakamali, R. Sepasdar, K. Nasrollahzadeh, I. Lourentzou, R. Sarlo, Toward a general unsupervised novelty detection framework in structural health monitoring, Computer-Aided Civil and Infrastructure Engineering 37 (9) (2022) 1128–1145.  doi:https://doi.org/10.1111/mice.12812.
  * [27] Y. Gao, K. M. Mosalam, Deep transfer learning for image-based structural damage recognition, Computer-Aided Civil and Infrastructure Engineering 33 (9) (2018) 748–768.  arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/mice.12363, doi:https://doi.org/10.1111/mice.12363.
URL https://onlinelibrary.wiley.com/doi/abs/10.1111/mice.12363

  * [28] K. Gopalakrishnan, H. Gholami, A. Vidyadharan, A. Choudhary, A. Agrawal, Crack damage detection in unmanned aerial vehicle images of civil infrastructure using pre-trained deep learning model, Int. J. Traffic Transp. Eng 8 (1) (2018) 1–14.
  * [29] H. Tang, Y. Xie, Deep transfer learning for connection defect identification in prefabricated structures, Structural Health Monitoring (2022) 14759217221119537.
URL https://doi.org/10.1177/14759217221119

  * [30] A. Rai, M. Mitra, A transfer learning approach for damage diagnosis in composite laminated plate using lamb waves, Smart Materials and Structures 31 (6) (2022) 065002.
  * [31] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, Imagenet: A large-scale hierarchical image database, in: 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255.  doi:https://doi.org/10.1109/CVPR.2009.5206848.
  * [32] X. Wang, Y. Xia, Knowledge transfer for structural damage detection through re-weighted adversarial domain adaptation, Mechanical Systems and Signal Processing 172 (2022) 108991\.  doi:https://doi.org/10.1016/j.ymssp.2022.108991.
URL https://www.sciencedirect.com/science/article/pii/S0888327022001698

  * [33] L. Bull, P. Gardner, J. Gosliga, T. Rogers, N. Dervilis, E. Cross, E. Papatheou, A. Maguire, C. Campos, K. Worden, Foundations of population-based shm, part i: Homogeneous populations and forms, Mechanical Systems and Signal Processing 148 (2021) 107141.  doi:https://doi.org/10.1016/j.ymssp.2020.107141.
URL https://www.sciencedirect.com/science/article/pii/S0888327020305276

  * [34] J. Gosliga, P. Gardner, L. Bull, N. Dervilis, K. Worden, Foundations of population-based shm, part ii: Heterogeneous populations – graphs, networks, and communities, Mechanical Systems and Signal Processing 148 (2021) 107144.  doi:https://doi.org/10.1016/j.ymssp.2020.107144.
URL https://www.sciencedirect.com/science/article/pii/S0888327020305306

  * [35] P. Gardner, L. Bull, J. Gosliga, N. Dervilis, K. Worden, Foundations of population-based shm, part iii: Heterogeneous populations – mapping and transfer, Mechanical Systems and Signal Processing 149 (2021) 107142.  doi:https://doi.org/10.1016/j.ymssp.2020.107142.
URL https://www.sciencedirect.com/science/article/pii/S0888327020305288

  * [36] G. Tsialiamanis, C. Mylonas, E. Chatzi, N. Dervilis, D. Wagg, K. Worden, Foundations of population-based shm, part iv: The geometry of spaces of structures and their feature spaces, Mechanical Systems and Signal Processing 157 (2021) 107692\.  doi:https://doi.org/10.1016/j.ymssp.2021.107692.
URL https://www.sciencedirect.com/science/article/pii/S088832702100087X

  * [37] M. H. Soleimani-Babakamali, R. Soleimani-Babakamali, R. Sarlo, M. F. Farghally, I. Lourentzou, On the effectiveness of dimensionality reduction for unsupervised structural health monitoring anomaly detection, Mechanical Systems and Signal Processing 187 (2023) 109910.  doi:https://doi.org/10.1016/j.ymssp.2022.109910.
URL https://www.sciencedirect.com/science/article/pii/S0888327022009785

  * [38] A. Mendler, C. Ventura, S. Allahdadian, The yellow frame: experimental studies and remote monitoring of the structural health monitoring benchmark structure, in: Special Topics in Structural Dynamics, Volume 5, Springer, 2019, pp. 233–244.  doi:https://doi.org/10.1007/978-3-319-75390-4_21.
  * [39] Artemis modal (Aug 2020).
URL https://svibs.com/

  * [40] R. Brincker, C. Ventura, Introduction to operational modal analysis, John Wiley & Sons, 2015.  doi:https://doi.org/10.1002/9781118535141.
  * [41] W. Soo Lon Wah, J. S. Owen, Y.-T. Chen, A. Elamin, G. W. Roberts, Removal of masking effect for damage detection of structures, Engineering Structures 183 (2019) 646–661.  doi:https://doi.org/10.1016/j.engstruct.2019.01.005.
  * [42] H. Sarmadi, K.-V. Yuen, Early damage detection by an innovative unsupervised learning method based on kernel null space and peak-over-threshold, Computer-Aided Civil and Infrastructure Engineering 36 (9) (2021) 1150–1167.  doi:https://doi.org/10.1111/mice.12635.
  * [43] E. Reynders, G. Wursten, G. D. Roeck, Output-only structural health monitoring in changing environmental conditions by means of nonlinear system identification, Structural Health Monitoring 13 (1) (2014) 82–93.  arXiv:https://doi.org/10.1177/1475921713502836, doi:10.1177/1475921713502836.
URL https://doi.org/10.1177/1475921713502836

  * [44] E. Reynders, G. D. Roeck, Continuous vibration monitoring and progressive damage testing on the z 24 bridge, Encyclopedia of structural health monitoring (2009).
  * [45] E. Reynders, G. De Roeck, Vibration-based damage identification: the z24 benchmark (2014).
URL https://lirias.kuleuven.be/1725994?limo=0

  * [46] O. Avci, O. Abdeljaber, S. Kiranyaz, M. Hussein, D. J. Inman, Wireless and real-time structural damage detection: A novel decentralized method for wireless sensor networks, Journal of Sound and Vibration 424 (2018) 158–172.  doi:https://doi.org/10.1016/j.jsv.2018.03.008.
  * [47] O. Avci, O. Abdeljaber, S. Kiranyaz, M. Hussein, M. Gabbouj, D. Inman, A new benchmark problem for structural damage detection: Bolt loosening tests on a large-scale laboratory structure, in: K. Grimmelsman (Ed.), Dynamics of Civil Structures, Volume 2, Springer International Publishing, Cham, 2022, pp. 15–22.
  * [48] O. Abdeljaber, A. Younis, O. Avci, N. Catbas, M. Gul, O. Celik, H. Zhang, Dynamic testing of a laboratory stadium structure, in: Geotechnical and Structural Engineering Congress 2016, 2016, pp. 1719–1728.
  * [49] O. Abdeljaber, O. Avci, S. Kiranyaz, M. Gabbouj, D. J. Inman, Real-time vibration-based structural damage detection using one-dimensional convolutional neural networks, Journal of Sound and Vibration 388 (2017) 154–170.  doi:https://doi.org/10.1016/j.jsv.2016.10.043.
  * [50] H. Zhang, I. Goodfellow, D. Metaxas, A. Odena, Self-attention generative adversarial networks, in: K. Chaudhuri, R. Salakhutdinov (Eds.), Proceedings of the 36th International Conference on Machine Learning, Vol. 97 of Proceedings of Machine Learning Research, PMLR, 2019, pp. 7354–7363.
  * [51] J. Xu, X. Ren, J. Lin, X. Sun, Diversity-promoting GAN: A cross-entropy based generative adversarial network for diversified text generation, in: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Brussels, Belgium, 2018, pp. 3940–3949.  doi:10.18653/v1/D18-1428.
  * [52] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial nets, in: Advances in neural information processing systems, 2014, pp. 2672–2680.
  * [53] J. F. Nash, Equilibrium points in <i>n</i>-person games, Proceedings of the National Academy of Sciences 36 (1) (1950) 48–49.  doi:https://doi.org/10.1073/pnas.36.1.48.
URL https://doi.org/10.1073/pnas.36.1.48

  * [54] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, S. Chintala, Pytorch: An imperative style, high-performance deep learning library, in: H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, R. Garnett (Eds.), Advances in Neural Information Processing Systems 32, Curran Associates, Inc., 2019, pp. 8024–8035.
URL http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-
performance-deep-learning-library.pdf

  * [55] S. J. Raymond, D. B. Camarillo, Applying physics-based loss functions to neural networks for improved generalizability in mechanics problems, arXiv preprint arXiv:2105.00075 (2021).  doi:https://doi.org/10.48550/arXiv.2105.00075.
URL https://doi.org/10.48550/arXiv.2105.00075

  * [56] S. Amini Niaki, E. Haghighat, T. Campbell, A. Poursartip, R. Vaziri, Physics-informed neural network for modelling the thermochemical curing process of composite-tool systems during manufacture, Computer Methods in Applied Mechanics and Engineering 384 (2021) 113959.  doi:https://doi.org/10.1016/j.cma.2021.113959.
URL https://www.sciencedirect.com/science/article/pii/S0045782521002966

  * [57] P. Welch, The use of fast fourier transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms, IEEE Transactions on audio and electroacoustics 15 (2) (1967) 70–73.