The discrete sequence was obtained through the processing of the TKE operator. First, full-wave rectification was applied to the discrete sequence. The time window method was used to obtain the envelope to shield the small fluctuations in the discrete sequence and obtain the overall trend of the signal.

The time window length was set to be n, and the envelope at the time point i was defined as the maximum sequence amplitude in the time window. Since the minimum duration of the nerve action potential is 2 ms, the default value of *n* is 2 ms if the expert does not have an extra set.

$$Skne(i) = \max(\varphi(j)), j \in \left[\mathbf{i} - \frac{n}{2}, \mathbf{i} + \frac{n}{2}\right] \tag{4}$$

The characteristic of burst start/end is that the signal changes from stationary white noise signal to maximum peak value. Therefore, the start/end position of the burst can be preliminarily detected in the light of the magnitude *Sknae*(*i*) of the envelope and the derivative *d*(*i*) of the point, if:

$$d(i) = Sknac(i) - sknac(i-1)\tag{5}$$

$$\text{Sknae}(i) \ge t \text{ and } d(i) > 0 \tag{6}$$

$$
\mathcal{C}th = \lambda \times Skna\_{\text{max}}\tag{7}
$$

where eth is the envelope threshold, which is defined as an empirical value based on the maximum value of the baseline. *λ* is an empirical parameter, which is adjusted according to different subjects. Commonly, *λ* is set to 1. That is, the maximum value of the baseline is used as the threshold of the envelope by default. Similarly, the end of the burst is defined as the envelope value less than t, and the derivative is less than 0.

Through the above operation, the start/end position of the burst was detected. However, the signal rose unsteadily, so the start/end position needs to be adjusted slightly. The significance of using the two methods is that the envelope can be used for rough segmentation. The integral signal is used to adjust the starting point position in more detail to segment the signal more accurately. The integral of *ϕ*(*n*) is the integral area of the sequence in unit time. It can reflect the detailed changes of sequence better than the envelope. Therefore, the interference of the signal jitter on the envelope start/end position can be reduced to obtain more accurate burst detection results. The *ϕIntegral* (*n*) is defined as:

$$
\varphi\_{\text{Integral}}\left(n\right) = \frac{1}{\Delta t} \int\_{i - \frac{1}{2\Delta t}}^{i + \frac{1}{2\Delta t}} \varphi(j) \tag{8}
$$

where *t* is the length of the time window. In the above process, a larger time window of signal envelope can better reflect the overall trend of the sequence. In this operation, selecting the time window *t* as half of the envelope window n can obtain more accurate segmented positions based on rough segmentation.

The rough segmentation envelope length *L* can be calculated according to the following formula:

$$L = EP - SP\tag{9}$$

where *SP* is the starting position of the burst and *EP* is the ending position of the burst. The range of the moving time window is one-tenth of the envelope length *L* in the adjustment step. The segmented burst area in this step is shown in Figure 3d. Therefore, the new starting and ending position is:

$$SP\_{new} = \min\left(\varphi\_{\text{Integral}}\left(i\right)\right), i \in \left[SP - \frac{1}{10}L, SP + \frac{1}{10}L\right] \tag{10}$$

$$EP\_{new} = \min\left(\varphi\_{\text{Integral}}\left(i\right)\right), \ i \in \left[EP - \frac{1}{10}L\_\prime EP + \frac{1}{10}L\right] \tag{11}$$

#### 2.3.3. Discrimination of Artifact Bursts

Burst in the signal was detected through the above process, including true nerve burst and false burst. Since the characteristics of the false burst are similar to the trust burst, they are difficult to distinguish based on the envelope.

SKNA signal was obtained from the body surface by standard lead. Therefore, the ECG artifact was the major noise source in SKNA. Although most energies of ECG were filtered out after a 500–1000 Hz band-pass filter, some residual energy still existed as background noise, especially in QRS complexes. Firstly, the R-peaks detection was performed on ECG using an open-source QRS detector [27]. Then, adaptive thresholds were applied to the length signal to determine the onset and duration of the QRS complexes [28]. In this way, we obtained two pieces of information: the position of the ECG artifact and the maximum width of each ECG artifact [29]. If:

$$SP < Index\_R < EP \text{ and } \mathcal{W}\_{qrs} < L \tag{12}$$

where *IndexR* is the index of the position of the *R* wave and *Wqrs* is the duration of the QRS complexes. That is, in case the burst area contains the position of the *R* wave and the width of the burst area is smaller than the width of the QRS complex, the burst area is supposed to be a false positive.

As shown in Figure 3, false positives resulting from short bursts in SKNA are discriminated by applying time thresholds to the start/end positions. In other words, start/end positions that had time differences shorter than Ts seconds were removed. This discriminating operation was applied after the start/end positions had been detected. This operation

allowed the algorithm to have a sensitive threshold for the on-time [30]. Since the minimum duration of the nerve action potential is 2 ms [31], the default value of Ts is 2 ms if the expert does not have an extra set.

#### *2.4. SKNA Energy Ratio*

In [32,33], aSKNA was used to assess SKNA over a period. The length of the period depends on the length of time required for clinical analysis, 30 s [11], 5 min [12], and half an hour or more [22]. The feature can be calculated in two steps. First, iSKNA is obtained by calculating the sum of the areas under the SKNA curve in unit time, as shown in Formula (7). The unit time was set to 0.1 s in the previous research [12]. Then, the amount of SKNA can be quantified by calculating the average of iSKNA, which is defined as aSKNA.

$$aSKNA = \frac{\sum \text{i}SKNA(i)}{N} \tag{13}$$

where *N* is the ratio of the time window of the calculated feature to that of *iSKNA*. For example, *N* is 3000 if the time window of *aSKNA* is 5 min. Noise has a great impact on this feature, especially the baseline noise and impulse noise. Moreover, due to the differences in sensors and subjects, this feature is unstable in individual comparison. Therefore, according to the concept of signal-to-noise ratio (*SNR*), we defined a feature to estimate SNA by calculating the ratio of the detected burst sequence energy to the baseline energy. The *SKNA* energy ratio (*SKNAER*) is defined as:

$$SKNAER = 10 \log\_{10} \frac{P\_{burst}}{P\_{baseline}} \tag{14}$$

where *Pburst* is the total energy of the nerve burst area detected after step 4 as show in Figure 3. *Pbaseline* is the total energy of the baseline area detected after step 3. It is worth noting that the discriminated artifact burst in step 4 is neither baseline nor nerve burst. It is recognized as noise and is not used to calculate the feature. To avoid error calculation caused by extreme conditions, the signal segment is discriminated as abnormal for secondary processing when it is determined to be all baseline or burst.

#### *2.5. Evaluation Methods*

To verify the effectiveness and accuracy of the algorithm, the data were manually labeled by an expert. The expert scrolled through the data using a custom graphical user interface tool and manually placed onsets and offsets of SKNA burst. For the annotation of burst, firstly, we recorded the start/end time of the activation action in the standard signal recording experiment. Based on the start/end time of the activation action and the relative amplitude of the signal, experts trimmed the start/end position of the burst.

The first problem is as follows: what was the detection quality of the proposed algorithm compared with the expert? This was answered with the data of Experiment 1.

To answer the first question, we obtained detection rate and precision. These measures were defined in terms of the following quantities and expressed as percentages. The detection rate was obtained to quantify the difference in the number of detected pairs of onsets/offsets and the number of pairs manually labeled by the expert. It was defined as the percentage of the number of true onset/offset pairs detected by the algorithm to the number of bursts labeled by the expert.

True positives (TPs): numbers of burst areas classified as the burst by both the expert and the algorithm.

False positives (FPs): numbers of burst areas classified as the burst by the algorithm and not by the expert.

False negatives (FNs): numbers of burst areas not classified as the burst by the algorithm and classified as the burst by the expert.

The detection rate and precision were defined as follows.

$$DR = \frac{TP}{TP + FN} \times 100\% \tag{15}$$

$$P\_{+} = \frac{TP}{TP + FP} \times 100\% \tag{16}$$

In addition, we also calculated the coincidence to answer the first question. The coincidence was defined to compare the coincidence degree of the automatic/manual segmentation results by algorithm/expert. The coincidence was computed as the ratio of the manual segmentation length of the overlapping part of automatic segmentation length and manual segmentation length.

$$CO = \frac{\min\left(EP\_{d\cdot}, EP\_{\mathcal{C}}\right) - \max\left(SP\_{d\cdot}, SP\_{\mathcal{C}}\right)}{EP\_{\mathcal{C}} - SP\_{\mathcal{C}}} \tag{17}$$

where *SPa* and *EPa* are the automatic segmentation of starting and ending position by algorithm; *SPe* and *EPe* are the manual segmentation of starting and ending position by expert.

#### *2.6. Reference Features*

The second question is what is the clinical effect of the proposed feature, and whether it is more effective in evaluating SNA than other features? The second question can be answered by using the data from Experiment 1 and 2. In Experiment 1, the CPT and the VM are the standard procedures for triggering sympathetic discharges [32,33], which can increase blood pressure, heart rate, and SNA level. In Experiment 2, hemodialysis is to purify the blood by dispersing and circulating all kinds of harmful and redundant metabolic wastes and excess electrolytes out of the body to achieve the purpose of correcting water-electrolyte and acid-base balance. As a result, pressure on the autonomic nervous system decreases, and SNA changes with the release of toxic substances. In this study, we used two methods to process data to verify the effectiveness of SKNA-based evaluation. One method was to use the proposed method to calculate the SKNAER in different periods. The other was to calculate the aSKNA of different periods without additional processing.

In order to further compare the differences in features, we calculated low frequency/ high frequency (LF/HF), the standard deviation of NN intervals (SDNN), and very low frequency (vLF) based on HRV. These indicators can reflect the characteristics of SNA. The increase of LF/HF and vLF represented the increase in SNA [34]. The decrease in SDNN indicated an increase in SNA [35]. In addition, the paired t-test was performed to evaluate the difference in features before and after time.

#### **3. Results**

Table 2 shows the statistical results of Experiment 1 in detail. The observed burst number of the signal collected from the arm was less than that from the chest, especially in the biceps position. In addition, the false positives and false negatives of the signal from the arm were also more than those of the signal from the chest. This may be due to more EMG interference on the arm. From this point of view, the signals obtained from the chest position and the forearm may have better signal quality than that of the biceps position. The proposed algorithm was verified on standard datasets. We calculated the burst detection accuracy of signals from three acquisition positions, respectively. The results showed that the detection rate and precision of the proposed algorithm on the acquired signal from the chest were 100.0 ± 0% and 94.2 ± 5.0%, respectively. Although the detection rate and precision of the algorithm decreased on the acquired signal from the biceps and forearm, the detection rate and precision were still acceptable, 96.4 ± 5.5% and 87.3 ± 7.4%. In addition, the coincidence area was also calculated in the experiment. The coincidence of the signal from the chest was 96.4 ± 1.2%, and it was higher than that of the signal

from the arm. Experimental results showed that the proposed algorithm had a satisfactory performance on the acquired signal from a different position.

**Table 2.** The mean, standard deviation, and confidence interval (90% CI) of detection rate, coincidence, and precision of the proposed algorithm on the acquired signal of Experiment 1. \* *p* < 0.05, \*\* *p* < 0.01, \*\*\* *p* < 0.001.

Compared to the proposed method, there were more false positives and false negatives using the method with iSKNA. On the one hand, the increase of false negatives in the biceps position was more than that in other positions. In the acquisition position with high signal quality, the detection rate of the method with iSKNA was not much different from that of the proposed method. However, the detection rate of the signal collected at the biceps position was reduced to 87.1 ± 11.0%. On the other hand, the difference between the method with iSKNA and the proposed method was mainly reflected in the number of false positives, resulting in the difference in precision. Since there was no additional processing for ECG artifacts, there were a number of false positives using the method with iSKNA. The precision and CO of the method with iSKNA were much lower than those of the proposed method. In general, the proposed method had better performance in burst detection than the method with iSKNA, especially in the signal with low signal quality.

Figure 4a–d indicate the correlation between SKNAER and the other features of the ten patients before and after sympathetic activation in Experiment 1. The results showed that the SKNAER was positively correlated with SDNN (r = 0.60), LF/HF ratio (r = 0.67), vLF power (r = 0.47), and aSKNA (r = 0.67). Figure 5a–e show the box diagram of the features related to the SNA before and after sympathetic activation in Experiment 1. The HRV features and SKNA features of sympathetic activation one minute before and one minute after were calculated. All features showed an upward trend after sympathetic activation. For the HRV features, SDNN increased from 48.75 to 93.19 ms (*p* < 0.001). From the perspective of this feature, SNA showed a downward trend. vLF increased from 5510.96 to 12,213.52 ms2 (*p* < 0.05), and three outliers occurred in the experiment. LF/HF increased from 1.94 to 4.27 (*p* = 0.15). For the SKNA features, aSKNA increased from 0.91 to 1.42 μV (*p* < 0.001) and SKNAER increased from −13.76 to 8.29 dB (*p* < 0.001). The calculated values of SKNAER before and after sympathetic activation had no overlap. That is, they were almost unaffected by individuals.

**Figure 4.** (**a**–**d**) The correlation between SKNAER and HRV features of the ten patients before and after sympathetic activation in Experiment 1.

**Figure 5.** (**a**–**e**) The statistical results of the features related to the sympathetic nervous activity before and after sympathetic activation in Experiment 1. \* *p* < 0.05, \*\*\* *p* < 0.001.

The whole data process of renal dialysis for each patient lasted about 4 h. Taking 30 min as a window, data were divided into eight periods. Figure 6 shows the trend of SKNA and HRV features during dialysis in 20 patients with renal failure in the clinical experiment. The SKNA and HRV features of each period were calculated to assess the SNA of renal dialysis patients. For the SKNA features, these two features decreased the second time, aSKNA from 1.19 to 1.05 μV (*p* < 0.01), SKNAER from 1.99 to −3.04 dB (*p* < 0.001). Then, these features increased in the fourth time period, aSKNA from 1.03 to 1.15 μV (*p* = 0.07), SKNAER from −2.45 to 1.94 dB (*p* < 0.001), and began to decline in the sixth time period, aSKNA from 1.24 to 1.12 μV (*p* = 0.11), SKNAER from 2.25 to −0.87 dB (*p* < 0.01), and remained stable until the end of the operation.

For the HRV features, LF/HF maintained an upward trend in the first two hours, especially the fourth period, from 2.13 to 2.70 (*p* < 0.05), and dropped from 2.19 to 1.60 in the sixth period (*p* < 0.01), and then finally increased, reaching 2.01 in the eighth period. vLF fluctuated greatly in the first two hours. It decreased from 1043.50 to 728.80 in the third period (*p* < 0.05) and quickly increased to 918.3 in the fourth period (*p* < 0.05). The trend in the last two hours was similar to that of LF/HF, which decreased first and then increased. SDNN rose continuously in the first three periods and fell from 42.88 to 38.47 ms in the fourth period. It increased in the sixth period, from 38.49 to 40.79 ms, and remained stable. The common trend of the HRV features was that there was a significant difference in the fourth period, indicating the activation of SNA. There was also a significant difference in the sixth period, indicating inhibition of SNA. The variation tendencies of HRV features were consistent with those of aSKNA and SKNAER. However, SKNAER had a more significant difference than aSKNA in evaluating SNA at different times, especially in the fourth and sixth periods.

**Figure 6.** (**a**–**e**) The trend of the SKNA and HRV features in the twenty patients during four-hour hemodialysis. \* *p* < 0.05, \*\* *p* < 0.01, \*\*\* *p* < 0.001.

#### **4. Discussion**

In this work, we developed a burst area detection algorithm and verified the accuracy of the standard sympathetic nerve activation experiments, including the Valsalva experiment and the CPT experiment. We obtained SKNA signals by placing three groups of electrode sensors on the human surface. The data were automatically detected by the proposed algorithm and manually labeled by experts receptively. Experimental results indicated that the consistency of the detected burst area between the algorithm and the expert was high. However, the number of burst areas observed from different positions was different, and the burst number from the biceps was smaller than that of the chest. In other words, the proposed algorithm can effectively locate the burst area, but it cannot further decouple the mixed noise and neural signals. Although the SKNA signal originates from ganglion, the effective information obtained by the signal was different due to the influence of the interference. Preprocessing multichannel signals with principal component analysis and other methods may lay a better foundation before burst segmentation.

The accuracy of burst segmentation was verified on the sympathetic activation dataset. The results showed that the detection rate on the signal of the chest reached 100%. Although the detection rate decreased on the signal of the forearm and biceps, it was also greater than 95%. Therefore, the proposed algorithm showed good performance in different acquisition positions. In addition, the coincidence degree greater than 95% indicated that the sensitivity to detect the burst area was close to the manual label of the expert. Compared with the method with iSKNA, the advantages of the proposed method were the improvement of detection rate on low signal-to-noise ratio signals and the discrimination of false positives. However, due to the complex neural changes and other confounding factors in the experiment, the number of false positives was not satisfactory enough, especially for the acquisition signal at the biceps. This may be affected by the accuracy of the QRS complexes' detected algorithm and EMG artifact, which led to the unrecognized false burst. Further analysis of the difference between the true and false burst may provide a new train of thought to reduce the misjudgment rate.

SKNA-based SNA evaluation has great potential in pathogenesis research such as atrial fibrillation [32] and myocardial infarction [22]. The SNA evaluation is often quantified by the estimation of mean burst amplitude [36], total burst amplitude [37], or burst area [38]. However, direct calculation of the mean burst amplitude, such as aSKNA [22,32], greatly impacted the evaluation of SNA because of the signal noise. The outlier of aSKNA further proved this point during hemodialysis. We discriminated and removed the non-typical burst after detecting the burst area and calculated the burst area energy ratio to assess the SNA. This feature had better significance in reflecting the trend of the SNA during the Hemodialysis experiment. Observational results of the hemodialysis experiment showed that the SNA gradually decreased in the second period, which was the patients' process from dynamic to static. It increased in the fourth period. This is the period when the most malignant clinical events may occur. In the sixth period, the sympathetic nerves returned to calm. The patients gradually completed dialysis for harmful substances in the body during this period. In the eighth period, the SNA increased, and the patients' activity increased at the last moment. From the perspective of aSKNA, except for the second period, the sympathetic nerve of the patient did not change significantly compared with the previous time. However, we observed more time-to-time differences from SKNAER. From this point of view, it can help us better understand the changes in the sympathetic nerves of patients during hemodialysis.

For Experiment 1, VM and CPT are standard sympathetic stimulation procedures. Results showed that there was no significant difference between HRV-based features and SKNA-based features in indicating SNA. It is worth noting that although SDNN showed significant changes before and after sympathetic stimulation procedures, the indicated SNA was actually the opposite. Although SDNN can effectively indicate the changes in the sympathetic nerve in most clinical studies, it is difficult to explain in some specific clinical scenes [39], especially in short-term measurement. It may be because there are many interference factors for RR interval in short-term analysis, such as false heartbeat [40] and algorithm error. Therefore, SKNA-based features were better than HRV-based features in the short-term analysis of SNA. Furthermore, the values of SKNAER before and after sympathetic activation did not overlap, while those of other features overlapped to a certain extent. In other words, SKNAER can better avoid individual differences compared to the other features. This can help us compare the SNA of different individuals, not just individuals themselves. According to the experimental results of this study, the frequency domain index of HRV may have a better correlation and interpretation with SKNAER in short-term analysis, such as LF/HF.

For Experiment 2, the trends of all features were not completely consistent in reflecting changes in SNA. However, these features showed the same change trend in some specific time periods, the fourth and sixth time periods, respectively. In other words, although the two kinds of features were calculated based on different signals, they were consistent in reflecting the trend of SNA at some levels. For the SNA evaluation, many features provided indirect explanations. However, these features sometimes did not always show the same trend [41,42]. This may be because the signal was doped with various interference factors, resulting in inaccurate interpretation. For example, HRV features were calculated based on RR intervals. The RR interval is often affected by the accuracy of the R-wave-detecting algorithm. In addition, patients with atrial fibrillation and premature beats also have a negative impact on the calculation results because of their special heart rhythm. Similarly, SKNAER is also affected by these factors. The wrong R-wave location result may lead to the misjudgment of the SKNA burst, which reduces the precision of the algorithm, especially in the signal with low signal quality. However, the positive outcome is that we discriminated and removed some interference in the calculation to provide a more accurate evaluation of SNA. In addition, SKNAER is calculated based on the energy of burst and baseline, so the influence of R-wave positioning accuracy on SKNA is indirect and less than that of the HRV index. With the advanced QRS positioning algorithm, the influence of R-wave positioning error on the SKNA feature can be further reduced. Furthermore, we established an evaluation method based on SKNA. Since SKNA is the real-time signal transmitted from the sympathetic nerve to the body surface, while ECG is the change of potential cardiac signal caused by the sympathetic nerve affecting cardiac function, thus the method based on SKNA is more direct than that of ECG in evaluating SNA. This also provides an auxiliary basis for the analysis of disease mechanisms.

The findings from this study should be considered in light of several factors. First, the proposed feature assumes that there is sympathetic activation in the measured data segment. The feature may output an error in case there is no burst area in the data segment or the noise covers the burst. Second, the shape and frequency of the burst were not considered for the proposed feature. This more in-depth information should be further processed and characterized to be applied to some specific clinical scenarios. Last, although the TKE factor is used to enhance the amplitude-frequency change of the signal, this operation may also enhance the amplitude-frequency change of some artifacts, resulting in false positives. Research on the conduction mode of nerve signals to the body surface and then distinguishing between effective nerve burst and noise is conducive to further finding effective information from SKNA. In addition, SKNA is greatly affected by motion artifacts The impact comes from the electrodes used to collect signals and the lack of appropriate standards to distinguish effective signals from motion artifacts. This may limit the further application of SKNA. Therefore, SKNA can be used as an effective supplement rather than a substitute for HRV in some specific scenarios, especially in short-term measurement.

#### **5. Conclusions**

This paper proposed an SNA evaluation method based on SKNA burst area detection. This method exhibited good performance in terms of detection rate, concordance, and precision on the sympathetic activation dataset manually labeled by the experts. The trend of SNA during hemodialysis was analyzed quantitatively based on the detected burst area. The results showed that SKNAER has a consistent trend in evaluating SNA compared with HRV features. Moreover, it had a more significant difference in the long-term SNA evaluation than aSKNA, consistent with the HRV features. The automatic burst detection algorithm proposed in this work can accurately locate the position of the active SNA, which is helpful to evaluate SNA more accurately. With further development, this new modality could play an important role in continuous monitoring of the autonomic nervous system status, as well as preventing correlated diseases. Future work will explore more useful physiological features for evaluating the autonomic nervous system, such as time-frequency analysis and non-linear dynamic evaluation.

**Author Contributions:** Conceptualization, C.L. and Y.X.; methodology, Y.X.; software, J.L. (Jiayi Li); validation, Z.X. and J.L. (Jiayi Li); formal analysis, H.C.; investigation, Y.Z.; resources, C.C. and J.W.; data curation, Y.Z.; writing—original draft preparation, Y.X.; writing—review and editing, Y.X. and C.Y.; visualization, Y.X.; funding acquisition, J.L. (Jianqing Li) and. C.L. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was supported by the National Key Research and Development Program of China (2019YFE0113800), the National Natural Science Foundation of China (62171123, 81871444, 62001111, and 62071241), and the Natural Science Foundation of Jiangsu Province of China (BK20190014, BK20192004, and BK20200364).

**Institutional Review Board Statement:** The study was conducted in accordance with the Declaration of Helsinki, and approved by the Ethics Committee of the First Affiliated Hospital of Nanjing Medical University, understudy number 2020-SRFA-183.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study. Written informed consent was obtained from the patient(s) to publish this paper.

**Data Availability Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

## *Article* **An Intelligent ECG-Based Tool for Diagnosing COVID-19 via Ensemble Deep Learning Techniques**

**Omneya Attallah**

Department of Electronics and Communications Engineering, College of Engineering and Technology, Arab Academy for Science, Technology and Maritime Transport, Alexandria 1029, Egypt<EMAIL_ADDRESS>
**Abstract:** Diagnosing COVID-19 accurately and rapidly is vital to control its quick spread, lessen lockdown restrictions, and decrease the workload on healthcare structures. The present tools to detect COVID-19 experience numerous shortcomings. Therefore, novel diagnostic tools are to be examined to enhance diagnostic accuracy and avoid the limitations of these tools. Earlier studies indicated multiple structures of cardiovascular alterations in COVID-19 cases which motivated the realization of using ECG data as a tool for diagnosing the novel coronavirus. This study introduced a novel automated diagnostic tool based on ECG data to diagnose COVID-19. The introduced tool utilizes ten deep learning (DL) models of various architectures. It obtains significant features from the last fully connected layer of each DL model and then combines them. Afterward, the tool presents a hybrid feature selection based on the chi-square test and sequential search to select significant features. Finally, it employs several machine learning classifiers to perform two classification levels. A binary level to differentiate between normal and COVID-19 cases, and a multiclass to discriminate COVID-19 cases from normal and other cardiac complications. The proposed tool reached an accuracy of 98.2% and 91.6% for binary and multiclass levels, respectively. This performance indicates that the ECG could be used as an alternative means of diagnosis of COVID-19.

**Keywords:** deep learning; COVID-19; ECG trace image; transfer learning; Convolutional Neural Networks (CNN); feature selection

#### **1. Introduction**

At the end of December 2019, the world faced a new type of threatening disease called coronavirus, commonly known as COVID-19 [1]. Based on statistics announced by the World Health Organization (WHO) [2], more than 190 million cases of COVID-19 and more than 4 million cases of mortality have been reported worldwide on 31 July 2021. Due to the rapid propagation and the massive increase in the number of new infections of such a disease, the world faced new challenges [3]. These challenges involved travel constraints, countries' lockdown, social distancing, and curfews. Most importantly, healthcare associations of many countries were about to collapse due to the superfluous number of COVID-19 infections that needed beds and deficiencies in vital medical kits and supplies. Consequently, the rapid and precise diagnosis of COVID-19 is important to lower mortality rates and avert the encumbrance on health organizations.

Based on the COVID-19 diagnosis provided by the Chinese government, the realtime reverse transcription-polymerase chain reaction (RT-PCR) test is the gold standard for the diagnosis of COVID-19 [4]. However, late sample acquisition, firm laboratory setting restrictions, and the requirement of qualified experts to perform the RT-PCR exam could lead to a prolonged and inaccurate diagnosis [5]. Therefore, more efficient methods are needed to achieve a more precise and faster diagnosis. Among these approaches are antigen tests and medical imaging, including computed tomography (CT) and X-ray imaging techniques. Although COVID-19 antigen tests are faster and cheaper than the RT-PCR test, they very often produce inaccurate results. The major limitation of antigen

**Citation:** Attallah, O. An Intelligent ECG-Based Tool for Diagnosing COVID-19 via Ensemble Deep Learning Techniques. *Biosensors* **2022**, *12*, 299. https://doi.org/ 10.3390/bios12050299

Received: 8 March 2022 Accepted: 24 April 2022 Published: 5 May 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

tests is their low sensitivity, leading to high false negative outcomes, therefore it is not recommended by the WHO [6]. Moreover, it has been reported to have lower sensitivity compared to RT-PCR tests [7]. On the other hand, CT and X-ray imaging modalities play an important role in the diagnosis of lung-related abnormalities. Numerous research articles have proven the ability of the X-ray and CT modalities to achieve more accurate results than RT-PCR [8,9]. However, these imaging modalities require the presence of a skilled specialized radiologist to perform the diagnosis. Furthermore, the COVID-19 diagnosis procedure is difficult due to the symmetry among the patterns of the new coronavirus and other sorts of similar diseases [10]. Furthermore, the manual investigation requires a long time and thus automatic diagnostic tools are compulsory to decrease observation time and exertion achieved by experts to perform the diagnosis and produce a precise decision.

Artificial intelligence (AI) techniques aim to create automated diagnostic tools capable of analyzing medical data (such as images and bio-signals) simply and fast. They have been utilized successfully to enhance prognosis and diagnosis of various disorders and diseases [11–20]. The ability of AI techniques to facilitate the new coronavirus has been proven in the survey article [21]. Currently, deep learning (DL) approaches are widely used to construct automated diagnostic tools using radiograph images to support the diagnosis of COVID-19 and avoid the challenges of manual inspection [22,23]. Regardless of the achievements of DL methods in diagnosing COVID-19 using radiographic images, these scanning techniques have some limitations. These shortcomings include high cost, immobility, exposure to a large amount of radiation, and the requirement for qualified technicians to acquire these images [24]. Hence, new diagnostic tools based on other modalities are needed to assist in COVID-19 diagnosis whilst the epidemic persists.

It is well-known that COVID-19 primarily affects the respiratory system; however, it also affects the cardiovascular system [25,26]. Numerous research articles have shown various types of cardiovascular alterations in people with COVID-19. These variations involve divergence of the ST segment of the PR interval [27], arrhythmias [28], QRST changes, and conduction disorders [25]. These cardiac variations can be visualized on the electrocardiogram (ECG) of patients with COVID-19. Such cardiovascular modifications [29] have promoted the study of ECG data as a new means of diagnosing the novel coronavirus. Looking at the huge advantages of using ECG, including low cost, mobility, simplicity of use, safe, harmless, and providing real-time monitoring, automated diagnostic tools for COVID-19 based on ECG data could be of significant value in addition to imaging modalities and PCR exams. Thus, further investigation is needed to verify the feasibility of using ECG for the diagnosis

#### *Related Studies*

The conventional method to study ECG data by AI is to mine traditional handcrafted features and employ them to train machine learning classifiers. These methods have previously been used to identify cardiac anomalies from ECG records. Numerous research articles used such methods based on 1D ECG signals to detect several cardiac problems [30–34]. However, these methods generally require a trade-off between accuracy and computation load and are subjective to errors [11,35]. Conversely, DL was recently employed to examine ECG by automatically attaining valuable features, thus avoiding the disadvantages of handcrafted methods [36,37]. Many studies have shown that ECG 1D signals converted to 2D demonstrations have better performance and benefits than 1D-based models [38,39]. Several studies analyzed and converted 1D to 2D ECG using transform domains such as short-term frequency transform and wavelet transform [38–44] and used them with DL techniques. Many studies used several forms of DL models to detect abnormalities in ECG signals [45–49]. It is worth mentioning the great efforts that were made by the PhysioNet/Computing in Cardiology Challenge in 2020 and 2021 to stimulate the multitype arrhythmia classification over annotated databases with thousands of 12-lead ECG recordings [50,51]. Despite the success of previous studies in detecting cardiac complications from ECG signals, it could not be easily used in real clinical practice

as the above methods mainly rely on ECG signals; however, in real medical practice, this is regularly not the usual scenario. Because the ECG data taken in real clinical practice are acquired and stored as 2D ECG trace images [52]. Unlike the digital ECG signal acquired using wearable sensors, which contain multiple clean and well-detached leads, the ECG trace image data acquired in real practice are ambiguous. Such a trace image has an overlay between ECG waveforms collected from different leads and the rigid surrounding minor axes that raise hardness in mining significant information precisely. Furthermore, in digital ECG signals, that data is collected in hundreds of hertz as a sampling frequency; however, in real medical practice, the ECG data are acquired in few hertz, which results in a huge degradation in the quality of the data which correspondingly impacts the classification performance of AI-based models. One possible solution to resolve that issue is to turn the trace image into a digital ECG signal [53]. However, this conversion is complex, and the converted signal is of low quality due to the extensive noise generated by the conversion [54]. Even with the great capabilities of DL methods, this noise hinders DL techniques in detecting the small variations among different cardiac anomalies, which is the major component of cardiac complications diagnosis.

The abovementioned issues obstruct the digital ECG signals from being used in realworld clinical practice which collects ECG records as trace images. Therefore, some research articles used direct ECG trace images to identify several cardiac complications using AI techniques. The authors in [55] proposed a system to detect myocardial infarctions from ECG trace images. Their system contained multiple divisions based on shallow artificial neural networks (ANN) that used 12-lead ECG, achieving an accuracy of 94.73%. In [56], a discrete wavelet transform (DWT) was used to obtain significant features from the trace images using the 'Haar' wavelets. An ANN was constructed to differentiate between normal and abnormal ECG patterns, obtaining an accuracy of 99%. In [57], five hand-made feature extraction methods along with five classifiers were used to recognize two categories of cardiac arrhythmias. The highest accuracy of 96% was achieved using local binary patterns and ANN. On the other hand, Du et al. [58] proposed a DL pipeline to identify several cardiac diseases. The pipeline determined the prospective distinctive regions and adaptively merged them. Next, a recurrent neural network was employed and attained a sensitivity and precision of 83.59% and 90.42%, respectively. The MobileNet v2-deep DL method was utilized in [59] to identify four cardiac complications with 98% accuracy. In [60], DenseNet was trained with ECG trace images to predict strokes and achieved 85.82% accuracy.

The promising performance achieved using the formerly discussed methods based on ECG trace images triggered the investigation of the possibility of employing this type of ECG data with DL techniques to diagnose COVID-19. An acknowledgment must be made of the recently published public data [61] which has helped to achieve the suggested target. This data has ECG images of patients with COVID-19 and other cardiac findings. To the best of our knowledge, up to today, four studies have utilized this dataset to examine the potential of using ECG trace images in the new diagnosis of coronavirus. This dataset was used in [62] to study the impact of employing various enhancement methods on the diagnosis of COVID-19 using EfficientNet trained with ECG trace images. The paper concluded that augmentation methods are useful to some extent; nevertheless, exceeding this extent will lower the performance. An 81.8% maximum accuracy was achieved. Whereas in [63], six DL approaches were utilized to identify COVID-19 from other cardiac findings in two classification categories. Alternatively, in [64], hexaxial feature and Gray-Level Co-occurrence Matrix (GLCM) approaches were employed to extract considerable features and generate hexaxial mapping images. The created images were fed to DL methods to distinguish COVID-19 from other images as a binary classification category with a precision of 96.2%. The study [65] extracted deep features from two layers of several CNNs to an accuracy of 97.73% and 98.8% for multiclass and binary classification problems.

These previous studies experienced numerous shortcomings. Initially, the tool implemented in [64] performed only using binary classification category; however, the multiclass

problem is more complicated and essential but was not considered. Furthermore, the hexaxial feature mapping utilized in it is quite sensitive to image quality, which correspondingly influences the extraction process of the GLCM procedure. The classification results obtained in the tool presented in [62] were considerably low and therefore cannot be reliable. The number of samples and features used in the testing process of the tool introduced in [63] was small, leading to a probable bias. The study [65] used a huge number of features to build their model. On the other hand, the tools proposed in [62,63,66] were based on individual DL models to perform the feature extraction or classification procedure. However, the research articles [67,68] confirmed that the incorporation of features of numerous DL approaches has the capacity to improve the classification results.

This study examined the viability of utilizing ECG information for COVID-19 diagnosis via presenting a novel diagnostic tool using various AI methods. The proposed tool attempts to overcome the limitations of the previous studies by incorporating several DL techniques and using a hybrid feature selection approach to reduce the number of features used to train the classification models. The classification procedure of the proposed tool is performed on two levels. The primary level aims to classify the ECG data to COVID-19 and normal cases (binary class level). The second level is multiclass to distinguish COVID-19 cases from normal and other cardiac complications.

#### **2. Materials and Methods**

#### *2.1. ECG Dataset*

The proposed diagnostic tool uses a recent dataset that is public [61], including images of ECG records for patients with COVID-19 and other cardiac problems. Until now, to the best of our knowledge, this is the primary and single public dataset for ECG records of COVID-19. ECG images available in the dataset are 1937 of distinct categories. The dataset consists of 250 scans of cases with the novel coronavirus, 300 trace records of cases with a present or former myocardial infarction (MI), 548 ECG records of irregular heartbeats, and 859 normal images without any heart complications as shown in Table 1. Data were acquired using a 12-lead system with a sampling frequency equal to 500 Hz through an EDAN SE-3 series 3-channel electrocardiograph. Table 1 also illustrates the number of images used for the training and validation sets of the proposed tool. The dimension of the images varied from 952 × 1232 to 2213 × 1572. The x-scale is 25 mm/s, and the y-scale is 10 mm/volt. Six ECG electrodes were placed on the chest representing six precordial leads. Another three electrodes were placed on the two arms and left leg representing six limb leads, including augmented voltage right (AVR), augmented voltage left (AVL), augmented voltage foot (AVF), Lead I, II, and III. The images of the dataset were evaluated by medical professionals using a telehealth ECG diagnostic scheme. This evaluation was carried out under the supervision of expert cardiologists who had long experience in ECG annotation and exploration. These medical experts removed all uncertain, ambiguous, and misleading images from the dataset.

In the binary classification level (normal versus COVID-19), 250 normal and 250 novel coronavirus records were utilized. Whereas in the multiclass classification level, a total of 750 images were employed, 250 for cardiac complications, 250 for normal cases, and 250 for COVID-19 cases. To avoid the classification bias that occurs due to the class imbalance structure of the ECG dataset (the number of images per class is not equal) that affects the classification process, an equal number of images was selected and used for each class to train the classification models. An ECG trace record sample for a COVID-19 patient is shown in Figure 1.

**Table 1.** Description of the ECG dataset including the number of available ECG images per class and the number of images used in the proposed study.

**Figure 1.** An ECG trace record sample for a COVID-19 patient.

#### *2.2. Proposed Tool*

The proposed automated tool consists of four steps: ECG trace image preprocessing, deep feature extraction and feature incorporation, hybrid feature selection, and classification. The proposed method used ten DL approaches. Figure 2 shows a diagram that describes the steps of the proposed diagnostic tool.

DL is an emerging technology that has been widely employed in several fields. DL approaches are the recent class of machine learning (ML). They consist of numerous architectures; however, convolution neural networks (CNNs) are the architectures most widely used for medical images [69]. Therefore, the proposed diagnostic tool utilizes ten CNNs of various architectures. These networks include InceptionResNet, ResNet-18, ResNet-50, ShuffleNet, Inception V3, MobileNet, Xception, DarkNet-19, DarkNet-53, and DenseNet-201.

**Inception V3** Google proposed the Inception CNN architecture in 2016 [70]. It is a newer version of GoogleNet [71], but with some modifications. It was first introduced to run well with reduced memory requirements and computational cost. Its principal component is the inception unit which merges numerous filters into a novel filter structure which correspondingly lowers the number of parameters. To expand the information stream into the network, the Inception module considered the depth as well as the width of the layers during the construction of the network [72].

**Figure 2.** A diagram describing the steps of the proposed diagnostic tool.

**ResNet** is one of the time-efficient CNNs that gained popularity due to its novel structure created by He et al. in 2015 [73]. ResNet counts on the residual block which embeds crosscuts in the interior layers of a standard CNN to cross several convolution layers which quickens and eases the convergence procedure of the CNN despite the huge number of convolution layers.

**Xception** is a new version of the Inception network introduced in 2017 [74]. The inception layers contain depthwise convolution layers, followed by a pointwise convolution layer. The Xception structure involves double layers of convolutional, then several depthwise separable convolution layers, and standard layers of convolutional and fully connected. The Xception module is more robust and powerful than the Inception module and can perform cross-channel and spatial interaction correlations while fully dissociated [75].

**Inception-ResNet-V2** presented a mixture of residual network architecture and the inception module [76]. It has a number of filters of various dimensions that are merged with residual joints. The main advantage of this fused architecture is enhancing the performance of the network and pace of convergence.

**DenseNet** was created by Huang et al. [77] in 2017, who extended the idea of shorter connections between layers near the input/output layers. The key building block of this network is the 'dense block'. The major difference between the residual block and dense block is that the latter attaches every layer to each layer having a similar input resolution, whereas the former generates shorter links among adjacent layers. Second, each layer of

DenseNet accomplishes a concatenation of the earlier outputs; in contrast, ResNet performs a summation. DenseNet-201 was utilized in this article, containing 201 layers.

**ShuffleNet** is an effective CNN primarily designed by Zhang et al. in 2018 [78]. ShuffleNet was initially produced to serve fields that require low computational capability. It contains two key blocks known as pointwise group convolution and channel shuffle. The first block utilizes convolution layers of dimension 1 × 1 to reduce training speed while attaining adequate precision. The second block supports the data flowing across feature channels by allowing a cluster of layers to control input data belonging to distinct groups, where the output/input channels are connected.

**DarkNet** is a new DL architecture designed by the authors of [79]. It employs YOLO-V2 as the backbone of its structure. DarkNet uses filters of dimension 3 × 3 and then doubles the number of channels after every pooling phase. It employs a pooling stage to perform detection and classification as well as 1 × 1 filters to reduce the feature presentation between 3 × 3 convolutions. Darknet-19 involves 19 convolutional layers, whereas DarkNet-53 contains 53 convolutional layers.

**MobileNet** is a fine and time-efficient DL architecture that was originally designed in [80]. It can decrease the complexity of the training model by lowering the number of parameters while maintaining an acceptable performance. These are convolutional layers of dimensions 3 × 3 and 1 × 1, respectively. MobileNet has 53 deep layers.

#### 2.2.1. ECG Image Preprocessing

Initially, the dimensions of the ECG images are changed according to the input layer dimension of each CNN model. Then, those ECG records are augmented to increase the amount of records available in the data set and prevent the likelihood of overfitting that could occur in the case of small data. Those augmentation methods included in the proposed diagnostic tool are flipping in both the x- and y-orientations, and translation in both the x- and y-directions where the range of the translation distance is picked randomly within the range (−30, 30). The scaling augmentation method is also applied to the image in the x- and y-directions where the image is scaled with a scale factor chosen randomly from the range (0.9, 1.1). Table 2 demonstrates the dimensions of the input layers of each of the CNN models and the extracted features length. Table 2 shows that the number of features extracted from the last fully connected layer of each CNN for the binary classification and multiclass classification levels is 2 and 3, respectively.

**Table 2.** The dimensions of the input layers of each of the CNN models and the mined features dimensions.

#### 2.2.2. Deep Features Extraction and Feature Incorporation

Some complications may occur while CNNs are being trained, including convergence and overfitting. These issues impose the adjustment of a few parameters in the CNNs to guarantee that the weights of the CNN layers are updated at the same rate during the training process. Transfer learning (TL) is a method that can solve this problem. TL reemploys a CNN that was previously learned with a huge dataset like ImageNet for another classification problem [81]. In other words, TL uses a pretrained CNN that has learned feature representations from a large dataset to solve another classification problem dealing with a small dataset (similar to the dataset used in this paper). This process can enhance detection accuracy if used for comparable problems [81]. For that reason, this paper used ten CNNs that were pretrained. Before retraining the ten CNNs, the number of their output layers was changed to 3 or 2 which is equal to the number of classes in the case of the multiclass and binary class classification categories of the proposed diagnostic tool. In other words, the DL models were retrained for the novel classification task. Then, after the retraining process was finished, deep features were extracted from the last fully connected layers of the ten pretrained CNNs. The number of features extracted from each CNN was 2 in the case of the binary classification category and 3 in the multiclass classification category. Afterward, the proposed tool incorporated the deep features extracted from the ten DL models in a concatenation way to form one feature vector consisting of 20 and 30 features in the case of the binary and multiclass classification categories, respectively.

#### 2.2.3. Hybrid Feature Selection

Feature selection (FS) is an essential step to selecting the most valuable features available in the feature space to reduce its dimension, which correspondingly boosts the diagnostic accuracy and avoids overfitting [82,83]. FS methods can be categorized into three categories: filter, wrapper, and hybrid [84]. Hybrid FS merges filter and wrapper methods. This category combines the benefits of previous FS types [84]. Thus, a hybrid FS approach was presented and employed in this study.

The hybrid FS step presented in the diagnostic tool combines the chi-squared test filter FS approach with a wrapper FS approach based on three search strategies. The chi-squaredtest is a well-known and commonly used FS method [85]. It attempts to determine the significant features *tk* that best differentiate positive and negative sets of instances of class *Ci*. The chi-squared test score is calculated using Equation (1).

$$\text{Chi} - \text{Squared Test} = \frac{\text{N}(\text{AD} - \text{CB})^2}{(\text{A} + \text{C})(\text{B} + \text{D})(\text{A} + \text{B})(\text{C} + \text{D})} \tag{1}$$

where N is the total number of ECG records (samples in a dataset); A = the number of samples in class *ci* that contain the feature *tk*; B = the number of samples that contain the feature *tk* in other classes; C = the number of samples in class *ci* that do not contain the feature *tk*; D = the number of samples that do not contain the feature *tk* in other classes.

The hybrid FS method initially ranks deep features extracted from the ten CNN models utilizing the chi-squared test filter FS. Then, it employs this ranking to guide the three feature search strategies within the wrapper FS approach. These three search strategies are backward, forward, and bidirectional. The first searching approach starts with all features in the feature space and then ignores features of lower ranks iteratively. Conversely, the forward approach begins with one feature having the greatest rank and then adds the following features one by one. The bidirectional alternates between the forward and backward strategies. Note that for the three strategies, only the features that improve the classification results are kept, while others are deleted.

#### 2.2.4. Classification

The classification phase was performed in two schemes. The first scheme was an endto-end deep learning classification with ten CNNs, including InceptionResNet, ResNet-18, ResNet-50, ShuffleNet, Inception V3, MobileNet, Xception, DarkNet-19, DarkNet-53, and DenseNet-201. The second scheme used several machine learning classifiers trained with deep features extracted from the last fully connected layers of the ten CNNs. These classifiers involved a support vector machine (SVM), random forest (RF), K-nearest neighbor (KNN), the linear discriminate classifier (LDA), quadratic discriminate analysis (QDA), and decision tree (DT). The classification step included two levels: binary and multiclass. At the former level, classifiers were used to identify COVID-19 and normal patients. The multiclass level classified images into normal, COVID-19, and cardiac complications. The 10-fold cross-validation method was used to validate the results. The classifiers were run 10 times and the average classification performance of all these runs is displayed in the results section. Classification was carried out in two phases. Phase I used the deep features extracted from the ten CNN models to train the classifiers. Phase II employed the hybrid FS approach to select features used to train the classifiers.

**LDA** is a popular machine learning technique used for both classification and feature reduction. It searches for the linear combinations of features that have a high ability to explain the data. LDA separates class labels of data using hyperplanes. These planes are achieved by looking for the projection of data points that can minimize their variance and maximize the distance between class labels.

**K-NN** is a commonly used classifier in the field of machine learning due to its simplicity, straightforwardness, and effectiveness even with noisy data. Although it is simplistic, it has the ability to reach good classification accuracy in medical applications. It allocates a label to every instance in the test data equivalent to the label amongst the k nearest neighbors included in the training data. This label is chosen according to the distance measured between the instance being classified and those instances in the training data. This distance shows that instance in the test data to those in the training data. The distance used in our approach was the Euclidean similarity measure and the number of neighbors (k) was equal to 1 and 5 for binary and multiclass classification levels, respectively, with equal distance weights.

**Decision Trees** are well-known machine learning classifiers that are widely used in medical applications due to several reasons. They are capable of visualizing interactions between extracted features. This visualization process enables a doctor to easily understand how the classifier decision is made. The DT classifier creates instances of data according to conditions. The DT has a tree structure with a root node whose leaves demonstrate class labels, and the branch nodes present the extracted features and reasons that result in this class label. The nodes of a tree are connected by an arc that represents the condition of the feature. The tree is divided into branches and leaves based on a metric such as information gain, gain ratio, or Gini index. The maximum number of splits in this study was 100, and the splitting criterion was the Gini diversity index.

**Random Forest** is an ensemble classifier that consists of multiple decision trees. RF uses the divide-and-conquer approach (DAC) to perform classification. The DAC method divides the input feature space into several partitions depending on a goodness metric. Subsequently, the classification outputs of all trees are averaged to produce a final decision. The Gain ratio metric was used in the proposed tool. There, the number of trees was 100.

**SVM** is a robust machine learning classifier. It transforms linear or nonlinear input data points into a new domain that can easily separate between classes of data. A hyperplane is employed to separate between classes of input data to facilitate classification. A kernel function maps the similarity between the input vector and the new higher-dimension feature space. The linear kernel function was employed.

On the other hand, for retraining the CNNs for end-to-end classification, the learning rate, number of epochs, and minimum batch size were adjusted to 0.0003, 10, and 4, respectively. Whereas the validation frequency was modified to 87 and 131 for binary and

multiclass classification levels, respectively. The ten CNNs were trained with the stochastic gradient descent with a momentum algorithm. The other hyperparameters were kept unchanged. The proposed diagnostic tool was implemented using the Weka Data Mining Tool [86] and MATLAB R2020a.

#### *2.3. Performance Evaluation*

The overall performance of the proposed diagnostic tool was measured using multiple metrics involving the Mathew correlation coefficient (MCC), the F1 score, precision, specificity, and sensitivity calculated using Equations (2)–(7). In addition to confusion, the receiver operating characteristics curve (ROC) and the area under ROC (AUC) were also determined.

$$\text{Accuracy} = \frac{\text{TP} + \text{TN}}{\text{TN} + \text{FP} + \text{FN} + \text{TP}} \tag{2}$$

$$\text{Sensitivity} = \frac{\text{TP}}{\text{TP} + \text{FN}} \tag{3}$$

$$\text{Precision} = \frac{\text{TP}}{\text{TP} + \text{FP}} \tag{4}$$

$$\text{MCC} = \frac{\text{TP} \times \text{TN} - \text{FP} \times \text{FN}}{\sqrt{(\text{TP} + \text{FP})(\text{TP} + \text{FN})(\text{TN} + \text{FP})(\text{TN} + \text{FN})}} \tag{5}$$

$$\text{F1} - \text{Score} = \frac{2 \times \text{TP}}{(2 \times \text{TP}) + \text{FP} + \text{FN}} \tag{6}$$

$$\text{Specificity} = \frac{\text{TN}}{\text{TN} + \text{FP}} \tag{7}$$

where FN refers to the false negative which is the amount of COVID-19 records wrongly categorized as nonCOVID-19, TN is the true negative representing the nonCOVID-19 records correctly recognized. TP is the true positive, which is equal to the number of COVID-19 scans properly identified. Finally, FP is the false positive equivalent to the sum of nonCOVID-19 records improperly classified as COVID-19.

#### **3. Results**

#### *3.1. Phase I Classification Results*

Phase I represents the use of deep features extracted from the ten CNNs and fused to train the machine learning classifiers. Table 3 illustrates the classification accuracy of phase I for the binary class and multiclass classification levels, respectively. Table 3 shows that the maximum accuracy of 97.78% was achieved for the binary classification level using the RF model. All other classifiers obtained an accuracy that ranged from 97.36% to 97.6%. The highest accuracy of 90.88% was achieved using the RF classifier for multiclass classification. The SVM, LDA, and KNN achieved the next-highest accuracies of 90.43%, 90.35%, and 89.39%. Finally, the DT and QDA classifiers reached the lowest accuracies of 86.56% and 85.6%, respectively. The confusion matrices attained using the LDA and SVM classifiers are shown in Figures 3 and 4 for binary and multiclass classification, respectively. The ROC curve for the SVM and LDA classifiers are shown in Figures 5 and 6 for binary and multiclass, respectively. Figure 7 shows that the AUC for the LDA and SVM classifiers were 0.99 and 0.99 for the binary class classification level. For the multiclass classification level, the AUCs for the LDA and SVM classifiers were 0.97 and 0.98, respectively.

**Table 3.** Phase I classification accuracy (%) and standard deviations obtained using machine learning classifiers.

**Figure 3.** Confusion matrices for the binary class classification level: (**left**) LDA, (**right**) SVM classifiers.

**Figure 4.** Confusion matrices for the multiclass classification level: (**left**) LDA, (**right**) SVM classifiers.

The two Figures S1 and S2 have been attached to the Supplementary Materials representing a two-dimensional scatter plot of the first two features of the feature space for the binary and multiclass classification levels used as inputs to the classifiers. Also, the two Figures S3 and S4 have been added to the Supplementary Materials representing a two-dimensional scatter plot of the LDA classifier predictions using the first two features of the feature space for the binary and multiclass classification levels. Moreover, the two Figures S5 and S6 have been added to the Supplementary Materials representing a twodimensional scatter plot of the LDA classifier predictions using the first two features of the feature space for the binary and multiclass classification levels

**Figure 5.** The ROC curves for the binary class classification level: (**left**) LDA, (**right**) SVM classifiers.

**Figure 6.** The ROC curves for the multiclass classification level: (**left**) LDA, (**right**) SVM classifiers.

To access and confirm the statistical significance of the performance of the ML classifiers, the one-way analysis of variance (ANOVA) test was applied to the results of the classifiers after a repeated 10-fold cross-validation process. The ANOVA test was performed on the classification accuracy results achieved using the classifiers of the binary classification level to test the statistical significance between them. The results are shown in Table 4. ANOVA was also performed for the results of the multiclass classification problem and the outputs of the test are shown in Table 5. It can be seen in Tables 4 and 5 that the p-values attained from the test were lower than α, where α = 0.05. Consequently, it could be concluded that there is a statistically significant difference in the classification accuracies of the classifiers for both the multiclass and binary classification levels.

**Figure 7.** The classification accuracy of the proposed diagnostic tool using the RF classifier of phase I compared to the end-to-end DL classification for the binary level.

**Table 4.** One-way analysis of variance test details for the binary classification level.

**Table 5.** One-way analysis of variance test details for the multiclass classification level.

Figures 7 and 8 compare the phase I performance of the RF classifier of the proposed diagnostic tool with the end-to-end DL classification for the binary and multiclass levels. Figure 7 proves that the deep features extracted from the last fully connected layers of the ten CNNs had a higher classification accuracy compared to end-to-end pretrained CNNs for the binary classification level. On the other hand, for the multiclass classification level, the RF classifier of the proposed diagnostic tool obtained 90.88% accuracy, which is higher than all other pretrained CNNs. As can be seen in Figure 8, the accuracy of the RF classifier of the proposed diagnostic tool was greater than the 76.44%, 75.56%, 72.89%, 73.33%, 72.89%, 71.59%, 71.11%, 67.11%, 69.33%, and 64.44% achieved using ResNet-50, ResNet-18, Inception-ResNet, Inception, Xception, DenseNet-201, DarkNet-53, DarkNet-19, MobileNet, and ShuffeNet, respectively.

**Figure 8.** The classification accuracy of the proposed diagnostic tool using the phase I RF classifier compared to the end-to-end DL classification for the multiclass level.

#### *3.2. Phase II Classification Results*

Phase II of the proposed diagnostic tool presented the features selected after the hybrid FS approach used them to train the classification models. The following section presents the results of the hybrid feature selection approach based on the three search strategies using three classifiers. First, it shows the rank scores of features using the chi-square test filter FS method. Then, it shows the number of selected features as well as the classification accuracy attained for the binary and multiclass classification levels. Tables 6 and 7 represent the ranking score for each feature attained using the chi-square test FS method for the binary and multiclass classification levels, respectively.

**Table 6.** The ranking score for each feature attained using chi-square FS along with its order in the feature vector and the name of the feature for the binary classification level.

**Table 6.** *Cont.*

**Table 7.** The ranking score for each feature attained using chi-square FS along with its order in the feature vector and the name of the feature for the multiclass classification level.

Table 8 shows the binary class classification level after the hybrid FS approach of the proposed diagnostic tool using the three search strategies (phase II) compared to phase I (before FS) for the DT, RF, and QDA classifiers as they achieved the highest accuracies in phase I. Table 8 shows that the hybrid FS approach of the proposed diagnostic tool improved the classification accuracy compared to phase I. This was obvious as the accuracies attained using the forward and bidirectional strategies were 98.2%, 98%, and 97.8%, which are better than those attained before FS. In addition, the accuracies attained using the backward strategy were 98.2%, 98%, 97.6% for the DT, RF, and QDA classifiers, which were higher than those achieved before FS using the same classifier except for the QDA which is equal to that achieved before FS. Some performance measures were calculated for the binary classification level and are illustrated in Table 9. Table 9 reveals the results for the sensitivity (0.968, 0.96, 0.956), specificity (0.996, 1, 1), precision (0.996, 1, 1), F1-score (0.982, 0.961, 0.978), and MCC (0.964, 0.989, 0.957) for the DT, RF, and QDA models, respectively, using the forward search strategy.

**Table 8.** The accuracy of binary-level classification (%) of the DT, RF, and QDA classifiers that obtained the highest accuracy in phase I compared to after using the three search strategies of the hybrid FS approach (phase II of the proposed diagnostic tool).

**Table 9.** The binary-level performance metrics (%) of the DT, RF, and QDA classifiers that achieved the highest accuracy using the forward search strategies of the hybrid FS approach.

On the other hand, the results of the multiclass classification level of phase II of the proposed diagnostic tool are displayed in Tables 10 and 11. Table 10 shows the multiclass accuracy of the hybrid FS approach of the proposed diagnostic tool using the three search strategies (phase II) compared to phase I (before FS) for the RF, LDA, and SVM classifiers which achieved the highest accuracy in phase I. The accuracies displayed in Table 10 verify that the hybrid FS approach based on the three search methods increased the capacity of the classification model compared to phase I (before FS). This was clear as the forward and bidirectional strategies achieved better accuracies of 91.6% and 90.93% for the RF classifiers, 91.07% and 91.33 for the LDA classifier, and 90.58% and 90.53% for the SVM classifier compared to 90.88%. Using the exact classifiers before FS, 90.35% and 90.43% accuracy was achieved. Similarly, the backward search method reached accuracies of 91.33%, 91.07%, and 90% using the RF, LDA, and SVM classifiers, which were higher than those attained before FS except for the SVM classifier, it remained the same. Table 11 indicates the sensitivity (0.916, 0.911, 0.905), specificity (0.958, 0.955, 0.953), precision (0.918, 0.918, 0.908), F1 score (0.917, 0.917, 0.906), and MCC (0.875, 0.875, 0.859) for the RF, LDA and SVM classifiers, respectively, using the forward search method.

**Table 10.** The classification accuracy (%) of the RF, LDA, and SVM that obtained the highest accuracy in phase I compared to after using the three search strategies of the hybrid FS approach (phase II of the proposed diagnostic tool).

**Table 11.** The multiclass-level performance metrics (%) of the RF, LDA, and SVM that achieved the highest accuracy using the forward search strategies of the hybrid FS approach.

Figure 9 shows the number of features selected for the binary and multiclass levels using the forward search strategy that reached maximum accuracy (using the DT classifier for binary and the RF model for multiclass). Figure 9 indicates that the number of features after the hybrid FS for the binary classification problem is three. The three features include Feature 2 of MobileNet, Feature 1 of InceptionResNet, and Feature 2 of ResNet-50. The figure also shows that the number of features after FS for the multiclass classification level is eight. These eight features are Feature 1 of MobileNet, Feature 3 of Inception, Feature 3 of ResNet-18, Feature 1 of Xception, Feature 2 of DarkNet-53, Feature 3 of DarkNet-53, Feature 1 of DarkNet-19, and Feature 3 of InceptionResNet.

**Figure 9.** The number of features of phase I and phase II for the binary and multiclass classification levels using forward search strategies (using the classifiers which attained the peak performance).

#### **4. Discussion**

Recent relevant studies revealed various forms of cardiovascular variations in ECG data acquired from patients infected by the novel coronavirus as ST-segment changes, QRST irregularities, and arrhythmias. On the other hand, several research articles discussed that COVID-19 could not be the leading reason for these cardiovascular deformities; nevertheless, it should be emphasized that it could reveal the intrinsic conditions or lower them [87]. The entire cardiac findings indicated in the literature have been observed on all the ECG data utilized in this study.

This paper presented a novel diagnostic tool to automatically diagnose COVID-19 by incorporating multiple DL and hybrid FS approaches. This diagnostic tool consists of two classification levels: binary and multiclass. The first level consists of distinguishing COVID-19 and normal cases, while the second level consists of recognizing COVID-19, normal, and other cardiac abnormalities. The proposed tool extracted deep features from the last fully connected layers of ten CNNs models. Next, it fused these features, used several classifiers in the two classification levels, and compared their performance with the end-to-end DL classification. The previous step is known as phase I of the proposed diagnostic tool. Afterward, a hybrid FS method was presented based on three search approaches. This process is called phase II of the proposed diagnostic tool. The results achieved in phase I showed that the deep feature incorporation is better than end-to-end DL classification as shown in Figures 7 and 8. Phase I of the proposed tool attained an accuracy of 97.78% and 90.88% for the binary and multiclass classification levels, respectively. These accuracies are greater than those obtained by the end-to-end deep learning classification, having a range of 87.33–96.67% and 64.44–76.44% for the binary and multiclass classification levels, respectively.

In the second phase of the proposed tool, only classifiers that attained the highest accuracies for either the binary or the multiclass classification levels were employed in the hybrid FS procedure. Table 8 compares the results before and after feature selection for the three classifiers which attained the highest accuracies for the binary classification level. Table 8 shows that the highest accuracy of 98.2% was achieved using DT trained with only three features selected during the FS process of the binary classification level. This accuracy is greater than the 97.62% achieved before FS using the same classifier trained with 20 features. Similarly, Table 10 compares the results before and after FS for the three classifiers which attained the highest accuracies for the multiclass classification level. Table 10 indicates that the maximum accuracy of 91.6% was reached using the RF classifier trained with only eight features chosen during the FS procedure of the multiclass classification level. This accuracy is greater than the 90.56% accomplished before FS using the same classifier learned with 30 features. Thus, the performance of phase II of the proposed tool verifies that the presented hybrid FS method had a further enhancement in classification performance. It also reduced the number of features successfully.

It is worth mentioning that ECG detection requires more physical contact between patients and physicians than the RT-PCR test or CT imaging, which will increase the risk of virus transmission. Therefore, ECG may be more suitable as an auxiliary inspection means of COVID-19 than a primary screening tool.

#### *4.1. Comparison with Related Studies*

The performance of phase II of the proposed diagnostic tool versus other relevant tools that are directly copied from published papers is demonstrated in Table 12. The ECG records used to construct the proposed diagnostic tool are added to Appendix A. The results illustrated in Table 12 show that the proposed tool could be used to distinguish between normal and COVID-19 cases. It can also differentiate between normal, COVID-19, and other cardiac abnormalities. The table also indicates that the proposed tool has a performance comparable to those of other related studies. It is worth mentioning that, for the binary classification level, the specificity and precision of the proposed tool are higher than in the other studies [63,64]. However, it has lower sensitivity than other studies. However, for the multiclass classification level, the proposed tool achieved higher sensitivity, specificity, and precision than the studies [62,63]. These results indicate that the proposed tool based on ECG data could be used to diagnose COVID-19. It could be considered a possible novel solution that might be utilized in actual medical scenarios. It can be considered an alternative to current diagnostic tools.

**Table 12.** The results of phase II of the proposed diagnostic tool versus other related studies that are directly copied from published papers.

#### *4.2. Limitations*

This study has several limitations. The first limitation is the small database for training/validation, which is quite insufficient for deep learning of thousands of hyperparameters. In addition, the lack of an independent test dataset is considered another limitation. Furthermore, this study did not consider methods that handle the class imbalance problem. Furthermore, the baseline rhythm of each patient is not available, and the effect of the baseline rhythm is not explored. Additionally, this study did not take into account optimization techniques for the selection of deep learning hyperparameters. In addition, the dataset used in the study is from confirmed COVID-19 patients. The detection of asymptomatic infections may not achieve the same level of sensitivity. Thus, the extension of the scope of the results is to some extent limited. Finally, this study did not consider the uncertainty of the input data.

#### **5. Conclusions**

The current study explored the prospect of employing ECG trace images for diagnosing the novel coronavirus. It proposed a novel automated ECG-based diagnostic tool that incorporates deep features from ten DL models. The proposed diagnostic tool used several well-known ML classifiers for classification. The classification procedure was performed on two levels. The primary level aimed to distinguish patients with COVID-19 from normal cases (binary class level). Whereas the second level was multiclass to distinguish cases of COVID-19 from normal and other cardiac complications. The major contributions of the diagnostic tool were, first, the construction of a novel automatic, inexpensive, harmless, susceptible, and quick diagnostic tool as a replacement to the present diagnostic tools to support the automatic detection of COVID-19. In addition, the novel tool relied on 2D ECG trace images to diagnose COVID-19, which is a new approach to achieving a diagnosis. Moreover, in view of the disparities in the performances between DL models, the proposed

tool utilized ten DL models of distinctive structures to merge their benefits, not a single architecture. Additionally, it extracted features from the last fully connected layers of the ten DL models instead of end-to-end DL classification (as in previous studies). The proposed tool merged these features to investigate the impact of merging on diagnostic accuracy. Furthermore, it presented a hybrid FS approach based on three search strategies to select the most significant deep features and the lower dimensions of the feature space. Finally, it explored whether the hybrid FS approach boosts the performance of the proposed diagnostic tool. The results achieved using the proposed tool could be evidence that ECG records can be used in diagnosing the new coronavirus. The presented tool may prevent the shortcomings of chest imaging techniques, antigen, and PCR exams. It could be considered an easy, inexpensive, quick, portable, and sensible approach. Therefore, it might help clinicians in diagnosing COVID-19 accurately and automatically. Upcoming experiments will test the efficiency of the proposed tool in actual clinical procedures. Further work will consider using resampling techniques that handle the class imbalance problem. Future work will explore more deep learning techniques as well as hyperparameter optimization approaches. In addition, the uncertainty of the input data will be taken into consideration in future work.

**Funding:** This research received no external funding.

**Supplementary Materials:** The following supporting information can be downloaded at: https: //www.mdpi.com/article/10.3390/bios12050299/s1, Figure S1: a two-dimensional scatter plot of the first two features of the feature space for the binary classification level; Figure S2: a two-dimensional scatter plot of the first two features of the feature space for the multiclass classification level; Figure S3: a two-dimensional scatter plot of the LDA prediction using the first two features of the feature space for the binary classification level; Figure S4: a two-dimensional scatter plot of the LDA prediction using the first two features of the feature space for the multiclass classification level. Figure S5: a two-dimensional scatter plot of the QDA prediction using the first two features of the feature space for the binary classification level; Figure S6: a two-dimensional scatter plot of the QDA prediction using the first two features of the feature space for the multiclass classification level.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** The dataset employed in this paper can be found in the Mendeley [https://data.mendeley.com/datasets/gwbz3fsgp8/2]. (accessed on 1 September 2021).

**Conflicts of Interest:** The author declares that they have no conflict of interest.

#### **Abbreviations**

#### **Appendix A**

The patients' records that have been used are as follows:

For the normal class: records utilized are normal (1) to normal (250).

For the COVID-19 class: all records available in the dataset are utilized

For cardiac abnormalities class: abnormal heartbeats(HB) records utilized are HB(1) to HB (88), myocardial infarction (MI) records used are MB (1) to MB (77), previous myocardial infarction (PMI) records used are PMI (1) to PMI (88).

#### **References**

## *Article* **The Relevance of Calibration in Machine Learning-Based Hypertension Risk Assessment Combining Photoplethysmography and Electrocardiography**

**Jesús Cano 1, Lorenzo Fácila 2, Juan M. Gracia-Baena 3, Roberto Zangróniz 4, Raúl Alcaraz 4 and José J. Rieta 1,\***

**Abstract:** The detection of hypertension (HT) is of great importance for the early diagnosis of cardiovascular diseases (CVDs), as subjects with high blood pressure (BP) are asymptomatic until advanced stages of the disease. The present study proposes a classification model to discriminate between normotensive (NTS) and hypertensive (HTS) subjects employing electrocardiographic (ECG) and photoplethysmographic (PPG) recordings as an alternative to traditional cuff-based methods. A total of 913 ECG, PPG and BP recordings from 69 subjects were analyzed. Then, signal preprocessing, fiducial points extraction and feature selection were performed, providing 17 discriminatory features, such as pulse arrival and transit times, that fed machine-learning-based classifiers. The main innovation proposed in this research uncovers the relevance of previous calibration to obtain accurate HT risk assessment. This aspect has been assessed using both close and distant time test measurements with respect to calibration. The k-nearest neighbors-classifier provided the best outcomes with an accuracy for new subjects before calibration of 51.48%. The inclusion of just one calibration measurement into the model improved classification accuracy by 30%, reaching gradually more than 96% with more than six calibration measurements. Accuracy decreased with distance to calibration, but remained outstanding even days after calibration. Thus, the use of PPG and ECG recordings combined with previous subject calibration can significantly improve discrimination between NTS and HTS individuals. This strategy could be implemented in wearable devices for HT risk assessment as well as to prevent CVDs.

**Keywords:** high blood pressure; hypertension; photoplethysmography; electrocardiography; calibration; classification models; machine learning

#### **1. Introduction**

High blood pressure or hypertension (HT) is the most significant risk factor for many cardiovascular diseases (CVDs) including cardiac arrhythmias, coronary disease, renal failure and stroke [1]. To this, it must be added that most patients with HT are undiagnosed, as in the early stages and even in the elevated blood pressure stage, HT rarely causes symptoms. For these reasons, regular blood pressure monitoring and the assessment of blood pressure levels is crucial for the prevention and early diagnosis of asymptomatic HT and the study of its evolution over time for diagnosed subjects [2].

Arterial blood pressure (BP) values have two components: systolic blood pressure (SBP), determined by the impulse generated by the contractions of the left ventricle, which

**Citation:** Cano, J.; Fácila, L.; Gracia-Baena, J.M.; Zangróniz, R.; Alcaraz, R.; Rieta, J.J. The Relevance of Calibration in Machine Learning-Based Hypertension Risk Assessment Combining Photoplethysmography and Electrocardiography. *Biosensors* **2022**, *12*, 289. https://doi.org/10.3390/ bios12050289

Received: 25 March 2022 Accepted: 28 April 2022 Published: 1 May 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

indicates how much pressure the blood is exerting against the arterial walls when the heart contracts, and the diastolic blood pressure (DBP), which depends on the resistance of the arteries to the passage of blood and indicates the pressure exerted against the walls when the heart relaxes [3]. BP depends mainly on two variables: the volume propelled by the heart in a unit of time and the resistances offered by the arteries to the passage of blood [4]. In turn, these variables depend on the activity of the autonomic nervous system (ANS), which governs heart rate and the resistance of the arterioles, and, on the other hand, the balance of water and salt filtered through the kidneys, which modulates blood volume.

Traditionally, BP has been measured through invasive as well as non-invasive strategies. Invasive BP measurement has been usually reserved for patient hospitalization, especially in Intensive Care Units (ICUs), where the availability of precise and time-continuous BP measurements is relevant [5]. For non-invasive BP estimation, conventional cuff-based measurement devices, which use oscillometric and auscultation methods, are known to be able to offer adequate accuracy. However, they are not designed to be wearable and only offer a one-off measure. Therefore, they are not compatible with continuous measurement throughout the day due to mobility limitations caused by the device, they are uncomfortable, and their measurement procedure, with the repeated inflation and deflation of the cuff, is somewhat tedious, cumbersome and requires patient attention [6].

Machine Learning classifiers provide many advantages to clinical medicine in general and to biosignal-based HT risk assessment in particular over non-invasive traditional measures, as they can be embedded in wearable devices such as smartwatches, facilitating uninterrupted monitoring throughout the day. This allows both the detection of asymptomatic hypertensive patients and the monitoring of diagnosed patients in their daily lives outside the clinical setting by screening changes in blood pressure.

As a consequence of the above factors, work in this field is focused on the development of cuff-less systems that can provide the user with information about the BP condition in near real time [7]. New wearable devices, such as wristbands or smartwatches capable of monitoring physiological signals that change according to BP level, as the electrocardiogram (ECG) and photoplethysmogram (PPG) do, may facilitate the development of these BP measurement systems [8,9]. The most promising signal is the PPG, an optical measurement technique that can be used to detect changes in blood volume in the micro vascular bed of tissues as a result of cardiac pumping. This technique is based on illumination of the skin measuring changes in light absorption [10]. It is typically implemented with a lightemitting diode (LED) to illuminate the skin and a photodetector to measure the amount of light transmitted or reflected through the skin. The change in tissue light absorption is governed by the amount of protein and hemoglobin in blood and the hemodynamic and physiological condition caused by the change in the properties of the artery [11].

In recent years, many studies have investigated methods to estimate BP using PPG signals. The first work that studied the correlation between the PPG and BP was conducted by Teng and Zhang [12], where a linear regression model was used to evaluate the relationship between four PPG features an BP. Once this relationship was known and established, later studies focused on the use of propagation theory, which extracted key features from ECG and PPG signals, simultaneously collected, for BP estimation.

Propagation features, as pulse transit time (PTT) and pulse arrival time (PAT), have been extensively used in previous works [13,14]. PTT was defined as the time taken for the pressure wave to travel between two arterial sites. Thus, it could be estimated as the time delay between a PPG wavefront measured by two separate sensors located in two distal sites of the body. For its part, PAT was defined as the delay between the electrical activation of the heart (R peak of ECG) and the PPG wavefront at the foot, maximum slope point and peak of the PPG signal, which represents the arrival of the pulse at the measurement location. Cavalcante et al. [15] applied this methodology for the first time using the start and end pulse points of these signals as well as PTT, PAT and pulse wave velocity (PWV) to determine the cardiovascular condition. Furthermore, Chen et al. [16] used ear and toe sensors to determinate PTT and its strong relationship with BP.

Other methodologies used the changes in PPG morphology to estimate BP. In this way, Kurylyak et al. [17] extracted 21 features from the PPG waveform, and demonstrated that PPG features could significantly decrease BP estimation error. Li et al. [18] and Kachuee et al. [19] also combined PAT and morphological parameters of the PPG, improving the accuracy of estimation of BP in comparison to only PAT-based features. After analyzing the different proposed methods to estimate BP, this work introduces a combination of both approaches, propagation theory features and morphological PPG features for enhanced HT risk assessment.

In the studies providing a BP value from PPG recordings, this value was just an estimation, so these methods need medical supervision. Thus, the present work introduces an alternative way to solve the problem of BP classification models with reliability, so that they can automatically provide in a continuous and non-invasive way the subject's blood pressure condition and can trigger alarms in case of an asymptomatic hypertensive condition. In this same way, Visvanathan et al. [20] used a support vector machine (SVM) to classify BP and Liang et al. [21] used PAT and PPG features and four distinctive classifiers, these being logistic regression, AdaBoost tree, Bagged tree and K-nearest neighbors, for the classification of subjects as a function of BP estimated values.

However, it has been demonstrated that the relationship between the aforementioned PPG-based propagation parameters and BP depends on many physiological factors, such as arterial walls' thickness and elasticity, age and gender, posture and risk factors of CVDs. Thus, calibration is needed when BP levels from a new subject are going to be evaluated by an automated classification method [22]. Moreover, calibration before measurement is essential to adapt the algorithms to the variations on PPG waveforms, as they are easily corrupted by fluctuations in blood circulation state, affecting the connection between BP and peripheral pulses [23].

The aim of the present study is to develop a classification system for discriminating between normotensive (NTS) and hypertensive (HTS) subjects and to evaluate the need and relevance of per-subject calibration. For this purpose, PPG and ECG simultaneous recordings have been analyzed and processed and propagation features, such as PTT and PAT, combined with other PPG morphological features have been extracted and used to train advanced classification models. The manuscript is organized as follows. Section 2 presents the database, the Machine Learning (ML) method procedure and preprocessing, the analysis techniques and the methods to evaluate the need for calibration. Section 3 presents the results, which will be analyzed in Section 4. Finally, in Section 5, the main scientific contributions of this study are remarked upon.

#### **2. Materials And Methods**

#### *2.1. Materials*

In this study, the recordings used were obtained from the MIMIC database, which contains information from ICU patients admitted to Beth Israel Deaconess Medical Center in Boston, USA [24]. This database was chosen as it contains ECG, PPG and invasive BP signals recorded simultaneously in ICU. BP signals in which the systolic or diastolic waves were indistinguishable, ECG signals where QRS morphology was distorted or PPG signals in which the systolic and diastolic waves were indistinguishable and the morphology was distorted were dismissed due to the presence of artifacts.

The BP values were labelled according to the report of the Joint National Committee on the prevention, detection, evaluation and treatment of high blood pressure [25]: as normotensive (NTS) for SBP lower than 120 mmHg, prehypertensive (PHT) for SBP between 120 and 140 mmHg and hypertensive (HTS) for SBP higher than 140 mmHg.

After labelling MIMIC recordings according to SBP values, it was observed that several subjects had stable stretches with different labels. One reason that explained these alterations in SBP values was that all patients were in an ICU, so they may have received treatment or medication that significantly altered SBP levels. Moreover, there were subjects with distant stretches, at different time points, whose SBP values were on the borderline between two labels, so that they had a different label across time even though the changes in SBP were only a few mmHg.

Because of the aforementioned reasons, those subjects with huge alterations of their SBP values (labels including NTS and HTS across time) were dismissed, as they were not suitable to train a classification model aimed at assessing the risk of HT. As a result, subjects maintaining the same label across the recording time were selected. A total of 913 recordings from 69 subjects, 45 being NTS and 24 being HTS, with acceptable signal quality conditions were selected from the MIMIC database. The signals were all recorded simultaneously with a duration of 120 s, a common sampling frequency of 125 Hz and a resolution of 8–10 bits [26].

#### *2.2. Signal Preprocessing*

The PPG signals were processed by a fourth-order Chebyshev II bandpass filter with cutoff frequencies between 0.5 and 10 Hz [27] to remove minor noises and artifacts caused by sensors' bad contacts, patient movements or any other interfering physiological activity, such as the respiratory activity, that did not provoke signal dismissing in the previous selection stage of minimum signal quality. Furthermore, the mean value of the filtered PPG was removed to prevent drifts and to allow a better comparison between different signals.

Since the waveform of the PPG signal itself is rather simple and not very informative, the derivatives of the signal were also used to better assess the changes in the signals caused by BP. They represent the velocity plethysmogram (VPG) and the acceleration plethysmogram (APG) and were obtained by applying the first and the second order derivatives, respectively, to the processed PPG signal [28].

The ABP signals which reflected the change in BP over the cardiac cycle were clear and did not require any processing to be applied. For its part, standard preprocessing was applied to each ECG [29]. Thus, they were high-pass filtered with cutoff frequency of 0.5 Hz to remove the baseline, and then low-pass filtered with a cutoff frequency of 50 Hz to reduce high-frequency muscle noise and power line interference, in this case, 60 Hz [29].

#### *2.3. Fiducial Points Identification*

After signal preprocessing, fiducial points from PPG, VPG and APG were extracted as illustrated in Figure 1. The systolic peaks of the three signals (S, W, a), the onset point of the PPG signal (O), and two local maxima and minimum of the APG signal (b, c, d, e) were extracted [28,30]. Fiducial points in the precessed signals were obtained based on searching local minima and maxima, calculated by establishing threshold and slope criteria in each of the pulses composing every signal.

**Figure 1.** Graphical definition of fiducial points detected from photoplethysmogram (PPG), velocity plethysmogram (VPG) and acceleration plethysmogram (APG) signals.

The maximum systolic blood pressure (SBP) was extracted as the maximum point of each ABP pulse. SBP was used to label every subject as NTS or HTS. Subjects whose selected segments had SBP < 130 mmHg were labeled as NTS, and subjects whose selected segments had SBP > 130 mmHg as HTS. Finally, for each ECG recording, an R-peak detector based on the phasor transform was applied to the processed ECG signal to obtain the position of each beat [30].

#### *2.4. Definition of Discriminatory Features*

After the detection of R-peaks in ECG recordings and the fiducial points for each PPG, VPG and APG signals, discriminatory features were defined based on the pulse wave propagation models, such as pulse arrival times (PAT) or pulse transit time (PTT), and other morphological features from the signals that are listed below [28,31,32]. Figure 2 illustrates the definition of the features.

**Figure 2.** Representation of *PATf oot*, *PATderivate* and *PATpeak* features obtained by the time interval between ECG R-peak and fiducial points of PPG signals as well as PPG morphological parameters: Systolic peak amplitude, TPP, rise time, areas under the pulse, width and TPI.

#### *2.5. Feature Selection*

The aim of the feature-selection stage was to select only those features, from the original 23 discriminating parameters, that presented relevant information for solving the classification problem optimally.

Firstly, since all the features were continuous quantitative variables, it was necessary to carry out a normalization, since each one could take on a different range of values and more weight would be given to the variables with higher values, not necessarily being more important. The normalization was carried out using "zscore" centering the variables so that they had zero mean and scaling so that they had unit standard deviation, as represented in the following equation

$$z = \frac{x - \overline{X}}{S},\tag{1}$$

where *x* is a concrete value of a given feature, *X* is the mean of all values of that feature and *S* the standard deviation.

Once the variables were normalized, *ReliefF* algorithm was applied to rank predictors by importance, determining which ones had the best discriminatory power. The key idea of this method is to estimate the quality of predictors according to how well instances near to each other are distinguished, rewarding predictors that give different values to neighbors of a different class [33]. Furthermore, by means of positive and negative correlation, the independence between pairs of variables was analyzed in order to discard those that did not provide new information for the classification task. Figure 3 illustrates the matrix whose entries are the correlation coefficients obtained by matching pairs of variables, so that highly correlated features can be discarded.

**Figure 3.** Correlation matrix of the 23 initial discriminatory features used in the study. Dark red values represent higher correlation coefficients and dark blue values represent lower correlation coefficients.

After analyzing the correlation matrix and *ReliefF* results, it was decided to remove three complex APG ratios (*b* − *c* − *d* − *e*)/*a*, (*b* − *e*)/*a* and (*c* + *d* − *b*)/*a* as they had high correlation coefficients with other features, as well as the last three *ReliefF* ranked features (TPP, TPI and pulse area), as the deletion of more features worsened classification performance. Finally, after the feature selection, a matrix of 17 normalized features was obtained, which will be used as inputs to train the classification models with ML techniques.

#### *2.6. Implementation Details*

The experiment was executed under MATLAB (MathWorks, Natick, MA, USA), a scientific and engineering computing software, running on a computer equipped with an Intel i7-8700 CPU @ 3.2 GHz, 16 GB of memory. The implementation for HT risk assessment combining PPG and ECG signals has been based on testing ML classification strategies such as logistic regression, Naive Bayes, discriminant analysis, support vector machines (SVM), k-nearest neighbors (KNN), ensemble classifiers and various types of decision trees [34]. Finally, SVM, Bagging Ensemble classifier and KNN were selected as they provided the highest percentages of classificatory accuracy.

SVM aims at finding the optimal separating hyper-plane between classes by focusing on the training cases that lie at the edge of the class distributions, the support vectors, so only training samples that lie on class boundaries are needed for discrimination [35]. The Bagging technique builds multiple classifiers based on a number of bootstrap samples. The outputs are decided by majority voting [36]. Finally, the KNN classifier obtains the k-nearest neighbors of the data to be classified and, as the Bagging technique, majority voting among the neighborhood is used to decide the output classification [37].

As stated before, the main objective of this study was testing whether HT risk assessment of new subjects could be improved with previous calibration. However, before addressing this goal, the classification of subjects as NTS or HTS, based on discriminant features extracted from PPG and ECG signals, was tested. In so doing, comparison with previous studies without subject-based calibration could be made. The experiment employed a leave-one-out cross-validation strategy. The classification algorithm was applied as many times as segments in the database, using each segment of 2 min in length as a single validation set and all other segments from the same subject, together with the other subjects, as a training set.

Classification performance was assessed with statistical tests for accuracy (*Acc*), sensitivity (*Se*), specificity (*Sp*) and *F1-Score*. *Acc* represented the percentage of correctly assessed PPG segments. *Se* was defined as the ability to detect as positive HTS subjects, whereas *Sp* was defined as the ability to detect as NTS healthy subjects. Finally, *F1-Score* was considered to be the harmonic mean of *Se* and *Acc*. These statistical tests were mathematically computed as

$$Acc = \frac{TP + TN}{TP + TN + FP + FN} \tag{2}$$

$$Sc = \frac{TP}{TP + FN} \tag{3}$$

$$Sp = \frac{TN}{TN + FP} \tag{4}$$

$$F1\text{-}Score = \frac{2 \cdot Se \cdot Acc}{Se + Acc} = \frac{2 \cdot TP}{2 \cdot TP + FP + FN} \tag{5}$$

where *TN* was the number of correctly classified NTS segments, *TP* the number of correctly classified HTS segments, *FN* the number of segments that the model predicted as NTS and were actually HTS and *FP* the number of patients that the model predicted as HTS and were actually NTS.

#### *2.7. Need for Calibration of New Subjects*

Calibration was defined here as the inclusion of at least one previous measurement of the subject under study in the training set. Aimed at studying the importance of calibration in the classification of new subjects as NTS or HTS, three approaches were taken:

	- a. Signal segments with a duration of 2 min were divided into 12 sub-segments of 10 s in length.

This way, a sequential calibration and validation was performed with the idea being to analyze the improvement in classification as the model was gradually calibrated by introducing previous measurements of the same subject very close in time.

3. Finally, the effectiveness of calibration for the classification of distant measurements was studied. To control the distance between measurements, groups of segments of the same patient that were less than 1 h, between 1 h and 6 h, between 6 h and 24 h and more than one day apart were selected. A sequential validation similar to the described for consecutive sub-segments was also followed in this approach in order to study whether classification results improved as the model was calibrated by introducing previous measurements of the same patient far away in time.

The aforesaid three approaches were developed employing the ML classification model that provided the best classification result in a leave-one-out cross-validation strategy.

#### **3. Results**

Statistical results of classification from the cross-validation strategy to discriminate between NTS and HTS segments are shown in Table 1. As can be seen, all three models provided outstanding classification results, with KNN being the model that obtained the best classification performance with a total accuracy of 93.54%, sensitivity of 92.31%, specificity of 94.35% and F1 score of 91.93%.

**Table 1.** Classification performance to distinguish between NTS and HTS individuals for the best models analyzed with the selected features.

Regarding results about the need to calibrate each model to provide the best classification outcome with new subjects, the KNN model was chosen as provided the best classification results with leave-one-out cross-validation. First of all, following the first approach detailed back in Section 2.7, the segments were classified without any previous calibration, in other words, with the training dataset only consisting of segments from other subjects, with each analyzed segment from the subject under study being tested for validation. In this case, classification accuracy with no previous calibration was 51.48%. This proved that hypertension risk assessment of subjects without a prior calibration provided low accuracy results, as has been also reported by previous studies [22,23].

Next, applying the second approach of Section 2.7, aimed at demonstrating whether poor classification results could be improved with calibration, Figure 4 shows the classification accuracy between NTS and HTS individuals, in the form of box-and-whisker plots, employing sequential validation of consecutive sub-segments. Moreover, the Figure indicates in each square the mean accuracy for all selected subjects according to the number of consecutive sub-segments in the training dataset acting as subject calibration. It can be seen that, with the sole incorporation of one prior close in time sub-segment in the training dataset for calibration, the classification performance increased by 30% with respect to

the case without calibration. Furthermore, accuracy improved progressively until it was stabilized above 96%, when more than six prior and close in time sub-segments from the same subject were present in the training dataset.

**Figure 4.** Classification performance provided by the KNN classifier in the discrimination between NTS and HTS individuals. Results obtained for sequential validation of consecutive sub-segments for each of the selected subject segments. In each box, the red line indicates the median, and the bottom and top edges indicate the 15th and 85th percentiles, respectively. The whiskers cover the most extreme data points not considered outliers, and the red symbol (+) stands for outliers. Black squares inside each box indicate mean accuracies.

Finally, for the third option defined in Section 2.7, performing calibration distant from measurements, Figure 5 shows classification outcomes of sequential validation with different distances between segments. It was demonstrated that calibration improved the classification task discriminating between NTS and HTS subjects because, as the number of measurements of the same subject in the model increased, so did the accuracy rate. Figure 5 also shows that with calibration and measurement separated by less than 1 h, the model was able to classify with an accuracy beyond 94% from the sixth calibration measurement onwards. As expected, these outcomes decreased as the distance between calibrations and test measurement increased, thus requiring up to five calibration measurements with distances between 6 h and 24 h to obtain classification accuracies above 75%. In any case, the need to perform several calibration measurements to achieve very good classification accuracy with test measurements, which could be many hours or even days away from calibration, does not seem to be a serious limitation. On the other hand, it is worth mentioning that only five subjects had recording lengths longer than five days, so that less stable results in Figure 5d can be considered as normal, because any misclassification would significantly affect the final accuracy. Although the number of subjects was not quite elevated in this last case, the obtained results demonstrate that classification performance was very good even with distances between calibration and test measurement of several days, which is very promising for real-world applications based on embedding these methodologies into wearable devices.

**Figure 5.** Classification accuracy to distinguish between NTS and HTS individuals using KNN sequential validation of segments with calibration distant from test measurements. (**a**) Distance below 1 h. (**b**) Distance between 1 and 6 h. (**c**) Distance between 6 and 24 h. (**d**) Distance above 24 h. In each box, the red line indicates the median, and the bottom and top edges indicate the 15th and 85th percentiles, respectively. The whiskers extend to the most extreme data points not considered outliers, and the red symbol (+) stands for outliers. Black squares inside each box indicate mean accuracies.

#### **4. Discussion**

The continuous measurement of BP is of great importance as it facilitates the early detection and prevention of hypertension, being the main risk factor for many CVDs. With the eruption in recent years of the Internet of Things [38] and cuff-less devices that are able to continuously measure and process physiological signals applying artificial intelligence techniques, such as ML and Deep Learning (DL), alternatives to traditional cuff-based single-time BP measurement methods have been proposed. The main signal used in related studies has been the PPG, as its morphological variations are related to the heart's activity and vascular walls condition, being similar to BP morphology both in frequency and time domains [39]. Furthermore, PPG signal can be acquired by non-invasive low-cost devices as smart watches, obtaining a continuous and real time measurement.

The monitoring of BP through PPG has mainly been studied by two different approaches: (i) addressing the problem of monitoring BP as a regression task estimating systolic and diastolic values; and (ii) addressing the problem of detecting hypertensive subjects as a classification task. In this study, the second approach has been developed, as estimations from the first approach still have serious limitations, so that it is more clinically beneficial to alert hypertensive subjects, acting as a support for clinical decision making.

Tjahjadi et al. [23] proposed the use of KNN technique and PPG signal without ECG, requiring the extraction of 2100 PPG feature points from 2.1 s of data. Their classification results achieved an F1-score of 100% for NTS and PHT patients and 90.80% for HTS patients. Although the authors affirm that this method achieved higher classification performance than other ML and DL methods, obtaining 2100 PPG feature points in such a short period of time required a sampling frequency of 1 kHz, which is a serious drawback for embedding this method in wearable devices, as it significantly increases the amount of data sampled, saved and transmitted which, unavoidably, will involve a considerably high power consumption.

Most studies for HT risk classification use both PPG and ECG signals, as PAT value is directly related to BP value. Although previous works have studied the efficiency of employing PAT as the only parameter to estimate BP [14,40], Liang et al. [21] reported a higher correlation with BP levels by combining PAT with additional PPG features. Dividing the dataset into 70% for training and 30% for validation, the KNN classification model obtained the best performance compared to bagged tree, logistic regression and AdaBoost tree. The F1 scores comparing NTS vs. PHT, NTS vs. HTS and NTS + PHT vs. HTS were 84.34%, 94.84% and 88.49%, respectively. As a consequence, the HT risk-discrimination performance between NTS and HTS was similar to the one achieved in the present study, where F1 score with the KNN classification model was 91.93%, employing leave-one-out cross-validation strategy. Therefore, both in previous studies and in the present work, the KNN classifier has been the best model to assess HT risk, combining PPG recordings and Machine Learning techniques. However, until now, there is no agreement about the discriminant features to be used, since it depends considerably on patient selection and database, mode of acquisition and signal quality.

In recent years, DL approaches have obtained outstanding performance extracting information from images [41,42]. Liang et al. [43] used the continuous wavelet transform of PPG signals and convolutional neural networks to classify BP. The dataset was divided in 80% for training and 20% for testing. The F1 scores for the binary classification comparing NTS vs. PHT were 80.52%, NTS vs. HTS were 92.55% and (NT + PHT) vs. HT trials were 82.95%. The main disadvantages of DL approaches are the requirement of a high computational cost, the extra duration of the training stage and the need for a large number of recordings.

One important consideration introduced by this work, that was not specified in related studies, has been the study of subjects with stable labels of BP. Usually, BP levels vary slightly throughout the day depending on the activities carried out by each person and many other factors, however, each subject would have to be labelled with a single and stable label. For example, any subject cannot be diagnosed as HTS at certain moments of the day, PHT at others and NTS at others. This is a problem when using databases such as the MIMIC, as it consists of recordings from ICU patients that, as a consequence of their unstable condition or the administration of drugs, may have altered and variable SBP levels.

Furthermore, any previous study about hypertension risk assessment has taken into account the relevance of calibration as a factor improving significantly classification results. In this respect, calibration has been only considered in other studies addressing BP estimation in combination with other patient data such as age [44], distance and area of arteries between measure sites or other factors that increase BP as exercise or postural changes [40]. Recently, Schlesinger et al. [45] used convolutional neural networks and PPG signals for BP estimation, achieving a reduction in mean absolute difference of 2.54 mmHg after calibration, using a single 30 s window of PPG signal and the associated BP reading.

The present study has proposed two calibration approaches, trying to improve the poor initial classification accuracy of 51.48% when a new subject entered the method without any previous calibration. The first approach investigated if the method improved classification accuracy when consecutive sub-segments of each subject were used both for calibration and classification, employing sequential validation. The assumption here was the supposed high similarity between calibration measurements and test measurement. Figure 4 showed that the presence of just one calibration sample was enough to increase classification performance more than 30%, which was enhanced even more as the number of calibration measurements raised.

The second approach studied the benefit of calibration for distances between calibration time and measurement time varying from less than 1 h to more than 24 h. This way, it was considered if PPG signal properties from the same patient were kept across time or changed along the day or week. For distances to calibration below 1 h, classification accuracy improved by 30% with just one calibration, keeping these results until more than 6 segments from the same patient were in the training dataset. The improvement of hypertension risk classification decreased slightly as the distance between calibration and

measurement increased, although the use of calibration always improved classification results compared to classifying a new uncalibrated subject. Thus, after the fifth calibration, all the experiments provided high accuracy.

These approaches have demonstrated that the properties of each patient's PPG features were variable over time, as worse results were obtained with measurements distant from calibration than with those very close in time to calibration. Therefore, in order to ensure high classification accuracy, several recalibrations performed at distant recording times and, if possible, in different situations are recommended to accurately asses the risk of HT with PPG and ECG recordings, which can be obtained in a simple way through wearable devices.

Finally, this study has certain limitations that are worth considering. Even though more than 900 recordings were analyzed, the total number of patients was not too large and there was no information available on factors that may imply a higher risk of HT such as age, sex or physical condition. In this respect, Mukkamala et al. [46] studied the age factor in calibration predicting a maximum calibration interval of 1 year for subjects of 30 years of age, that declined linearly to 6 months for subjects at the age of 70, using the PTT as discriminatory feature. In addition, this study has only applied the artificial intelligence technique of ML. Future works will address the application of DL classifiers in order to discern whether they are able to improve hypertension risk assessment of current ML classifiers employing calibrated PPG recordings.

#### **5. Conclusions**

The combined extraction of discriminant features from PPG and ECG recordings, together with the use of machine learning classification models such as KNN, has been able to perform outstanding hypertension risk assessment in the discrimination between NTS or HTS subjects. The application of per-subject calibration, both in close and distant measurements, has proved its relevance for accurate classification. The implementation of these artificial intelligence techniques in wearable devices would improve the early diagnosis and prevention of cardiovascular diseases associated to hypertension.

**Author Contributions:** Conceptualization, J.C., R.A. and J.J.R.; methodology, J.C., R.A. and J.J.R.; software, J.C. and J.J.R.; validation, J.C., L.F., J.M.G.-B., R.Z., R.A. and J.J.R.; resources, L.F., J.M.G.-B., R.Z. and J.J.R.; data curation, J.C., L.F., J.M.G.-B. and R.Z.; original draft preparation, J.C.; review and editing, J.C., L.F., J.M.G.-B., R.Z., R.A. and J.J.R. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received financial support from grants PID2021-00X128525-IV0, PID2021- 123804OB-I00 and TED2021-129996B-I00 of the Spanish Government 10.13039/501100011033 jointly with the European Regional Development Fund (EU), SBPLY/17/180501/000411 from Junta de Comunidades de Castilla-La Mancha and AICO/2021/286 from Generalitat Valenciana.

**Institutional Review Board Statement:** Data is available upon request and through MIMIC approval. MIMIC project was approved by the Institutional Review Boards of Beth Israel Deaconess Medical Center (Boston, MA, USA) and the Massachusetts Institute of Technology (Cambridge, MA, USA).

**Informed Consent Statement:** Requirement for individual patient consent was waived as the study did not impact clinical care and all data were de-identified.

**Data Availability Statement:** The data supporting reported results and presented in this study are available on request from the corresponding author.

**Conflicts of Interest:** The authors have no association with commercial entities that could be viewed as having an interest in the general area of the submitted manuscript. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

#### **References**

## *Article* **Proof of Concept in Artificial-Intelligence-Based Wearable Gait Monitoring for Parkinson's Disease Management Optimization**

**Robert Radu Iles, an 1,2, Claudia-Georgiana Cordos, 3, Laura-Ioana Mihăilă 3, Radu Fles, ar 4, Ana-Sorina Popescu 1, Lăcrămioara Perju-Dumbravă 1 and Paul Faragó 3,\***

**Abstract:** Parkinson's disease (PD) is the second most common progressive neurodegenerative disorder, affecting 6.2 million patients and causing disability and decreased quality of life. The research is oriented nowadays toward artificial intelligence (AI)-based wearables for early diagnosis and long-term PD monitoring. Our primary objective is the monitoring and assessment of gait in PD patients. We propose a wearable physiograph for qualitative and quantitative gait assessment, which performs bilateral tracking of the foot biomechanics and unilateral tracking of arm balance. Gait patterns are assessed by means of correlation. The surface plot of a correlation coefficient matrix, generated from the recorded signals, is classified using convolutional neural networks into physiological or PD-specific gait. The novelty is given by the proposed AI-based decisional support procedure for gait assessment. A proof of concept of the proposed physiograph is validated in a clinical environment on five patients and five healthy controls, proving to be a feasible solution for ubiquitous gait monitoring and assessment in PD. PD management demonstrates the complexity of the human body. A platform empowering multidisciplinary, AI-evidence-based decision support assessments for optimal dosing between drug and non-drug therapy could lay the foundation for affordable precision medicine.

**Keywords:** artificial intelligence; sensors; convolutional neural networks; Parkinson's disease; biomedical monitoring; accelerometer; pressure sensor; disease management; electromyography; correlation

#### **1. Introduction**

More than 200 years ago, in 1817, Dr. James Parkinson published a scientific work entitled "The Essay on Shaking Palsy" [1] and with it the foundation of the disease that bears his name. After two centuries, we still struggle to understand and treat neurodegenerative diseases, such as Parkinson's disease (PD), which are growing exponentially, especially in industrialized regions, and given that no one is immune to them, specialists are concerned about "the Parkinson pandemic" [2].

In 2017, we started the PDxOne project, intending to optimize the management of PD by offering sustainable automated or semiautomated solutions.

**Citation:** Iles,an, R.R.; Cordos, , C.-G.; Mih ˘ail ˘a, L.-I.; Fles,ar, R.; Popescu, A.-S.; Perju-Dumbrav ˘a, L.; Faragó, P. Proof of Concept in Artificial-Intelligence-Based Wearable Gait Monitoring for Parkinson's Disease Management Optimization. *Biosensors* **2022**, *12*, 189. https:// doi.org/10.3390/bios12040189

Received: 31 January 2022 Accepted: 17 March 2022 Published: 23 March 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

#### *1.1. Considerations on Parkinson's Disease*