**Author details**

Rajeev Singh1 \* and Teek Parval Sharma2

1 G.B. Pant University of Agriculture and Technology, Pantnagar, Uttarakhand, India

2 National Institute of Technology, Hamirpur, Himachal Pradesh, India

\*Address all correspondence to<EMAIL_ADDRESS>
© 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

**65**

pp. 157-163

*Security in Wireless Local Area Networks (WLANs) DOI: http://dx.doi.org/10.5772/intechopen.89857*

[1] IEEE 802.11i. Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications: Medium Access Control (MAC) Security Enhancements. 2004; IEEE Standard

Tenth IEEE International Conference on Computer and Communication Technologies, WOCN'13; July 2013;

[9] Newman LH. The Secure Wi-Fi Standard Has a Hugh Dangerous Flaw [Internet]. 2017. Available from: https://www.wired.com/story/ krack-wi-fi-wpa2-vulnerability/

[10] Wi-Fi Alliance. Discover Wi-Fi Security [Internet]. Available from: https://www.wi-fi.org/discover-wi-fi/

[11] Li X, Bao F, Li S, Ma J. FLAP: An efficient WLAN initial access authentication protocol. IEEE

Systems. 2013;**25**(2):488-497

[12] Martinovic I, Zdarsky FA,

Computing. 2013;**2**(3):176-187

Transactions on Parallel and Distributed

Bachorek A, Schmitt JB. Measurement and analysis of handover latencies in IEEE 802.11i secured networks. In: Proceedings of the European Wireless Conference (EW2007); April 2007;

[13] Singh R, Sharma TPA. Secure WLAN authentication scheme. IEEK (Institute of Electronics Engineers of Korea) Transactions on Smart Processing and

[14] Tang C, Wu DO. An efficient mobile authentication for wireless networks. IEEE Transactions on Wireless

Communications. 2008;**7**(4):1408-1416

[16] Singh R, Sharma TP. A key hiding communication scheme for enhancing the wireless LAN security. Springer Wireless Personal Communications.

2014;**77**(2):1145-1165

[15] Park CS. Two-way handshake protocol for improved security in IEEE 802.11 wireless LANs. Computer Communications. 2010;**33**(9):1133-1140

Bhopal. pp. 1-4

security

Paris. pp. 1-7

[2] Singh R, Sharma TP. On the IEEE 802.11i security: A denial of service perspective. Wiley Journal of Security and Communication Networks.

[3] Lepaja S, Maraj A, Efendiu I and Berzati S. The impact of the security mechanisms in the throughput of the WLAN networks. In: Proceedings of the 7th Mediterranean Conference on Embedded Computing; June 2018;

[4] Clancy TC. Secure handover in Enterprise WLANs: CAPWAP, HOKEY, and IEEE 802.11r. IEEE Wireless Communications. 2008;**15**(5):80-85

(IRJET). 2019;**06**(01):537-545

[6] Singh R, Sharma TPA. Location based method for restricting the flooding DoS effect in WLANs. Journal of Location Based Services. 2016;**9**(4):273-295. Taylor and Francis

[7] Singh R, Sharma TP. A key refreshing technique to reduce 4-way handshake latency in 802.11i based networks. In: Proceedings of the Fourth IEEE International Conference on Computer and Communication Technologies, ICCCT'13; September 2013; Allahabad.

[8] Singh R, Sharma TP. A sequence number based WLAN authentication scheme for reducing the MIC field overhead. In: Proceedings of the

[5] Asante M, Akomea-Agyin K. Analysis of security vulnerabilities in Wifiprotected access pre-shared key (WPA-PSK/ WPA2-PSK). International Research Journal of Engineering and Technology

2015;**8**(7):1378-1407

**References**

Budva, Montenegra

*Security in Wireless Local Area Networks (WLANs) DOI: http://dx.doi.org/10.5772/intechopen.89857*


**4.3. Remarks**

In spite of the scarcity of the log data, the proposed method achieved extremely good results. In case of data abundance, if the data is not well calibrated it could compromise the quality of the convergence and the influence of the not calibrated wells in the final model would be noticeable.

In this case, both synthetic seismic data and acoustic impedance cube captured the main geologic features of these complex reservoirs, noticeable in the correlation coefficients between the seismic and the synthetic amplitudes. The quality of the seismic data takes a minor role since the method overcomes the situation of imposition of artificial correlations as it happens in the standard methods;

Since the co-simulation of the impedances uses a local coefficient correlation, it is possible to compute the local uncertainty associated to the seismic acoustic impedances;

The uncertainty of the seismic acoustic impedance could be used to access the uncertainty associated with the porosity model, as presented in [24].

Tests prove that the variation of the final correlation coefficient is about 2% with the modification of the initial seed, it means that others parameters such as the number of layers and the size of it, as other parameters can be optimized to produce better results. But these results are more difficult to reach when the complexity of the geology and the structural model became more elaborated.

To handle different geology scenarios such channels or specific shape reservoirs, an adapted approach is proposed in the next part of this work.


**Table 6.**

*Robustness.*

DAD comes first because it is a non-destructive detection method. MSD not only provides information on the qualitative chemical composition of the sample but also in certain situations allows us to determine the quantitative content of the component (glutathione) that is not visible to DAD. The calibration curve for the MSD is not linear, but parabolic, which probably reflects the mutual repulsion of like-charged particles in the gas phase. The use of MSD for quantitative analysis requires further study and justification.


**Learning Entropy: Multiscale Measure for Incremental Learning**

## **Ivo Bukovsky**

Czech Technical University in Prague, Technicka 4, 166 07, Prague 6, Czech Republic; E-Mail<EMAIL_ADDRESS>Tel.: +420-224-352-529; Fax: +420-224-352-674

*Received: 26 July 2013; in revised form: 17 September 2013 / Accepted: 22 September 2013 / Published: 27 September 2013*

**Abstract:** First, this paper recalls a recently introduced method of adaptive monitoring of dynamical systems and presents the most recent extension with a multiscale-enhanced approach. Then, it is shown that this concept of real-time data monitoring establishes a novel non-Shannon and non-probabilistic concept of novelty quantification, *i.e.*, Entropy of Learning, or in short the Learning Entropy. This novel cognitive measure can be used for evaluation of each newly measured sample of data, or even of whole intervals. The Learning Entropy is quantified in respect to the inconsistency of data to the temporary governing law of system behavior that is incrementally learned by adaptive models such as linear or polynomial adaptive filters or neural networks. The paper presents this novel concept on the example of gradient descent learning technique with normalized learning rate.

**Keywords:** incremental learning; adaptation plot; multiscale; learning entropy; individual sample learning entropy; approximate learning entropy; order of learning entropy; learning entropy of a model; non-Shannon entropy; novelty detection; chaos; time series; HRV; ECG

#### **Nomenclature**

#### **1. Introduction**

Prediction and novelty detection of dynamical system behavior is a vital topic today. Time series are common representatives of observed behavior of complex dynamical systems and the non-stationarity and perturbations are the real-world drawbacks. Such variously caused novelties of newly measured samples of data affect the prediction accuracy and thus they can affect, e.g., control, diagnostics, medical treatment accuracy, and can interfere with many other signal processing objectives.

It has been shown more recently in [1,2] that novelty of individual samples of time series or even intervals of behavior of complex, high-dimensional and nonlinear dynamical systems can be efficiently monitored by relatively simple adaptive models (*i.e.*, low-dimensional neural network architectures). By real-time adaptation of such short-term predictors and by observing also the behavior of adapted parameters (neural weights), we are able to cognitively monitor and evaluate every new measured sample or even whole intervals of behavior with varying complexity (e.g., varying levels of chaos, noise, perturbations). Therefore, in this approach every new measured sample of data is evaluated with respect to its consistency to temporary governing law (dynamics) of a system, which is different from common statistical measures and furthermore, different from entropy based approaches that do not consider consistency of data with the governing law of behavior of data. Moreover, the cognitive approach presented in this paper is also different from existing methods of novelty detection that use learning systems, because this approach does not operate with residuals of the learning system. The terms adaptation, learning and incremental learning can be understood to be equal for clarity of explanations in this paper. However, the learning process of a learning system is generally understood to be a more complex cognitive process than just a parameter adaptation technique [3,4]. The novel concept of entropy in this paper is not limited only to the supervised adaptation, but the principal is applicable to any learning systems in general.

In literature, two fundamental streams of evaluating the entropy of data in dynamical systems in the sense of information contents (novelty) that is carried by measured samples of data can be tracked down, *i.e.*, the probability based approaches, e.g., [5] and the learning system based approaches, e.g., [6].

The first (probabilistic) stream is represented by the statistical approaches of novelty measures and by probabilistic approaches for evaluation of entropy. The Sample Entropy (SampEn) and the Approximate Entropy (ApEn) are the very typical and relevant examples [7,8]. These approaches are closely related to the multi-scale evaluation of fractal measures as discussed in [9–12] and thus to the power-law [13] concept, which is also the partial inspiration for the presented matter in this paper. The usefulness of multi-scale approach is also apparent from the coarse-graining based multi-scale extensions to SampEn in [14,15] and its further and very recent extension in [16]. Some more case studies utilizing SampEn, ApEn, and Multiscale Entropy (MSE) can be found in [17,18].

Another probabilistic approach to the evaluation of entropy as the conditional mutual information between present and past states is proposed as the Compensated Transfer Entropy in [19]. Work [20] can be referenced for the fault detection using a probabilistic entropy approach, and a probabilistic entropy approach to the concept shift (sometimes the concept drift) detection in sensory data is reported in [21].

The second stream is represented by the utilization of learning systems such as neural networks and fuzzy-neural systems, and this is also the main area relevant to the presented work in this paper. During the last three decades of 20th century, the works that were in focus regarding learning systems are [22–25], and for incremental learning approach can be referenced for example also the work [26]. Then, a particularly focused approach toward the utilization of learning systems has been rising with works [27–29], where nonlinear estimators and learning algorithm were utilized for the fault detection via the proposed utilization of a fault function that evaluates behavior of residuals of a learning system. Currently, significant research that shall be referenced is adaptive concept drift detectors proposed in [30–32] and the cognitive fault diagnosis system for sensory data published in [33]. Some readers might also see some analogies of the proposed approach in this paper to the Adaptive Resonance Theory [34]. Because, the proposed approach in this paper utilizes a memory of data behavior, which is represented by the online learning parameters of a learning system, and the unusual behavior of incrementally learning parameters is quantified and introduced as the novel entropy concept in this paper.

Up to the best of my knowledge, I am not aware of any works by other authors on non-probabilistic approaches for evaluation of entropy which are, in their very principal, free from any use of output residuals of a learning model; so that only the behavior of incrementally learning parameters of even imprecise learning models would serve for novelty evaluation in sense of information contents quantification (entropy).

This paper introduces novel concept of entropy and its calculation that neither is based on statistical approaches nor is it based on evaluation of error residual. This new approach operates only on parameter space of incrementally learning systems. The presented principle is purely based on evaluation of unusual behavior of incrementally learning parameters of a pre-trained model, regardless the error residual, *i.e.*, in principle regardless the prediction error itself or its behavior in time. This paper demonstrates the novel approach on Gradient Descent (GD) adaptation that is one of the most comprehensible incremental learning techniques. The very original and funding principals and some related results with Adaptation Plot (AP) have been published in [1,2,38,39] and the first multi-scale extension was proposed in [40]; those are the funding concepts of *Learning Entropy* (LE) and the *Approximate Individual Sample Learning Entropy* (*AISLE*) that are introduced in this paper.

The paper is organized as follows: the second section recalls two fundamental principles (techniques) that are necessary for evaluating the LE, *i.e.*, GD—a comprehensible example of an incrementally learning technique, and the technique of visualization of learning energy, *i.e.*, the AP. The third section derives the calculation of the novel measure of learning activity, *i.e.*, the measure of learning energy that an incrementally learning model displays for each newly measured sample of data and, thus, the principle of the *Individual Sample Learning Entropy* (ISLE). Then, a practical cumulative-sum technique for estimation of ISLE is introduced as the Approximate Individual Sample Learning Entropy. Consequently, the concept of the *Order of Learning Entropy* is introduced according to the order of the estimated time derivative of neural weights that serve to calculate the LE.

The fourth section shows experimental demonstrations including real-world data application of AISLE. The fifth section discusses results, furthermore, theoretical and practical aspects of LE, and it also discusses the fact that LE is not necessarily correlated to the magnitude of prediction error.

In terms of mathematical notation, variables are denoted as follows: small caps as "*x*" for a scalar, bold "**x**" for a vector and, bold capital "**X**" for a matrix. Lower indexes as in "*xi*" or "*wi*", indicate the element position in a vector. If a discrete time index is necessary to be shown, it comes as "*k"* in round brackets such as **x**(*k*), *y* denotes measured time series and stands for a predictor output. Further notation, as such **w**, represents a vector that contains all adaptable parameters, *i.e.*, weights of a predictor and, **w** is a vector of all adaptive weight increments that are the cornerstone quantities for evaluation of LE by incrementally learning models. The meaning of other symbols is given at their first appearance throughout the text. Time series of constant sampling are considered. *y*-

#### **2. Funding Principles**

This section reviews two fundamental principles for the latter introduced LE concept. The very fundamental principle is the supervised incremental learning of predictive models, *i.e.*, sample-by-sample adaptation of adaptive parameters (neural weights) to the evaluated signal. As the very cornerstone approach, the GD (incremental) learning is recalled such as for linear or polynomial predictors and neural networks. The second fundamental principle is the binary-marker visualization of how much must the (initially pre-trained) predictor adapt its weights to each sample of data to capture contemporary governing law, *i.e.*, this is the technique of the AP [1–2].

#### *2.1. Predictive Models and Adaptive Learning*

Though not limited to, the GD adaptation algorithm is the most fundamental technique for the evaluation of LE. Moreover, GD learning is very efficient especially when used with linear filters or low-dimensional neural network architectures (predictors). The use of GD is recalled particularly for linear predictors (filters) and for polynomial predictors (also called Higher-Order Neural Units HONUs [1,35–37]) in this subsection.

As for time series, let us consider the representation of a general prediction scheme as follows;

$$\tilde{\mathcal{Y}}(k+h) = f\left(\mathbf{x}(k), \mathbf{w}\right) \tag{1}$$

where denotes predicted value at prediction horizon of *h* samples; *f*(.) is a general differentiable function (linear or polynomial predictor or a neural network) mapping the input vector **x**(*k*) to the predicted output; vector **w** contains all adaptable parameters (weights) of a predictor. *y*-( ) *k h*-

To unify the terminology about general predictors (1) that are used for the purpose of LE, the following lemmas are given:

L1. Predictor (1) is a static model that performs direct prediction if input vector **x**(*k*) contains only the recent history of measured data.

L2. Predictor (1) is a dynamical (recurrent) model that performs indirect prediction if input vector **x**(*k*) contains also step-delayed values of . *y*-

L3. Dimensionality of predictor (1) corresponds to the state space dimension for which the mapping *f*(.) is defined (for Equation (1) this relates to the numbers of inputs in **x** including step-delayed feedbacks of if a predictor is dynamical one). *y*-

L4. Dimensionality of a real data corresponds to the order of dynamics of real-data-generating system and it is further extended by other real inputs (that further increase dimensionality of behavior of real data).

L5. A low-dimensional predictor of time series is such predictor that is considerably less-dimensional than the embedding dimension of the time series itself.

Sample-by-sample GD adaptation scheme of predictor (1) can be defined using prediction error *e*, which is given as:

$$e(k) = \mathcal{Y}(k) - \tilde{\mathcal{Y}}(k) \tag{2}$$

so the individually adapted weight increments are calculated in order to decrease the square error criteria as follows:

$$
\Delta\boldsymbol{\omega}\boldsymbol{\eta}(k) = -\frac{1}{2}\cdot\boldsymbol{\mu}\cdot\frac{\widehat{\boldsymbol{\mathcal{C}}}\boldsymbol{e}(k)^{2}}{\widehat{\boldsymbol{\mathcal{C}}}\boldsymbol{\eta}\boldsymbol{\eta}} = \boldsymbol{\mu}\cdot\boldsymbol{\mathcal{C}}\cdot\frac{\widehat{\boldsymbol{\mathcal{O}}}\boldsymbol{\mathcal{Y}}(k)}{\widehat{\boldsymbol{\mathcal{C}}}\boldsymbol{\eta}\boldsymbol{\eta}} = \boldsymbol{\mu}\cdot\boldsymbol{\mathcal{C}}\cdot\frac{\widehat{\boldsymbol{\mathcal{O}}}\boldsymbol{f}\left(\mathbf{x}(k-h),\mathbf{w}\right)}{\widehat{\boldsymbol{\mathcal{O}}}\boldsymbol{w}\boldsymbol{\eta}}\tag{3}
$$

where, is an adaptive weight increment of *i* th weight,  is the learning rate, and *k* is the discrete index of time that also denotes the reference time, *i.e.*, is currently measured error, is predicted value *h* samples ahead. For completeness, the updates of all weights at each sample time *k* can be in its simplest form (no momentum or regularization term) as follows: *wi*( ) *k e*( ) *k y*-( ) *k h* -

$$\mathbf{W}(k+1) = \mathbf{W}(k) + \Delta \mathbf{W}(k) \tag{4}$$

Recall, stability of the reviewed GD algorithm (2–4) can be practically improved by proper scaling input and output variables (e.g., z-score) and by various approaches for rescaling the learning rate *-*(e.g., [41,42]).

In case the predictor is a dynamic model, *i.e.*, lemma L2, we may alternatively refer to GD as to Real Time Recurrent Learning (RTRL) technique [43] if the above GD scheme (1–4) is applied with recurrently calculated derivatives for feedback elements in input vector **x**.

Equation (1) gives only a general form of a predictor for LE. The particular form of the mapping *f*(.) and configuration of inputs and feedbacks in input vector **x** as in Equation (1), as well as the proper sampling period, are all case specific.

Nevertheless, it can be reasonable to start with linear adaptive filters as they are simplest and computationally efficient especially when vector **x** should contain relatively higher number of inputs (~ > 20). In case of linear filters, the predictor (1) yields the vector multiplication form of row vector **w** and column vector **x** as follows:

$$
\tilde{\mathbf{y}}(k+h) = \mathbf{w}(k) \cdot \mathbf{x}(k) \tag{5}
$$

The weight updates are directly calculated for a linear model as follows:

$$\mathbf{A}\mathbf{w}(k) = \mu \cdot \mathbf{e}(k) \cdot \mathbf{x}(k)^T \tag{6}$$

where, *T* denotes vector transposition and where recurrently calculated partial derivatives (as if according to RTRL) are neglected, *i.e.* , . **xw0**

As regards selection of learning rate *-*, the first technique that should be considered is the learning rate normalization that practically improves the stability of the weight update system (4, 6), so the weight updates can be actually calculated at every sample time as follows:

$$\Delta \mathbf{W}(k) = \frac{\mu}{1 + \left\| \mathbf{x}(k) \right\|} \cdot e(k) \cdot \mathbf{x}(k)^T \tag{7}$$

where "|| ||" denotes a vector norm—more on techniques for learning rate normalization and adaptation of a regularization term (the unit in the denominator) can be found, e.g., in [41,42].

For evaluation of LE of nonlinear time series, polynomial adaptive predictors such as Higher-Order Neural Units can be recommended [1,35–37]; HONUs are attractive adaptive models because their mapping is customable as nonlinear while they are linear in parameters => optimization of HONUs is of a linear nature, so HONUs do not suffer from local minima problem in the way as conventional neural networks do when GD learning technique is used.

Of course, because of various systems and according to various user experience, other types of predictors such as perceptron neural networks or any other kind of adaptive models, suitable for GD (but not limited to GD) adaptation, can be used as a cognitive (here the supervised), incrementally learning tool, for evaluation of LE. This subsection recalls the GD rule as a straightforward example of incremental learning technique, that is a comprehensible option for AP and latter for calculation of LE.

#### *2.2. Adaptation Plot (AP)*

The variability of weight increments **w** in (3) resp. (7) reflects the novelty of data that corresponds to the difficulty with which an adaptive model learns to every new sample of data. Therefore, the AP was introduced for GD and adaptive models (HONUs) in [1] and further in [2,38,39] as a visualization tool of adaptation activity (or novelty in data) of adaptive predictors.

AP is based on evaluation and visualization of unusual weight increments of sample-by-sample GD adapted models. It was shown through [1,2,38–40] that low-dimensional predictors can capture and evaluate important signal attributes. As such, unusual samples, very decent perturbations, unusual appearance or variations of level of chaos or noise, incoming inter-attractor transitions of hyper-chaotic systems, also hidden repeating patterns can be revealed and intervals of a similar level of chaos can be revealed in otherwise seemingly, similarly complicated signals.

To clarify the principle of AP, the sensitivity parameter for marker detection of AP has to be recalled. A governing law variability marker (a dot) in AP (Figure 1) is drawn at every sampling time *k*, if the corresponding weight increment exceeds its contemporary, usual magnitude, which can be in principle sketched by the following rule:

$$\text{if } \dot{f}\left(\left|\Delta w\_{\hat{l}}(k)\right|>\alpha \cdot \overline{\left|\Delta w\_{\hat{l}}(k)\right|}\right) \Rightarrow \text{draw a marker for weight } w\_{\hat{l}} \text{ at time } k \tag{8}$$

where, is the detection sensitivity parameter, and is a floating average of absolute values of recent *m* neural increments of *i* th neural weight as follows: ( ) *k wi*

$$\left| \overline{\Delta \mathbf{w}\_{\bar{l}}(k)} \right| = \frac{1}{m} \cdot \left( \sum\_{j=k-m+1}^{k} \left| \Delta \mathbf{w}\_{\bar{l}}(j) \right| \right) \tag{9}$$

**Figure 1.** Adaptation Plot (AP) is a tool for universal evaluation of information hidden in adapted neural weights via transformation into a binary space (time series is cognitively transformed to patterns of binary featuresüAP markers (the dots)), for more on functionality of AP please see [1,2].

The mutually alternative explanations of the sensitivity are as follows:

The major single-scale weakness of AP is the need for manual tuning of the sensitivity parameter , so the first multi-scale solution to AP has been proposed in [40] without connotations to any entropy concept*.*

In the next section, it is proposed that the multi-scale solution to novelty detection via AP and over a whole range of sensitivity detection establishes a novel entropy concept of LE.

#### **3. Learning Entropy (LE)**

In this section, the concept of Learning Entropy (LE) is introduced for supervised incremental learning. This novel entropy concept can utilize sample-by-sample adaptation of low-dimensional predictors [1,2,36–39] and uses the technique of the AP. Notice, the GD with normalized learning rate (e.g., [41,42]) is used in this paper for its clarity and for its good performance; however, LE is not principally limited to only GD technique, nor to supervised techniques in general.

In fact, LE is a cognitive entropy measure concept because the cognitively obtained knowledge about variation of temporary governing laws of the evaluated data is utilized.

Important distinction of this concept is that if a system behavior is very complex from statistical point of view, but it is deterministic from the point of view of its governing law, the information content (complexity, entropy) of the data is lower, the more deterministic the behavior is (deterministic chaos, forced nonlinear (chaotic) oscillator).

For example, if a predictive model can adapt fully to a governing law of deterministically chaotic time series, then the further data of time series have no new information to us (the new data are redundant because we know a governing law). However, if a deterministic (chaotic) time series becomes perturbed, the perturbed data (samples or intervals) have new information, *i.e.*, novel data have entropy that can be adaptively (cognitively) detected (e.g., by supervised GD learning).

#### *3.1. Individual Sample Learning Entropy (ISLE)*

In this subsection, LE is approached via GD (supervised learning) and it is demonstrated on the example of deterministically chaotic time series obtained from Mackey-Glass equation [44] in chaotic mode as particularly given in Equation (10):

$$\frac{d\mathbf{y}(t)}{dt} = \mathbf{b} \cdot \mathbf{y}(t-\tau) \cdot \left(\mathbf{1} + \mathbf{y}(t-\tau)^{10}\right)^{-1} - \mathbf{g} \cdot \mathbf{y}(t) \tag{10}$$

where *t* denotes the continuous index of time, and the chaotic behavior results from the setup of; *b* = 0.2, *g* = 0.1, and the lag = 17. The time series was generated by Equation (10) and data were sampled with period *t* = 1 [time unit] as {*y*(*k*); *k* = 0,1,…,700} where *k* denotes the discrete time index.

Lets introduce 1% perturbation at sample *k* = 500 as follows:

$$\mathbf{y}(500) = \mathbf{y}(500) + \mathbf{0}.01 \cdot \mathbf{y}(500) \tag{11}$$

The time series with the detail of the perturbed sample at *k* = 500 is shown in Figure 2. First 200 samples is used to pre-train a low-dimensional predictive model (given random initial weights **w**) by GD in 300 epochs. Then, the adaptive model runs adaptation only once on further data *k* = 200…700.

As a nonlinear and low-dimensional predictive model, static quadratic neural unit (QNU, [1,35,36]) is chosen for its good quality of nonlinear approximation and in-parameter linearity that theoretically avoids problem of local minima for adaptation [37]. QNU can be expressed in a long-vector multiplication form as follows:

$$\tilde{\mathbf{y}}(k) = \sum \left\{ \mathbf{w}\_{\bar{i},j} \cdot \mathbf{x}\_{\bar{i}} \cdot \mathbf{x}\_{\bar{j}} \; ; \; \mathbf{i} = \mathbf{0} \dots n, \; j = i \dots n, \; \mathbf{x}\_{\mathbf{0}} = 1 \right\} = \mathbf{w} \cdot \mathbf{colx} \tag{12}$$

where, the length of recent history of time series in input vector **x** is chosen as *n* = 4 as follows:

$$\mathbf{x} = \begin{bmatrix} \mathbf{x}\_0 = \mathbf{1} & \mathbf{x}\_1 & \dots & \mathbf{x}\_n \end{bmatrix}^T = \begin{bmatrix} \mathbf{1} & \mathbf{y}(k-1) & \mathbf{y}(k-2) & \mathbf{y}(k-3) & \mathbf{y}(k-4) \end{bmatrix}^T \tag{13}$$

where, *T* stands for vector transposition and **colx** is the long column vector representation of quadratic multiplicative terms that are pre-calculated from **x** as follows:

$$\mathbf{col} \mathbf{x} = \begin{pmatrix} \mathbf{x}\_i \cdot \mathbf{x}\_j \ \vdots \ i = 0 \dots n, j = i \dots n, \ \mathbf{x}\_0 = 1 \end{pmatrix} = \begin{bmatrix} \mathbf{x}\_0 \ \mathbf{2} & \mathbf{x}\_0 \mathbf{x}\_1 & \mathbf{x}\_0 \mathbf{x}\_2 & \dots \ \mathbf{x}\_i \mathbf{x}\_j & \dots & \mathbf{x}\_n \end{bmatrix}^T \tag{14}$$

Furthermore, **w** is a row weight vector of the same length as **colx**.

The sample-by-sample updates of **w,** can then be calculated by the GD according to (1–6) as follows:

$$\mathbf{A}\mathbf{w}(k) = \mu \cdot e(k) \cdot \mathbf{c} \mathbf{ol} \mathbf{x}(k)^T \tag{15}$$

In regards to selection of the learning rate *-*, a variation of the learning rate normalization (e.g., [41]) practically improves the stability of the weight update system (6), so the weight updates Equation (15) can be actually calculated at every sample time as follows:

$$\Delta \mathbf{w}(k) = \frac{\mu}{1 + \mathbf{colx}(k)^T \cdot \mathbf{colx}(k)} \cdot e(k) \cdot \mathbf{colx}(k)^T \tag{16}$$

where *-* = 1 is used in following experiments for initial pre-training, and *-* = 0.1 is used for adaptive detection online.

**Figure 2.** Mackey-Glass time series in chaotic mode (10) with the detail of perturbation (11) at *k* = 500.

For the time series shown above and for the given setup, the QNU has 15 neural weights and the corresponding AP is shown in Figure 3. The AP in the bottom axis of Figure 3, shows that the six weight increments *w*9 and *w*11-15 of adaptive model were unusually large for the sensitivity = 5.5, *i.e.*, the pre-trained adaptive model (12) captured the perturbed sample at *k* = 500, while the prediction error *e*(*k* = 500:503) was of even a smaller than usual magnitude. Therefore, the markers in AP, visualize, activity in which the model learns to each newly measured sample, even when the adaptive model is not absolutely accurate (for another example please see Figure 13 in [40]).

**Figure 3.** The bottom axis is the AP with manually tuned for perturbed time series in Figure 2. The six AP markers for weights *w*9 and *w*11…15 at *k* = 503 correspond to the perturbed sample at *k* = 500 (11) (while the above magnitudes of prediction error and the adaptation weight increments do not indicate anything at first sight).

The particular time series in top axes in Figure 3 has a significant frequency component of about 60 samples and the most of the signal events chaotically recur within this interval. Therefore, parameter *m* that calculates recently usual magnitudes of weight increments Equation (9) is pragmatically set to *m* = 60 for this time series in this paper. More discussion on choice of parameter *m* follows in Section 5 of this paper.

The most critical parameter to obtain a meaningful and useful result with the AP (such as in Figure 3) is the detection sensitivity . To overcome this single-scale weakness of AP, *i.e.*, the dependence on proper selection of -, a multi-scale approach can be adopted. Naturally, the markers in AP appear according to , and the more sensitive detection is (*i.e.*, the smaller ), the more markers appear for more unusual samples of data [40], *i.e.*, for samples of higher LE. The dependence of AP markers is demonstrated in Figure 4.

**Figure 4.** The APs of the chaotic time series (Figure 2) with perturbed sample at *k* = 500 for increasing detection sensitivity (*i.e.*, decreasing = 6.08, 6.01, 5, 2); the number of AP markers *N*() related to perturbed sample at *k* = 500 tends to increase with increasing sensitivity more rapidly than for usual (not novel) samples.

For - = 6.08 in Figure 4, there are three AP markers that indicate some unusually large learning activity at *k* = 311,503,678. However, for constant , the detection must be made more sensitive to reveal the perturbed sample (*k* = 500) in contrary to the other seemingly novel points, which is manually made in Figure 4 by redrawing AP for = 6.01 and = 5. For = 2, the detection becomes too sensitive and the AP is not useful anymore.

To become independent of single-scale issue of manual selection of -, the multi-scale approach was proposed in [40]. It is further recalled in this section for the example of the above time series, and it is newly related to the concept of LE.

In order to enable us to perform multi-scale analysis of AP markers, we may consider the power-law concept, e.g., [13]. For AP markers at instant time *k* as a function of sensitivity parameter *,* where the detection sensitivity is increasing with decreasing , *i.e.*, we can assume theoretical power-law approximation as follows:

$$N(a) \cong \left(a\right)^{-H}, \quad \log\left(N(a)\right) \cong -H \cdot \log\left(a\right) \tag{17}$$

where, the exponent *H* characterizes the nonlinear change of quantity of AP markers along the varying sensitivity of detection , and *N* is the quantity of AP markers (here the vertical sum at instant *k* as in Figure 4) for the given sensitivity of detection .

Similarly to common fractal measure approaches, the change of quantity of AP markers along the increasing sensitivity parameter , can be quantified by estimation of characterizing exponent *H* as the slope of log-log plot as:

$$H = \lim\_{\alpha \to \alpha\_{\max}} \left( -\frac{\log(N(\alpha))}{\log(\alpha)} \right) \tag{18}$$

where, *max* is a specific (theoretical) value of detection sensitivity for which the very first AP marker would appear for evaluated samples *k.* Thus, *max* can be loosely defined as follows:

$$\sum\_{k} \left\{ N(\alpha \le \alpha\_{\max}) \right\} \ge 1; \; \sum\_{k} \left\{ N(\alpha > \alpha\_{\max}) \right\} = 0 \\ \; \; ; \; \sum\_{k} \left\{ N(\alpha\_{\max}) \ge 1 \right\} \tag{19}$$

For any arbitrarily close to*max*, *H* becomes large if a measured sample of data is novel, *i.e.*, if data is inconsistent with the governing law that has been temporarily learned by a predictor. For a particularly used predictor (1), the theoretical maximum *H* = could be obtained correctly only for those samples of data where all AP markers vertically appear instantly, *i.e.*, if where and where *n* is the number of all adaptable weights used for the AP. -% *N n* ( ) *max N*( ) 0,  & *max*

If we introduce a new variable *E* to normalize *H* as follows:

$$E(k) = \frac{2}{\pi} \cdot \arctan(H(k)) \implies E \in \{0, 1\} \tag{20}$$

then, we have arrived in Equation (20) to the normalized entropy measure *E* that quantifies learning activity of a sample-by-sample adaptive models (1–4). Thus, the variable *E* in Equation (20) is a novel non-Shannon and non-probabilistic measure for evaluation of novelty of each single sample of data in respect to it's consistency to the temporary governing law learned by a predictor. Thus, *E* in Equation (20) can be called the *Individual Sample Learning Entropy (ISLE).*

ISLE can be evaluated for all samples in a window of AP and consequently even only for a custom selection of particular samples. For example, Table 1 and Figure 5 shows comparison of ISLE for three specific samples of data for which AP markers appeared as the very first for *max* 6.08 in Figure 4, but where only *k* = 503 has AP markers due to the novelty in data. We can see in Figure 6 that sample at *k* = 503 has much larger *E* than the other two data samples. !

**Table 1.** The number of AP markers *N*() increases fastest for *k* = 503 because the incremental learning attempts to adapt the weights to the perturbation at *k* = 500 (Figure 2), *max* 6.08. !

**Figure 5.** The limit slope *H* (18) for Table 1 is largest when an adaptive model starts learning a new governing law (here the perturbation at *k* = 500) that causes unusually large weight increments (here calculated with normalized learning rate by Equation (16)).

**Figure 6.** AISLE calculated as *EA* by Equation (21) for the signal in Figure 2 and for sensitivities - *=* 3*\**[ 1.17 , 1.16 , 1.15 ], *m* = 60, *-*0 = 1, *-* = 0.1; the perturbation at *k* = 500 is followed by the rapid increase of LE.

Naturally, if an adaptive model is familiar with a temporary governing law of behavior of data, and if the measured samples of data are consistent with the governing law, then the adaptive model does not need to unusually adapt its parameters (weights) and *E* is low.

Importantly, evaluation of LE is not conditioned by the fact that the predictor must be precise and perfectly pre-trained. Inversely, if *E* = 0 is only achieved for all samples in an AP window, it does not necessarily imply that a predictor is precisely pre-trained and has zero prediction errors. *E* = 0 would just imply that the adaptive predictor (1) is familiar with data regardless its prediction error in Equation (2). *E* = 0 would mean that weights are constant (rounded to a decimal digit) during adaptation for all *k* even though the prediction error is not zero*;* this practically happens often with GD for not too large learning rate *-*(this can be practically verified on pre-training data).

In particular, this section demonstrated the calculation of the LE via the sample-by-sample adaptation and it is applicable to every new sample measured and it can detect novelty of individual samples. Therefore, this particular technique by Equations (1–20) results in evaluation of the novelty measure that can be called the *Individual Sample Learning Entropy (ISLE).*

However, the above estimation of ISLE via *E* by Equation (20) is rather a theoretical and explanatory matter, because proper estimation of slope *H* (18) depends on finding of *max*. The next subsection resolves this issue.

#### *3.2. Approximate Individual Sample Learning Entropy (AISLE)*

A practical technique to approximate *E* as a normalized measure of ISLE for every newly measured sample *y* is introduced in this subsection. This approximate technique does not require discovering of proper *max* complying strictly with (19). *E* can be approximated by *EA* for every sample of data *y*(*k*) as follows:

$$E\_A(k) = \frac{1}{n \cdot n\_{\alpha}} \sum \left\langle N(\alpha); a \in \mathfrak{a} \right\rangle, \ \mathfrak{a} = \left[ a\_1, a\_2, \dots, a\_{n\_{\alpha}} \right], \ \alpha\_1 > \alpha\_2 > \dots > \alpha\_{n\_{\alpha}} \tag{21}$$

which is a sum of markers over a range of sensitivities that is normalized by the number of weights *n* and by the number of selected sensitivity parameters *n*'(thus 1/(*n ·n* is a normalization term to achieve , *i.e.*, *n* markers can appear *n*times for a sample of data *y*(*k*). *EA* 0,1

For every individual sample, the measure *EA* in Equation (21) approximates *E* in (20), because larger values of *EA*, corresponds to a steeper slope *H* (Figure 5). Thus *EA* can be called the *Approximate Individual Sample Learning Entropy (AISLE).*

Also similarly to *E* in (20), *EA* introduced in Equation (21) is also a normalized measure and it's possible maximum value *EA* = max = 1 can be obtained if all AP markers for a sample *y*(*k*) appear for all detection sensitivities in . The possible minimum value of *EA* of a sample *y*(*k*) is zero if no markers appear [similarly as to *E* in Equation (20)].

For the example given in Table 1, the setup for calculation of *EA* is *n* = 15, *n* =7, )'\*+()(,(-(.(/(0(for which *EA* results as follows: *EA*(*k* = 311) = 0.105, *EA*(*k* = 503) = 0.629, *EA*(*k* = 678) = 0.095 (see corresponding slopes *H* in Figure 5). However, to better approximate the limit slope *H*, the elements of shall be selected closer to the approximate neighborhood of *max*, *i.e.*, around *max* 6 while = 2 is already too far from *max ,* as it is shown in Figure 4. Thus, *EA* can be estimated for that is reduced to the neighborhood of and both cases are thus *max* ! 6

compared by Figure 5 and 6. Even though that the result in Figure 6 is dependent on selection of (it clearly reveals perturbation of sample *y*(*k* = 500).

In Figure 6, we can see that other samples of data show also some smaller LE (AISLE in particular). Those data are not perturbed and they are fully consistent with governing law (10), yet they display nonzero LE. The reason for this is that the predictor (12–16) is a low-dimensional one and so it is not able to fully learn the governing law. However, it clearly detects and evaluates an inconsistent (here the perturbed) sample. In the next subsection, the further extension to the above introduced definition and calculation of LE that improves its evaluation accuracy is proposed.

#### *3.3. Orders of Learning Entropy (OLEs)*

When weights of a learning system are adapted by an incremental learning, the weights fluctuate in the weight state space with energy that the weight-update system has.

The weight update system receives its (learning) energy, from the measured data, *i.e.*, from the input vector **x**(*k*) and the target *y*(*k*) in case of supervised learning. The more inconsistent newly measured samples *y*(*k*) to the current knowledge of the learning system, *i.e.*, the higher LE of the samples, the more energy the weight increments **w** receives.

In other words, the weight update system resembles an engine, with its fuel being the input data. Then, the LE is the actual (time-varying) octane number of the fuel.

Weight increments **w** are the key variables for LE. During incremental learning, each weight behaves with energy of various orders that can be defined for the AP and thus for evaluation of the LE as follows: *wi*

$$\left| \frac{d^4 w\_{\vec{l}}(k)}{dt^4} \right| \approx \Delta^4 w\_{\vec{l}}(k) = \left| \Delta w\_{\vec{l}}(k) - 3 \cdot \Delta w\_{\vec{l}}(k-1) + 3 \cdot \Delta w\_{\vec{l}}(k-2) - \Delta w\_{\vec{l}}(k-3) \right| $$

1 *etc*.

From the above point of view, the originally introduced rule of AP (8) can be extended for the second order LE as follows:

$$\left| \text{if } \left( \left| \Delta \boldsymbol{\nu}\_{\hat{l}}(k) - \Delta \boldsymbol{\nu}\_{\hat{l}}(k-1) \right| > \alpha \cdot \left| \overline{\Delta^2 \boldsymbol{\nu}\_{\hat{l}}(k)} \right| \right) \Rightarrow \text{draw a marker for weight } \boldsymbol{\nu}\_{\hat{l}} \text{ at time } k \tag{22}$$

where, the recently usual second derivative (acceleration) of weights is as:

$$\left| \overline{\Delta^2 w\_{\bar{l}}(k)} \right| = \frac{1}{m} \left( \sum\_{j=k-m}^{k-1} \left| \Delta w\_{\bar{l}}(j) - \Delta w\_{\bar{l}}(j-1) \right| \right) \tag{23}$$

Similarly, for the 3rd order LE it would be as follows:

$$\left| \overline{\Delta^3 \boldsymbol{w}\_{\hat{I}}(k)} \right| = \frac{1}{m} \left( \sum\_{j=k-m}^{k-1} \left| \Delta^2 \boldsymbol{w}\_{\hat{I}}(j) - \Delta^2 \boldsymbol{w}\_{\hat{I}}(j-1) \right| \right) \tag{24}$$

and so on for higher orders.

To distinguish among the above modifications in which the LE is calculated via adaptation plot rule as shown in Equation (8) or Equation (22), the *Order of Learning Entropy* (OLE) is introduced in this section and its most common cases are summarized in Table 2, where the details of the formulas has been indicated above in this section.

**Table 2.** Orders of LE (OLE) and Corresponding Detection Rules, see Equations (8,9,22–24).

Figure 7 and Figure 8 show the results of AISLE for the above first five Orders of Learning Entropy estimates for data in time series (10), this time with two perturbations as follows:

$$\text{y}(k=47\text{S}) = \text{y}(k=47\text{S}) - 0\text{S} \quad \text{and} \quad \text{y}(k=500) = \text{y}(k=500) - 0.05\tag{25}$$

Figure 7 demonstrates the impact of various LE orders as they can significantly improve detection of inconsistent samples of data. Moreover, Figure 7 and Figure 8 also demonstrates that Zero-Order Learning Entropy, which deals just with weights themselves, does not have the cognitive capability to evaluate the learning effort of the predictor, *i.e.*, does not detect the unusual samples at *k* = 475,500, nor the AP (bottom Figure 8) reflects inconsistent data. 0 *EA*

This subsection introduced Orders of LE as they relate to the time derivatives orders of adaptable parameters *wi*.It was demonstrated that useful LE Orders are especially starting from 1st Order and higher (Figure 7) which is consistent to the results of experiments that were made through recent years with the AP [1,2,38–40].

**Figure 7.** AISLE of Orders of time series (10) with two perturbations of magnitude 0.05 at *k* = 475 and *k* = 500, - = [15, 14, …, 1], *m* = 60, *-*0 = 1, *-* = 0.1; the zero order AISLE shown in top axes is not capable to capture the inconsistent data at *k* = 475, 500 (see the in Figure 8), the higher orders can improve novelty detection significantly. 0 *EA*

**Figure 8.** (Top) calculated for *-* = {1.01[15, 14, … 1,]} that is closer to corresponding of zero order ISLE and (bottom) the AP for = 1.01; Zero-Order learning entropy does not capture the inconsistent data (signal and other setup as in Figure 7). 0 *EA max* 0 *E*

#### **4. Experimental Analysis**

#### *4.1. A Hyper-Chaotic Time Series*

Another theoretical example of time series where the AISLE can be clearly demonstrated is the time series obtained from hyper-chaotic coupled Chua's circuit [45] (with some more details and results on AP in [2]). The dimension of the used coupled Chua's circuit in continuous time domain is 6 and its embedding dimension shall be at least 2 × 6 + 1 = 13 according to the Taken's theorem. Let's choose the static QNU of lower dimension (embedding *n* = 6) as the learning model in sense of Equation (1) and in particularly according to Equations (12–14). Its proper pre-training by GD (16) for the first only 100 samples of the time series is shown in Figure 9; the sum of square errors (SSE) approaches 1E-5 after last epoch of pre-training.

**Figure 9.** Pre-training of low-dimensional QNU (12–16), on first 100 samples of hyper-chaotic time series in 10,000 epochs; *-* = 1; *n* = 6 => 28 weights; quality of pretraining (SSE) affects LE (Figure 10 *vs.* Figure 11).

Let us now introduce a slight perturbation in two samples as follows:

$$\text{y}(k=475) = \text{y}(k=475) - 0.2 \quad \text{and} \quad \text{y}(k=500) = \text{y}(k=500) - 0.02 \tag{26}$$

Then, the evaluation of AISLE for the less properly and more properly pre-trained learning model (Figure 9) is given in Figure 10 and Figure 11, respectively.

**Figure 10.** AISLE for the less properly pre-trained model; only 200 epochs of pre-training (Figure 9) naturally result in that the LE appears larger for more samples than just the perturbed ones at *k* = 457, 500 Equation (26), see also Figure 11.

**Figure 11.** AISLE for more properly pre-trained learning model than in Figure 10. (here 1E + 4 epochs, Figure 9); two slight perturbations at *k* = 457, 500 Equation (26) are followed by larger AISLE (esp. of 4th order).

However, the design of a proper learning model and its correct pre-training are crucial and non-trivial tasks for correct evaluation of LE and it may require an expert in adaptive (learning) systems or in neural networks. Nevertheless, from our experiments with AP and HONU [1,2,36–40] it appears that the very precise pre-training of the learning model is practically not always too crucial and that the structure of a learning model can be designed quite universally, e.g., with HONUs as they are nonlinear mapping predictors that are naturally linear in parameters. A practical rule of thumb for the above introduced HONU and GD can be to keep pre-training as long as the error criteria keep decreasing, *i.e.*, until the learning model learns what it can in respect to its quality of approximation *vs.* the complexity of the data. The effect of more proper pre-training can be demonstrated by comparison of Figure 10 with Figure 11, where we can see that more proper pre-training naturally filters out the inconsistent samples at *k* = 475, 500, from all the other samples that are naturally generated by the hyper-chaotic behavior.

This subsection demonstrated the calculation of the LE on a theoretical hyper-chaotic time series and it demonstrated the influence of the quality of pre-training on its evaluation. Two real-world data examples are given in the next subsection.

#### *4.2. Real Time Series*

Heart beat tachograms (R-R diagrams) and ECG signals are complex and non-stationary time series that are generated by a multidimensional and multilevel feedback control system (the cardiovascular system) with frequent external and internal perturbations of various kind. First, this subsection demonstrates potentials for real-world use of LE on real-time novelty detection of heart beat samples in R-R diagram retrieved from [46] using static QNU and GD learning via Equations (12–16).

**Figure 12.** AISLE for R-R diagram [46]; the learning model is static QNU, *n* = 5, pre-training samples = 200, epochs = 100, *-* = 0.001; the peaks of AISLE correspond or directly follow the inconsistent sample at *k* = 652.

The results and the pre-training setup is given in Figure 12, which shows that the new pattern of heart rate behavior starting at *k* = 652 has been detected. Second, demonstration of potentials of LE is shown on real-time sample-by-sample monitoring of ECG with algorithm modification for quasi-periodic signals. LE is shown for a real-time sample-by-sample monitoring of ECG time series with spontaneous onset of ventricular tachycardia (233 Hz, data courtesy of [47]) using Linear Neural Unit (LNU) with normalized GD, *i.e.*, adaptive linear filter (5–7), (13). The dimensionality of the used LNU is *n* = 80 Equation (13), so the calculation of AISLE for also a higher dimensional predictor is demonstrated below in Figures 13, 14 and A1.

**Figure 13.** Capturing the onset of spontaneous ventricular tachycardia in animal ECG by the AISLE of static Linear Neural Unit (LNU, *n* = 80, pre-training epochs = 500, pre-training samples *k* = 0.500, full data in Figure 16), data courtesy of [47].

Because ECG is a quasi-periodical signal and the used LNU is not able to fully learn the governing law that drives the ECG, the evaluation of LE can be modified to compare neural weight increments with respect to the prevailing periodicity of the signal. The floating average of absolute values of recent *m* neural increments of *i* th neural weight (originally given in (9)), can be modified as follows:

$$\left| \overline{\Delta \nu\_{\hat{I}}(k)} \right| = \frac{1}{m} \cdot \left( \sum\_{j=k-90}^{k-70} \left| \Delta \nu\_{\hat{I}}(j) \right| \right) \tag{27}$$

where, the actual time range of averaging **w** is located into the neighborhood of maximum autocorrelation of ECG signal (here for the lag of 80 samples).

**Figure 14.** The typical feature of LE: while prediction error and weight increments reach seemingly regular magnitudes or even smaller ones, the LE can be correctly high regardless the actual error of the learning model (see Figures 13 and 15).

#### **5. Discussion**

The straightforward difference of the introduced LE from statistical entropy approaches is that behavior of a system may be statistically complex, but if the behavior is deterministic (e.g., deterministic chaos) and newly measured samples are consistent with the governing law, then these data does not carry any novel information. Due to the incremental learning (adaptation) during detection, the LE approach is potentially suitable for real time non-stationary systems.

The weights in **w** and especially their higher-order time derivatives (estimated via weight increments **w**) correspond to the (learning) energy of an incrementally learning system, and it appears that higher orders of AISLE shall be generally considered. Naturally, higher orders of AISLE appear more reliable for novelty detection than the Zero-Order AISLE; however, higher orders AISLE appeared one sample delayed behind the first order AISLE (Figure 12).

The LE is a relative measure, that is related to the data to which it is applied as well as, it is related to the learning model and to its capability to extract the governing law from training data. Obtaining a useful LE can be a non-trivial task that requires a suitable (though not perfect) learning model (e.g., a low-dimensional model) that is capable to approximate a temporary governing law in data.

It is generally difficult to provide readers with the pragmatic rule of thumb that can be suggested for selection and pre-training of a model for LE, because the possible consequences of the insufficient training or the overtraining vary from case to case and may depend on used type of learning model and applied learning algorithm and on setup parameters, so the user's experience might be necessary. Nevertheless, linear filters or relatively simple polynomial predictors that are linear in parameters can be a good option to start with experiments on the LE.

Regarding supervised learning (*i.e.*, predictors), it was demonstrated already in [40] on chaotic logistic equation that the actual prediction error is not necessary correlated to the inconsistency of data that is adaptively monitored. Details of adaptive predictor output, prediction error, and the magnitude of weight increments during incremental learning of ECG signal for Figure 13 are given in detail in Figure 14. By comparison of the two figures, we see that the onset of ventricular tachycardia does not introduce larger prediction error (Figure 14); however, the LE of the starting arrhythmia is high (Figures 13 and 15) for the learning model used.

**Figure 15.** Profile of LE of 4th order and the *Learning Entropy of a Model* (LEM) evaluated at *k* = 700 for QNU and for the time series from Figure 12.

While this papers resolves the single-scale issue of sensitivity detection parameter via the multiscale approach, another point of discussion is selection of parameter *m* in formula (9) for a particular signal. A pragmatic rule for setting *m* according to the lowest frequency component of chaotic signals with quasi-periodic nature was indicated in subsection 3.1. Then an interval modification for choosing *m* for a quasi-periodical signal with significantly distinct intervals of behavior (e.g., the ECG signal) was demonstrated in Equation (27), where formula (9) was customized to calculate average increment magnitudes within the lag corresponding to the first maximum of autocorrelation function. For chaotic signals where the periodicity and maximum lag of autocorrelation function is not clear (e.g., discrete time chaos of the logistic equation or R-R

diagrams), the choice of parameter *m* can be a case dependant and future research is to be carried. Also, a possible multi–scale approach for *m* for LE can be considered for future research. Nevertheless, the author has observed that calculation of LE is practically much more robust to the selection of parameter *m* than to the sensitivity detection . 3herefore, the multi–scale approach for and not for *m* is primarily introduced for LE in this paper.

The LE is also a promising concept for development and research of new measures that would evaluate particular learning models and data. While this paper introduces individual sample focused LE, *i.e.*, ISLE and AISLE, there is strong potential for interval-based measures of LE. For example, by introducing the accumulated plot of LE, *i.e.*, the cumulative sum of AISLE, as follows:

$$AccSum\left(E\_A(k)\right) = \sum\_{i=1}^{k} \left(E\_A(i)\right) \tag{28}$$

one may obtain the LE profile of a particular adaptive model and of particular data. When evaluated for the whole time series, *AccSum*(*EA*) would summarize the signal and might be possibly used to distinguish between different models or adaptation techniques. Then the very last point of the profile, *i.e.*, *AccSum*(*EA*(*k* = max)), could be interpreted as the *Learning Entropy of a Model (LEM),* see Figure 15.

In other words, the LEM quantifies the familiarity of an adaptive model with data. Practically, the familiarity of a learning model with data corresponds to the generalization ability of the model. Adopting the fact that the best learning model shall have the best generalization ability (LEM = *min*) and the lowest prediction error, a general function of the convolution of Learning Entropy and the prediction error appears be a promising direction for continuing research that, however, exceeds the scope of this paper.

There are certainly many other issues that should be discussed regarding evaluation of LE and the further introduced concepts. The proper evaluation of LE depends on a number of factors where users experience with learning systems and signal processing is important. However, the objective of this paper is to introduce the LE as a new non-probabilistic concept of online novelty detection via evaluation of data sample inconsistency with contemporary governing law that is incrementally learned by a learning system.

#### **6. Conclusions**

This paper is the first work that introduces the concept of LE as a non-probabilistic online measure for relative quantification of novelty of individual data samples in time series. This normalized and multi-scale measure evaluates the inconsistency of individual samples of data as an unusual learning activity of an incrementally learning model over all adaptable parameters. It is a multi-scale measure because the learning activity is evaluated over the whole range of detection sensitivity parameter that is the key parameter for online visualization of unusual learning activity (in AP). Evaluation of unusual learning activity was proposed for estimation of various orders of time derivatives of weights that reflects the learning energy of an incrementally learning model, thus Orders of LE were introduced.

A particular technique for calculation of Approximated Individual Sample Learning Entropy was introduced. AISLE represents a loose analogy to approximate sample entropy in sense of using cumulative sums instead of approximation of the slope in a log-log coordinates. The whole explanation and the technique of calculation of AISLE is demonstrated on a straightforward example of supervised incremental learning, *i.e.*, on GD in this paper. As learning models, static linear and polynomial neural units (quadratic polynomials) were demonstrated in this paper as they are good to start with LE for their comprehensibility and in-parameter-linearity which is a good feature for GD learning. Examples of calculating the AISLE for theoretical chaotic time series as well as for two bio-signals were presented.

The major objective of this paper was a comprehensible introduction of the LE and its calculation rather then competition to conventional entropy approaches. The LE is introduced as a missing concept among probabilistic entropy approaches that usually do not consider a governing law in otherwise statistically complex data. In principle, the concept of LE is not only limited to supervised learning. There are strong potentials for LE for neural networks, signal processing, adaptive control, fault and concept drift detection, and big data applications. The evaluation of Learning Entropy will depend on many factors including users experience with adaptive systems and the detailed summary exceeds the introductory focus of a single paper.

#### **Acknowledgments**

The author would like to thank Madan M. Gupta from the University of Saskatchewan for introducing him to higher–order neural units and for continuous support and consultancy. The author would also like to thank Witold Kinser from the University of Manitoba for introducing him to multi–scale analysis approaches and for continuous support and consultancy, the colleagues from Tohoku University, namely from the Yoshizawa-Sugita Lab (formerly Yoshizawa-Homma Lab) and from the Department of Radiological Imaging and Informatics, for their continuous and vital cooperation and support. Special thanks are due to The Matsumae International Foundation (Tokyo, Japan) that funded the author's cooperation with colleagues at Tohoku University in Japan in 2009, and this cooperation is still vitally continuing. The author would like to thank anonymous reviewers for their insightful and constructive remarks. The author has been partly supported by grant SGS12/177/OHK2/3T/12 and partly by project TA03020312. The author would like to acknowledge the language Python [48] and the related open-source communities for developing these open-source software and its libraries.

#### **Conflicts of Interest**

The authors declare no conflict of interest.

## **Appendix**

**Figure A1.** Full data for Figure 13 (excluding first 500 of pre-training samples), arrhythmia spontaneously starts around *k* = 2200), data courtesy of [47].

## **References**

Reprinted from *Entropy*. Cite as: Zhao, J.; Zhao, M.; Yu, H. Effect of Prey Refuge on the Spatiotemporal Dynamics of a Modified Leslie-Gower Predator-Prey System with Holling Type III Schemes. *Entropy* **2013**, *15*, 2431-2447.

## *Article*


**2. Methodology**

The narrative review approach was used to draft this chapter. Information was retrieved from the internet-based search for original articles and reviews published between the years 2000 and 2021, using PubMed and Google scholar portals, and from textbooks. The keywords used for the literature search and inclusion criteria were 'control, diagnostics, exposure, soil-transmitted helminths, low-and middleincome countries, and personal protective equipment'. In-text references have been cited appropriately and full references are provided in the bibliography section.


*Article* **Femtosecond Laser-Micromachining of Glass Micro-Chip for High Order Harmonic Generation in Gases**

#### **Anna G. Ciriolo 1, Rebeca Martínez Vázquez 1,\*, Alice Roversi 2, Aldo Frezzotti 3, Caterina Vozzi 1, Roberto Osellame 1,2 and Salvatore Stagira 1,2**

Received: 8 January 2020; Accepted: 1 February 2020; Published: 4 February 2020

**Abstract:** We report on the application of femtosecond laser micromachining to the fabrication of complex glass microdevices, for high-order harmonic generation in gas. The three-dimensional capabilities and extreme flexibility of femtosecond laser micromachining allow us to achieve accurate control of gas density inside the micrometer interaction channel. This device gives a considerable increase in harmonics' generation efficiency if compared with traditional harmonic generation in gas jets. We propose different chip geometries that allow the control of the gas density and driving field intensity inside the interaction channel to achieve quasi phase-matching conditions in the harmonic generation process. We believe that these glass micro-devices will pave the way to future downscaling of high-order harmonic generation beamlines.

**Keywords:** femtosecond laser micromachining; high order harmonic generation; de laval gas micro nozzle; attosecond science

#### **1. Introduction**

Since its first observation more than twenty years ago, it has been known that an intense and ultrashort laser pulse, focused in a gaseous medium, drives the emission of a burst of coherent radiation, collinear to the driving beam, with a spectral content ranging from the vacuum ultraviolet to the soft X rays [1]. This process is called high-order harmonic generation (HHG) since the spectrum of the emitted radiation appears as a combination of numerous odd harmonics of the fundamental laser field. In the temporal domain, this emission is structured as a train of attosecond light pulses, and indeed, HHG is routinely exploited in the fields of extreme ultraviolet spectroscopy and Attosecond Science [2].

The manipulation of HHG beams is done in grazing incidence on bulky and expensive optics that require careful alignment and even active stabilization systems, and, as a consequence, these beam lines extend over several meters, occupying entire rooms. Up to now, HHG-based eXtreme Ultra Violet (XUV, from 100 nm down to 10 nm) and soft X (from 10 nm down to 1 nm) coherent light sources are confined within a few advanced laboratories [3] because of their technological complexity. In this framework, a substantial breakthrough in ultrafast technology can be achieved by the miniaturization of HHG beamlines, which could remarkably foster the application of HHG sources in numerous novel fields.

A strategy to obtain such compact sources is to generate high-order harmonics in gas-filled glass capillaries of a few centimeters length, which behave as hollow waveguides [4–6]. Hollow waveguides are already widely exploited in the field of ultrafast laser sources mainly for the compression of intense pulses [6], but also in the field of HHG, they lead to impressive results allowing hundreds times enhancement of the intensity of the harmonics to be reached [4] and spectral components up to the keV photon energy [5] in tabletop systems.

A route for achieving even higher miniaturization is based on femtosecond laser micromachining, which is a powerful fabrication technique that has already demonstrated its high potential in the fabrication of miniaturized lab-on-chip devices [7]. Ultrashort laser pulses are focused inside a transparent sample and, due to nonlinear absorption processes, such as multiphoton and avalanche ionization, they produce a permanent modification of the material only at a small region around the focus where the highest intensity is reached [8,9].

The photogenerated hot-electron plasma during fs-laser irradiation induces high temperatures and pressures that give rise to different phenomena, such as densification, direct photo structural modifications, and color-center formation. Under suitable conditions, the combination of such effects may lead to a local increase in the etching speed of the material over a micrometer-sized volume [10]. By moving the laser focus inside the substrate, one can use the laser beam to define three-dimensional regions of increased etching speed. A powerful method for the fabrication of microfluidic networks, in a 3D geometry, directly buried in the glass substrate is femtosecond laser irradiation of fused silica glasses followed by chemical etching (FLICE). Until now, the devices fabricated by FLICE have been extensively used for the manipulation of liquids, but they are perfectly suitable for the manipulation of gases as well.

In this work, we will present gas-filled microfluidic devices, fabricated through the FLICE technique, demonstrating efficient emission of extreme ultraviolet radiation produced by HHG. We exploit the flexibility, accuracy, and 3D capabilities of the FLICE technique to create glass micro-devices for manipulating gas fluxes and for guiding laser beams through microchannels.

#### **2. Materials and Methods**

For the femtosecond laser micromachining of the glass devices, the second harmonic (515 nm) of a femtosecond laser beam (Satsuma, Amplitude Systemes S.A., Pessac, France) was focused inside a fused silica sample using a 63 × (0.65 NA) microscope objective (LD-plan Neofluar, Zeiss, Oberkochen, Germany). As represented in the scheme of Figure 1, the glass sample was mounted onto a high-resolution 3D movement system (Aerotech, ANT, Hampshire, UK) and moved with respect to the laser beam following the desired trajectory. The laser repetition rate was set at 1 MHz, with a pulse duration of 230 fs and pulse energy between 200 nJ and 300 nJ depending on the dimensions of the irradiated geometry. After the laser irradiation, the sample was immersed in an ultrasonic bath of a 20% HF aqueous solution at 35 ◦C.

HHG experiments were performed under vacuum conditions in a beamline composed of an interaction chamber (working pressure 10−5 mbar) and a grazing incidence XUV spectrometer (working pressure 10−6–10−7 mbar) (see Figure 2). The beam of a Ti:Sapphire laser (25 fs, 1 kHz, 10 mJ) was coupled into the microchannel, which was located inside the interaction chamber. The driving laser beam was focused at the entrance of the hollow waveguide by a 20-cm focal lens. The spatial coupling of the beam with the microchannel was achieved with the help of a high-precision motorized alignment stage. The gas pressure was accurately controlled by a system composed of a needle valve and a pressure gauge placed in the gas line after the needle valve. The high-order harmonics radiation was acquired by a spectrometer composed of grazing-incidence optics. A first toroidal mirror was used for generating an intermediate focus for spectroscopy purposes. The second toroidal mirror focused the HHG beam on a dispersion grating. The dispersed HHG signal was acquired by means of a vacuum compatible detector (Photek, VID140, East Sussex, UK) incorporating one micro-channel plate (MCP) and a P20 phosphor screen. The image displayed on the phosphor screen was acquired by a CCD camera (Apogee Ascent A2150, Andor, Belfast, UK). The same chip was used for HHG for several weeks, with no evident degradation.

**Figure 1.** Femtosecond laser micromachining setup. The laser beam passed through an attenuation module, made of a λ/2 waveplate and a polarizer, and then through a telescope and a non-linear crystal (BBO) for second harmonic (515 nm) generation; the second harmonic beam was focused onto the glass sample. The sample was mounted on to a 3-dimension translation stage.

**Figure 2.** High-order harmonics generation and acquisition setup. The chip was mounted onto a home-built 5-axis translation stage to allow a precise coupling of the driving laser.

We used Comsol Multiphysics software to model the gas speed, temperature, and density (in the steady-state) along the hollow waveguide. We applied the high Mach number flow model coupled with the so-called k- turbulence model in order to properly describe the turbulent flow behavior of the gas inside the device. For the discretization of the geometry, we used a triangular mesh, optimized by the software for the physics of the problem.

#### **3. Results and Discussion**

The basic design of the glass chip for HHG is shown in Figure 3a; it consisted of a cylindrical top central reservoir (gas pipe) from which several small microchannels (gas distribution channels) departed and reached the cylindrical horizontal main channel (hollow waveguide). The gas was inserted through the top reservoir and, thanks to the homogeneously distributed small microchannels, it uniformly filled the hollow waveguide. In fact, the driving laser that propagated inside the waveguide encountered a uniform gas density.

The final device was made in a fused silica plate (dimensions 6 × 10 × 1 mm3) and contained two parallel microchannels; the upper one served as an auxiliary channel for beam alignment, whereas the lower one was devoted to HHG. The reservoir for gas input had a radius of 1.6 mm, the small channels for gas distribution had a variable diameter from ~100 μm (at the reservoir base) to ~30 μm at the main channel surface. The main channel, which acted as a hollow waveguide, had a 120 μm diameter. Figure 3b,c show, respectively, the microscope image of the device after laser irradiation and a picture after chemical etching. The in-volume irradiated paths, which look darker in Figure 3b, led to embedded and interconnected empty channels after etching. The irradiated central channel presented a sinusoidal radius to compensate for inhomogeneous exposition to acid during the etching process, and to obtain a central channel with a constant radius after the etching step (see Figure 3c). It is important to point out that the FLICE technique allows combining structures with millimeter size

(gas reservoir with 3 mm diameter) and small structures with micrometer size (microchannels) in the same device with just one fabrication process.

**Figure 3.** (**a**) Scheme of the high-order harmonic generation (HHG) glass chip, (**b**) microscope image of the irradiated sample, the different components are evidenced: 1-hollow waveguide, 2-gas redistribution channels, 3-gas pipe reservoir, 4-lateral cutting walls, (**c**) picture of the device after the HF etching.

To investigate the gas distribution in the microchip, we modeled the gas flow through the device in the final steady-state using Comsol Multiphysics. The mesh cell dimensions varied between a maximum value of 140 μm and a minimum value of 15 μm. The results, depending on the gas backing pressure, are shown in Figure 4. In particular, the gas temperature, density, and Mach number along the capillary axis are reported. With the exception of small peaks next to the outlets of the distribution channels, the gas flow exhibited the typical features of gas flows within micro-channels with a high length to diameter ratio. The gas flow was nearly isothermal and subsonic in most of the flow region. The density profile smoothly decreased from the center towards the waveguide outlets at the lowest backing pressure value, but it showed stronger spatial modulation at higher pressures. Abrupt changes in the behavior of all flow properties were observed in correspondence to the exits towards the vacuum chamber, where the gas underwent strong expansion.

The gas density inside the channel scaled linearly with the backing pressure. Thus, by properly monitoring the backing pressure, it is possible to achieve accurate control of the density of the generating medium inside the microchannel and, as a consequence, of the generated harmonics intensity.

For the HHG experiments, the gas pipe was connected to the glass chip, which was positioned into the interaction chamber, and a fraction of the laser beam (400 μJ pulse energy) was coupled inside the hollow waveguide. Figure 5 shows a comparison among different harmonic spectra generated inside the microchannel filled with neon gas. A decreasing yield was observed when the gas backing pressure was increased from 50 to 210 mbar. The reduction is more evident for low-order harmonics, as indicated by the reshaping of the harmonic spectrum. This pressure dependence is due to both phase-matching effects and absorption from the gas: as the pressure increases, the XUV radiation is strongly absorbed; moreover, the phase mismatch between the fundamental and the harmonic field worsens and leads to a dramatic reduction in the yield mainly in the low-energy part of the spectrum [11,12].

**Figure 4.** Numerical simulation of the Ne gas evolution inside the micromachined device reported in Figure 2, showing (**a**) gas temperature, (**b**) density, and (**c**) Mach number as a function of the gas backing pressure. (**d**) Gas volume geometry and mesh. The main channel and the small lateral features reproduce the internal chip structure. The two cylindrical structures placed at the extremities were used as output volume for simulating gas expansion in the interaction chamber. The simulations were performed using COMSOL.

**Figure 5.** HHG spectra generated inside the chip using Ne gas with a backing pressure of 50 (blue), 110 (red), and 210 mbar (yellow); this latter curve is magnified 10 times.

We compared the optimal HHG yield inside the gas-filled channel with that achieved in the most commonly used interaction geometry based on a gas jet. Figure 6 shows single-shot harmonics spectra generated inside the channel filled with neon at a 50-mbar backing pressure (a) or helium at a 300-mbar backing pressure (b). The results are compared to HHG in a gas jet produced by a 1-mm diameter valve (Parker) that is routinely used for gas-pulse generation in HHG, and that can be operated both in a steady and a pulsed mode. In the steady regime (red spectra), the valve was feed with gas pressures comparable to that used in the channel; in particular, 140 mbar of Neon and 300 mbar of Helium were used. In the pulsed regime (yellow spectra), higher backing pressures, on the multi-bar scale, were used for reproducing the standard working conditions of HHG experiments. Both with Helium and Neon, an extended cutoff, up to 7 nm (160 eV) was obtained inside the microchannel, whereas a cutoff of about 11 nm (110 eV) was observed in the jets. Moreover, an estimation of the harmonic generation

yield in the spectral region detected by the spectrometer can be performed by the integration of the HHG spectra. As a result, a higher generation yield was achieved in the microchannel for both gases, which is up to 20 times that achieved with the pulsed jet and about 104 times that obtained with the steady jet. The improved performances achieved in the microchannel are related both to the extension of the interaction region and the different interaction geometry with respect to the gas jets.

**Figure 6.** Comparison between HHG spectra produced inside the microchannel (blue), in a continuous gas jet (red), and in a pulsed gas jet (yellow) for Ne (**a**) and He (**b**) gases.

Although there is an overall improvement in the generation yield with respect to traditional HHG in gas, we are confident that a further improvement in the harmonic generation can be obtained by overcoming the phase-matching limitations. In fact, as in many nonlinear processes, HHG efficiency depends both on the single-atom response [13] and on the macroscopic response during propagation in the medium [14], so for generating a bright output beam the emission of a large number of atoms over an extended region must add in phase. The phase relationship between the fundamental and the harmonics field depends on several factors that are mainly related to the generating medium and generation geometry. In particular, the gas neutral and free-electron plasma optical dispersion is wavelength-dependent, so that the fundamental field and the newly generated harmonics can accumulate a phase-mismatch during the propagation. The geometrical phase effects depend instead on the space-structure of the fundamental beam, either tight-focusing or waveguide-assisted confinement.

Practical approaches exploiting quasi phase-matching (QPM) are challenging [5]. In this sense, we propose prototyping different microchips for HHG endowed with multiple gas micro-jets, with the attempt of exploiting QPM. We follow two main strategies which are known to be effective in the improvement in the HHG yield, i.e., the modulation of the driving field intensity or the modulation of the gas density [15,16].

Modulation of the driving field intensity is achievable through modulation of the hollow waveguide diameter. In fact, the diameter of the propagating mode will depend on the waveguide diameter and as a consequence, also the peak intensity [17]. In Figure 7, panels (a) and (b) show the scheme and microscope image of the microchip with the modulated radius. The gas delivery inside the channel is similar to the one in the previous device; it is inserted through the top reservoir, flows through a network of microchannels, and reaches the hollow waveguides at some specific positions. In addition, in this device, each delivering microchannel faces a small exit channel in order to create a modulation of gas density. The strong capabilities of the FLICE technique are evidenced in this glass, which has a 20% radius modulation (110–135 μm) and 30 small gas inlets and outlets with micrometer size.

**Figure 7.** (**a**) Cartoon of the quasi phase-matching device with 32 gas inlets and modulated diameter. (**b**) Microscope image of the device in (**a**). (**c**) Cartoon of the device with four de Laval micro-nozzles for gas density periodic modulation. (**d**) Numerical simulation (Comsol) of gas density inside the chip shown in (**c**).

It is possible to achieve an even higher modulation of the gas density inside the hollow waveguide by injecting the gas through a series of de Laval micro-nozzles whose supersonic flow condition, at the end of the divergent section, helps produce a more focused gas jet. The scheme of the chip is shown in Figure 7c. In this device, there is no gas reservoir, and the convergent section of each nozzle is almost directly facing the gas inlet. Different nozzle geometries were numerically investigated, using Comsol software, in order to enhance/control the density modulation along the axis of the hollow waveguide.

It should be noticed that the nozzle flow behavior in the considered conditions deviates from the classical one, as described by the inviscid theory. Although the inlet pressure is high enough to avoid rarefaction effects, viscous forces are very pronounced (the nozzle throat Reynold's number is about 300) and delay the supersonic transition within the divergent section whose effective area is strongly reduced by a thick boundary layer [18]. So far, the best nozzle configuration consists of a simple double cone geometry with an inlet diameter of 220 μm, a 60 μm throat diameter, and a 90 μm exit diameter, whose size is the largest as possible which preserves the guiding characteristics of the hollow waveguide. The overall nozzle length is 131 μm. Figure 7d shows the simulated gas density profile along the waveguide axis. In order to perform an accurate discretization of the problem, we used different meshes for the nozzle region, central waveguide, and gas exits. In particular, for the micro-nozzles, we used mesh cell dimensions that varied between a maximum element dimension of 5 μm and a minimum element dimension of 0.08 μm (near the walls). A strong density increase in correspondence with the nozzle position is clearly visible (~2 × 1024 molecules/m3). This density modulation is expected to be high enough to obtain QPM, given that the nozzle periodicity is the correct one.

#### **4. Conclusions**

In the present work, we demonstrated the application of femtosecond laser micromachined glass microchips to the generation of high-order harmonics in gas. The working principle relays in a main cylindrical microchannel that acts as a hollow waveguide where the driving laser propagates. This microchannel is filled with gas through a three-dimensional network of gas distribution channels all embedded in the glass device. The gas flow was numerically studied and characterized. As a

result of the laser-gas interaction, an increase in the HHG signal was observed, which surpasses the performances of the standard generation configuration based on pulsed gas-jets both in terms of efficiency and spectral extension.

By exploiting the extreme flexibility of the FLICE technique, different generation prototypes, including gas-micro-jets and periodically structured capillaries, were realized, aiming at further improving the harmonics generation yield in a quasi phase-matching regime.

We foresee that the FLICE technique could be exploited to downscale an entire HHG-based beamline to a glass chip, integrating several additional functionalities, such as the separation of the IR driving beam from the HHG radiation by waveguide effects, and the inclusion of an interaction module for HHG spectroscopy in liquid and gaseous samples.

**Author Contributions:** Conceptualization, C.V., R.O., and S.S.; Formal analysis, A.G.C., R.M.V., and A.F.; Funding acquisition, C.V. and S.S.; Investigation, A.G.C., R.M.V., and A.R.; Methodology, A.G.C., R.M.V., and A.F.; Validation, A.G.C., R.M.V., and S.S.; Writing—original draft, A.G.C. and R.M.V.; Writing—review and editing, A.G.C., R.M.V., A.F., C.V., R.O., and S.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the European Union's Horizon 2020 research and innovation program under the Marie Skłodowska-Curie projects ASPIRE (Grant No. 674960), by the European Research Council Starting Research Grant UDYNI (Grant No. 307964), by the Italian Ministry of Research and Education (ELI project—ESFRI Roadmap).

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).


*2.3. Experimental Procedures*

#### 2.3.1. Chemical Precipitation

Chemical precipitation procedure was used to remove potassium in this experiment. In the process, the potassium which was in simulated brine would be selectively precipitated as potassium perchlorate by adding perchloric acid. Removal of potassium could reduce the co-extracted effect in the solvent extraction process. The parameter of chemical precipitation such as effect of pH value was investigated. Precipitation rate was calculated according to Equation (1) and the flow diagram is shown in Figure 1.

$$\mathbf{P}(\%) = \frac{[\mathbf{M}]\_0 - [\mathbf{M}]}{[\mathbf{M}]\_0} \times 100\tag{1}$$

**Figure 1.** Flow diagram of chemical precipitation.

P is Precipitation rate, [M]0 is metal concentration of leach liquor, [M] is metal concentration of leach liquor after precipitation.