In DB adoption, there are also strong backwards-compatibility constraints. For instance, in EMV, the public-key infrastructure or the restrictions of keeping as close as possible to old-generation EMV cards/readers are such that a DB protocol, following the designs we saw in Sect. 7.3, is simply un-adoptable out of the box.

# *7.6.3 Specialist Implementations and Slow Adoption*

On the one hand, PKES with relay-protection are finally becoming commercial arguably due to relay attacks being exploited by fraudsters in the automotive sector. On the other hand, in DB-enhanced EMV contactless protocols (*à la* PaySafe), a dishonest party already has a tangible incentive to mount a distance-fraud attack;—a purchase receipt carries an intrinsic proof that the card was in the range of the reader. Yet, EMV with relay-protection is not widely deployed and, indeed, the markets do not appear to call for protocols to be enhanced with full DB-protection yet.

Should such DB frauds appear in practice, would we then see fully-fledged DB solutions being implemented for commercial purposes? Or, will the 5th generation of mobile networks (5G) and its increased spectrum and higher bands lead to the true rise of DB technology in the ubiquitous systems of the 2020s, and raise new DB research questions?

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Part IV Hardware Implementation and Systems**

# **Chapter 8 It Started with Templates: The Future of Profiling in Side-Channel Analysis**

**Lejla Batina, Milena Djukanovic, Annelie Heuser, and Stjepan Picek**

**Abstract** Side-channel attacks (SCAs) are powerful attacks based on the information obtained from the implementation of cryptographic devices. Profiling side-channel attacks has received a lot of attention in recent years due to the fact that this type of attack defines the worst-case security assumptions. The SCA community realized that the same approach is actually used in other domains in the form of supervised machine learning. Consequently, some researchers started experimenting with different machine learning techniques and evaluating their effectiveness in the SCA context. More recently, we are witnessing an increase in the use of deep learning techniques in the SCA community with strong first results in side-channel analyses, even in the presence of countermeasures. In this chapter, we consider the evolution of profiling attacks, and subsequently we discuss the impacts they have made in the data preprocessing, feature engineering, and classification phases. We also speculate on the future directions and the best-case consequences for the security of small devices.

# **8.1 Introduction**

In 1996, Kocher demonstrated the possibility to recover secret data by introducing a method for exploiting leakages from the device under attack [338]. In other words, implementations of cryptographic algorithms leak relevant information

L. Batina Radboud University, Nijmegen, The Netherlands

S. Picek (-) Delft University of Technology, Delft, The Netherlands about the data processed through physical side-channels such as timing [338], power consumption [339], EM emanation [493], and sound [225].

Side-channel attacks (SCAs) exploit weaknesses in the physical implementation of cryptographic algorithms rather than the algorithms themselves [389]. Those weaknesses stem from the physics of the underlying computing elements, i.e., CMOS cells, which makes it hard to eliminate such threats.

Numerous evaluation techniques, which generally involve some form of digital signal processing and statistical computations, have been proposed in the literature. Some of the most important methods include Simple Power Analysis (SPA) [339], Differential Power Analysis (DPA), and Template Attacks (TA) [135].

The SPA technique implies that the attacker aims at reconstructing the secret key using just a single trace of side-channel information, and it often exploits the difference in basic public-key operations such as double-and-add, or add-andmultiply [339]. Still, SPA is not possible if the observed signal-to-noise ratio (SNR) is not high enough. Consequently, most of the time developed countermeasures make SPA futile.

DPA techniques are based on the evaluation of many traces with varying input data for the targeted algorithm. After that step, a brute-force attack, testing sub-key hypotheses, is performed on a part of the algorithm (so-called "divide and conquer"). In the DPA approach, a large number of samples are used in order to reduce noise by averaging, and a single-bit power model is commonly adopted [339]. On the other hand, Correlation Power Analysis (CPA) represents a multi-bit power model in order to reduce the influence of noise on the possibility to execute a successful attack [115]. The main difference between these two techniques is that DPA is based on computing the difference between two trace sets, while CPA uses the correlation coefficient in order to calculate the dependency test. We often also say that the two use different side-channel distinguishers. Side-channel attacks using the above three techniques have been reported on a wide variety of cryptographic implementations, see, e.g., [154, 402, 410, 412, 434, 500] including some real-world applications [196].

In contrast to DPA, TA requires a profiling stage, i.e., a step during which the cryptographic hardware is under full control of the adversary to estimate the probability distribution of the leaked information and make better use of all the information present in each sample [135]. In this way, TA can provide a promising model of the real device, instead of using some a priori model.

TA is the best (optimal) technique from an information-theoretic point of view if the attacker has an unbounded number of traces and the noise follows the Gaussian distribution [277, 367]. After the template attack, the stochastic attack emerged using linear regression in the profiling phase [515]. In the years that followed, researchers recognized certain shortcomings of template attacks and tried to modify them in order to deal better with the complexity and portability issues. An example of such an approach is the pooled template attack where only one pooled covariance matrix is used in order to cope with statistical difficulties [142]. Alongside such techniques, the SCA community realized that a similar approach to profiling is used in other domains in the form of supervised machine learning. Consequently, some researchers started experimenting with different machine learning (ML) techniques and evaluating their effectiveness in the SCA context. Although mainly considering distinct scenarios and various ML techniques, all those papers tend to establish different use cases where ML techniques can outperform the template attack and establish themselves as the best choice for profiled SCA. More recently, we are witnessing the relevance of deep learning (DL) techniques in the SCA community with strong results in side-channel analyses, even in the presence of countermeasures.

# **8.2 Profiled Side-Channel Attacks**

Profiled side-channel attacks estimate the worst-case security risk by considering the most powerful side-channel attacker. In particular, one assumes that an attacker can possess an additional device of which he or she has nearly full control. From this device, he obtains leakage measurements and is able to control the used secret key or at least knows which one is used. Knowing the secret key enables him to calculate intermediate processed values that involve the secret key for which he is estimating models. These models can then be used in the attacking phase to predict which intermediate values are processed and therefore carry information about the secret key. Commonly used models are the identity value or Hamming weight/distance.

**Uniformly Distributed Classes** Targeting intermediate variables, e.g., when loaded or manipulated on the device and resulting mostly in 2*n* uniformly distributed classes where *n* is the number of bits of the intermediate variable.

**Binomial Distributed Classes** The Hamming Weight (HW) or the Hamming Distance (HD) of a uniformly distributed intermediate variable results in *n* + 1 binomially distributed classes.

# *8.2.1 Definition of Profiling Attacks*

In this section, we consider side-channel attacks on block ciphers for which a divide and conquer approach can be utilized. Note that, as there exist operations within the block cipher which manipulate each block/chunk (e.g., bytes in Advanced Encryption Standard (AES)) independently and most importantly involving only one block/chunk of the secret key, an attacker only needs to make hypotheses about the secret key block/chunk instead of the complete secret key at once.

More formally, let *k*∗ denote (a chunk of) the fixed secret cryptographic key that is stored on the device and let *t* denote (a chunk of) the plaintext or ciphertext of the cryptographic algorithm. The mapping *y* maps the plaintext or the ciphertext *t* ∈ *T* and the key *k*∗ ∈ *K* to an intermediate value that is assumed to relate to the deterministic part of the measured leakage *x*. For example,

$$\mathbf{y}(t,k) = \mathsf{Succ}[T \oplus k],\tag{8.1}$$

where Sbox[·] is a substitution operation. The measured leakage *x* can then be written as

$$\mathbf{x} = \varphi(\mathbf{y}(t, k^\*)) + r,\tag{8.2}$$

where *r* denotes independent additive noise and *ϕ* is a device-specific (unknown) deterministic function mapping the intermediate variable to the leakage space. In the rest of this chapter, we are particularly interested in multivariate leakage **x** = *x*1*,...,xD*, where *D* is the number of time samples, i.e., features (also called attributes or points of interest).

Now, it is considered that the attacker has the following information at his disposal to conduct the attack:

In the attacking phase the goal is to make predictions about *y(ta*1 *, k*∗ *a ), . . . , y(taN , k*∗ *a )*, where *k*∗ *a* is the secret key on the attacking device. Note, even before running the attack, there are several steps one could do in order to make the attack more powerful. These phases are depicted in Fig. 8.1.

# *8.2.2 Data Preprocessing*

In the data preprocessing phase, the aim is to prepare the data in a way to increase the performance of side-channel analysis. There are several papers considering various data augmentation techniques in order to artificially generate measurements so as to increase the size of the profiling dataset. Cagli et al. propose two data

**Fig. 8.1** Depiction of an end-to-end profiling attack

augmentation techniques they call Shifting and Add-Remove [122]. They use convolutional neural networks (CNN) and find data augmentation to significantly improve the performance of CNN. Pu et al. use a data augmentation technique where they randomly shift each measurement in order to increase the number of measurements available in the profiling phase [489]. They report that even such simple augmentation can effectively improve the performance of profiling SCA. Picek et al. experiment with several data augmentation and class balancing techniques in order to decrease the influence of highly unbalanced datasets that occur when considering HW/HD models [478]. They show that by using a wellknown machine learning technique called SMOTE, it is possible to reduce the number of measurements needed for a successful attack by up to 10 times. Kim et al. investigate how the addition of artificial noise to the input signal can be beneficial to the performance of the neural network [329].

# *8.2.3 Feature Engineering*

When discussing the feature engineering tasks, we can recognize a few directions that researchers follow in the context of SCA:

When discussing feature engineering, it is important to mention the curse of dimensionality. This describes the effects of an exponential increase in volume associated with the increase in the dimensions [71]. As a consequence, as the dimensionality of the problem increases, the classifier's performance increases until the optimal feature subset is reached. Further increasing the dimensionality without increasing the number of training samples results in a decrease in the classifier performance.

In the SCA community, there are several standard techniques to conduct feature selection:

• Pearson Correlation Coefficient. The Pearson correlation coefficient measures the linear dependence between two variables, *x* and *y*, in the range [−1*,* 1], where 1 is a total positive linear correlation, 0 is no linear correlation, and −1 means a total negative linear correlation. The Pearson correlation for a sample of the entire population is defined by [301]:

$$Pearson(\mathbf{x}, \mathbf{y}) = \frac{\sum\_{l=1}^{N} ((\mathbf{x}\_{l} - \bar{\mathbf{x}})(\mathbf{y}\_{l} - \bar{\mathbf{y}}))}{\sqrt{\sum\_{l=1}^{N} (\mathbf{x}\_{l} - \bar{\mathbf{x}})^{2}} \sqrt{\sum\_{l=1}^{N} (\mathbf{y}\_{l} - \bar{\mathbf{y}})^{2}}},\tag{8.3}$$

where *x*¯ and *y*¯ are the empirical means of *x* and *y*, respectively.

• SOSD. In [230], the authors proposed as a selection method the sum of squared differences, simply as:

$$SOSD(\mathbf{x}, \mathbf{y}) = \sum\_{i, j > l} (\bar{\mathbf{x}}\_{\mathbb{Y}i} - \bar{\mathbf{x}}\_{\mathbb{Y}j})^2,\tag{8.4}$$

where *x*¯*yi* is the mean of the traces where the model equals *yi*. Because of the square term, SOSD is always positive. Another advantage of using the square is that it enlarges big differences.

• SOST. SOST is the normalized version of SOSD [230] and is thus equivalent by the pairwise student T-test:

$$SOST(\mathbf{x}, \mathbf{y}) = \sum\_{i, j > i} \left( (\bar{\mathbf{x}}\_{\mathbf{y}\_i} - \bar{\mathbf{x}}\_{\mathbf{y}\_j}) / \sqrt{\frac{\sigma\_{\mathbf{y}\_i}^2}{n\_{\mathbf{y}\_i}} + \frac{\sigma\_{\mathbf{y}\_j}^2}{n\_{\mathbf{y}\_j}}} \right)^2 \tag{8.5}$$

with *nyi* and *nyj* being the number of traces where the model equals to *yi* and *yj* , respectively.

There are several more relevant works in the domain of feature selection and SCA. The work of Lerman et al. [367] compared template attacks and machine learning on dimensionality reduction. They concluded that template attacks are the method of choice as long as a limited number of features can be identified in leakage traces containing most of the relevant information. Zheng et al. looked into feature selection techniques but they did not consider machine learning options [600]. Picek et al. conducted a detailed analysis of various feature selection techniques where some are also based on machine learning (so-called wrapper and hybrid methods) [477]. They concluded that commonly used feature selection techniques in SCA are rarely the best ones and they mentioned L1 regularization as a powerful feature selector in many scenarios.

# **8.3 Template Attacks**

In this section, we start by explaining the details of template attacks, and after that we give details about two techniques that emerged from template attacks—pooled template attacks and stochastic attacks.

# *8.3.1 Context of Template Attack*

In the pioneering template attacks article of Chari, Rao, and Rohatgi, it is shown that template attacks apply advanced statistical methods and can break implementations secure against other forms of side-channel attacks [135].

In some works template attacks are built to classify the state of a byte, e.g., a key byte in RC4 [135, 498]. The weakness of these papers is the need to create 256 templates for each byte. Additionally, the template building process can only be guided by partial attack results. In [498], the authors reduce the number of points of a trace by using an efficient algorithm instead of the standard principal component analysis method, which increases the speed of selecting points of interest. Also, by introducing a preprocessing phase with the use of discrete Fourier transformation on traces, the authors improve the template attack results in practice.

Agrawal et al. develop two new attack techniques that extend the work of the previously mentioned research results [11]. The first is a single-bit template attack technique that creates templates from peaks observed in a DPA attack resulting with a high probability value of a single DPA-targeted bit. Their second, templateenhanced DPA attack technique can be used to attack DPA protected cards and consists of two steps: a profiling phase and a hypothesis testing phase. In the first, profiling phase, the attacker, who is in possession of a smart card with a biased RNG, builds templates, and in the hypothesis testing phase the attacker uses previously built templates to mount a DPA-like attack on a target card which is identical to the test smart card, but has a perfect RNG. The authors illustrate these two attack techniques considering unprotected implementations of DES and AES on smart cards.

Archambeau et al. take template attacks techniques a step further by transforming leakage traces in order to identify important features (i.e., transformed time instants) and their number automatically. Actually, they use the optimal linear combination of the relevant time samples and execute template attacks in the principal subspace of the mean traces creating a new approach, the principal subspace-based template attack (PSTA) [25]. The authors validate this approach by attacking the RC4 stream cipher implementation and an FPGA implementation of AES.

In the literature, the main focus is on template attacks aiming at recovering the secret key of a cryptographic core from measurements of its dynamic power consumption. But with scaling of technology, static power consumption grows faster and creates new issues in the security of smart card hardware. Therefore, Bellizia et al. proposed Template Attack Exploiting Static Power (TAESP) in order to extract information from a hardware implementation of a cryptographic algorithm using temperature-dependence of static currents as a source of information leakage [70].

# *8.3.2 Standard Template Attack*

The template attack is based on the Bayesian rule and works under the simplifying assumption that the measurements are mutually independent among the *D* features given the target class. More precisely, given the vector of *N* observed attribute values *x*, the posterior probability for each class value *y* is computed as:

$$p(Y=\mathbf{y}|\mathbf{X}=\mathbf{x}) = \frac{p(Y=\mathbf{y})p(\mathbf{X}=\mathbf{x}|Y=\mathbf{y})}{p(\mathbf{X}=\mathbf{x})},\tag{8.6}$$

where **X** = **x** represents the event that **X**1 = **x**1 ∧ **X**2 = **x**2 ∧ *...* ∧ **X***N* = **x***N* .

Note that the class variable *Y* and the measurement *X* are not of the same type: *Y* is discrete while *X* is continuous. So, the discrete probability *p(Y* = *y)* is equal to its sample frequency where *p(***X** = **x**|*Y* = *y)* displays a density function. In most state-of-the-art models *p(***X** = **x**|*Y* = *y)* is assumed to be based on a (multivariate) normal distribution and is thus parameterized by its mean and its covariance matrix:

$$p(\mathbf{X} = \mathbf{x} | Y = \mathbf{y}) = \frac{1}{\sqrt{(2\pi)^D |\Sigma\_\mathbf{y}|}} e^{-\frac{1}{2}(\mathbf{x} - \bar{\mathbf{x}}\_\mathbf{y})^T \Sigma\_\mathbf{y}^{-1} (\mathbf{x} - \bar{\mathbf{x}}\_\mathbf{y})}.\tag{8.7}$$

# *8.3.3 Pooled Template Attack*

In practice, the estimation of the covariance matrices for each class value *y* can be ill-posed mainly due to an insufficient number of traces for each class. The authors of [142] propose to use only one pooled covariance matrix to cope with statistical difficulties and thus a lower efficiency. Accordingly, Eq. (8.7) changes to

$$p(\mathbf{X} = \mathbf{x} | Y = \mathbf{y}) = \frac{1}{\sqrt{(2\pi)^D |\Sigma|}} e^{-\frac{1}{2}(\mathbf{x} - \bar{\mathbf{x}}\_Y)^T \Sigma^{-1} (\mathbf{x} - \bar{\mathbf{x}}\_Y)}. \tag{8.8}$$

The works in, e.g., [142, 476, 477, 481] showed that indeed the pooled version is more efficient, in particular for a smaller number of traces in the profiling phase.

# *8.3.4 Stochastic Attack*

Compared to TA, the stochastic attack (SA) utilizes linear regression instead of probability density estimation [515]. One critical aspect of SA is the choice of regressors (aka base functions), as for example shown in [275]. A natural choice in the context of side-channel analysis is the bitwise selection of the intermediate variable, i.e., let [·]*b* define the function selecting the *b*th bit and using the same intermediate variable as in Sect. 8.2.1 then

$$(\mathsf{[\\$box[T\oplus k]{T\oplus k}]})\_{\mathsf{l}} \quad \mathsf{[\\$box[T\oplus k]{T\oplus k}]}\_{\mathsf{2}} \quad \dots \quad \mathsf{[\\$box[T\oplus k]{T\oplus k}]}\_{\mathsf{n}} \quad \dots \quad (8.9)$$

is an *n*-dimensional vector used as regressors. One benefit of SA is the constructive feedback of side-channel leakage detection it might bring to the evaluator (see, e.g., [278]).

# **8.4 Machine Learning-Based Attacks**

Machine learning encompasses a number of methods used for classification, clustering, regression, feature selection, and other knowledge discovering methods [423]. A typical division of machine learning algorithms is into supervised, semi-supervised, and unsupervised approaches. Each of those paradigms can also be used in SCAs—supervised (profiling) attacks, semi-supervised attacks (profiling), unsupervised (non-profiling) attacks.

In Fig. 8.2, we depict differences in the supervised and semi-supervised cases.

#### **Supervised Techniques**

The supervised approach assumes that the attacker first possesses a device similar to the one under attack. Having this additional device, he is then able to build a precise profiling model using a set of measurements while knowing the plaintext/ciphertext and the secret key of this device. In the second step, the attacker uses the earlier profiling model to reveal the secret key of the device under attack. For this, he additionally measures a new set of traces, but as the key is secret he has no further information about the intermediate processed data and thus builds hypotheses. The only information that the attacker transfers between the profiling phase and the attacking phase is the profiling model he builds.

When considering supervised machine learning and SCA, in recent years there have been numerous papers considering various targets, machine learning algorithms, and scenarios. Actually, the most common denominator for most of the work

**Fig. 8.2** Profiling side-channel scenario: supervised (left), semi-supervised (right)

is the fact that they attack AES [235, 274, 279, 285, 363–365, 367, 475, 476, 479, 481]. More recently, deep learning (DL) techniques started to capture the attention of the SCA community. Accordingly, the first results confirmed expectations, with most of the early attention being paid to convolutional convolutional neural networks [122, 329, 386, 482].

As far as we know, when considering machine learning-based attacks on other ciphers, there are only a few papers. Heuser et al. consider Internet of Things scenarios and lightweight ciphers where they compare 11 lightweight ciphers and AES in terms of their SCA resilience and conclude that lightweight ciphers cannot be considered to be significantly less resilient than AES [274, 276].

#### **Semi-supervised Techniques**

Semi-supervised learning is positioned in the middle between supervised and unsupervised learning. There, the basic idea is to take advantage of a large quantity of unlabeled data during a supervised learning procedure [517]. This approach assumes that the attacker is able to possess a device to conduct a profiling phase but has limited capacities. This may reflect a more realistic scenario in some practical applications, as the attacker may be limited by time or resources, or also face implemented countermeasures, which prevent him from taking an arbitrarily large amount of side-channel measurements while knowing the secret key of the device.

The first application of semi-supervised SCA was done by Lerman et al., where the authors conclude that the semi-supervised setting cannot compete with a supervised setting [366]. Note, the authors compared the supervised attack with *n* + *m* labeled traces for all classes with a semi-supervised attack with *n* labeled traces for one class and *m* unlabeled traces for other unknown classes (i.e., in total *n* + *m* traces). Picek et al. conduct an analysis of two semi-supervised paradigms (self-training and graph-based learning) where they show that it is possible to improve the accuracy of classifiers if semi-supervised learning is used [480]. What is especially interesting is that they show how semi-supervised learning is able to significantly improve the behavior of the template attack when the profiling set is (very) small.

# *8.4.1 Conducting Sound Machine Learning Analysis*

Since it is not possible (in general) to expect machine learning techniques to give us theoretical observations or proofs of results, we need to rely on a set of procedures to run experiments such that the results are convincing and easy to reproduce. In the next section, we briefly discuss several steps to be considered in order to make the analysis more reproducible.

#### **Datasets**

When preparing the data for machine learning analysis, it is necessary to discuss the number of measurements, the number of features, and the number of classes (if known). Additionally, if the data come from different distributions, one needs to discuss those. If not all data from datasets are used, it is necessary to state how the samples are chosen and how many are used in the experiments. One needs to define the level of noise appearing in the data in a clearly reproducible way, e.g., using the signal-to-noise ratio (SNR). Finally, if some feature engineering procedure is used, it needs to be clearly stated in order to know what features are used in the end.

## **Algorithms**

When discussing the choice of algorithms, first it is necessary either to specify which framework and algorithms are used or provide pseudo-code (for example, when custom algorithms are used). As a rule of thumb, more than one algorithm should always be used: the algorithms should ideally belong to different machine learning approaches (e.g., a decision tree method like Random Forest and a kernel method like Support Vector Machine (SVM)). Next, all parameters that uniquely define the algorithm need to be enumerated.

## **Experiments**

Regarding the experiments, it is first necessary to discuss how the data are divided into training and testing sets. Then, for the training phase, one needs to define the test options (e.g., whether to use the whole dataset or cross-validation, etc.) After that, for each algorithm, one needs to define a set of parameter values to conduct the tuning phase. There are different options for tuning, but we consider starting with the default parameters as a reasonable approach and continue varying them until there is no more improvement. Naturally, this should be done in a reasonable way, since the tuning phase is the most expensive from the computational perspective and it is usually not practical to test all combinations of parameters.

#### **Results**

For the tuning phase, it is usually sufficient to report the accuracy. For the testing results, one should report the accuracy but also some other metric like the area under the ROC curve (AUC) or the F-measure. The area under the ROC curve is used to measure the accuracy and is calculated via Mann-Whitney statistics [580]; the ROC curve is the ratio between the true positive rate and the false positive rate. An AUC close to 1 represents a good test, while a value close to 0.5 represents a random guess. The F-measure is the harmonic mean of the precision and recall, where precision is the ratio between true positive (TP, the number of examples predicted positive that are actually positive) and predicted positive. The recall is the ratio between true positives and actual positives [488]. Both the F-Measure and the AUC can help in situations where accuracy can be misleading, i.e., where we are also interested in the number of false positive and false negative values.

# **8.5 Performance Metrics**

When considering profiling SCA, there are three performance metrics we mention: accuracy, guessing entropy, and success rate. The accuracy is the proportion between the correctly classified measurements and all measurements:

$$ACC = \frac{TP + TN}{TP + TN + FP + FN}.\tag{8.10}$$

TP refers to true positive (correctly classified positive), TN to true negative (correctly classified negative), FP to false positive (falsely classified positive), and FN to false negative (falsely classified negative) instances. TP, TN, FP, and FN are well-defined for hypothesis testing and binary classification problems. In the multiclass classification, they are defined in a one class vs. all other classes manner, and are calculated from the confusion matrix.

A side-channel adversary *AEK ,L* conducts an experiment Exp*AEK ,L* , with timecomplexity *τ* , memory complexity *m*, and making *Q* queries to the target implementation of the cryptographic algorithm. The attack outputs a guessing vector *g* of length *o*, and is considered a success if *g* contains correct key *k*∗. *o* is also known as the order of the success rate. The *o*th order success rate of the side channel attack AEK*,*L is defined as:

$$\text{SR}\_{A\_{E\_K,L}}^o(\tau, m, k^\*) = \text{Pr}[\text{Exp}\_{A\_{E\_K,L}} = 1] \tag{8.11}$$

The Guessing entropy measures the average number of key candidates to test after the attack. The Guessing entropy of the adversary *AEk ,L* against a key class variable *S* is defined as:

$$GE\_{A\_{E\_{K},L}}(\tau, m, k^\*) = \mathsf{E}[\mathrm{Exp}\_{A\_{E\_{K},L}}] \tag{8.12}$$

# **8.6 Countermeasures Against SCA**

There are various countermeasures against SCAs that have been proposed over the years. A general approach focuses on decreasing the information gathered from the measurements:

• Noise Addition. Introducing external noise in the side-channel, shuffling the operations or inserting dummy operations in cryptographic implementations is often used as a countermeasure against SCAs. The basic objective is to reduce the signal-to-noise ratio (SNR) and thereby decrease the information gathered from measurements. Still, as shown already by Durvaux et al. [194], these countermeasures become insecure with increasing attack time.

# **8.7 Conclusions**

In this chapter, we discussed profiling side-channel attacks where we started with data preprocessing and feature engineering. Then we presented several templatelike techniques and afterward machine learning techniques. Next, we discussed how to conduct a sound machine learning analysis that should result in reproducible experiments. We finished the chapter with a short discussion on how to test the performance of SCA and what are some of the possible countermeasures to make such attacks more difficult.

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 9 Side Channel Assessment Platforms and Tools for Ubiquitous Systems**

**Apostolos P. Fournaris, Athanassios Moschos, and Nicolas Sklavos**

**Abstract** Side Channel Attacks are nowadays considered a serious risk for many security products and ubiquitous devices. Strong security solution providers need to evaluate their implementations against such attacks before publishing them on the market, thus performing a thorough assessment. However, this procedure is not straightforward and even with the appropriate equipment, it may require considerable time to provide results due to the slow process of collecting measurements (traces) and the inflexible way of controlling the tested implementation. In this chapter, we explore and overview the trace collection landscape for generic devices under test (including ubiquitous systems) highlighting and overviewing the latest trace collection toolsets and their shortcomings, but also proposing a trace collection approach that can be applied on the most recent, open source toolsets. We showcase our proposed approach on the FlexLeco project architecture, which we have developed in our lab, and manage to practically describe how an evaluator using the proposed methodology can collect traces easily and quickly without the need to completely redesign a control mechanism for the implementation under test.

# **9.1 Introduction**

The transition of computing devices to the ubiquitous era, where the cyber world is merging with the physical world to bring an ever present layer of computer intelligence to everyday life objects, is bringing, among other things, cybersecurity and privacy into the physical world as an issue to be constantly considered. Due to IP interconnected cyberphysical systems, attackers can gain access not only to a victim's data but also to the victim's life itself, by controlling "smart"

University of Patras, Patras, Greece

A. P. Fournaris (-)

Industrial Systems Institute/R.C. ATHENA, Patras, Greece e-mail<EMAIL_ADDRESS>
A. Moschos · N. Sklavos Georgia Institute of Technology, Atlanta, USA

devices of urban and industrial systems, critical infrastructures and individual households [540]. This highlights the need for strong security features that must be installed on embedded systems. However, ubiquitous devices have several nonfunctional constraints like small processing power, low power consumption or small memory footprint, that prohibit the use of several traditional security and cryptography schemes (e.g., asymmetric cryptography). This has led to the redesign and simplification of existing security solutions (like TLS schemes and their cipher modes) or the development of cryptography algorithms especially designed for low performance devices (lightweight cryptography schemes [197]).

The design and evaluation of security schemes and cryptographic algorithms must take into account several parameters that are related to a scheme's cryptographic strength (addressed by cryptanalysis, formal security verification methods etc.), to its algorithmic performance (addressed by collecting and comparing performance and resource measurements) and to its resistance against implementation attacks like side channel and fault injection attacks [337]. Side channel attacks (SCAs) have a leading role in the design of modern cryptosystems, since they are the weakest point and they have been exploited by many well-known attacks in order to break otherwise unbreakable security/cryptography algorithms. SCAs can be easily applicable to ubiquitous, cyberphysical systems, where devices are left unattended in potentially security "hostile" environments (in remote, secluded areas, inside malicious user premises, etc.)

There is a broad research field related to various SCAs, aiming to exploit various physical characteristics of a device including timing, power consumption, electromagnetic emission, etc. The flagship SCA analysis methods are of statistical nature and can be categorized into horizontal attacks (using one or a few collected inputs) or vertical attacks (using many collected inputs). Among the most potent and successful such attacks are Differential Power Analysis (DPA) or Correlation DPA [390] as well as template, online template [58] and Mutual Information Attack (MIA) [229]. A simple review of the above advanced SCAs reveals that all of them require a considerable amount of collected leaked physical characteristic inputs in order to be effective.

Assessing if a security/cryptography scheme on a ubiquitous device is SCA resistant, is not a straightforward process. It usually follows two directions. In the first direction, a security scheme is evaluated against specific SCAs while the second direction is based on performing a generic information leakage assessment based on some statistical test. Student's or Welch t-test are two such tests that use statistical hypothesis testing in order to detect if one of a series of sensitive intermediate processes during a security procedure significantly influences the measurement data or (more often in a non-specific test) detect how different is a collected trace from random noise (indicating a bias addressed to exploitable information leakage).

Collecting inputs for SCA analysis is done using specialized equipment in a fairly cheap and easy way. However, when a huge number of inputs need to be acquired then the processing of an input (or trace, as they are usually denoted in the literature) becomes a very slow and cumbersome process. A restricted number of tools that help the acquisition of the needed traces for advanced SCA attacks exist. Most of them still require a considerable amount of time for trace collection and employ custom (to the cryptosystem at hand) control mechanisms. There are open source SCA trace collection setups widely used by the research community [451] but have either very primitive support or are built on low-cost equipment that cannot endure very sophisticated attacks without having software developed by an attacker [427]. Also, adjustments for different cryptographic devices under test are always needed, sometimes to a great extent. There exist a few commercial companies in this research field (Riscure [501], CRI[495]) that offer their own proprietary equipment to the prospective crypto-analysts at a prohibitively high cost, affordable only by high budget labs.

In this chapter we review the latest commercial, open source and experimental trace acquisition tools and focus on their benefits and drawbacks. We also highlight the need for a cheap, fast and automated process for sending and receiving data between a Device under Test (DUT) and a controller. We comment on the importance of efficient control loops of DUT in order to speed-up the trace acquisition process. Finally, we propose and describe a three step approach for controlling, functionally validating and SCA assessing a DUT that goes beyond the traditional control model that other solutions follow. Finally, we describe the use of our solution using an open source hardware/software trace acquisition platform that we have developed in our lab (FlexLeco project [427]) that consolidates the latest trends in trace collection tools and platforms.

The rest of the chapter is organized as follows. In Sect. 9.2, the major SCA categories and leakage assessment methodologies are described and evaluated according to the number of traces needed. In Sect. 9.3 we overview existing open source and commercial trace collection platforms and toolsets, and the proposed trace collection approach is described. In Sect. 9.4, we describe the use of the proposed approach on the FlexLeco project architecture, and discuss the benefits of the solution. In Sect. 9.4 conclusions are provided.

# **9.2 Side Channel Attacks, Leakage Assessment Methods and Problems**

A typical side-channel attack (SCA) measurement and analysis setup consists of several components, including measuring equipment (e.g., a digital signal oscilloscope), a DUT controller component that handles attacker-to-DUT communication as well as a Personal Computer (PC). The PC is used by an attacker/evaluator for providing input to the controller and for analyzing, with the help of programming tools, the collected leakage trace measurements by applying signal processing techniques on them [390]. Possible additions to this classic SCA measurement collection and analysis setup could be some kind of pre-amplifier to boost the signals acquired by the measuring equipment (e.g., an oscilloscope), a differential cable to help reduce acquired measurements noise, electromagnetic probes for acquiring electromagnetic emanation signals, and some kind of tool that will speed up leakage trace capturing.

 Under the ubiquitous computing framework, small embedded system devices (e.g., RFID tags, wireless sensor network nodes, smart card devices), having lightweight versions of cryptographic algorithms, are the most prominent candidates to have SCA vulnerabilities. Assessing the SCA resistance of such devices can be achieved by evaluating the device when it is deployed and is fully operational (in that case the DUT is the whole device) or can be achieved by evaluating individually, in a controlled environment, a specific cryptography or security implementation that is meant to be deployed on the ubiquitous device. In the first case, SCA assessment is very hard to perform since, apart from security functions, on the DUT there are many operations, unrelated to security, that are being executed in parallel [219]. Such operations can be considered as hard-toremove noise inside the collected traces [219]. In the second case, the DUT is not the full ubiquitous device but a specific security hardware or software implementation deployed on this device. No other operations are executed in the second case control environment, thus noise is minimized, enabling the evaluator to test many SCA resistance scenarios in depth.

When trace collection is needed for SCA analysis or assessment, the attacker/evaluator needs to pass the cryptography/security algorithm's expected input data each time from his control point (usually a personal computer) to the control component, which is responsible for sending the data to the DUT for one security/cryptography process to begin. After having set the right settings in the oscilloscope (e.g., sampling rate, time window capture, resolution capture) the attacker arms it. The attacker then sends a command to the DUT, for it to start performing the evaluated security process. Before the start of the process the DUT sends a triggering signal to the oscilloscope, warning it that the process is about to start and a trace capture must be performed. As soon as the trigger signal reaches the oscilloscope, it captures a leakage trace measurement (e.g., power consumption or the electromagnetic emanation, depending on the used probes) of the DUT. The attacker/evaluator then requests from the oscilloscope the captured trace for analysis at his PC. When the captured trace reaches the PC, various signal processing techniques are applied on it and it is used for side-channel analysis of the DUT. The above procedure (called loop round) is repeated for each new security/cryptography process we want the DUT to perform.

# *9.2.1 Side Channel Attack Categories*

Adopting the formulation approach described in [60, 61, 220] we can model each security/cryptography computation *C* as a series of *n* different *Oi* operations (for *i* ∈ {0*,* 1*,...n* − 1}) that each require inputs *Xi* (thus *Oi(Xi)*). We can also assume that each operation output can be considered as input to another operation during the *C* computation. Each operation *Oi* is linked to an information leakage variable *Li*. A side channel attack is possible if there is some secret information *s* that is shared between *Oi* and its leakage *Li*. The ultimate goal of a side channel analysis is, by using some strategy, to deduce *s* from a series of information leakage *Li* values [220]. To achieve that we collect leakage traces *L* which are sequences (in time) *L* = {*L*0*, L*1*,...Ln*−1} and try to associate them with the computation *C* as a sequence of operations *C* = {*O*0*, O*1*,...On*−1}.

SCAs follow either the vertical or horizontal leakage collection and analysis strategy, as originally described in [60]. In the vertical approach, the implementation is used *N* times using either the same or different inputs each time in order to collect leakage traces. Each trace is associated with the *j* -th execution of the computation. In the horizontal approach, leakage traces and related analysis is collected/performed from a single execution of the computation and each trace corresponds to a different time period within the time frame of this execution. As expected, in horizontal attacks the implementation input is always the same.

By associating and distinguishing specific patterns of *Oi* operations (consisting of a single or multiple operations) in a leakage trace *L*, an attacker can recover the secret *s*. Such an attack approach is typically followed in simple SCAs (SSCAs) that are mostly horizontal attacks meaning that they are mounted using a single leakage trace that is processed in time. Vertical SSCAs that need more than one leakage trace rely on comparative analysis for pattern matching between computations with different inputs. Vertical SSCAs require a few hundred leakage traces to be collected considering that averaging technique for noise reduction is applied between leakage traces of the same computation.

There exist several countermeasures that can thwart SSCAs, thus, more advanced attacks have been devised (i.e., Advanced SCAs (ASCAs)). ASCAs do not focus only on single operations or series of operations (e.g., *Oi*) but also on the computation operands (i.e., the operation inputs *Xi*) [61, 220]. ASCAs are focused on a specific subset of the calculation *C* (e.g., some *Oi* operations that are strongly associated with *s*) and examine how this subset behaves over a large collection of (e.g., *N*) leakage traces *Lj* associated with computations *Cj* with different inputs (where *j* ∈ {1*,...,N*}). ASCAs on this subset of operations and associated leakage traces, exploit the statistical dependency between the calculation *C* for all *Xj* and the secret *s*. ASCAs follow the hypothesis test principle [60, 335] where a series of hypothesis *s*´ on *s* (usually on some bit *j* of *s* i.e., *s*´ *j* =0 or 1) are made and a series of leakage prediction values are found based on each of these hypotheses using an appropriate prediction model. The values of each hypothesis are evaluated against all actual leakage traces using an appropriate distinguisher *δ* for all inputs *Xi* so as to decide which hypothesis is correct.

Most ASCAs are of vertical nature and their success is highly related to the number of processed leakage traces. The most widely used ASCA vertical attack is Differential Attack (DSCA) originally proposed by Kocher in [336], which was later expanded into the more sophisticated Correlation SCA (requiring fewer traces to reveal the secret than DSCA) [21] and the collision correlation attack [99, 210, 425], which can be mounted even if the attacker does not have full control of the implementation inputs. There are, however, also horizontal ASCAs that apply differential or correlation statistics on a single leakage trace assuming that a subset of operations associated with the secret *s* appear many times in this trace. Finally, ASCAs can bypass strong SCA countermeasures (e.g., randomization/blinding) by combining horizontal and vertical approaches [59, 210, 220].

Following the above categorization (Vertical vs. Horizontal, SSCA vs. ASCA), we can include profiling SCAs like Template Attacks or Machine Learning Attacks [367] among the vertical ASCAs. Profiling attacks operate in two phases: Initially, they need to collect a series of leakage traces from a device under the full control of the attacker (with known inputs and secrets) so as to create a leakage model. In the second phase, the leakage model is used as a template or as the training set of a machine learning algorithm in order to recover a secret from a series of traces collected from a device (similar to the one used for profiling) not under attacker control.

# *9.2.2 Leakage Assessment Using t-Test*

In addition to SCA resistance assessment based on the success of various SCAs, a generic methodology for finding information leakage from a DUT has been gaining ground. The dominant, generic, leakage assessment methodologies are based on Student's t-distribution following specific and non-specific t-tests [67, 516]. The goal is to detect any type of information leakage that occurs during the computation of security/cryptography functions in the DUT, at a certain *n*-th SCA order. [An SCA attack of order *n* appears when there exists an *n* set of intermediate variables that appear during the computation of the algorithm, such that knowing a few key bits (in practice fewer than 32 bits) allows us to decide whether two inputs (respectively two outputs) give the same value for a known function of these *n* variables.] Any sensitive computational intermediate operation *Oi* series that appears on the side channel as significantly different from random noise can potentially be detected using the above leakage assessment approach without conducting any specific SCA. This significant difference is enough to mark a DUT implementation as leaky and SCA insecure.

Test Vector Leakage Assessment (TVLA) is one of the most promising, generic, non-specific leakage detection techniques, initially proposed by Cryptography Research (CRI) [67]. The method is practically used as the first action towards assessing a system's SCA leakage. It consists of a univariate test that is performed on a series of traces obtained from a DUT. The DUT implementation is evaluated as non-leaky if the test throughout the duration of the DUT trace remains below a certain threshold, independently of the leakage model that might be used. More precisely, we test the case where there is no leakage (null hypothesis) versus the case where there is some leakage at a certain intermediate point *L(t)* at time *t*. Let *nt r* be the number of traces that the evaluator collects and *ns* the number of samples in each trace. Following the notation of [597], and assuming that we have *N* traces of *ns* samples each, let **L** = {*L*1*,...,Lns* } be the leakage traces of a security/cryptography implementation, having mean values *L*¯*i*. For the traces null hypothesis to be true the expected leakage value *L*¯ *exp* should be the same as the measured value, if there is no leakage, i.e., *L*¯ *exp* = *L*¯*i* ∀*i* ∈ {1*,...,ns*}. In practice, to perform the TVLA we conduct two experiments. In both experiments, we collect *N/*2 traces where the variable to be leakage assessed has a known, fixed, value and *N/*2 traces where this variable has a random value. This trace collection is done in an interleaved way by randomly choosing to acquire either a fixed trace or a random trace each time. After collecting the necessary traces, we perform a Welch's t-test as described in [516] and check the outcome against a threshold.

TVLA and all similar leakage assessment tests can provide an initial indication regarding SCA leakage but require huge amounts of collected traces in order to provide an accurate assessment result. In the case of symmetric key cryptography algorithms (e.g., in AES implementations) this number is in the order of millions (in [516] *N* =100M traces for an AES evaluation). This number is reduced in Public Key algorithms (in the order of thousands of traces) yet still is hard to collect since each public key algorithm implementation trace consists of a very large number of samples *ns*. From a trace collection perspective, TVLA is a very slow assessment method due to this high *N* number, and can become very frustrating for an SCA evaluator.

# *9.2.3 Practical Considerations in SCA Trace Collection*

When, in practice, the above described SCA attacks are applied to actual, raw collected traces using real hardware or software implementations, their success rate is very low. This happens since in a trace, leakage information is mingled with a considerable amount of SCA-useless signals that we can consider here as "noise". Noise, as very accurately described in [411], can be external, intrinsic, quantization or algorithmic. External noise that is picked up from external to the DUT sources as well as intrinsic noise, due to the DUT's physical characteristics (capacitance, conduction, transistor, non-linear behavior, etc.), are not under the control of the attacker and the DUT SCA resistance designer, and must be removed or reduced by some appropriate trace preprocessing technique. On the other hand, quantization noise can be considerably reduced by using better trace collection equipment (with small A/D quantization for example). Algorithmic noise is usually designer infused on a DUT in an effort to increase randomness in data processing but also infused by computation functionalities unrelated to security, such as interrupts, pipelining, OS system calls, etc., these can very often appear in ubiquitous devices.

In order to practically make a successful SCA, noise cancellation techniques must be applied during or after trace collection in order to have clear and useful leakage traces. Traditional noise reduction techniques can be applied on traces after collection based on low-, band- or high-pass filtering after finding dominant frequencies using, for example, Fast Fourier Transform analysis or based on trace resampling where sample windows are specified and they are replaced by their average value [501]. However, many researchers apply the averaging technique to increase the signal-to-noise ratio and mount a successful attack. Using averaging, we collect *T* leakage traces with the same inputs and secret, then average them sample-per-sample to come up with a single, averaged leakage trace that contains a lot less noise than each individual trace itself. The technique is very popular (almost mandatory) in embedded system SCAs to remove noise, but it increases the needed number of collected traces by a factor of *T* since we need *T* trace collections for each SCA useful, averaged, leakage trace.

Another practical factor to be considered during trace collection, which also has an impact on the number of collected traces, is trace misalignment. This phenomenon happens when, after triggering a security/cryptography computation, the various *Oi* operations do not always appear at the same point in time in all leakage traces of this computation, even if the same inputs and secret are used. Misalignment can appear frequently in software based security computations (a common case in ubiquitous devices) since such computations run in parallel to other securityunrelated processes, or random events may happen that influence the computation execution sequence. Misalignment can also appear in hardware-implemented security computations when related SCA countermeasures are introduced in the computation flow. To solve the problem, SCA theory states that traces should be realigned using postcollection trace processing techniques [390]. However, in practice, for several cases of ubiquitous devices, traces are so misaligned that they become SCA useless and cannot be effectively realigned. Thus, there is a percentage of collected traces that due to misalignment should be discarded. This makes it imperative to collect more traces than those needed, having in mind that some of them will be useless due to misalignment. A rule of thumb in such cases is to collect 20% more traces than needed. This percentage can increase to 50% in highly misaligned traces (usually on ubiquitous software implementations).

# **9.3 Side Channel Attack Trace Collection Platforms**

There exist several Security Test Labs and individual researchers who have proposed and manufactured their own ad hoc hardware boards [399, 532, 565] for trace collection. In the years following the discovery of SCAs, several Side-Channel Analysis measurement boards and evaluation setups emerged in the security community. The purpose of such boards is to provide a common platform for the aspiring attackers to mount their attacks and help them get low noise measurements in an easy way. Typically, they accommodate a general purpose device (a microprocessor, an ASIC, or an FPGA) serving as the DUT, connected with a controlling device (control component) on the same board (mainly some sort of microcontroller). There were also boards that accommodated signal-enhancing mechanisms on the same board to ease the oscilloscope's work [117]. Gradually the quality of the boards improved to such a degree that several of them found their way to the market for commercial use, with significant success.

There is limited variety of commercial SCA boards, each with its pros and cons. The Cryptographic Engineering Research Group (CERG) presented the FOBOS board [565, 566], which consists of two FPGA's, one for handling control tasks and one for the implementation of the cryptographic algorithm. The FOBOS board contains a Digital Clock Manager that produces frequencies in the range of 31.25 kHz up to 50 MHz for the Victim Board. The communication with the Control FPGA is performed through a software control loop that is responsible for transmitting the appropriate commands and the necessary data to the victim FPGA. The software loop is also responsible for the communication with the oscilloscope (currently supporting only Agilent oscilloscopes). Unfortunately, the FOBOS approach relies on old FPGA boards, is not capable of automated multiple trace collection through the provided control loop, is only applicable to a specific type of oscilloscope, and involves the PC in each trace collection (part of the control loop) which considerably burdens the collection speed.

The Tamper-resistance Standardization Research Committee (TSRC) released two trace collection hardware boards [399]. The first is INSTAC-8 which uses an 8-bit microcontroller and the second is INSTAC-32 with a 32-bit microcontroller as the control component and an FPGA chip as the DUT. Unfortunately, very limited information is provided on the clock frequencies and the communication methods that those two boards employed. Similarly to FOBOS, they also featured some custom-made software loops for the communication between the users and the victim chips. An evolution of the above approaches is the Sakura-Sasebo Project [354], which provides several choices regarding measurement boards. Initially, low-end-based chip boards were implemented, leading to the Sasebo-G, Sasebo-GII and Sasebo-W solutions, which for several years constituted the most widely used platforms for trace collection. Later, more sophisticated versions of those boards were launched, the Sakura-G, Sakura-X and Sakura-W. Both Sakura-G and Sakura-X contain two FPGAs each (a Control FPGA acting as the control component and a cryptography FPGA acting as the DUT), making them perfect for evaluation of hardware-implemented cryptographic algorithms, while Sakura-W was suitable for evaluation of smartcard security. Unfortunately, the boards are still supported by a primitive interface on the Control FPGA that enables the interfacing of a particular cryptographic algorithm with limited key-length. Also, the provided software loop in charge of data and command transmission to the Cryptographic FPGA is slow, oriented towards a specific algorithm with fixed key-length, and offers through a PC program very basic functionality only for a single trace capture per loop round.

An attempt to remedy this issue was made in the IAMeter project, which is focused exclusively on developing an efficient control loop for commercial and custom FPGA board platforms (including the Sasebo-G and GII boards) [307]. However, even this attempt can provide only a single trace collection per loop round, it is not capable of adjusting/controlling the DUT clock, and it relies heavily on PC-based configurations (including Python scripts along with MySQL databases queries) which slow the trace collection as a whole.

Recently, in an effort to provide a cheap SCA setup, the ChipWhisperer project [450] managed to provide a diverse tool-chain for embedded hardware security research. The ChipWhisperer featuring a CW1173-Lite board is suitable for capturing traces and attacking software cryptographic algorithms implemented in its Atmel XMEGA microcontroller, and it was recently upgraded to the CW1200- Pro version that offers additional ease-of-use on the process. ChipWhisperer though also provides the CW305-Artix FPGA board that specifically targets attacks on hardware cryptographic algorithms implemented in its ARTIX 7 FPGA as well as the OpenADC dongle that enables trace capturing directly from hardware devices (without the need for an oscilloscope). Both CW1173 and CW305 allow the modification of Victim's Chip frequency (e.g., the frequency range for the CW305- Artix board starts from 5 MHz and goes up to 160 MHz due to the onboard PLL (Phase Locked Loop)). The communication between the user and the Victim Chip in both boards relies on a software control loop called Chipwhipserer-Capture. This is a PC-based loop responsible for sending the appropriate data and commands to the Victim board. In the case of the CW305-Artix FPGA board the PC software loop is assisted by an onboard AVR microprocessor (partially acting as a control component).

Complete measurement and evaluation setups are provided by experienced security test laboratories with longtime presence in the hardware security community. BrightSight delivers the Sideways acquisition center. CRI-Rambus offers the DPA Workstation [495] which is a complete platform that enables the testing and analyzing of cryptographic chips and systems vulnerabilities, to power and electromagnetic (EM) side-channel attacks. Finally, Riscure launched the Inspector [501] which is an integrated test tool for side channel analysis and fault injection attacks on security devices. The above commercial setups offer SCA resistance evaluation/assessment on individual security/cryptography hardware and software implementations as well as on fully working DUT ubiquitous devices (e.g., embedded systems).

The gap between the measurement boards and the complete evaluation setups is huge when it comes to cost. Trace collection and SCA evaluation setups providing a complete package that includes measurement collection, side-channel attacks and DUT security analysis, have a considerable cost. On the other side, the cost of the measurement boards is much more affordable, nonetheless exploiting their capabilities to the full extent is not always supported by their manufacturers in terms of software or hardware tools.

# *9.3.1 Proposing a Fast Trace Collection Approach Beyond the Traditional Model*

As the need for a huge number of traces for SCA evaluation and leakage assessment becomes considerable (in the presence of DUT with SCA countermeasures) the traditional SCA trace collection control loop model is not fast enough for efficient, practical, trace acquisition. The main bottleneck in such control loops is the presence of a PC device for providing inputs, collecting outputs and controlling each execution of the DUT security/cryptography implementation [427]. The solutions presented in the previous subsection, although they do not manage to eliminate the need for a PC inside the control loop, clearly indicate a tendency to migrate traditional PC-related functionality to other hardware or software entities that are closely associated with the DUT. In some solutions, the control loop operations are implemented in hardware and are downloaded on a dedicated control FPGA that is physically connected to the DUT. This is done in the Sasebo/Sakura project and in some ChipWhisperer (NewAE) technologies, just to name some examples. Hardware, however, is not flexible and thus cannot be easily adapted to different algorithms or assessment techniques. Similarly, control loop functionality is partially migrated on dedicated control loop ASIC microcontrollers or microprocessors that operate alongside a PC in order to implement the DUT control in software. Such solutions lack speed when transmitting DUT test vectors to the DUT itself since PC usage is still needed.

Therefore, it has become apparent that a different, updated approach to realize DUT control loops for SCA evaluation and leakage assessment is needed. In this work, considering the previous paragraph's described hardware and software control loop limitations, we propose an approach that relies on a hardware/software co-design solution for controlling the trace acquisition process. Extending and generalizing the work in [427], we propose a three-step SCA trace collection process. For this process to be possible, the control loop is not executed on a PC but exclusively on a microcontroller that is directly connected to the DUT. The microcontroller can be an ASIC (hard-core) or FPGA based (softcore) depending on the SCA trace collection board at hand. Using software that is executable on the microcontroller we gain the flexibility of a software control loop solution. Using the microprocessor that is directly connected (through a bus interface) to the DUT we gain very high control loop speed, which is not achievable using PC-based control loops. More information on such an architecture can be found in the use case example presented below.

In the first step of the proposed control trace collection process, denoted as the design phase, the evaluator can describe in a programming or scripting language (e.g., C, Python, JavaScript), using some developed Application Program Interface (API), the SCA trace acquisition experiment that needs to be performed. The experiment includes inputs that need to be provided to the DUT, specification of the security/cryptography operations that need to be executed, the execution sequence, the delay between experiment executions (in case the experiment needs to be executed more than once) and DUT output storage. The goal of the design phase is to fully specify the inputs, parameters and execution sequences of the experiment. The outcome of this phase is an executable file that can be transmitted to the microcontroller bootloader for execution.

The second step of the proposed control trace collection process, denoted as the execution phase, is focused on the execution of the designed experiment. This phase does not include any non-trivial transmission delays between the control loop entity (i.e., the microcontroller) and the DUT, since the bus connecting them is extremely fast, in contrast to the PC-based control loop where such transmission is done serially. During execution, the microcontroller control loop is responsible for transmitting the appropriate signals to the DUT so as to execute one or multiple times a security/cryptography operation as well as to generate appropriate trigger signals for trace collection by a DSO.

In the final step of the proposed control trace collection process, denoted as the trace processing phase, the execution of the experiment has been concluded and the experiment traces have been collected by the DSO. In this phase, post-collection operations are performed on the collected traces, like averaging and alignment but also operations related to the specificities of a particular experiment. This phase is performed on the DSO or on a PC with an appropriate digital signal processing toolbox, and it can be slow (depending on the post-collection operations that are executed). However, the performance delay is considerably smaller than when a PC is included in the control loop during an experiment.

# **9.4 A Use Case of a Flexible and Fast Platform for DUT SCA Evaluation**

To showcase the applicability and effectiveness of the above-proposed three-step trace collection architecture and mechanism in action, we focus on the Flexible Leakage Collection (FlexLeco) project, which was recently published in [427]. the FlexLeco project was designed to match the latest trace collection challenges and to introduce a unified mechanism for applying various trace collection scenarios. It provides an architecture that tries to blend the reconfigurability of Software Control loops with the speed of Hardware Control loops. Taking advantage of the latest trace collection boards that utilize schemes with two FPGAs (one acting as the Control Unit and the other as the Device Under Test), the project created two generic hardware interfaces that enable fast communication between the two FPGAs, and managed to include an embedded softcore processor inside the Control FPGA, which is in charge of the Control Loop during the execution of trace collection scenarios. The project is currently instantiated on the boards of the Sakura/Sasebo project (Sakura-X and Sakura-G), but it can be modified to fit any board that adopts the approach of two distinct, hardware-isolated FPGA chips.

The FlexLeco architecture of Fig. 9.1, consists of two generic interfaces and an embedded softcore processor. Inside the Cryptographic FPGA a generic cryptographic interface is implemented for the communication of the Control Unit with the DUT. This interface contains two variable memory spaces (called "Hyperegisters") that handle the inputs and outputs of the DUT.

Inside the control FPGA exists a generic control interface that is directly connected with the embedded microprocessor (a Xilinx Microblaze for the Spartan-

**Fig. 9.1** General architecture

6 Control FPGA of the Sakura-X board), residing in the same FPGA. With this approach, the softcore microprocessor undertakes the duty of control and communication with the Cryptographic FPGA, through a hardware Finite State Machine implemented inside the control interface, which utilizes a custom hardware protocol between the two FPGAs. Data inputs to the DUT can be provided by the softcore processor, as well as from a hardware Pseudo Random Generator (PRNG) module (to support TVLA leakage assessment scenarios). The communication between the two FPGAs is performed using a 16-bit address bus, a 16-bit data bus for sending data to the crypto side and a 16-bit data bus for receiving data from the Cryptographic FPGA side.

The use of an embedded system design inside the control FPGA provides support for the proposed three-step trace collection approach thus allowing the execution of a software API on the microprocessor for the realization of the trace collection control loop. This API consists of reusable code functions that fit multiple trace collection scenarios and DUTs. Through these functions the control FPGA remains unchanged (no need for redesign or reprogram), regardless of the DUT inside the cryptographic FPGA, as it can be quickly reconfigured only by passing certain values to software registers inside the control interface. By setting code values to these registers, the control interface's Finite State Machine is ready to serve any updated cryptographic component inside the cryptographic FPGA. In this way, the control component can be permanently be downloaded inside the control FPGA's flash memory, thus negating the inflexibility issues that other hardware control loops present in their adaptability to different DUTs and scenarios.

The above-mentioned software API provides functions that, beside the initialization of the control loop, set up the leakage trace collection parameters (inputs/outputs number, bit-length, randomness), trigger encryption/decryption, send or receive plaintext/ciphertext values to FIFOs, and register and randomize

input values.1 By doing so, the attacker/evaluator can use the API functions to create any SCA or leakage assessment scenario to be executed inside the softcore microprocessor, omitting the slow, PC-based software control loops. In that sense, the FlexLeco solution fully supports and favors the design phase of the proposed trace collection approach of this chapter and provides to an SCA evaluator all the tools required to design diverse and complex SCA evaluation experiments.

The cryptographic interface on the cryptography FPGA side is primarily designed for testing specific security/cryptography hardware implementations. It is an open source HDL (Hardware Design Language) design that can be adapted to the DUT's cryptographic algorithm specifications during design time. By simply assigning appropriate values for five HDL generic parameters, the number of inputs, outputs and their bit length is adjusted to that of the DUT's algorithm. The generic interface is synthesized and downloaded whenever a new hardware implemented cryptography algorithm is tested.

Inside the cryptographic interface, a Digital Clock Manager has been included that provides different clock frequencies to the DUT and the interface. The frequency of the interface is the same as that of the other one inside the control FPGA, while the frequency of the DUT can be clocked as high as the component's critical path and the FPGA chip's functional specifications allow (or as low as the attacker/evaluator desires). By raising the frequency of the DUT, the evaluator is now able to use DSOs with low memory size buffers, thus fitting more traces on the time interval the DSO offers (as long as the DSO's sampling frequency allows it). The DCM's output frequencies update is a straightforward process done by changing a single parameter during the cryptographic FPGA's synthesis phase.

Presenting such flexibility and scalability, the FlexLeCo mechanism allows the evaluator to perform various trace collection scenarios like a Single-Encryption, a Single-Encryption with Rapid Block Mode (if an oscilloscope with such a feature is available) and a Multi-encryption mode [427], for different DUTs and with minimum overhead between the mode updates.

During any trace-collection scenario (Fig. 9.1), at design phase, the softcore microprocessor is set up so as to initiate communication transactions with the cryptographic FPGA, in which it either reads and sends the contents of the corresponding test vector records (i.e., plaintexts) or signals a random value generation (using an API function or the hardware PRNG) and transmits it to the DUT. We can design an experiment where this procedure continues until all of the test vectors on the microprocessor's memory have been sent to the cryptographic device (DUT) or until the needed number of random inputs is reached. After the design phase, the actual experiment is executed in the softcore microprocessor and post-collection operations may be performed. As an example of such postcollection operations, we showcase the Multi-encryption scenario, detailed in [427], which is enabled in the FlexLeco project in case an RBM (Rapid Block Mode) Digital

1Both software- and hardware-based randomization is supported through the PRNG module.

**Fig. 9.2** Multi-encryption trace with 1000 AES encryptions

**Fig. 9.3** Post-collection extraction outcome as a single AES leakage trace

Signal Oscilloscope is not available to the evaluator. During this scenario, the DSO starts capturing a continuous waveform (Fig. 9.2) of leakage traces starting from the first de/encryption and continuing for all de/encryptions until the end of the DSO's chosen time window. By setting up the appropriate time window, after execution phase, we capture a single continuous waveform that should contain the leakage traces of all the cryptographic processes we have instructed the DUT to execute. The split of this continuous multi-encryption waveform into individual single de/encryption traces is done during the trace processing phase. The outcome single encryption trace from this post-collection operation can be seen in Fig. 9.3. The whole process is considerably faster than if we tried to capture each de/encryption trace autonomously during the execution phase [427].

To qualitatively compare the proposed three-step trace collection approach of Sect. 9.3.1 as was realized using the FlexLeco project, we present Table 9.1 where our approach is compared with recent open source trace collection projects in terms of flexibility, usability and various post-collection feature supports. The presented results are collected from actual experimentation of the authors with the compared

**Table 9.1** Leakage trace collection architectures qualitative features comparisons

a Hardware interfaces are unique for each cryptography DUT

bHard to implement software control loop and tedious, time-consuming interfacing of different cryptographic DUTs

cMulti-encryption only with constant or random plaintexts

projects or from personal communication with the projects' developers. The table indicates that the three-step approach is flexible enough to rival existing and wellestablished solutions offered to the SCA community by Sakura and ChipWhisperer.

# **9.5 Conclusions**

In this book chapter we focused on an important aspect of SCA analysis, evaluation and leakage assessment, which is the efficient and easy collection of needed SCA traces. We presented the traditional mechanism for collecting traces from DUT ubiquitous devices and commented on the drawbacks of this approach. After briefly describing dominant SCA attack categories and leakage assessment methodologies in view of their needed number of traces, considering also the high level of trace noise and possible misalignments that are ever present in ubiquitous devices, we concluded that the traditional model is not practically useful for ubiquitous systems SCA evaluation. To further explore the recent SCA trace collection and analysis landscape, we described the most prominent open source and commercial toolsets, both research and commercial. Most have shortcomings in terms of controlling in a flexible and easy manner the DUT to be SCA evaluated, thus giving us the motivation to propose a three-step trace collection methodology using a design, an execution, and a trace processing phase. To validate the applicability, efficiency, and ease-of-use of this proposed approach, we applied it to the FlexLeco project open source solution, which is highly compatible with our proposal. Using this use case we managed to easily design experiments and collect results even when we applied complex design scenarios, like the multi-encryption mode where multiple inputs are provided on the DUT, multiple traces are collected as one, and the actual single traces (that are usable for SCA evaluation or leakage assessment) are extracted after postprocessing. To conclude, the evaluation of our proposal and the exploration of the recent toolset landscape indicate that there is a need for a different model for trace collection. In this model, the trace collection DUT control functionality is migrated close to the DUT (on a device physically connected to the DUT) and not on a remote control entity (like a PC).

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 10 Challenges in Certifying Small-Scale (IoT) Hardware Random Number Generators**

## **Darren Hurley-Smith and Julio Hernandez-Castro**

**Abstract** This chapter focuses on the testing and certification of Random Number Generators (RNG). Statistical testing is required to identify whether sequences produced by RNG demonstrate non-random characteristics. These can include structures within their output, repetition of sequences, and any other form of predictability. Certification of computer security systems draws on such evaluations to determine whether a given RNG implementation contributes to a secure, robust security system. Recently, small-scale hardware RNGs have been targeted at IoT devices, especially those requiring security. This, however, introduces new technical challenges; low computational resources for post-processing and evaluation of onboard RNGs being just two examples. Can we rely on the current suite of statistical tests? What other challenges are encountered when evaluating RNG?

# **10.1 Introduction**

Randomly generated values are sought after for a variety of applications, in which they are often vital. Cryptographic systems require random values to ensure that generated keys are unpredictable, making brute force attacks against those keys unfeasible. Even in the entertainment industry, there is a demand for randomness: lotteries and games both rely on random number generation to guarantee the fairness of the game in question.

However, random number generation is a non-trivial task. Deterministic Random Number Generators (DRNG), also known as Pseudo-Random Number Generators (PRNG), are incapable of truly random output [514]. PRNG achieve an *appropriate degree of randomness* by using an initial seed value to populate a proportionally far longer sequence of apparently random output. This form of random number generation is only unpredictable if the seed value remains unknown. To this end,

University of Kent, Canterbury, UK

D. Hurley-Smith (-) · J. Hernandez-Castro

e-mail<EMAIL_ADDRESS>
© The Author(s) 2021

G. Avoine, J. Hernandez-Castro (eds.), *Security of Ubiquitous Computing Systems*, https://doi.org/10.1007/978-3-030-10591-4\_10

most PRNG algorithms are periodically re-seeded from a natural source of entropy. The primary benefit of PRNGs is that they are usually extremely fast, especially when compared to the natural entropy sources used to seed them. This makes them highly attractive for use in computer systems, and in applications requiring highvolume RNG.

Physical sources of entropy can provide what is referred to as *true randomness*. True Random Number Generators (TRNG) use a broad array of different entropy sources as their key component but share several common characteristics. They do not require seeding to generate randomness and use a natural phenomenon as their entropy source. TRNGs can be classified further, as classical or Quantum Random Number Generators (QRNGs). To simplify matters, TRNG will refer to classical methods, and QRNG will refer to quantum methods from this point. TRNG utilize microscopic phenomena that generate statistically random noise signals. The photoelectric effect and thermal noise are two examples of classical entropy sources. QRNG operate on similar principles but instead make use of quantum phenomena. These include photon-counting, using a beam-splitter, or the observation of quantum shot-noise in MOS/CMOS devices.

All random number generators can be evaluated using statistical test batteries. Dieharder, Federal Information Processing Standard (FIPS) 140-1/2, and National Institute of Standards and Technology (NIST) SP800-22 [448] represent the three most common test batteries used for professional testing of random number generators. Manufacturers often use such tests to demonstrate the correct functioning of their products, but they are also used by third-parties to independently verify the randomness of a device. NIST and Common Criteria [407] provide guidelines and tests that have been independently developed to ascertain whether an RNG is nonrandom. These tests evaluate RNGs by identifying whether there is any observable bias, structure or predictability in an RNG's output. It is not possible to identify randomness, but non-randomness can be detected. Certification schemes make use of such tests to publicly acknowledge the robustness of computational security systems. Specific methodologies have been devised to guide and ensure the quality of these evaluations in the area of RNG validation.

Significant trust is placed in statistical testing to determine whether an RNG provides sufficiently random output. The aim of this chapter is to demonstrate that the challenges of statistical testing of randomness are far from solved. We evaluate a selection of contemporary TRNG to highlight issues in data collection, test correlation and the overuse of older test batteries to the exclusion of newer tests. As minuscule, integrated TRNG become more prolific through their use in Internet of Things (IoT) products, these considerations become all the more important.

The following sections discuss, in order: certification of RNGs and the standards/testing procedures that apply, the challenges faced during the collection of data from RNGs, and two sets of experimental results demonstrating issues in the appropriate selection of statistical tests for RNG evaluation.

# **10.2 Certification, Standards, and Testing**

Many companies employ their own testing teams, to whom the responsibility of carrying out company mandated quality control falls. ID Quantique (IDQ) and NXP are two examples, both of whom perform varying degrees of testing on their products. In the case of products implementing cryptography, RNG testing is vital for the validation of the cryptosystem in question. However, in-house testing is insufficient for certification, with the exception of self-certification (as performed by IDQ). Testing must be performed by a third-party to ensure impartiality.

NIST is one example of a standards and testing institution. This US institute concerns itself with the advancement of measurement science, standards and technology. This body does not conduct testing or reward certificates itself but is responsible for the publication and impartial development of statistical test suites for randomness tests. Special Publications (SP) are created to circulate accepted developments in the field of RNG testing and formal verification of RNG. Of particular note are SP800-90B [449] and SP800-22 [448]. SP800-90B details specific tests for the entropy source and final outputs of PRNG and TRNG. SP800- 22 details an extensive test battery suitable for use over PRNG and TRNG (including QRNG by association with TRNG).

Common Criteria (CC) is an international standard (ISO/IEC 15408). Unlike the NIST SP documents discussed previously, CC is a broad framework for the verification of computer security systems [407]. Functionality, construction, and assurance requirements are the core tenets of the CC. It is important to emphasize that this is a whole-system-security verification: RNG testing is only part of a larger verification process. However, it can be argued that RNG validation is a keystone for the certification of a computer-based security system. If the RNG is incapable of providing the appropriate output, then it is unlikely that the security system will be robust to the degree demanded by the CC.

To differentiate between different applications and their security requirements, the CC has developed the Evaluation Assurance Level (EAL) scheme. These numbered levels, from 1 to 7, reflect an increasing security requirement. At level 1, testing is cursory and reports provided by manufacturers are acceptable. As higher certifications are sought, more third party and design-stage tests by third parties are required. At levels 5+, spot checks of manufacturing plants and implementation of security critical systems are performed. NXP produces two CC EAL certified devices: the DESFire EV1 (EAL4+), and the DESFire EV2 (EAL5+).

The test methodology employed by the CC when testing RNGs is outlined in AIS-31 [327]. AIS-31 outlines the test methodology for entropy sources in computer-based security systems [327]. AIS-20 is referred to as the source of information for recommended tests and parameters for TRNG evaluation. Both documents have a TRNG focus, as they are aimed at the evaluation of the formal verification of entropy sources, not the PRNG algorithms that they may seed. As a result, hardware RNGs are the focus of these documents.

**Table 10.1** Standards applied in the testing of selected RNGs

aReverse biased semiconductor junction

Table 10.1 shows a selection of RNGs and their associated certifications. CC EAL, METAS, CTL, and FIPS 140-2 are applicable as certifications from their respective institutions. NIST SP800-22 indicates that the NIST methodology and test battery were applied when testing the RNG in question (whether internally or externally). Any RNG testing process requires a set of statistical tests. One of the earliest examples of a statistical test battery for randomness is Marsaglia's Diehard battery [393]. NIST SP800-22 provides a more expansive series of tests developed by Rukhin et al. [508]. The NIST battery contains 15 tests, which are evaluated in terms of uniformity and proportion of *p*-values for each test. There has been some criticism of the accuracy of these results. Marton and Suciu observed that false alarms were common and that more tests that SP800-22 suggests can be failed by otherwise robust RNGs [396]. NIST itself states that any failure is cause for further investigation, but does not suggest any specific follow up procedures for RNG testing. It is implied that further data collection and testing a larger number of target devices are initial approaches to the problem.

Dieharder is an extension of Diehard, integrating the SP800-22 tests and the original Diehard tests [116]. This brings the battery up to a total of 30 tests, with 76 variant tests in total. This battery requires a much larger body of test data than its predecessors. To test a stream of data with no rewinds with every test in the suite, one must collect 228 GB of data. This is far beyond the recommended parameters suggested by NIST and CC. A 4 GB sample would rewind 57 times under the same test conditions. If a sequence of repeats during a single execution of a given test, type-1 errors may be introduced. The test may report such repetition as a violation of its definition of randomness, and identify the sequence as non-random, when in fact it was just insufficiently large. This highlights the importance of appropriate data collection.

TestU01, developed by L'Ecuyer and Simard, is more of an RNG developer's toolbox than test battery [362]. However, it incorporates 5 different test batteries: Alphabits, Rabbit, Small Crush, Crush, and Big Crush. Each battery has a differing number of tests. Alphabits and Rabbit operate over bits, whilst the Crush batteries operate over floating point numbers between 0 and 1. Alphabits, Rabbit, and Small Crush complete in minutes over samples of 2 GB in size. Big Crush requires a large amount of data (or a constant stream of input from the target device) and can take hours to complete. McCullough et al. identify some potential issues with this toolset. Some tests are only able to read 32 bits and are more sensitive to errors in their most significant bits than their least significant bits [403]. To resolve this, they suggest that tests are performed over the sequences forwards and backwards. The issue here is that a test on live data cannot be performed in this manner. This limits many tests and prevents them from being used as live tests.

Another class of tests exists; continuous tests. These tests are designed to identify whether there have been hardware failures that lead to corruption or cessation of the entropy stream. FIPS 140-1/2 are designed with hardware in mind [121]. Both tests suites can be implemented in the circuitry of an RNG, providing a constant series of results regarding the health and functionality of the device. A core requirement of any continuous test is that no RNG should output two identical *N* bit blocks in succession. If this condition is not met, the device should cease function immediately and alert the user that it is not performing as expected. However, this does not detect more subtle flaws. The astute reader may also have deduced that requiring that no two *N* bit blocks be identical actually results in reduced entropy. This has an impact on the legitimacy of such tests when considering that the definition of an ideal RNG is one that is completely unpredictable. These tests are likely to be implemented alongside IoT TRNG implementations due to their efficient implementation in hardware, carrying the previous concerns to millions of potential devices.

The usage of NIST, Dieharder, TestU01, and other statistical test batteries can vary between institutions. NIST SP800-22 outlines minimum sample sizes and Dieharder implies these by rewinding samples if insufficient data is provided. However, during self-certification, some companies have been found to test small samples, below the suggested guidelines. This can cast doubt over the validity of their findings.

# **10.3 Challenges in Data Collection**

For standalone RNG, data collection may be simple. However, there are no official certifications for standalone RNG. FIPS and CC both certify whole security systems, not individual elements, so even though RNG testing is key to this process, a standalone RNG that passes these tests still cannot be certified. Regardless, RNG testing as a part of whole system certification is a critical consideration. Data collection from certain integrated RNGs may not be trivial. As IoT devices represent a whole-system security implementation, they may be certified; RNG evaluation forms a critical part of any such evaluation process.

Black-box design is often employed by companies using licensed technology, or who need to protect their Intellectual Property (IP). This means that schematics of their security implementation, including RNG, may not be publicly accessible. For lower EAL awards, such obfuscation of technical detail may extend to inspectors and CC officials. At higher levels, non-disclosure agreements are required as a part of the certification contract between the petitioning company and the evaluating body. Such arrangements are expensive. Inspectors and independent testers have to be compensated for their times and the cost falls to the company requesting an evaluation at a given EAL. As a result, self-certification is common.

The speed with which an RNG may be read depends on a great many factors. In situations where the RNG is fully integrated, there may be additional overheads such as post-processing, use of a PRNG to *clean* TRNG output used as an entropy source, or simply a hard limit on output size and speed. A poignant example of this is the DESFire EV1 and EV2. These RFID cards do not directly expose their internal TRNG to the user, requiring that the user extracts random numbers using the authentication protocol instead. This protocol requires that both the card and reader exchange random values as part of their authentication handshake [289]. The 16 bytes values transmitted by the card can be collected and stored in a file for analysis using statistical tests for randomness [288]. This is a time-consuming procedure, as Table 10.2 shows. To collect 64 MB of data from the DESFire cards, approximately 12 days were required. The primary bottleneck in this process was the need to complete the authentication protocol before a second handshake could be initiated to gather additional 16-byte sequences. Attempting to terminate the protocol by switching off the reader, thus resetting the card, proved to be even more time-intensive [288]. This issue is shared by IoT devices, many of which implement integrated TRNGs.

IoT devices have a plethora of ways in which PRNG and TRNG may be implemented. The FRDM K64F board implements a TRNG, though the output is limited to making calls internally for use, or outputting values over an I/O pin in the form of unsigned integers. Though significantly faster than the EV1 and EV2, this is still much slower than most standalone TRNGs. The Red Bear Duo does not implement a local entropy source. An on-board PRNG must be supplied with off-device entropy, with no checks or continuous tests performed on-device. In a full-system implementation, such a device can make it difficult to identify where the flaw in its RNG occurs.

The standalone generators (Quantis, Comscire and ChaosKey entries in Table 10.2) are substantially faster, making data collection trivial by comparison. However, this does not mean that samples of appropriate size were tested.

SP800-90B states that an entropy source must provide 1,000,000 bits of sequential output for testing [449]. Concatenation of smaller sequences is tolerable if contiguous output to that size is not possible, but is undesirable. 1000 such sequences must be concatenated, according to NIST guidelines. SP800-22 extends these requirements by recommending that 100 samples of the aforementioned size are tested to validate the results [508]. AIS-31 and AIS-20 do not stipulate minimum sample sizes. John Walker states that, in their default configuration, the Diehard tests should be run over at least 100 MB of data [568].

With this in mind, the test reports of several TRNG manufacturers can be more thoroughly analyzed. IDQ states that their Quantis devices pass the Diehard and NISTSP800-22 batteries with no failure.1 SP800-22 tests were conducted over 1000 samples of 1,000,000 bits in length. A significance level of 1% was maintained throughout this process. Diehard was used over a single sample of 1 · 109 bits. Our own tests confirm that IDQ's report of no failures is true, even for larger samples (ours were 2.1 GB in size). In this case, IDQ is a good example of a test protocol that is in line with the recommendations of test developers.

Comscire's PQ32MU, a QRNG that uses shot-noise as an entropy source, is a different story. Their NIST-Diehard report2 shows that the number of tests has been reduced. The reduced sample size is one issue, but reducing the number of tests can result in the loss of certain capabilities. Unless the removed tests are wholly redundant, it is likely that their removal will impact the capability of the battery to detect certain types of non-randomness. The insufficient sample size is cited as the reason for excluding those tests. Comscire only tests this QRNG using 2 samples; one of 8 · 107 bits and another of 1 · 106 bits. This is drastically below the suggested sample size for Diehard. Even though these samples meet the requirements of NIST SP800-90B in the most basic sense, they still fall short of SP800-22's additional recommendations requiring the testing of at least 100 samples. Considering the ease with which samples can be generated from standalone RNGs such as these, it is surprising that a more robust test process is not used.

# **10.4 Appropriate Selection of Tests**

The correlation between tests in a battery, and as a whole if the evaluation methodology involves multiple test batteries, must be considered. Statistical tests have a limited range of issues that they are able to identify in the target RNG. Test

1https://marketing.idquantique.com/acton/attachment/11868/f-004c/1/-/-/-/-/Randomness %20Test%20Report.pdf.

2https://comscire.com/files/cert/comscire-pq32mu-nist\_diehard-validation-tests.pdf.

**Fig. 10.1** Example of Ent default output in byte mode [272]

batteries are intended to mitigate this issue by providing many statistical tests that evaluate different aspects of the target RNG, providing a broader analysis.

Hernandez-Castro et al. identify a degree of correlation between tests in the Ent battery. The Ent battery is a simple set of tests included in most Linux distributions as a simple statistical testing tool [568]. Ent includes tests for estimated entropy, compression, *χ*2, arithmetic mean, Monte Carlo *π*, and serial correlation. Bit and byte level tests can be run over target sequences. Figure 10.1 shows the output of the Ubuntu 16.04 Ent utility in byte mode.

By degenerating an initially random sequence using a genetic algorithm, Hernandez-Castro et al. were able to observe the test results of Ent as the sequence slowly became more ordered and predictable [272]. The results demonstrate that many of the Ent tests have a degree of correlation. Entropy and compression tests analyze the same general attributes, both performing linear transformation and ceiling operations on a sequence. The *χ*2 and excess statistics provided by the *χ*2 test are also closely correlated. The conclusion of the paper recommends that the excess and compression statistics should be discarded.

Soto et al. explore the degree of correlation between tests in the NIST SP800-22 battery. Their work finds that the range of attributes evaluated by SP800-22 may be insufficient to recognize issues [538]. TRNG and QRNG are particular issues, as many examples of these RNGs have been developed since the development of SP800-22. Soto describes the independence of tests in this battery as an open problem.

Turan et al. provide a more recent analysis of SP800-22. Their work finds that the frequency, overlapping template (input template 111), longest run of ones, random walk height, and maximum order complexity tests produce correlated statistics [560]. This issue is most evident when using small samples or block sizes. Georgescu et al. build on Turan's work, identifying and examining the open problems in SP800-22 test correlation. The sample size is found to have a significant effect on the correlations between tests. The correlations identified by Turan et al. are confirmed, and their relationship with sample size explored in greater depth [226]. Such results demonstrate that every element of an RNG test methodology must be carefully examined to ensure a meaningful and unbiased result. Georgescu et al. conclude by stating that better tests than those implemented in SP800-22 may exist, as that battery is now quite old.

Researchers have commented on the ambiguity of SP800-22's hypothesis and statistical output, stating that more descriptive test output is required. Zhu et al. propose a *Q-value*, computed using test statistics prior to their consolidation to *p*values [601]. The proposed statistic is more sensitive to total variation distance and Kullback-Leiber divergence. This overcomes some of the issues caused by correlations between the non-*χ*2 level 2 tests of SP800-22 [601].

Dieharder implements many of the SP800-22 tests. As a result, it shares many of the criticisms levelled at SP800-22 [206]. TestU01 is a more recent battery aimed at allowing researchers to develop and evaluate their own RNGs, especially TRNGs. There is little critical literature regarding this battery at present, so the independence of tests in TestU01 is an open question at this time. Turan et al. comment on the presence of some tests that they have found to be correlated being implemented in the Crush batteries of TestU01 [560].

The diversity of a test methodology is related to, but separate from, the independence of tests. Where independence is a measure of how related the results from a set of tests may be, diversity is a measure of how many methods of evaluation are used in the analysis of an RNG. A common observation is that the isolated use of *p*-values is insufficient to fully characterize the randomness (or lack thereof) of a sequence. Research by Hurley-Smith et al. explores TRNG and QRNG in detail to identify flaws that were not detected by the most commonly used test batteries. In these analyses, test correlation and diversity are key topics.

# *10.4.1 Randomness Testing Under Data Collection Constraints: Analyzing the DESFire EV1*

The first of these in-depth analyses was conducted over the Mifare DESFire EV1, an RFID card produced by NXP [379]. The DESFire EV1 is used as a part of the Transport for London (TfL) Oyster card scheme, as well as other loyalty and ewallet schemes throughout Europe. As a device that can store cash value, it requires robust security to foster trust among vendors and users. The EV1 has achieved an EAL4+ certification, based on its full security implementation.

Table 10.3 shows the Dieharder results for 3 DESFire EV1 cards. As mentioned previously, data collection from the EV1 is challenging, requiring 12 days to obtain 64 MB of data. As a result, this was the largest amount of data able to be collected. A total of 100 cards were tested, with all 100 passing. The 3 cards shown in this table show the *p*-values reported by the Dieharder tests for all tests that can be performed on 64 MB of data without rewinds.

Card 3 shows a single failure of the Dieharder battery, for the count the ones test. However, this was not reproduced by any other card that was tested. Therefore, it is reasonable to conclude that the Dieharder battery does not identify any significant degree of non-randomness in the tested sequences.

Table 10.4 shows the pass rates for NIST tests. All SP800-22 tests were used over the EV1 samples we collected.

**Table 10.3** Dieharder results [289]

The bold values indicate those tests which fail, but such a degree that they are well outside the bounds of confidence established by NIST (or in the case of Ent, our extrapolation of the NIST SP800-22 confidence bound of *a* = 0*.*01)

**Table 10.4** NIST SP800-22 results [289]

The bold values indicate those tests which fail, but such a degree that they are well outside the bounds of confidence established by NIST (or in the case of Ent, our extrapolation of the NIST SP800-22 confidence bound of *a* = 0*.*01)

Cards 1 and 3 both show some borderline results, notably in the Runs and Nonoverlapping template tests. However, the majority of cards (98 of 100) passed this battery. This was a cause for concern: any failure is a cause for further investigation as stated by SP800-90B. As a result, further analysis was deemed necessary, and the

**Table 10.5** Mifare DESFire EV1 ENT results for 64 MB of TRNG output [289]

The bold values indicate those tests which fail, but such a degree that they are well outside the bounds of confidence established by NIST (or in the case of Ent, our extrapolation of the NIST SP800-22 confidence bound of *a* = 0*.*01)

humble ENT battery was used as a starting point for a more generalized approach to our EV1 TRNG evaluation.

Considering Hernandez-Castro et al.'s work on the independent of Ent tests, the compression and excess statistics should be discarded. However, the full results of the Ent battery over 3 EV1 cards are shown in Table 10.5 for the sake of completeness.

All tests are passed, with the exception of the *χ*2 test. For the 3 64-MB samples shown in Table 10.5, the *χ*2 statistic is exceptionally poor. By comparison, a sequence that passes this test should have a *χ*2 statistic of between 220 and 305. Even at a sample size of 1 MB, 100 DESFire EV1 cards failed this test. These results show that the values in the tested sequences are not uniformly distributed: there is a bias towards some byte values and away from others. Considering that the *χ*2 test is such a trivial (and widely used) test of the distribution of values in a sequence, it is surprising that it would highlight issues in the output of the EV1's TRNG while Dieharder and NIST SP800-22 do not.

Non-uniform distribution of bytes is not an automatic indicator of nonrandomness. It is not a good indicator of randomness, but it is also possible for a true source of randomness to produce a slightly biased sequence. However, as per the guidelines of AIS-20 and SP800-90B, a TRNG should provide an output that is functionally equivalent to that of a cryptographic PRNG. As a result, non-uniform byte distribution is a concern. The fact that there is bias is an important observation, but more important is the analysis of that bias.

Figure 10.2 provides a deeper examination of how bias is expressed by the TRNG output of 100 DESFire EV1 cards. Figure 10.2a shows the mean bias of 100 1- MB samples. The extreme deviation from the expected distribution of values is apparent in the square-wave of the plot. The expected distribution should result in a noisy, relatively evenly distributed set of byte values. A bias in the order of 10−5 is observed, with an almost evenly distributed bias among values that are deviated above or below the normal. To be precise, 127 values are biased above the normal, and 129 are biased below the normal.

Figure 10.2b refines the observations of the previous graph. Fourier approximation of the bias reveals that the distribution of byte values has a period (*w*) of

**Fig. 10.2** Analysis of DESFire EV1 bias [289]. (**a**) Mean bias of 100 1-MB samples. (**b**) Mean Fourier approx. of 1 MB samples. (**c**) Mask test results. (**d**) *χ*2 scores for 100 1-MB samples

−31.9918. This results in 8 oscillations throughout the 256 possible byte values, with a shift across the normal, observed every 32 values (approximately). Statistical analysis of the possible distribution of bits within these byte values shows that the under-occurrence of a specific bit-sequence can result in this very particular form of bias.

Figure 10.2c provides the results of a so-called *mask test*. The purpose of this test is to XOR each byte of a sequence with an 8-bit sequence, ranging from *0000000* through all intervening values to *11111111*. The sum of all sequences that resolve to zero after the XOR operation records the occurrence of that bit-sequence throughout a sample. This graph shows the composite of 100 1-MB sequences tested in this manner. It is immediately apparent that there's a significant deviation from the normal for mask *00011000*. For all cards, and for both 64 and 1 MB samples, this bias was observed. Following our responsible disclosure to NXP, it was suggested that this bias may be caused by an incorrectly implemented whitening-function: a function usually intended to remove bias from TRNG output.

Figure 10.2d shows the distribution of the *χ*2 statistic for 100 1-MB EV1 samples. The statistics are proportionally lower than those seen for the 64 MB samples (Table 10.5). This is because sample size has a direct effect on the expression of bias within a sequence. Early experiments conducted by Hurley-Smith et al. demonstrated that the bias of the DESFire EV1 could not be observed at sample sizes smaller than 7.5 KB [288, 289]. This emphasizes the point made in Sect. 3 regarding data collection. The amount of data collected needs to meet a minimum size to reliably identify issues in the RNGs being tested. This minimum threshold is test-specific, requiring that the highest minimum sample size is identified by analysts prior to conducting an evaluation of an RNG. Furthermore, these experiments showed that even well-respected, proven statistical test batteries such as Dieharder and NIST SP800-22 were unable to identify the issue with the DESFire EV1. It is clear that it is possible to design a TRNG to pass these tests, but is it wise to rely on tests that can be designed for? Does designing to meet the finite and narrow requirements of Dieharder and NIST SP800-22 actually provide any guarantees of randomness? We argue that it does not.

# *10.4.2 Identifying Issues with Quantum Random Number Generators*

The EV1 experiments provided an introduction to issues in using well-established statistical tests to identify non-randomness in TRNG. QRNGs are currently too large for RFID card or IoT device implementations, but miniaturization of quantum entropy sources is proceeding quickly, and proposals for IoT-scale QRNG have already been published. However, there are several open problems with the current generation of QRNGs and their evaluation.

Even when sample collection is not a problem, there can be issues. IDQ's Quantis range of QRNGs is based on an optical quantum source of entropy (a beam splitter). Comscire produces a rival product, the PQ32MU, which uses quantum shot-noise as its entropy source. Both companies provide multiple models of QRNG with varying speeds, all with appropriate statistical test results associated with their devices. As previously discussed, IDQ provides a relatively robust test report, though it is limited to Diehard and NIST SP800-22 tests. Comscire uses few and small samples, with a smaller number of tests, limiting the rigor of its test process significantly. None of the devices tested as a part of this work were validated using the AIS-31 methodology, nor were they certified (as there are no official certifications for standalone RNGs).

Data collection is not an issue from these devices, the Quantis devices provide data at a rate of 4 or 16 Mb/s, whiles the PQ32MU has an output speed of 32 Mb/s. As a result, collecting large amounts of data is trivial. A key difference in these two brands is that the Quantis generators do not implement on-board post-processing to remove bias, whilst the PQ32MU is an all-in-one product with post-processing performed on-device.

Table 10.6 shows the results for the EV1, Quantis generators and PQ32MU. Both raw and post-processed Quantis output is shown. EV1 data is tested for 64 MB samples over 100 cards. Quantis and Comscire QRNGs are tested over 100 2.1 GB

**Table 10.6** Dieharder, NIST and TestU01 results

samples collected from one of each type of device. Ideally, more devices would be tested, but the cost was a limiting factor (the cheapest device, a 4M, costs e900).

All devices pass Dieharder, while all but 2 EV1's pass the SP800-22 tests. The TestU01 toolkit has been used, with 4 of its statistical test batteries used to evaluate all tested devices, including the EV1. Due to the sample size requirements of the Crush tests, EV1 data has not been tested for either Crush test. Immediately, the EV1 shows critical issues, failing the Alphabits and Rabbit batteries. The average failure rate is 1 of 4 tests for Alphabits, and 5 of 16 tests for Rabbit. This shows how the simple addition of a new test battery can instantly reveal weaknesses that the better-known batteries cannot identify.

Raw Quantis samples, especially those of the 4M and its USB variant, also perform very poorly on Alphabits and Rabbit. They also perform very poorly in Crush, but a significant number of samples pass the Small Crush tests. This could be because the Small Crush battery has many tests in common with SP800-22, leading to a correlation between the results. Post-processing cleans up many of these issues, but not completely. Most notably, Alphabits, Rabbit and Crush test results improve dramatically, with the most drastic change being the jump from 3 passed tests for the 4M under Alphabits, to 91 passes. This shows that appropriate use of a QRNG is yet another factor to consider: improper use of a device may not be identified by the more well-known test batteries and incorrect configuration can be as damaging as any other form of non-randomness. The Comscire PQ32MU performs well on most tests but struggles with the Rabbit and Crush tests.

Table 10.7 shows the results of Ent for the QRNG. DESFire results are not shown to avoid repetition. A summary of the 100 samples tested shows that Post-processed Quantis data, and the PQ32MU, passes the *χ*2 and serial correlation tests with no issues. All devices pass the other tests, hence their omission from this table. However, the raw Quantis data fails the *χ*2 test dramatically. Furthermore, the 4M and its USB variant perform quite poorly on the serial correlation test at the bit level. This emphasizes the need to test sequences across multiple block sizes to identify issues that may occur at lower or higher orders of output. Unlike the EV1, raw Quantis data does not provide an easily identifiable or consistent bias across

**Table 10.7** ENT results

samples. The bias appears to drift between samples, with the only constant being a tendency to express a 10−6 bias above the normal for byte values 0–5. Even this is not a representative trend, with only 38% of samples showing this particular trait.

Figure 10.3 shows the *χ*2 statistics for raw Quantis samples from all the 16M and 4M devices. The results for the Quantis USB are omitted, as the USB is effectively a 4M in different packaging and provides similar results.

The 16M (a) fails the *χ*2 test for 90 of its samples. The mean statistic for the 16M is approximately 350. This is above the acceptable maximum threshold for this test. The 4M is significantly worse, with a mean statistic of 506. Unlike the 16M, the 4M shows no passes at all (the USB reports similar results). In fact, the minimum statistic for the 4M was 407. This is significantly above the maximum threshold for the *χ*2 test.

The experiments conducted over these QRNGs show that established tests do not always identify issues that more recent (or just less well-known) tests highlight. The TestU01 battery reinforces the results of the Ent test, by providing a wider variety of more sophisticated tests that prove that there are issues beyond simple deviation from the normal distribution of values at the byte and bit level. As TestU01 is designed to provide the tools to test TRNG, this battery would ideally be made a mandatory recommendation for TRNG and QRNG testing. Dieharder and NIST SP800-22 will remain in use, as they are effective at identifying egregious issues with RNG output, but the extension of the minimum recommended number of tests is very much needed at this time. Post-processed and raw data should be tested and the results clearly marked to show users how the improper configuration of software post-processing can be identified and resolved. One should also consider that if IoT QRNGs are sought-after, how does one implement a post-processing algorithm (which are known for their high memory requirements) in such a small package? Resource limitations may prevent effective post-processing of QRNG output, the consequences of which are made clear in the preceding work.

**Fig. 10.3** Distribution of *χ*2 scores for Quantis devices. (**a**) 16M. (**b**) 4M

# **10.5 Conclusion**

There are many complex issues to consider when evaluating RNGs for use in security systems. Device specifications, the use of off-device entropy pools, postprocessing, and output speed are all critical to the evaluation process. Each element should be tested in isolation, but only the whole device may be certified, leading to issues when considering black-box design philosophy and resource-constrained devices.

The DESFire EV1 has been found to output biased values from its TRNG, but this does not necessarily mean that the RNG itself is at fault. Subsequent work with the EV2 has found no issues with its RNG. Combined with conversations with NXP's engineers, this indicates that the issue may instead be in the whitening function employed to remove bias from the raw TRNG output. The EV1 results highlight two key issues; the role of black box design in complicating the evaluation process, and the quality control challenges facing small-scale robust TRNGs. Quantis QRNGs also require post-processing, as demonstrated by the exceptionally poor results shown by raw data over a variety of statistical tests. Rigorous testing of RNG with multiple input and processing dependencies should require results demonstrating the performance of both raw and processed output of such devices. This will aid in the identification of implementation errors.

Reliance on Dieharder and NIST SP800-22 cannot continue to the exclusion of new tests, such as those employed by TestU01. There is a wealth of academic literature on the subject of statistical tests of randomness and efforts must be made to identify which of these will provide the next wave of reliable tests of randomness. Finally, it is important to consider how tests may be evaded by manipulation of RNG output; future work will focus on how some simple manipulations result in predictable data that passes current statistical tests of randomness.

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Chapter 11 Finding Software Bugs in Embedded Devices**

**Aurélien Francillon, Sam L. Thomas, and Andrei Costin**

**Abstract** The goal of this chapter is to introduce the reader to the domain of bug discovery in embedded systems which are at the core of the Internet of Things. Embedded software has a number of particularities which makes it slightly different to general purpose software. In particular, embedded devices are more exposed to software attacks but have lower defense levels and are often left unattended. At the same time, analyzing their security is more difficult because they are very "opaque", while the execution of custom and embedded software is often entangled with the hardware and peripherals. These differences have an impact on our ability to find software bugs in such systems. This chapter discusses how software vulnerabilities can be identified, at different stages of the software life-cycle, for example during development, during integration of the different components, during testing, during the deployment of the device, or in the field by third parties.

# **11.1 The Challenges of Embedded Devices and Software**

We argue that the problem of embedded software security is due to multiple factors, including a systematic lack of transparency, control, and resistance to attacks. A particular way to improve this is to analyze the software of these devices, with the particular goal of identifying software vulnerabilities in order to correct them as early as possible.

A. Francillon (-)

EURECOM, Sophia Antipolis, France e-mail<EMAIL_ADDRESS>
S. L. Thomas University of Birmingham, Birmingham, United Kingdom

A. Costin Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland

# *11.1.1 Lack of Transparency*

Today, many smart devices are compromised during massive attacks, and may be abused to form large *botnets* (networks of compromised devices). Record-high Distributed Denial of Service (DDoS) attacks (i.e., network flooding) reportedly generated between 620 Gbps and 1 Tbps of traffic [241, 344]. These DDoS attacks were reported to use several hundred thousand compromised embedded/smart devices, comprising dozens of different models of Commercial Off-The-Shelf (COTS) products like IP/CCTV cameras and home routers. Most of those devices were compromised using default or hard-coded credentials set by the manufacturer [345].  Worse yet, the users or owners of the device are often not aware of the problem, and unable to solve it. In fact, devices are not designed to be inspected and modified by end-users (e.g., to perform forensics as discussed in Chap. 13).

# *11.1.2 Lack of Control*

Another important problem is that smart devices are generally provided as a fixed software (i.e., firmware) and hardware platform, often tied to a cloud service and bundled together as a closed system that the user has little control over. An example of the negative consequences of this customer lock-out is the *Revolv* smart thermostat. *Revolv*'s manufacturer was acquired by its competitor Nest, and after a year Nest stopped the cloud service, rendering the *Revolv* thermostats installed in homes impossible to use [271]. Users often have no choice regarding which software the device should run, or which cloud service to use, or what the device should do. Choosing, installing and using alternative software for such devices is difficult, if not impossible, often due to the single-purpose nature of the hardware and software design, the lack of public documentation, in addition to any anti-tampering measures imposed by the manufacturer.

# *11.1.3 Lack of Resistance to Attacks*

In practice, Internet scanning botnets are active enough that some devices will be compromised within a few minutes after being connected to the Internet [344]. To be considered trustworthy, devices need to have a certain level of resistance to attacks. This is astonishing, because in essence many of the recurring security issues with smart devices have already been "solved" for many years. If vulnerabilities and corresponding attack situations could ultimately be avoided, it is important to ask who is responsible for the damage caused by compromised smart devices, beyond the malware author. The device owner may be legally responsible, but often the enduser does not have any means to detect or prevent such compromises, or to apply a secure configuration. On the other hand, the manufacturers currently often have no legal liability, and thus no incentive (e.g., economic, legal) to prevent a potential vulnerability and compromise.

# *11.1.4 Organization of This Chapter*