# A Neural Network–Based Energy Management System for PV–Battery Based
Microgrids

Yusuf Gupta, Mohammad Amin The authors are with the Department of Electric
Power Engineering, Norwegian University of Science and Technology, 7491
Trondheim, Norway (e-mail<EMAIL_ADDRESS>mohammad.amin@ntnu.no).

###### Abstract

A neural network-based energy management system (NN-EMS) has been proposed in
this paper for islanded ac microgrids fed by multiple PV–battery based
distributed generators (DG). The stochastic and unequal irradiation results in
unequal PV output, which causes an unequal state-of-charge (SoC) among the
batteries of the DGs. This effect may cause the difference in the SoCs to
increase considerably over time, leading to some batteries reaching their SoC
limits. These batteries would no longer be able to control the dc-link of the
hybrid grid forming DG. The proposed NN-EMS ensures SoC balancing by learning
an optimal state-action mapping using the outputs of an optimal power flow
(OPF). The training dataset has been generated by executing a mixed-integer
linear programming based OPF for droop-based island microgrids considering a
practical generation-load profile. The resultant NN-EMS controller inherits
the information of optimal states and the network behaviour. Compared to
traditional time-ahead centralized methods, the proposed strategy does not
require accurate generation-load forecasting. Further, it can also respond to
the variations in the PV power in near-real-time without resorting to solving
an OPF. The proposed NN-EMS controller has been validated by case studies on a
CIGRE LV microgrid containing PV-battery hybrid DGs. The proposed concept can
also be extended to synthesize decentralized controllers that can cooperate
among themselves to achieve a global objective without communication.

###### Index Terms:

Centralized control, distributed generation, droop control, energy management,
microgrid, neural network, PV-battery, state-of-charge.

##  I Introduction

A microgrid is an electrical network containing distributed energy resources
such as distributed generators, storage and loads. It can operate as a subset
of a larger electrical network or as an energy island. Grid forming DGs are
beneficial for operating island microgrids with significant penetration of
power-electronic interfaced energy sources. Hybrid DGs consisting of
photovoltaic (PV) arrays and battery packs can be controlled as grid forming
units, and they can also provide a range of power-energy services when the
microgrid operates as an island [1].

Hierarchical control is a popular architecture for the control of microgrids.
In this structure, the control objectives are segregated into different
control layers and implemented at different time scales. For instance, an
optimal energy management system (EMS) is executed in the tertiary control
layer every few minutes [2]. In contrast, the instantaneous power control and
proportional power sharing among the DGs are implemented in a decentralized
manner inside each DG.

The focus of this paper is on the optimal power flow (OPF) executed in an EMS
for island microgrids. Researchers have investigated OPFs for microgrids
extensively using centralized, distributed and decentralized EMS [3]. The OPF
can be programmed using linear or nonlinear formulations and solved using
conventional solvers, heuristics, or advanced learning assisted solvers [4].

The OPF can be formulated to improve a technical (e.g. voltage regulation),
economical (e.g. operating cost) or environmental objective. For microgrids
with hybrid PV-battery based DGs, balancing the state-of-charge (SoC) of the
batteries of all the DGs is vital for increased availability and reliability.
In the absence of appropriate SoC balancing schemes, some batteries may reach
their minimum or maximum SoC sooner than others. Furthermore, existing control
strategies which aim for proportional active power sharing may not be
appropriate for continuously operating the hybrid DGs in grid forming mode.
Thus, it is vital to investigate advanced control strategies for balancing SoC
among the batteries and maintaining acceptable power sharing.

Figure 1: Overview of the steps for obtaining NN-EMS

The control strategies and the EMSs available in the literature for island
microgrids can be broadly classified into decentralized [5, 6, 7, 8, 9, 10,
11, 12, 13, 14], distributed [15, 16, 17], centralized [18, 19, 20, 21, 22,
23]. The popular decentralized techniques are based on adaptive droop, online
perturbations in power, adaptive virtual impedance, and intelligent control
[14]. Most of these strategies apply to standalone energy storage or do not
consider the renewable energy and storage portion at all. Moreover, their
performance is far from optimal as compared to centralized controllers. In
contrast, the centralized or distributed control strategies can optimally
coordinate hybrid DGs, but they operate at a slower time scale. However, the
changes in PV irradiation occur at a much faster rate, often on a seconds time
scale, resulting in unequal SoC on a longer time horizon. Moreover, these
strategies are model-dependent and also depend on the accurate generation-load
forecast. Furthermore, they also require extensive measurements from all over
the network. Thus, there is a need to investigate advanced SoC balancing
strategies to address these limitations. These strategies should respond
faster, do not rely on up-to-date models or forecast information during
operation and do not impose heavy computational burden or measurements.
Artificial intelligence (AI) techniques based on machine learning (ML) and
neural networks (NN) are well suited for deriving these controllers.

AI-based tools are being explored for power systems applications [24]. The
reference [25] provides an overview of AI applications for microgrids in
particular. There have been some efforts in using AI tools to improve the OPF
formulation and reach optimal solutions quickly [4]. These OPFs or the
conventional OPFs can be employed for obtaining optimal operation datasets.
This dataset can then be used for obtaining a controller which can mimic the
performance of the OPF [23, 26, 27, 28, 29, 30, 31]. Inspired by this
philosophy, a NN based EMS (NN-EMS) has been proposed in this paper focusing
on the SoC balancing aspect.

The broader steps of the proposed approach have been depicted in Fig. 1. A
multi-input, multi-output (MIMO) central controller has been synthesized by
training a NN with a dataset obtained by executing a mixed-integer linear
programming (MILP) based OPF. The key contributions of this paper are as
follows:

  1. 1.

A NN-EMS has been proposed where NNs are employed to learn the nonlinear
mapping between the cause (irradiation changes, load changes) and the control
action [nominal frequency setpoints in the $P-f$ droop control] for balancing
the SoC in near-real-time. The trained NN based central controller does not
require heavy computations for predicting near-optimal control actions in
near-real-time.

  2. 2.

A MILP based OPF has been formulated for a droop controlled microgrid
containing hybrid PV-battery DGs. This OPF is executed considering real
generation-load profile and a dataset mapping the cause-action has been
obtained for training the NN-EMS.

  3. 3.

The OPF is executed such that the droop effect between the active power of the
DG and the network frequency is retained. This step is crucial for generating
a functional dataset for training.

  4. 4.

Finally, for increased availability and reliability, decentralized intelligent
controllers are also introduced, which can be useful if the central controller
becomes unavailable.

The rest of the paper is organized as follows. Section II describes the system
configuration. Section III presents the concept of the proposed NN-EMS
structure, the OPF formulation and the steps required in synthesizing the
centralized NN-EMS. Section IV contains the results and discussion, followed
by the conclusions.

##  II System Configuration

An island ac microgrid consisting of multiple DGs and loads, as shown in Fig.
2, has been considered in this paper. Each DG consists of an inverter fed by a
parallel combination of a PV array and a battery pack, as depicted in Fig. 3.
The PV array consists of series-parallel strings of PV panels connected to the
dc-link through a dc-dc converter. Similarly, the battery pack consists of a
series-parallel combination of battery modules connected to the dc-link
through a dc-dc converter.

The battery converter controls the dc-link voltage and the PV operates in
maximum power point tracking (MPPT). The inverter operates in grid forming
mode using the traditional $P-f$ and $Q-V$ droop controllers shown in (1) and
(2), respectively.

Figure 2: A typical island ac microgrid: CIGRE LV microgrid Figure 3: Power
and control block diagram of a hybrid DG

$f_{i}^{*}=f_{n-i}-m_{p-i}P_{inv-i}$ (1)
$v_{o-i}^{*}=V_{n-i}-n_{q-i}Q_{inv-i}$ (2)

where $f_{i}^{*}$ and $v_{o-i}^{*}$ are the reference frequency and voltage
magnitude, respectively for the ith DG’s inverter. $f_{n-i}$ and $V_{n-i}$ are
the nominal setpoints of the frequency and the voltage magnitude, $m_{p-i}$
and $n_{q-i}$ are the droop slopes, and $P_{inv-i}$ and $Q_{inv-i}$ are the
active and reactive power output of the inverter, respectively. The reference
voltage vector generated by the droop controller is tracked by the high-
bandwidth inner voltage-current control loops of the inverter.

In a hierarchical control structure, $V_{n}$ and $f_{n}$ are updated by a
centralized or a distributed controller at a slower time scale. The focus of
the proposed strategy is to infuse the knowledge of the network, generation-
load variations and the optimal operating conditions in a controller which can
mimic the performance obtained from a centralized OPF in near-real-time.
Although the proposed approach can be applied to a variety of objectives such
as power sharing and voltage control, the focus will be on the objective of
balancing the SoC of the batteries.

The physical phenomenon can be described as follows. In the absence of a
suitable SoC balancing strategy for the DGs and with fixed droop parameters,
unequal PV powers would cause a proportional charging/discharging of their
batteries, resulting in unequal SoC. However, by adaptively varying the
nominal frequency setpoint ($f_{n}$) of all the DGs, a relative difference in
PV powers can be transferred on the inverter side rather than changing the
battery power outputs. In this way, the batteries’ SoCs can be kept equal, and
they will only respond to a long term generation-load imbalance. Furthermore,
the DGs can keep operating in grid forming mode as long as the batteries have
sufficient capacity to control the dc bus. It is to be noted that there exists
a trade-off between SoC balancing and power sharing.

##  III Proposed NN-EMS

This section explains the overall control philosophy and then details the OPF
and the steps needed to synthesize the NN-based controller, which can predict
optimal settings of $f_{n}$ for SoC balancing. The OPF formulation is used to
obtain a dataset containing optimal operating points for various generation-
load scenarios. This dataset is subsequently used to train the NN for
synthesizing a centralized controller in an offline manner. In effect, an
intelligent mapping is obtained between the inputs, states and the control
variable.

###  III-A Control Philosophy

The SoC of the batteries are kept equal by transferring the differential
amount of the PV power of the DGs to the inverter output side rather than
charging or discharging the batteries unequally. Moreover, the difference
between the available generation and the load demand also affects the SoC over
time. Thus, an adaptive control of the inverter output powers is required,
which responds to the varying generation and demand.

It is to be noted that synthesizing such a nonlinear adaptive controller is
not trivial. In this paper, a NN has been trained to obtain an intelligent
centralized controller, as depicted in (3).

$f_{n-i}\;=\;F(P_{pv-1},\;P_{pv-2},\;...,\;P_{pv-n},\;f)$ (3)

The inputs of the NN are the PV output power ($P_{pv}$) of all the DGs and the
microgrid frequency ($f$). The outputs of the NN or the predicted quantities
are the nominal frequency setpoints ($f_{n}$) for all the DGs. Once the NN
learns the input-output relation, it can replace the traditional OPF to
predict the $f_{n}$, as shown in Fig. 4.

Figure 4: Working of the NN-EMS

The choice of $P_{pv}$ and $f$ as inputs for predicting $f_{n}$ is strategic.
Firstly, the SoC imbalance will happen whenever there is a difference in the
irradiation ($Irr$) incident on the PV of the DGs, and the inverters still
keep sharing active power as dictated by a fixed droop setting. Hence,
$P_{pv}$ is the most important input for the NN. Secondly, the load also keeps
varying and these variations have little or no correlation with $P_{pv}$
variations. However, the load variations are reflected in the instantaneous
frequency of the microgrid due to the use of the $P-f$ droop. Hence, $f$ is
taken as an additional input for training the NN.

As depicted in Fig. 4, the frequency measurement also acts as feedback to the
NN-EMS while the microgrid is operating. This feedback helps the NN-EMS
predict suitable $f_{n}$ setpoints and converge towards an equilibrium
iteratively. Although this phenomenon works well, as will be demonstrated in
the results section, the controller’s optimality, convergence and stability
guarantees are challenging to prove theoretically. These aspects are out of
the scope of this paper and can be investigated in future.

###  III-B OPF formulation

This subsection describes the key equations of the OPF, which will be used to
generate the training dataset for the NN-EMS. The MILP based OPF proposed in
[20] has been used as a base, and it has been enhanced for our application.
For brevity and ease of understanding, the formulation of [20] has not been
discussed in this paper. The readers can refer to it for the OPF portion
related to the network equations (KCL, KVL), load modelling, droop control,
inverter output power and line loss calculation.

The fixed inputs are the network parameters (line impedances and the rated
load values), PV and battery ratings, initial SoC and the droop control
parameters. The variable inputs for each iteration are the forecasted
irradiation and load demand. The outputs from the OPF for further use are
$P_{pv}$, $P_{bat}$, $P_{inv}$, $f$, $f_{n}$, and SoC values.

####  III-B1 PV

The PV is assumed to operate in MPPT. The MPPT is assumed to operate ideally
and track the MPP dictated by the $P_{pv}$ versus irradiation ($Irr$) curves.
These curves are usually available from the PV manufacturer’s datasheet. This
allows us to employ a simple linear relation, depicted in (4), to calculate
the output power from a PV panel for varying irradiation.

$P_{panel-i}\;=\;c_{1}Irr_{i}\;+\;c_{2}$ (4)

where $P_{panel-i}$ and $Irr_{i}$ are the PV panel output in W and the
irradiation in W/m2 for the ith DG. The coefficients $c_{1}$ and $c_{2}$ are
the slope and the y-intercept, respectively, of the fitted linear relation. It
should be noted that during night-time when $Irr$ is zero and if $c_{2}$ is
negative, then (4) will provide a negative value for $P_{panel}$, which is
impractical. To avoid this, the $P_{panel}$ value must be set to zero when
$Irr$ = 0. The total $P_{pv}$ value is then determined by (5).

$P_{pv-i}\;=\;P_{panel-i}N_{pv-i}$ (5)

where $N_{pv}$ is the number of panels in the PV array.

####  III-B2 Battery

The dc-dc converter and inverter are assumed to be lossless. Thus, the
instantaneous power balance inside the DG can be expressed as follows:

$P_{inv-i}\;=\;P_{bat-i}+\;P_{pv-i}$ (6)

where $P_{bat-i}$ is the battery power output for the ith DG. A positive value
of $P_{bat-i}$ indicates that the battery is discharging and vice versa. The
SoC of the battery pack varies with time depending on the generation-load
balance. The SoC percentage can be determined as follows:

$SoC_{i-k}=\frac{C_{bat-i-(k-1)}-\frac{P_{bat-i}}{V_{bat-i}}\Delta T}{C_{bat-
i-rating}}\times 100$ (7)

where $C_{bat-i-(k-1)}$ is the available charge in the battery at the end of
$(k-1)^{th}$ time step and $C_{bat-i-rating}$ is the rated charge capacity.

The optimization objective is to balance the SoC of all the DGs for each time
step. In other words, the objective is to minimize the absolute value of the
sum of $dSoC$, where $dSoC$ is given by (8).

$dSoC_{i-j-k}=SoC_{i-k}-SoC_{j-k}$ (8)

where $SoC_{i-k}$ and $SoC_{j-k}$ are the state of charge of the ith and jth
DG, respectively, for the kth time step. Thus, the objective is to minimize
$N_{DG}\times N_{DG}$ terms of $dSoC_{i-j}$ for each time step. The objective
function can be written as follows:

$\displaystyle minimize\;\sum dSoC_{i-j}$ (9)

####  III-B3 Inverter

The SoC of all the DGs can be maintained equal if $P_{bat}$ remains equal for
all the DGs. It is evident from (6) that for unequal values of $P_{pv}$ for
different DGs, their $P_{bat}$ can be maintained equally by adaptively
controlling the $P_{inv}$. For example, for an increase in $P_{pv}$ for a DG,
its $P_{inv}$ can be increased by a suitable amount compared to other DGs
whose $P_{pv}$ are lower. Similarly, the $P_{inv}$ can be decreased in
response to a decrease in $P_{pv}$. However, the change in $P_{pv}$ is not the
same for all the DGs. Moreover, the change in $P_{pv}$ is stochastic. An
offline OPF can consider such unequal and stochastic irradiation changes and
provide optimal setpoints required for $P_{inv}$ for all the DGs.

It should be noted that the inverters are assumed to operate in grid forming
mode. Hence, the required change in $P_{inv}$ can be effected by changing the
$f_{n}$ value appropriately. Thus, the SoC balancing can be achieved by
dynamically predicting and controlling the $f_{n}$ values in response to a
change in irradiation and load. However, the active power sharing would get
disturbed if the SoC needs to be balanced. Thus, a trade-off exists between
the SoC balancing and the active power sharing. An alternative is to constrain
the error in the active power sharing to a tolerable limit, $\epsilon$, inside
the OPF. For example, for equally rated DGs, the following constraint can be
included:

$P_{i-err-max}=|\frac{P_{inv-i}-P_{avg}}{P_{avg}}|\times 100\leq\epsilon$ (10)

where $P_{avg}$ is determined as follows.

$P_{avg}=\frac{P_{inv-1}+P_{inv-2}+...+P_{inv-N_{DG}}}{N_{DG}}$ (11)

Furthermore, the line losses, $P_{loss}$, may also worsen due to the SoC
equalization process. Hence, it must also be limited; for instance to 10%, as
follows:

$P_{loss}\leq 0.1\;\sum\;P_{load-rating}$ (12)

where $\sum\;P_{load-rating}$ is the sum of the rated loads of all the buses.

As will be demonstrated in Section IV, the $f_{n}$ limits considered while
executing the OPF play a vital role in obtaining an easy-to-fit dataset for
the NN. Instead of limiting the $f_{n}$ with a fixed lower and upper bound,
these limits are set depending on the total load forecast during each OPF time
step. This also helps preserve the droop effect between the active power and
the frequency.

###  III-C Controller Synthesis using NN

The intelligent central controller is synthesized by training the NN with the
optimal operation dataset obtained by executing the OPF for historical or
forecasted irradiation and load profile.

The dataset is randomly divided into training, validation and testing sets. A
two-layer feedforward NN has been used to map a relation between $N_{DG}+1$
input variables and $N_{DG}$ outputs, as shown in Fig. 5. Enough number of
neurons should be chosen for the hidden layer for the desired accuracy in
prediction and also to avoid over-fitting. The NN adjusts the weights and
biases of the hidden layer and output layer ($W_{h}$, $B_{h}$, $W_{o}$,
$B_{o}$) during training for obtaining better accuracy and generalization. The
fitting accuracy between the actual and the predicted outputs can be evaluated
by determining the mean square error (MSE) and the regression coefficient (R)
value.

Figure 5: Structure of the employed feed-forward NN for training

##  IV Results and Discussion

A modified CIGRE low voltage microgrid, as shown in Fig. 2, has been used as
the test system [32]. The rated voltage and frequency of the microgrid are 400
V ph-ph and 50 Hz. It consists of 18 buses, 17 lines, five loads ($L_{11}$ –
$L_{15}$) and three DGs (DG1 – DG3). The DGs are placed on buses 1, 14 and 18
for reduced power demand, according to [20]. They are considered equal in
rating. The rated load values have been scaled down to one-third compared to
[32]. The load values are included in Table I.

Table I: Load data

Bus no. | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
---|---|---|---|---|---|---|---|---|---
P (kW) | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
Q (kvar) | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
Bus no. | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18
P (kW) | 0 | 1.62 | 0 | 0 | 0 | 16.15 | 6.52 | 1.62 | 7.08
Q (kvar) | 0 | 1.00 | 0 | 0 | 0 | 10.01 | 4.04 | 1.00 | 4.39

Each DG is fed by a PV array and a battery pack which have been sized
appropriately considering the PV profile [33] and load pattern [34] of May
2020 for the city of Trondheim in Norway. The available data have been
converted from hourly values to 5-minute values by linear interpolation. To
introduce different variations in the irradiation of the DGs, random gains
between 1 and 1.14 have been multiplied to the PV pattern of [33]. The
resultant irradiation, $Irr_{1}$, $Irr_{2}$ and $Irr_{3}$ are shown in Fig.
11(a). The load profile is also demonstrated in Fig. 11(c), where the peak and
the minimum load are approximately 1.5 and 0.85 times the rated total load of
approximately 33 kW.

The PV panels and the dc-dc converter have been sized to provide 70 kWp power
and 650 V at the dc-link of the inverter. The PV panels from SunPower (model
no: SPR-305E-WHT-D) have been considered. The power output data of the PV
panel versus the input irradiation is included in Table II. The MATLAB
function polyfit has been used to fit the data from this table. The fitted
line is shown in Fig. 6.

Table II: $Irr$ vs $P_{panel}$ for a 305 W PV module

$Irr$ (W/m2) | 0 | 25 | 50 | 75 | 100 | 125 | 150
---|---|---|---|---|---|---|---
$P_{panel}$ (W) | 0.0 | 6.7 | 13.9 | 21.2 | 28.6 | 36.1 | 43.6
$Irr$ (W/m2) | 175 | 200 | 225 | 250 | 275 | 300 | 325
$P_{panel}$ (W) | 51.2 | 58.8 | 66.4 | 74.1 | 81.8 | 89.4 | 97.1
$Irr$ (W/m2) | 350 | 375 | 400 | 425 | 450 | 475 | 500
$P_{panel}$ (W) | 104.8 | 112.5 | 120.2 | 127.8 | 135.7 | 143.4 | 151.1
$Irr$ (W/m2) | 525 | 550 | 575 | 600 | 625 | 650 | 675
$P_{panel}$ (W) | 158.9 | 166.6 | 174.3 | 182 | 189.8 | 197.5 | 205.2
$Irr$ (W/m2) | 700 | 725 | 750 | 775 | 800 | 825 | 850
$P_{panel}$ (W) | 212.9 | 220.6 | 228.3 | 236.1 | 243.8 | 251.5 | 259.1
$Irr$ (W/m2) | 875 | 900 | 925 | 950 | 975 | 1000 |
$P_{panel}$ (W) | 266.8 | 274.5 | 282.1 | 289.9 | 297.6 | 305.2 |

Figure 6: Actual vs fitted values for $P_{panel}$

###  IV-A Data Collection

The critical parameters of the OPF and the NN-EMS are included in Table III.
The OPF is fed with the 5-minute interval data of irradiation and load. The
OPF has been written in MATLAB and solved using the IBM CPLEX solver.

Table III: OPF and NN-EMS parameters Parameter | Value | Unit
---|---|---
$N_{pv}$ | 8$\times$29 | –
$c_{1},c_{2}$ | 0.3072, $-$2.1944 | –
$V_{bat}$ | 550 | V
$SoC_{min}$, $SoC_{max}$ | 10, 90 | %
$dSoC_{min}$, $dSoC_{max}$ | 0, 10 | %
$P_{i-err-max}$ | 60 | %
$m_{p}$ | 0.005 | rad/s/kW
$n_{q}$ | 0.1 | V/kvar
$\epsilon$ | 60 | %
$V_{min}$, $V_{max}$ | 0.95, 1 | p.u.
$f_{min}$, $f_{max}$ | 0.95, 1 | p.u.
Figure 7: Droop effect lost Figure 8: Plot of $f_{n}$ vs $P_{pv1}$ and $f$: No
steps for $f_{n}$ limits

The OPF is first executed sequentially for the 31 days (31$\times$24$\times$12
$\rightarrow$ 8928 time steps) considering fixed bounds of 0.99–1.01 per unit
(p.u.) for $f_{n}$. The optimal values of $f$ versus $P_{inv}$ are plotted for
all the DGs in Fig. 7. It can be observed that all the obtained optimal values
of the frequency lie near the lower bound of 0.99 p.u. Thus, the droop effect
is lost since all the $f$ values are nearly the same irrespective of the
inverter output power. For better visualization, the optimal values of $f_{n}$
versus $P_{pv}$ and $f$ for DG1 are plotted in Fig. 8. It should be noted that
it is challenging to fit a relation of $f_{n}$ with $P_{pv}$ and $f$ using
such a dataset.

To preserve the droop effect and also to spread the $f_{n}$ values, the OPF
was rerun with $f_{n}$ constrained in steps according to the p.u. value of the
forecasted load. The $f_{n}$ upper and lower limits have been successively
decreased from 1.01 to 0.99 p.u. in 20 equal steps of 0.001 p.u. For example,
the $f_{n}$ limits are set to 1.010–1.009 p.u. if the load value is between 1
p.u. and 0.95 p.u., and it is set to 1.009–1.008 if the load value is between
0.95 p.u. and 0.90 p.u. and so on. The drooping effect is preserved, as
illustrated in Fig. 9. Furthermore, as seen in Fig. 10, the data points are
spread out, thus facilitating easier and more accurate fitting by the NN in
the next step.

Figure 9: Droop effect preserved Figure 10: Plot of $f_{n}$ vs $P_{pv1}$ and
$f$: 20 steps for $f_{n}$ limits

The dataset consisting of 8928 optimal steady-state operating points is saved
for the training and validation steps. As seen in Fig. 11(b), the maximum
$P_{pv}$ is about 80 kW, whereas the maximum and minimum $P_{bat}$ are 14.47
kW and –65.28 kW, respectively. The $P_{err-avg}$ and $P_{err-max}$ are found
to be 2.54% and 32.71%, respectively, over the whole month. As seen from the
SoC and $P_{bat}$ waveforms in Fig. 11(e) and Fig. 11(f), the OPF can provide
a perfect SoC balancing. It should also be noted that the batteries operate in
charging mode most of the time.

Figure 11: The inputs and outputs of the OPF for 8928 sequential runs of 5
minute time step Figure 12: Comparison of the NN-EMS (dotted lines) with
conventional droop (solid lines): Battery charging case Figure 13: Comparison
of the NN-EMS (dotted lines) with conventional droop (solid lines): Battery
discharging case

###  IV-B Training the NN based controller

The training has been performed using the neural network fitting application
in MATLAB 2020b. The dataset has been randomly divided into 80%, 10% and 10%
cases for training, validation, and testing. The Levenberg-Marquardt algorithm
with ten neurons has been selected for training. Since there are three DGs in
the test microgrid ($N_{DG}$ = 3), the NN-EMS must be trained for mapping four
inputs to three outputs. The MSE and R of the fit are close to 2e–6 and 1,
respectively.

###  IV-C Working of the trained NN-EMS

A Simulink model of the CIGRE microgrid has been constructed. A detailed model
of the DG has been implemented. The DG consists of PV plus boost converter
(MPPT), battery plus bidirectional converter (dc-link voltage control) and an
inverter with LCL filter, as shown in Fig. 3. The inverter control system
consists of a droop controller and an inner current-voltage-PWM controller.
The simulation runs at a time step of 2.5 $\mu$s.

The NN-EMS scans the $f$ and $P_{pv-1,2,3}$ and predicts $f_{n-1,2,3}$
instantaneously. The $f_{n-1,2,3}$ setpoints are then communicated to the
respective DG’s droop controller. The NN-EMS can mimic the performance of a
centralized OPF on a much faster time scale. Its capability to act faster is
crucial because PV variations happen on seconds scale. Furthermore, the NN-EMS
needs fewer measurements compared to a centralized OPF.

####  IV-C1 Battery charging scenario

The NN-EMS has been tested for a generation-load combination which entails the
batteries to operate in charging mode, as depicted in Fig. 12. The NN-EMS is
disabled at the start of the simulation, and the DGs run with conventional
droops with a fixed nominal frequency of 50.25 Hz till 6 s. The SoC of all the
batteries are assumed to be 14% approximately. The irradiation is set to 600
W/m2 at the start and the load levels are set to 1 p.u. The waveforms in Fig.
12 correspond to the microgrid’s response with fixed droop and the proposed
NN-EMS based adaptive droop. They have been represented by solid and dotted
patterns, respectively.

The irradiations are changed unequally at 4 s to 575 W/m2, 600 W/m2 and 550
W/m2 for $DG_{1,2,3}$, respectively. As seen in Fig. 12, the $P_{pv}$
waveforms are stable due to the proper working of the MPPT. The dc-link
voltage ($V_{dc}$) is also seen to be tightly controlled at 650 V by the
battery dc-dc converter. Thus, the power balance is working well at the dc-
link. The inverters’ active power sharing is equal initially due to the $P-f$
droop. It is to be noted that the $Q-V$ droop parameters have been kept
constant throughout the case studies since the focus of the work is not on
reactive power sharing.

It can be further observed from Fig. 12 that after 4 s, the difference in the
$P_{pv}$ of the DGs causes their $P_{bat}$ output to become unequal. This
triggers a divergence in the SoCs, as seen by the $SoC_{dif}$ waveforms.
$SoC_{dif}$ is the deviation of the SoCs from their mean value $SoC_{avg}$ as
shown in (13).

$SoC_{dif}\;=\;SoC_{i}\;-\;SoC_{avg}$ (13)

$SoC_{avg}$ is also plotted in Fig. 12. Ideally, all the $SoC_{dif}$ values
should converge to zero. However, this difference will increase over time
without a suitable SoC balancing strategy. The NN-EMS is enabled at 6 s, and
it starts predicting and communicating the $f_{n-1,2,3}$ setpoints to the DGs,
as shown in Fig. 12(b). The difference in $f_{n-1,2,3}$ setpoints causes the
DGs to produce unequal power, as seen from the dotted waveforms of Fig. 12(e).
Thus the DGs try to extract unequal power from their dc links. This, in turn,
forces their batteries to absorb unequal powers, as seen in Fig. 12(b). It can
be seen that the $P_{bat-1,2,3}$ converge towards an almost equal value. In
this way, the SoCs do not diverge from each other and remain almost equal, as
seen in Fig. 12(f). This validates the effectiveness of the proposed NN-EMS in
maintaining near-perfect SoC balance.

####  IV-C2 Battery discharging scenario

It can be observed from Fig. 11(f) that the training dataset majorly contains
charging scenarios. Thus, it is also essential to test the proposed technique
for the battery discharging scenario, as depicted in Fig. 13.

The DGs run with conventional droop with a fixed nominal frequency of 49.38 Hz
till 6 s. The SoC of all the batteries are assumed to be 31% approximately.
The irradiation is set to 150 W/m2 initially, and the load levels are set to
1.66 p.u. It should be noted that the NN-EMS is not trained for this load
level.

The irradiations are changed at 4 s to 150 W/m2, 140 W/m2 and 130 W/m2 for
$DG_{1,2,3}$, respectively. It can be observed from Fig. 13(a) and 13(c) that
due to the difference in the $P_{pv}$ of the DGs, the $P_{bat}$ becomes
unequal. This triggers a sharp divergence in the SoCs, as seen in Fig. 13(f).

The NN-EMS is enabled at 6 s, and it starts predicting and communicating the
$f_{n-1,2,3}$ setpoints to the DGs, as shown in Fig. 13(b). The difference in
$f_{n-1,2,3}$ setpoints causes the DGs to produce unequal power, as seen from
the dotted waveforms of Fig. 13(e). The DGs extract unequal power from their
dc links. This, in turn, forces the batteries to provide unequal powers, as
seen in Fig. 13(b). The $P_{bat-1,2,3}$ outputs converge to almost equal
value. In this way, the SoCs of the batteries remain almost equal, as seen in
Fig. 13(f). Thus, the proposed NN-EMS works well in discharging scenarios
also, although it is trained majorly for the charging scenarios.

It is crucial that the NN-EMS also provides accurate predictions even when
there is a sudden change in load. To test this, a load of 5 kW and 3 kVA is
added on bus 15 at 30 s. It can be observed that the $f_{n-1,2,3}$ predictions
smoothly move to a new operating point in response to the load change. The SoC
balance is still preserved, as seen from the near-zero values of $SoC_{dif}$.
This further validates the effective operation of the NN-EMS.

###  IV-D Extending the NN-EMS for decentralized implementation

Although the centralized NN-EMS provides excellent performance, it is prone to
communication delay or failure. For better reliability, the proposed control
philosophy can also be extended in future to obtain decentralized NN based
controllers for the DGs. For instance, each DG’s controller can consider its
$P_{pv}$ and the network frequency as the inputs for predicting its $f_{n}$.
Thus, the following function needs to be mapped for each DG:

$f_{n-i}\;=\;F(P_{pv-i},\;f)$ (14)

Figure 14: Decentralized implementation of a trained NN model Figure 15:
Comparison of decentralized NN based controllers (dotted lines) with
conventional droop (solid lines): Battery charging case

As shown in Fig. 14, the decentralized approach would facilitate a completely
communication-less operation since the predictions happen locally in each DG
using the locally available measurements. However, it is very challenging for
the classical NN techniques to accurately fit (14) for all the DGs and perform
well during operation.

The main problem is that the $f_{n}$ setpoints obtained from the OPF for any
given generation-load scenario lie very close to each other. The proximity of
the $f_{n}$ setpoints makes the NN prone to inaccurate predictions, which is
undesirable. To demonstrate this, decentralized controllers were synthesized
according to (14), and the performance was tested for the charging scenario,
as depicted in Fig. 15.

The MSE and the R obtained were close to 1e–5 and 1, respectively. Although
the fitting accuracy is excellent, the performance during operation is not as
expected. It can be seen from Fig. 15(f) that the SoC balancing does not
improve much with the decentralized controllers. In the future, advanced NN
techniques like deep learning can be explored to enhance the performance of
the decentralized approach.

##  V Conclusions

A neural network (NN) based centralized EMS has been proposed for microgrids
containing PV-battery hybrid DGs. The NN learns the optimal control actions to
be taken to maintain the SoC balance among the batteries of different DGs
while the irradiation and load changes occur. The NN-EMS has been trained
using optimal operating points obtained using an OPF. Thus, it inherits the
information of optimal states and the network behaviour. The SoC of the
batteries are maintained nearly equal by transferring the differential amount
of the PV power to the inverter output side rather than charging/discharging
the batteries unequally. This is achieved by adapting the $P-f$ droop
controller setpoints using the trained NN-EMS. The proposed concept has been
validated by case studies on a CIGRE LV microgrid and is shown to maintain
almost equal SoCs. The NN-EMS predicts near-optimal setpoints in real-time
without depending on an accurate generation-load forecast or solving an OPF.
Further, it also responds effectively to the variations in the PV power and
load changes. The proposed concept can also be thoroughly investigated in
future to synthesize completely decentralized controllers that can cooperate
among themselves to achieve a network-wide objective.

## References

  * [1] M. Al-Saadi, M. Al-Greer, and M. Short, “Strategies for controlling microgrid networks with energy storage systems: A review,” _Energies_ , vol. 14, no. 21, 2021.
  * [2] J. M. Raya-Armenta, N. Bazmohammadi, J. G. Avina-Cervantes, D. Sáez, J. C. Vasquez, and J. M. Guerrero, “Energy management system optimization in islanded microgrids: An overview and future trends,” _Renewable and Sustainable Energy Reviews_ , vol. 149, p. 111327, 2021.
  * [3] M. Z. Kreishan and A. F. Zobaa, “Optimal allocation and operation of droop-controlled islanded microgrids: A review,” _Energies_ , vol. 14, no. 15, 2021.
  * [4] G. Ruan, H. Zhong, G. Zhang, Y. He, X. Wang, and T. Pu, “Review of learning-assisted power system optimization,” _CSEE J. Power Energy Syst._ , vol. 7, no. 2, pp. 221–231, 2021.
  * [5] H. Mahmood, D. Michaelson, and J. Jiang, “A power management strategy for pv/battery hybrid systems in islanded microgrids,” _IEEE J. Emerg. Sel. Topics Power Electron._ , vol. 2, no. 4, pp. 870–882, 2014.
  * [6] Y. Karimi, H. Oraee, M. S. Golsorkhi, and J. M. Guerrero, “Decentralized method for load sharing and power management in a pv/battery hybrid source islanded microgrid,” _IEEE Trans. Power Electron._ , vol. 32, no. 5, pp. 3525–3535, 2017.
  * [7] T. A. Fagundes, G. H. F. Fuzato, C. R. De Aguiar, K. D. A. Ottoboni, M. Biczkowski, and R. Q. Machado, “Management and equalization of energy storage devices for dc microgrids using a soc-sharing function,” _IEEE Access_ , vol. 8, pp. 78 576–78 589, 2020.
  * [8] F. Chen, H. Deng, and Z. Shao, “Decentralised control method of battery energy storage systems for soc balancing and reactive power sharing,” _IET Generation, Transmission & Distribution_, vol. 14, no. 18, pp. 3702–3709, 2020\.
  * [9] X. Lu, K. Sun, J. M. Guerrero, J. C. Vasquez, and L. Huang, “State-of-charge balance using adaptive droop control for distributed energy storage systems in dc microgrid applications,” _IEEE Trans. Ind. Electron._ , vol. 61, no. 6, pp. 2804–2815, 2014.
  * [10] H. Mahmood, D. Michaelson, and J. Jiang, “Decentralized power management of a pv/battery hybrid unit in a droop-controlled islanded microgrid,” _IEEE Trans. Power Electron._ , vol. 30, no. 12, pp. 7215–7229, 2015.
  * [11] N. Vazquez, S. S. Yu, T. K. Chau, T. Fernando, and H. H.-C. Iu, “A fully decentralized adaptive droop optimization strategy for power loss minimization in microgrids with pv-bess,” _IEEE Trans. Energy Convers._ , vol. 34, no. 1, pp. 385–395, 2019.
  * [12] H. Mahmood and J. Jiang, “Autonomous coordination of multiple pv/battery hybrid units in islanded microgrids,” _IEEE Trans. Smart Grid_ , vol. 9, no. 6, pp. 6359–6368, 2018.
  * [13] H. Mahmood and F. Blaabjerg, “Autonomous power management of distributed energy storage systems in islanded microgrids,” _IEEE Trans. Sustain. Energy_ , pp. 1–1, 2022.
  * [14] T. A. Fagundes, G. H. F. Fuzato, P. G. B. Ferreira, M. Biczkowski, and R. Q. Machado, “Fuzzy controller for energy management and soc equalization in dc microgrids powered by fuel cell and energy storage units,” _IEEE J. Emerg. Sel. Topics Power Electron._ , vol. 3, no. 1, pp. 90–100, 2022.
  * [15] C. Li, E. A. A. Coelho, T. Dragicevic, J. M. Guerrero, and J. C. Vasquez, “Multiagent-based distributed state of charge balancing control for distributed energy storage units in ac microgrids,” _IEEE Trans. Ind. Appl._ , vol. 53, no. 3, pp. 2369–2381, 2017.
  * [16] D. Li, Z. Wu, B. Zhao, and L. Zhang, “An improved droop control for balancing state of charge of battery energy storage systems in ac microgrid,” _IEEE Access_ , vol. 8, pp. 71 917–71 929, 2020.
  * [17] D.-D. Zheng, S. S. Madani, and A. Karimi, “Data-driven distributed online learning control for islanded microgrids,” _IEEE J. Emerg. Sel. Topics Circuits Syst._ , pp. 1–1, 2022.
  * [18] D. Wu, F. Tang, T. Dragicevic, J. C. Vasquez, and J. M. Guerrero, “Autonomous active power control for islanded ac microgrids with photovoltaic generation and energy storage system,” _IEEE Trans. Energy Convers._ , vol. 29, no. 4, pp. 882–892, 2014.
  * [19] Y. Guan, J. C. Vasquez, and J. M. Guerrero, “Coordinated secondary control for balanced discharge rate of energy storage system in islanded ac microgrids,” _IEEE Trans. Ind. Appl._ , vol. 52, no. 6, pp. 5019–5028, 2016.
  * [20] Y. Gupta, S. Doolla, K. Chatterjee, and B. C. Pal, “Optimal dg allocation and volt–var dispatch for a droop-based microgrid,” _IEEE Trans. Smart Grid_ , vol. 12, no. 1, pp. 169–181, 2021.
  * [21] G. Agundis-Tinajero, N. L. D. Aldana, A. C. Luna, J. Segundo-Ramírez, N. Visairo-Cruz, J. M. Guerrero, and J. C. Vazquez, “Extended-optimal-power-flow-based hierarchical control for islanded ac microgrids,” _IEEE Trans. Power Electron._ , vol. 34, no. 1, pp. 840–848, 2019.
  * [22] F. Cingoz, A. Elrayyah, and Y. Sozer, “Plug-and-play nonlinear droop construction scheme to optimize islanded microgrid operations,” _IEEE Trans. Power Electron._ , vol. 32, no. 4, pp. 2743–2756, 2017.
  * [23] S. Gao, C. Xiang, M. Yu, K. T. Tan, and T. H. Lee, “Online optimal power scheduling of a microgrid via imitation learning,” _IEEE Trans. Smart Grid_ , vol. 13, no. 2, pp. 861–876, 2022.
  * [24] S. M. Miraftabzadeh, M. Longo, F. Foiadelli, M. Pasetti, and R. Igual, “Advances in the application of machine learning techniques for power system analytics: A survey,” _Energies_ , vol. 14, no. 16, 2021.
  * [25] R. Trivedi and S. Khadem, “Implementation of artificial intelligence techniques in microgrid control environment: Current progress and future scopes,” _Energy and AI_ , vol. 8, 2022.
  * [26] R. R. Kolluri and J. de Hoog, “Adaptive control using machine learning for distributed storage in microgrids,” in _Proc. 11th ACM Intl. Conf. Future Energy Syst._ , 2020, p. 509–515.
  * [27] C. Keerthisinghe, A. C. Chapman, and G. Verbič, “Energy management of pv-storage systems: Policy approximations using machine learning,” _IEEE Trans. Ind. Informat._ , vol. 15, no. 1, pp. 257–265, 2019.
  * [28] R. Dobbe, O. Sondermeijer, D. Fridovich-Keil, D. Arnold, D. Callaway, and C. Tomlin, “Toward distributed energy services: Decentralizing optimal power flow with machine learning,” _IEEE Trans. Smart Grid_ , vol. 11, no. 2, pp. 1296–1306, 2020.
  * [29] P. Chen, S. Liu, B. Chen, and L. Yu, “Multi-agent reinforcement learning for decentralized resilient secondary control of energy storage systems against dos attacks,” _IEEE Trans. Smart Grid_ , pp. 1–1, 2022.
  * [30] W. Dong, Q. Yang, W. Li, and A. Y. Zomaya, “Machine-learning-based real-time economic dispatch in islanding microgrids in a cloud-edge computing environment,” _IEEE Internet Things J._ , vol. 8, no. 17, pp. 13 703–13 711, 2021.
  * [31] S. Karagiannopoulos, P. Aristidou, and G. Hug, “Data-driven local control design for active distribution grids using off-line optimal power flow and machine learning techniques,” _IEEE Trans. Smart Grid_ , vol. 10, no. 6, pp. 6461–6471, 2019.
  * [32] K. Strunz, “Medium voltage distribution network benchmark,” in _Benchmark Systems for Network Integration of Renewable and Distributed Energy Resources_. CIGRE, 2014, ch. 6, p. 33.
  * [33] European Union Joint Research Centre, “JRC Photovoltaic Geographical Information System - PVGIS 5.2,” https://re.jrc.ec.europa.eu/pvg_tools/en/, [Online; accessed 15-March-2022].
  * [34] Statnett, “Production and Consumption Data,” https://www.statnett.no/for-aktorer-i-kraftbransjen/tall-og-data-fra-kraftsystemet/last-ned-grunndata/, [Online; accessed 15-March-2022].


11institutetext: VERIMAG
University of Grenoble Alpes (UGA), Grenoble, France 11email:
<EMAIL_ADDRESS>
# A tool assisted methodology to harden programs against multi-faults
injections

Etienne Boespflug and Abderrahmane Bouguern and Laurent Mounier and Marie-
Laure Potet

###### Abstract

Fault attacks consist in changing the program behavior by injecting faults at
run-time in order to break some expected security properties. Applications are
hardened against fault attack adding countermeasures. According to the state
of the art, applications must now be protected against _multi-fault injection_
[1, 2]. As a consequence developing applications which are robust becomes a
very challenging task, in particular because countermeasures can be also the
target of attacks [3, 4]. The aim of this paper is to propose an assisted
methodology for developers allowing to harden an application against multi-
fault attacks, addressing several aspects: how to identify which parts of the
code should be protected and how to choose the most appropriate
countermeasures, making the application more robust and avoiding useless
runtime checks?

###### Keywords:

multiple fault-injection; code analysis; software countermeasure; dynamic-
symbolic execution.

##  1 Introduction

Fault injection is a powerful attack vector, allowing to modify the code
and/or data of a software, going much beyond more traditional intruder models
relying “only” on code vulnerabilities and/or existing side channels to break
some expected security properties. This technique initially targets security
critical embedded systems, using physical disturbances (e.g., laser rays, or
electro-magnetic fields) to inject faults. However, it may now also concern
much larger software classes when considering recent hardware weaknesses like
the so-called Rowhammer attack [5, 6], or by exploiting some weaknesses in the
power management modules [7, 8, 9]. Furthermore, in the growing domain of IoT,
security is based on very sensitive operations such as boot-loading or Over-
the-Air firmware update which must be protected against fault injections[10,
11].

As a result, programs must be hardened against fault injection, combining
hardware and/or software _countermeasures_ aiming to detect runtime security
violations. According to the state of the art, applications must now be
protected against (spacial or temporal) _multi-fault injection_ [1, 2], namely
when several faults can be injected at various times or locations during an
execution. As a consequence developing applications which are robust becomes a
very challenging task and a cat-and-mouse game, in particular because
countermeasures can be also the target of attacks [3, 4]. The aim of this
paper is to propose an assisted methodology for developers allowing to harden
an application against multi-fault attacks, addressing several aspects: how to
identify which parts of the code should be protected and how to choose the
most appropriate protection schemes, making the application more robust and
avoiding useless runtime checks?

There exist tools adding countermeasures, generally at compilation-time. They
are dedicated to particular fault models (data modification, instruction skip,
flow integrity) [12, 13, 14], for instance by adding redundant checks or
duplicating idempotent instructions. Nevertheless these tools target single
fault robustness, where countermeasures themselves cannot be faulted.
Furthermore such countermeasures are added in a brute force approach, based on
(coarse-grained) directives given by the developers, indicating which parts of
the code must be protected. Such an approach is no longer realistic when
multi-faults must be taken into account, since more complex countermeasures
must be considered, potentially adding unnecessary performance/size costs.

The objective of this paper is to address the issue of assisting security
developers in the countermeasure insertion process. In particular, we provide
the following contributions:

  1. 1.

we formulate the problem of robustness comparison in presence of multi-faults;

  2. 2.

we propose a methodology to analyze countermeasures “in isolation”, both in
terms of classes of attacks they detect and considering their own attack
surface;

  3. 3.

we propose an algorithm allowing to harden applications starting from
identified vulnerabilities, depending on countermeasure properties;

  4. 4.

we provide an implementation of this approach, based on the Lazart tool [15]
and evaluate our approach on a benchmark of code examples.

Section 2 introduces multiple fault-injection and countermeasures through a
motivating example and presents the Lazart tool. Section 3 proposes
definitions for robustness evaluation and comparison dedicated to multi-
faults. Section 4 describes the proposed methodology for analyzing
countermeasures in term of their adequacy and weakness against established
fault models. Section 5 presents our countermeasure placement algorithms and
illustrates them on several examples. Finally, Section 6 discuss related
works, limitations of our solution and future directions.

##  2 Fault injection attacks

In this section we introduce fault injection attacks through a simple example
and the tool Lazart which is used to analyze robustness against fault attacks.
Then we propose a hardened version of our example illustrating classical
countermeasures and introduce our contributions.

###  2.1 A fragile byteArrayCompare version

Listing 1 is an excerpt of a byteArrayCompare function used in the verifyPIN
collection taken from the FISSC public benchmark[16]. Such a function compares
two PIN codes and returns true if they are equal. BOOL_TRUE and BOOL_FALSE are
robust encoding of boolean constants true and false111Classically 0xAA and
0x55 to be resistant to bit flip.. The security property we want to guarantee
is that result is true if only if the two PIN codes are equals
(a1[0..size-1]==a2[0..size-1].

⬇

1BOOL byteArrayCompare(UBYTE* a1, UBYTE* a2, UBYTE size){

2 int i;

3 int result = BOOL_TRUE;

4 for(i = 0; i < size; i++)

5 if(a1[i] != a2[i]) result = BOOL_FALSE;

6 return result; }

Listing 1: Fragile byteArrayCompare function

This example is insecure against fault injection consisting in inverting
control flow condition. To show that we use Lazart [15], a tool analyzing the
robustness of a software under multi-fault injections. It takes as input a C
code, a fault model, a security property and exhaustively find all attack
paths. Lazart relies on a 2-steps approach:

  * •

First, a _high order mutant_ is generated from the LLVM representation of the
program. This mutant statically encodes all the possible injected faults (as
symbolic boolean values) according to a chosen fault model

  * •

Then, a _dynamic-symbolic exploration_ , performed by Klee [17], produces a
representative path for all the successful attacks violating the security
property.

Various fault models are supported by Lazart targeting control flow or data
modifications (as illustrated in Section 4). Table 1 line V1 gives the results
supplied by Lazart for this first version when size is fixed to 4 assuming all
bytes of the two input arrays a1 and a2 are different. We consider here a
limit of 8 faults. All these attacks violate the security property.

Table 1: Lazart results for the byteArrayCompare versions Order | 0 fault | 1 fault | 2 faults | 3 faults | 4 faults | 5 faults | 6 faults | 7 faults | 8 faults
---|---|---|---|---|---|---|---|---|---
V1 | 0 | 1 | 1 | 1 | 2 | 0 | 0 | 0 | 0
V2 | 0 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 2

The 1-fault attack consists in inverting the loop condition $i<size$ (line 4
of Listing 1), stopping directly the array comparison. The 2-faults, 3-faults
and one of the 4-faults attack consist in inverting the test of line 5, one,
two or three times respectively, and then stopping the loop. The last 4-faults
attack consists in inverting four times the test of line 5 and exiting the
loop normally.

###  2.2 A robust byteArrayCompare version

We propose in Listing 2 a more robust version of the function
byteArrayCompare, adding classical redundant-check countermeasures: 1) a test
line 10 checking the exit value of the loop and 2) a systematic verification
of the result of condition (lines 6 and 8). Calls to the function atk_detected
stop the execution, signaling the detection of an abnormal behavior.

⬇

1BOOL protected_byteArrayCompare(UBYTE* a1, UBYTE* a2, UBYTE size){

2 int i;

3 int result = BOOL_TRUE;

4 for(i = 0; i < size; i++) {

5 if(a1[i] != a2[i]) {

6 if(a1[i] == a2[i]) atk_detected();

7 result = BOOL_FALSE;}

8 else if(a1[i] != a2[i]) atk_detected(); }

9 if(i != size) atk_detected();

10 return result; }

Listing 2: Example: Secure byteArrayCompare function

In a multi-fault context, the attacker is able to bypass countermeasures. For
instance here she can invert the extra tests (lines 6, 8 and 9). Results
supplied by Lazart for this new version (still for the test inversion fault
model) are given on Table 1, line V2. Therefore, the 1-fault attacks found in
the fragile function is no longer possible and it now requires two test
inversions: the loop condition and its associated check line 9. More generally
each previous attack now requires to double the number of injected faults. As
we can see, added countermeasures make our function more robust, but
unfortunately they also come with their own attack surface (here our encoding
of countermeasures is sensitive to test inversion).

###  2.3 Attack information supplied by Lazart

Attacks will be represented by sequences of fault occurrences consisting in
pairs (injection point, fault model). For instance the first attack of Table 1
is represented by < IP4(TI), IP9(TI)>, where IP4 is the fault injection point
of line 4 and TI denotes “test inversion”. From that we can compute how many
times a fault injection point occurs into successful attacks. Table 2 shows
results associated to Table 1. In particular we can observe that attacking
IP6, corresponding to an added countermeasure, never results in a successful
attack. As a consequence this countermeasure can be removed, without impacting
the robustness of our implementation (for the considered security property).
In the same way if we only consider order 2 attacks, countermeasures lines 5
and 8 are no longer useful. Other configurations where some countermeasures
can be safely removed can be found in [18].

Table 2: Lazart Hotspots analysis for Listing 2 IP line | 0 fault | 1 fault | 2 faults | 3 faults | 4 faults | 5 faults | 6 faults | 7 faults | 8 faults | Total
---|---|---|---|---|---|---|---|---|---|---
line 4 | 0 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 4
line 5 | 0 | 0 | 0 | 0 | 1 | 0 | 2 | 0 | 7 | 10
line 6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
line 8 | 0 | 0 | 0 | 0 | 1 | 0 | 2 | 0 | 7 | 10
line 9 | 0 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 4

In the context of multi-faults, some attacks can be considered as redundant.
An attack $b$ will be consider as redundant with respect to an attack $a$ iff
$a$ is a proper prefix of $b$, denoted by $a\leq b$, $\leq$ being a partial
order relation.

vspace*-1em

###### Definition 1

Minimal attacks

Let $E$ be a set of attacks. $Minimal(E)$ is the smallest subset of $E$ such
that every attack $b$ in $E$ is represented by a unique attack $a$ in $Min(E)$
such that $a\leq b$.

In order to give a more synthetic view Lazart can compute the subset of
minimal attacks and how many attacks they represent. For the example of
Listing 1 no attack is redundant, but considering at first minimal attacks
happens to be useful for larger examples [19]. Lazart supplies also others
results allowing to evaluate the coverage of the analysis, such as the number
of explored paths, if some timeout has been raised, etc. These information are
useful to assess the completeness of the analysis.

###  2.4 Our objectives and contributions

In the context of single fault, hardening a code generally follows a “try and
test” approach: countermeasures are added and robustness is checked through a
new analysis (as such a secure implementation is now expected to become robust
against single-fault attacks). Nevertheless this process is not worth
considering in a multi-fault context since it would introduce a very
detrimental overhead. The aim of this paper is to propose several placement
algorithms allowing to add countermeasures advisedly targeting to avoid
unnecessary code. To do that we propose a methodology for stating and
analyzing countermeasures “in isolation” in order to formally establish
properties such as the adequacy of a countermeasure w.r.t. a given fault model
and its inherent attack surface. Countermeasures we target are systematic
countermeasures detecting attacks, like test or load duplication, and, more
generally, systematic countermeasures allowing to detect errors in data or
control flows.

To the best of our knowledge, this approach is innovative. As pointed out
before, generally tools add countermeasures in a systematic way (e.g., stack
canaries are added by compilers to any functions satisfying some basic
syntactic criteria) without precisely taking into account which control
locations should be protected w.r.t. the security properties which have to be
enforced. This situation introduces potentially serious and unnecessary
overheads. On the other hand, methods has been proposed to prove hardened
programs [20, 21, 4], or to verify the efficiency of a given form of
countermeasure against attacker models [22, 23]. All these approaches are
developed in the context of single fault and adapted to a particular
countermeasure or fault models. On the contrary, the approach we consider here
is general, modular, able to cope with multi-fault attacks and it addresses
the problem of optimization of countermeasures placements. As a first step we
propose in the next section a framework to properly compare the robustness of
several versions of a same code in a multi-faults context.

##  3 Robustness metrics for multi-faults

In the context of certification for high levels of security [24] applications
are submitted to vulnerability analyses (the AVA class of Common Criteria),
conducted by experts team (as ITSEF laboratory). For instance rating physical
attacks for smart cards or similar devices is established by the JIL Hardware
Attacks Subgroup [25] on the based on a set of relevant factors: elapsed time,
expertise, knowledge of the target of evaluation, access to it, …

At the level of code-based simulation tools some metrics are used such as the
number of successful attacks, potentially weighted by the total number of
attack or the attack surface [26]. Nevertheless these metrics are not really
used in practice to compare robustness of applications: the try and test
approach turns out to be suffisant when single fault is considered. On the
contrary, in the context of multi-faults we have to formalize how
implementations can be compared, that we propose here. Furthermore we will use
our formalization to validate our methodology and algorithms showing that we
improve the robustness of applications in adding countermeasures.

###  3.1 Definition of multi-fault robustness

Generally simulation tools give the number of attacks for a given golden run
starting from a fixed initial state. A simulated trace $t$ is a finite
sequence of transitions corresponding to a nominal execution steps or faulted
steps, starting from an initial state. In the following $init(t)$ denotes the
initial state of $t$ and $fault(t)$ the number of faulted transitions into
$t$. For a success condition $S$ (generally the negation of a security
property), a set of fault models $m$, simulated traces $T$ of a program $P$
can be partitioned into the following way:

  * •

$T_{N}(m,S)$ : traces obtained under the nominal execution (without fault)

  * •

$T_{D}(m,S)$ : faulted traces that are detected by a countermeasure

  * •

$T_{S}(m,S)$ : successful attacks (verifying $S$ and not detected)

  * •

$T_{F}(m,S)$ : non detected faulted traces that do not verify $S$

###### Definition 2

Input vulnerability characterization

Let $P$ be a program, $i$ an initial state (including inputs), $m$ a set of
fault models and $S$ a successful condition. We define as
$Vuln_{\mu}(P,i,m,S)$ the attack function associated to the input $i$ defined
from the rank 0 to $\mu$:

$Vuln_{\mu}(P,i,m,S){\hat{=}}f\in 0..\mu\rightarrow\mathbb{N}\mid\ \forall
n\in 0..\mu~{}(f(n)=\\#\\{t\in T_{S}(m,S)\mid\ init(t)=i\ \land\
fault(t)=n\\}$

Definition 2 can be extended to a set of initial states $I$. For instance
Table 1 describes $Vuln_{\mu}(P,I,m,S)$ with $\mu=8$,
$I=\\{a2[0..3],a1[0..3],4)\mid\forall i\in 0..3~{}a1[i]{\not=}a2[i]\\}$,
$m=\\{Test\\_inversion\\}$ and $S$ be result==BOOL_TRUE.

###### Definition 3

Robustness level

If $I$ is the set of all initial states and $Vuln_{\mu}(P,i,m,S)(i)=0$ for all
$i\in\ 1..\mu$ and for all $i\in I$ we say that _$P$ is robust up to $\mu$
faults_.

###  3.2 Robustness comparison

Here we target to compare a program $P$ with an hardened version of $P$. For
instance we want to state that the secure version of function byteArrayCompare
is more secure than the initial version, up to order 8.

###### Definition 4

Robustness comparison

Let $P$ be a program, ${I}$ a set of initial states, $\mu$ the order to
consider, $m$ the set of fault models and $S$ the successful condition. Let
$P^{\prime}$ be an hardened version of $P$. Robustness comparison is defined
as follow:

$P^{\prime}\geq^{rob}_{\mu,{I},m,S}~{}P\ {\hat{=}}$ $\forall i\in\
{I}~{}\forall n\in\
1..\mu\sum_{j=1}^{n}Vuln_{\mu}(P^{\prime},i,m,S)(j)\leq\sum_{j=1}^{n}Vuln_{\mu}(P,i,m,S)(j)$

Our definition does not only compare the total number of attacks for a given
order but all intermediary sums. Thanks to this definition, several nice
properties hold as monotonicity ($P^{\prime}\geq^{rob}_{n+1,{I},m,S}\ P\
\Rightarrow P^{\prime}\geq^{rob}_{n,{I},m,S}\ P$ ) and transitivity
($P^{\prime\prime}\geq^{rob}_{\mu,{I},m,S}P^{\prime}\ \land\
P^{\prime}\geq^{rob}_{\mu,{I},m,S}P\ \Rightarrow
P^{\prime\prime}\geq^{rob}_{\mu,{I},m,S}P$).

###  3.3 Discussion

A classical approach to evaluate the robustness of a code against fault attack
is to count the number of faults issued from a given golden run, where both
the inputs and the initial states are fixed in advance. Several works [27, 28,
16] use metrics to compare protected program versions in this setting. The
robustness comparison relation we propose can be seen as a generalization of
these metrics, taking into account a set of initial states as well as multi-
faults and still offering good properties. Note that this comparison relation
is relevant only for two programs having the same nominal behaviors (i.e. the
same functionality without fault). Moreover, it is a partial relation.
Typically, two programs $P$ and $P^{\prime}$ can become incomparable up to
$n+1$ faults if $P^{\prime}$ is more robust than $P$ up to $n$ faults and, for
$n+1$ faults, the sum of the attacks of program $P^{\prime}$ is greater than
the one of $P$. Finally, thanks to our symbolic execution based approach,
computing this relation needs to consider in practice only one representative
attack per execution path (even potentially reduced to minimal attacks as
defined in definition 1) hence allowing to reduce metrics explosion due to the
added countermeasures (according to the dilution paradox [26]).

##  4 Countermeasures analysis

In this section, we propose a systematic approach for analyzing
countermeasures " in isolation " in terms of classes of attacks they detect.
Combined with their proper attack surface (section 5.1) we are able to
construct what we called the "countermeasure catalog", used later to choose
adapted countermeasures for hardening implementations. Our approach is based
on the encoding of several generic schemes, described by C codes, associated
with their pre and postconditions. We use the weakest-precondition WP plugin
of Frama-C and the ACSL annotation language [29].

  1. 1.

Mutation schemes will characterize a fault model as how a fault impacts the
behavior of sensible instructions. Postcondition states the nominal behavior
of the instruction as well as the behavior is impacted by a fault.

  2. 2.

Countermeasure schemes will describe how abnormal behavior will be detected.
Postcondition characterized behaviors that are not blocked.

  3. 3.

Protected schemes will describe how countermeasures are inserted w.r.t.
mutation schemes. The expected postcondition is the nominal behavior
associated to the mutation scheme (all faults provoking a variation of the
nominal behavior must be blocked).

We will illustrate our approach on three fault models: test inversion, data
load modification (when we read a memory value) and Else-following-Then. These
fault models are very classical ones, in particular the Else-following-Then
model describes when the jump at the end of the then block is skipped (or
transformed into a nop operation). We also consider three countermeasures:
test duplication, load duplication and a lightweight control flow integrity
based on block signature.

###  4.1 Mutation schemes

We consider a fault model as a set of mutation schemes describing how the code
is transformed under a fault injection. The fault parameter represent the
presence of a fault (0 if no fault, different from 0 otherwise). The
postcondition describes the expected nominal behavior (when fault equals 0) as
well as the result of a fault. Thanks to the ACSL language we can clearly
distinguish nominal and faulty behaviors. Listing 3 describes the mutation
scheme for test inversion, when inverting the result of a conditional jump.
Other fault models are given in Appendix 0.A.

⬇

1/*@ assigns \nothing ;

2 @ behavior nominal :

3 @ assumes fault==0 ;

4 @ ensures C!=0 ==> \result==br_then ;

5 @ ensures C==0 ==> \result==br_else ;

6 @ behavior faulted :

7 @ assumes fault!=0 ;

8 @ ensures C!=0 ==> \result==br_else ;

9 @ ensures C==0 ==> \result==br_then ; @*/

10int mutation_ti(int C, int br_then, int br_else, int fault)

11 { if((C && !fault) || (!C && fault)){ return br_then; }

12 else { return br_else; } }

Listing 3: Test inversion mutation scheme

All fault schemes (listings 3 and 7, 8 in Appendix 0.A) are proved by the
plugin Frama-C/WP.

###  4.2 Countermeasure schemes

Countermeasures calls a detection function (atk_detected) which stops the
execution. Listing 4 describes the detection function and a countermeasure
duplicating the condition. Listing 5 describes a more sophisticated
countermeasure, with a proper variable representing which block must be
reached. Function sign records the name of the next block and function check
verifies that the reached block corresponds to the expected one.

⬇

1/*@ ensures \false;

2 @ assigns \nothing ;@*/

3void atk_detected() { exit(0); }

4

5/*@ ensures C!=0;

6 @ assigns \nothing ; @*/

7void TestDup (int C) { if(!C) atk_detected(); }

Listing 4: Test duplication countermeasure

⬇

1int RTS;

2

3/*@ ensures ((C!=0 ==> RTS==id_true)) ;

4 @ ensures ((C==0 ==> RTS==id_false)) ;

5 @ assigns RTS; @*/

6void sign(int C, int id_true, int id_false)

7{ if(C) RTS = id_true; else RTS = id_false; }

8

9/*@ ensures (RTS==id) ;

10 @ assigns \nothing; @*/

11void check(int id) { if(RTS!=id) atk_detected(); }

Listing 5: Signature based countermeasure

Function atk_detected never normally terminated. Each countermeasure targets
to block an erroneous control flow. The postcondition describes the final
state ensured by the countermeasure. All these functions are proved by the
WP/Frama-C plugin, as well as the load duplication countermeasure given in
Appendix 0.A.

###  4.3 Protected schemes

The last step consists to prove the adequacy of a countermeasure against a
fault model. To do that we combine countermeasures with a mutation scheme
targeting to block abnormal behaviors introduced by the mutation. Listing 6
shows how a condition can be protected by test duplication against test
inversion. The postcondition is build in a systematic way stating that we
terminate normally with the nominal behavior of the mutation scheme (the
nominal behavior of Listing 3 for our example below). That means that no fault
occurs or a faults occurs but without impact on the nominal behavior.

⬇

1/*@ behavior nominal :

2 @ ensures ((C!=0 ==> \result==br_then)) ;

3 @ ensures ((C==0 ==> \result==br_else)) ;

4 @ assigns \nothing ; @*/

5

6int protected_TI_DT(int C, int br_then, int br_else, int fault){

7 if((C && !fault) || (!C && fault))

8 { TestDup(!C); return br_then; }

9 else { TestDup(C); return br_else; } }

Listing 6: The insertion of the TD CM in the TI mutation scheme

Listings in Appendix 0.A section 0.A.3 respectively give protected versions
for the load modification protected by load duplication (listing 9) and for
the test inversion and the Else-following-Then fault models protected by the
Signature countermeasure (listings 11 and 12). All these protected versions
(including listing 6) are proved by the WP/Frama-C plugin.

Contrary to [4] we do not prove the reduced postcondition fault==0 but we
impose to preserve the nominal behavior of the instruction impacted by a
fault. Our approach is more general: for instance we admit a fault that does
not impact the nominal behavior, as an attack by test inversion when the two
targeted blocks are identical or the data load modification when we inject the
right value. On the contrary, for listing 13, we can strengthen the
postcondition with the clause ensures fault==0.

###  4.4 Results and discussions

###### Proposition 1

Adequate countermeasure.

Let $m$ be a fault model characterized by a mutation scheme $MS$ with a
nominal behavior NB. A protected scheme PS (embedding a countermeasure) is
adequate for $m$ if and only if PS verifies the postcondition $NB$. Then,
according to definition 3 we can state that PS is robust up to order 1, for
the fault model $m$, for all $I$ verifying the precondition of PS and for the
success condition $\lnot NB$.

Table 3: Adequacy of countermeasures against fault models Fault model Countermeasure | Test duplication | Load duplication | Block signature
---|---|---|---
Test inversion | adequate | - | adequate
Else following Then | KO | - | adequate
Data load modification | - | adequate | -

Table 3 describes the result of proofs of protected programs we developed.
They are all proved by the WP/FRama-C plugin (adapted) except Listing 13 that
corresponds to an erroneous countermeasure (KO). An empty case means that it
does not make sense to consider this combination of fault model and
countermeasure. Due to the modular proofs in WP/Frama-C we can replace a
sensitive instruction (as a test, a load or and then-else structure) by one of
its protected versions without modifying its nominal behavior. For instance we
can replace the instruction if(C) return br_then; else return br_else; by
if(C) {TestDup(!C); return br_then;} else {TestDup(C); return br_else;}.

To conclude, the notion of adequacy introduced in Proposition 1 tells us
whether a countermeasure protects a sensitive code pattern against a single-
fault attack model. The proposed methodology is based on the preservation of
the nominal behavior, that can be carefully specified (the semantics of the
code pattern as well as the assigns clause that allows to embed our protected
scheme in a larger code without unpredictable side effects). On the contrary
the impact of a fault in mutation scheme is not really useful here but can be
exploited later if we want to establish some properties in the presence of
faults.

##  5 Our methodology for hardening program

Starting from a set of attacks, the aim is to propose a methodology for
assisting countermeasures placement in the more appropriate way. In the
context of multi-faults, countermeasures come with their proper attack surface
(the code which is added), and fault injections on this code can produce new
execution traces, potentially producing new successful attacks. Because our
objective is to propose an alternative to the unrealistic brute force "try and
test" approach when multi-faults is considered we want to study the weakness
of countermeasures in a modular way. The next section explains how this
analysis can be conducted.

###  5.1 Countermeasure protection level

We define (and compute) the protection level of a countermeasure, which
characterizes the minimal number of faults allowing to bypass a
countermeasure. Bypassing a countermeasure means the countermeasure terminates
normally (without call to the function atk_detect) but violating its
postcondition (i.e. states that must be normally blocked).

###### Definition 5

Protection level

Let $CM$ be a countermeasure scheme defined by its postcondition $R$ and $M$ a
set of fault models. Let $pl(CM,M)=l$ the level of protection of $CM$ w.r.t.
$M$ defined as:

Condition 1. l is the minimal number of necessary faults producing an attack
violating $R$: $l=min\\{(fault(t)~{}|~{}t\in T_{S}(M,\lnot R)\\}$ (or $\infty$
if no successful attack exists).

Condition 2. any traces t such that 0<fault(t)<l are blocked : $\forall
t~{}(0<fault(t)<l\Rightarrow t\in T_{D}(M,\lnot R))$

Sets $T_{S}(M,\lnot R)$ and $T_{D}(M,\lnot R)$, introduced section 3,
respectively represent the set of successful attacks w.r.t. $\lnot R$ and the
set of detected attacks (with a call to function atk_detected). Our tool
Lazart allows us to establish the protection level. First we compute
$T_{S}(M,\lnot R)$ starting from order 1 to a given bound $max$ until an
attack is found. On the contrary Lazart does not directly compute
$T_{D}(M,\lnot R)$. Then we will compute $T_{S}(M,R)$, the set of attacks
verifying $R$ and not detected, that must be empty for 1 to $l-1$ faults
(implying that all attacks are detected). Table 4 shows the protection level
of our different countermeasures (listings 4, 5, 9) for the two fault models
test inversion and data load modification and their combination (denoted Comb1
here).

Table 4: Protection levels of countermeasure schemes Countermeasure Fault model | Test inversion | Load modification | Comb1
---|---|---|---
Test duplication | 1 | 1 | 1
Load duplication | 1 | 1 | 1
Block signature | 1 | 1 | 1
$i$ Test duplication | $i$ | 1 | $i$
$i$ Load duplication | $i$ | $i$ | $i$
$i$ Block signature | $i$ | $i$ | $i$

Because all single fault models produce a protection level of 1 it is not
mandatory to compute the protection level for the combination (combinations of
models can sometimes allow to produce more efficient successful attacks, but
it is not the case for 1). We extend our protection level when $i$ new
instances of countermeasures are considered, because duplicating
countermeasures preserves the adequacy of protection scheme222It is also
possible to redefined new protection schemes combining or duplicating
countermeasures and prove their adequacy as before..

Discussion/limitation: Protection level is a partial function: it is not
always possible to compute a protection level when the second condition does
not hold. That means that this countermeasure against the considered fault
models can not be analyzed in a modular way. For instance if we consider a
fault model jumping anywhere in the code our countermeasures do not fulfill
this condition. In this case countermeasures must embed local verification
(for instance on the the value of the program counter).

###  5.2 Our methodology to harden programs

The approach we propose allows to harden a program $P$ up to $n$ faults, given
as a set of fault models $M$ and a success condition $S$. It relies on a
catalog $C$ of available countermeasures associated to the model for which
they are adequate and their protection levels with respect to the fault models
in $M$. This approach is based on the following steps:

  1. 1.

Run Lazart to get the set of faults attacks $T_{S}(M,S)$ up to $n$. These
attacks characterize the robustness of $P$ with respect to $M$ and $S$.

  2. 2.

Compute the subset $A$ of minimal attacks (definition 1) and the set $IP$ of
their associated injection points (section 2.3). $IP$ gives a superset of the
injection points to be protected in order to harden $P$.

  3. 3.

Select both a subset $IP^{\prime}\subseteq IP$ of injection points to protect,
together with the corresponding countermeasures available in $C$.

  4. 4.

Build a hardened program $P^{\prime}$ obtained by protecting each element of
$IP^{\prime}$ using the results of step 2.

The question is now to decide whether $P^{\prime}$ is robust by construction
up to $n$ faults. Two sufficient conditions are given below (propositions 2
and 3) leading to the algorithm proposed in Section 5.3.

###### Proposition 2

Let $A$ a set of attacks obtained on a program $P$ with respect to fault
models $M$ and a security objective $S$. Let $P^{\prime}$ obtained from $P$ by
protecting a single $ip$ of each attack trace $a$ of $A$ with an adequate
countermeasure of protection level $pl>n$. Then $P^{\prime}$ contains no
longer any attack of $A$.

To prove this proposition let us consider a $k$-faults attack
$a=[ip_{1},ip_{2},\cdots,ip_{k}]$ of $A$. According to Section 5.1, faulting
$ip_{i}$ in $P^{\prime}$ requires now $pl+1$ faults, hence each attack of $A$
does no longer succeed in less than $n$ faults. Note that this proposition
(trivially) holds when the countermeasures cannot be attacked (e.g, they are
implemented using tamper-resistant hardware protections), since their
protection levels can be considered as infinite.

###### Proposition 3

Thanks to Definition 1 on minimal attacks, if we protect every minimal attacks
up to $n$, all attacks in $T_{S}(M,S)$ will be protected up to $n$.

###  5.3 Countermeasure placement algorithm

Algorithm 1 aims to protect one $ip$ per execution trace using a
countermeasure with a maximal available protection level. It iterates over the
attacks of $A$ by increasing order of faults injected. For each attack $a$, it
builds first the set $IP$ of injection points of $a$ with an adequate
countermeasure $c$ with a minimal $l$ greater than $n$. Then, it selects a
candidate $ip$ of $IP$ with the most occurrences in the set of remaining
attacks. If $pl$ is equal to $n$ then attack $a$ becomes protected, together
with a maximal number of other attacks sharing this injection point. Otherwise
$a$ is left unprotected. We denote by $(c,l)\in C(ip)$ a countermeasure and
protection level of $C$ of fault model related to the injection point $ip$
(encoded into IP representation as exemplified in Section 2.3).

Algorithm 1 Heuristic-based hardening algorithm

a program $P$, a set of attack traces $A$ of $P$ up to a $n$ faults, a catalog
$C$

$Protected\leftarrow\emptyset$ $\triangleright$ set of protected traces of $A$

$IpProtected\leftarrow\emptyset$ $\triangleright$ set of protected injection
points of $A$

for k in $1$ to $n$ do

$A_{k}\leftarrow$ attacks of $A$ with $k$ faults

for $a$ in $A_{k}\setminus Protected$ do $\triangleright$ for all not yet
protected attacks

$IP\leftarrow\\{ip\in a\mid\exists(c,l)\in C(ip)~{}\mbox{s.t. $c$ is adequate
for $ip$ and}~{}l\geq n\\}$

if $IP\neq\emptyset$ then $\triangleright$ keep the ips of $IP$ with a minimal
protection level $\geq n$

$IP\leftarrow\\{ip\in IP\mid\exists(c,l)\in C(ip)~{}\mbox{s.t.}~{}\forall
ip^{\prime}\in a.\;\forall(c^{\prime},l^{\prime})\in
C(ip^{\prime}).\;l^{\prime}>l\\}$

else$\triangleright$ take the ips of $a$ with a maximal protection level

$IP\leftarrow\\{ip\in a\mid\exists(c,l)\in C(ip)~{}\mbox{s.t.}~{}\forall
ip^{\prime}\in a.\;\forall(c^{\prime},l^{\prime})\in
C(ip^{\prime}).\;l^{\prime}<l\\}$

end if

choose $ip\in IP$ with a maximal number of occurrences in $A\setminus
Protected$

if $pl+1>n$ then

$IpProtected\leftarrow IpProtected\cup\\{(ip,c)\\}$ $\triangleright$ $ip$ will
be protected with $c$

$Protected\leftarrow Protected\cup\\{a^{\prime}\in A\mid ip\in a^{\prime}\\}$

end if

end for

end for

$P^{\prime}\leftarrow P$

for all $(ip,c)$ in $IpProtected$ do

$P^{\prime}\leftarrow$ protect $ip$ of $P^{\prime}$ with $c$

end for

return $(P^{\prime},Protected)$

###  5.4 Formal results

We can now state results on Algorithm 1 based on Propositions 2 and 3.

###### Proposition 4

Assurances supplied by algorithm 1

  1. 1.

Algorithm1 terminates.

  2. 2.

We have $P^{\prime}\geq^{rob}_{n,{I},m,S}~{}P$ (attacks in $Protected$ or with
a prefix in $Protected$ are no longer successful attack up to $n$ and no new
attack is introduced)

Leveraging the result obtained from $A$ to $P$ needs $A$ to be representative
of $P$ (i.e., each possible concrete attack of $P$ is represented by a path in
the set $T_{S}(M,S)$). In our case this means that the path enumeration
performed by Lazart to produce $A$ is complete. Under this hypothesis, if
$Protected=A$, then $P^{\prime}$ is robust up to $n$ faults.

A degraded version. If, for some $ip$, $C$ does not contain an adequate
countermeasure with a protection level $pl$ greater than $n-1$ proposition 4
still holds when replacing $n$ by $x$ faults, where $x$ is the minimal
protection level used to protect traces of $A\setminus Protected$.

In the general case we cannot say anything beyond $x$ faults, because
injecting more than $x$ faults on a countermeasure may lead to new execution
states and hence new attacks in $P^{\prime}$. In this case $P^{\prime}$ should
be hardened as well by repeating the same procedure. This iterative process
terminates either if we get rid of all attacks with strictly less than $n$
faults, or if some $k$-faults attacks (for $k\leq n$) will always remain
unprotected due to a lack of available countermeasure in $C$. Furthermore, for
particular fault models (as test inversion and data load modification by any
value) we could be able to show that fault injections on the protected scheme
does not introduce new paths (showing that successful attacks for the level of
protection does not produce states that are not covered by the associated
mutation scheme). Then, if $A$ is representative of $P$, deviant paths will be
already analyzed333It is not the case for instance if our data load
modification is reduced to increment the value which is read..

###  5.5 Experimentations

In this section, we give the results obtained with various countermeasure
placement algorithms. We consider the test inversion and data load mutation
fault models, with their combination, and the $i$ Test duplication and $i$
Load duplication countermeasures. Two programs are used for experimentations:
VP the version 4 of the verifypin program in the FISSC benchmark [16] and FU
an implementation of a firmware updater. VP is a generalization of our
introductive examples (section 2) and all these programs are already be used
for large experimentations [18] and are freely available444lazart.gricad-
pages.univ-grenoble-alpes.fr/home/fdtc20/index.html.

Our results are summarized in Table 5. Columns $PM$ gives the program to
protect and the fault models to consider. Column IPs gives the total number of
injection points of $P$ with respect to $m$. We consider three placement
algorithms:

  * •

$Naive$: protection of level $n$ for all injection points of $P$.

  * •

$All$: protection of level $n$ for all injection points involved in at least
one successful attack

  * •

$Single$: protection of a single injection point at level $n$ per sucessful
attack (Algorithm 1).

Column “#added-cms” indicates the number of countermeasures added, for each
algorithm, up to $n=4$ faults. As expected, Algorithm 1 gives much better
results than the two other ones. In particular the $Naive$ one, used by all
tools adding countemeasures in a systematic way (disregarding actual attacks)
is clearly outperformed. Moreover, Algorithm 1 significatively reduces the
number of added countermeasures, in particular on the FU example555still
considering a relevant security property.

Table 5: Countermeasures added by placement algorithms | algorithm | #added-cms
---|---|---
PM | IPs |  | 1 faults | 2 faults | 3 faults | 4 faults
VP | 8 | Naive | 8 | 16 | 24 | 32
with test inversion |  | All | 3 | 8 | 12 | 16
|  | Single | 3 | 6 | 9 | 12
FU | 42 | Naive | 42 | 84 | 126 | 168
with Test inversion |  | All | 0 | 28 | 42 | 88
|  | Single | 0 | 14 | 21 | 28
FU | 2 | Naive | 2 | 4 | 6 | 8
with data load mutation |  | All | 1 | 4 | 6 | 8
|  | Single | 1 | 2 | 3 | 4
FU | 44 | Naive | 44 | 88 | 132 | 176
with test inversion |  | All | 1 | 32 | 60 | 96
\+ data load mutation |  | Single | 1 | 16 | 24 | 32

##  6 Conclusion

In this paper we propose a global methodology assisting developer for
hardening applications against multi-fault attacks, which is nowadays the
state of the art [1, 2]. Generalizing the existing single fault hardening
approach is far from easy due to the he attack surface introduced by
countermeasures, potentially introducing new attacks and new paths to explore.
In order to master this inherent complexity we propose an innovative approach
based on two properties: the adequacy of a countermeasure against a given
fault model and the protection level, characterizing it weakness against a set
of fault models (number of faults greater than the protection level could
produce out of control behaviors). The proposed approach is limited to
countermeasures with a local detection power, as for all automatic tools
adding countermeasures in the single fault context.

At first we give a formal framework for specifying and proving fault models,
countermeasures and protected schemes to establish the adequacy property. The
key is the specification of the nominal behavior of instructions, how it can
be impacted by faults and how deviations can be detected. Our work can be seen
as a generalization of the approach described in [4], also based on the
Frama-C/WP plugin, as explained in Section 4.3. Formally establishing the
robustness of a countermeasure scheme against an attacker model is not new. In
the context of Control flow integrity many countermeasures have been proposed
and proved, as in [30]. In the context of fault injection, formal methods have
been used to establish the effectiveness of countermeasures [22, 23]. But
these works are dedicated to particular form of countermeasures and specific
fault models, and they address single faults only.

Based on the new notion of protection level we propose algorithms allowing to
automatically insert countermeasures, going beyond automatic algorithms not
taking into consideration successful attacks (the naive approach). Furthermore
we are able to characterize the level of assurance of the protected programs
(property 4). Nevertheless, computing which injection points must be
protected, w.r.t. successful properties requires a first analysis giving some
exhaustive insurances. Here we based this analysis on an symbolic execution
engine (Klee [17] and Lazart [15]), which is sensitive to the code complexity
(number of paths and number of fault injections). We are currently exploring
several ways for mastering this complexity based on static analysis [19] and a
smart encoding of faults [31], which could be combined with the approach
proposed here.

As pointed out, a such generic framework is new and must be more largely
evaluated, considering existing fault models and countermeasures, together
with more sophisticated countermeasures and more flexible placement
algorithms. In particular we are studying an algorithm, based on Integer
Linear Programming, allowing to combine several countermeasures per attacks,
with lower protection levels, still preserving our robustness properties.

## Acknowledgment

This work is supported by SECURIOT-2-AAP FUI 23 and by the French National
Research Agency in the framework of the "Investissements d’avenir” program
(ANR-15-IDEX-02) and the project ARSENE of PEPR Cybersecurité.

## References

  * [1] C. H. Kim and J. Quisquater, “Fault attacks for CRT based RSA: new attacks, new results, and new countermeasures,” in _Information Security Theory and Practices. WISTP 2007, Heraklion, Greece, 2007, Proceedings_ , 2007, pp. 215–228.
  * [2] E. Trichina and R. Korkikyan, “Multi fault laser attacks on protected CRT-RSA,” in _2010 Workshop on Fault Diagnosis and Tolerance in Cryptography, FDTC 2010, Santa Barbara, California, USA, 21 August 2010_ , 2010, pp. 75–86.
  * [3] ANSSI, Amossys, EDSI, LETI, Lexfo, Oppida, Quarkslab, SERMA, Synacktiv, Thales, and T. Labs, “Inter-cesti: Methodological and technical feedbacks on hardware devices evaluations,” in _SSTIC 2020, Symposium sur la sécurité des technologies de l’information et des communications_ , 2020.
  * [4] T. Martin, N. Kosmatov, and V. Prevosto, “Verifying redundant-check based countermeasures: a case study,” in _SAC ’22: The 37th ACM/SIGAPP Symposium on Applied Computing_. ACM, 2022.
  * [5] M. Seaborn and T. Dullien, “Exploiting the DRAM Rowhammer bug to gain kernel privileges: how to cause and exploit single bit errors,” in _Black Hat_ , 2015.
  * [6] D. Gruss, C. Maurice, and S. Mangard, “Rowhammer.js: A remote software-induced fault attack in javascript,” in _DIMVA 2016, San Sebastiá_ , 2016, pp. 300–321.
  * [7] A. Tang, S. Sethumadhavan, and S. J. Stolfo, “CLKSCREW: exposing the perils of security-oblivious energy management,” in _26th USENIX Security Symposium, USENIX Security 2017_. USENIX Association, 2017.
  * [8] P. Qiu, D. Wang, Y. Lyu, and G. Qu, “Voltjockey: Breaching trustzone by software-controlled voltage manipulation over multi-core frequencies,” in _Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS_. ACM, 2019.
  * [9] K. Murdock, D. Oswald, F. D. Garcia, J. Van Bulck, D. Gruss, and F. Piessens, “Plundervolt: Software-based fault injection attacks against Intel SGX,” in _Proceedings of the 41st IEEE Symposium on Security and Privacy (S &P’20)_, 2020.
  * [10] N. Timmers and A. Spruyt, “Bypassing secure boot using fault injection,” in _Black hat Europe 2016_.
  * [11] S. Delarea and Y. Oren, “Practical, low-cost fault injection attacks on personal smart devices,” _Applied Sciences_ , vol. 12, no. 1, 2022.
  * [12] C. Hillebold, “Compiler-assisted integrity against fault injection attacks,” Master’s thesis, University of Technology, Graz, December 2014.
  * [13] J. Proy, K. Heydemann, A. Berzati, and A. Cohen, “Compiler-assisted loop hardening against fault attacks,” _ACM Trans. Archit. Code Optim._ , vol. 14, no. 4, pp. 36:1–36:25, 2017.
  * [14] N. Belleville, K. Heydemann, D. Couroussé, T. Barry, B. Robisson, A. Seriai, and H. Charles, “Automatic application of software countermeasures against physical attacks,” in _Cyber-Physical Systems Security._ , 2018, pp. 135–155.
  * [15] M.-L. Potet, L. Mounier, M. Puys, and L. Dureuil, “Lazart: A symbolic approach for evaluation the robustness of secured codes against control flow injections,” in _Seventh IEEE International Conference on Software Testing, Verification and Validation, ICST 2014_. IEEE, 2014, pp. 213–222.
  * [16] L. Dureuil, G. Petiot, M. Potet, T. Le, A. Crohen, and P. de Choudens, “FISSC: A Fault Injection and Simulation Secure Collection,” in _Computer Safety, Reliability, and Security - 35th International Conference, SAFECOMP 2016_ , 2016, pp. 3–11.
  * [17] C. Cadar, D. Dunbar, and D. R. Engler, “KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs,” in _8th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2008, USA_ , 2008, pp. 209–224.
  * [18] E. Boespflug, C. Ene, L. Mounier, and M.-L. Potet, “Countermeasures Optimization in Multiple Fault-Injection Context,” in _”Fault Diagnosis and Tolerance in Cryptography” FDTC 2020_ , Sep. 2020.
  * [19] G. Lacombe, D. Feliot, E. Boespflug, and M.-L. Potet, “Combining static analysis and dynamic symbolic execution in a toolchain to detect fault injection vulnerabilities,” _Journal of Cryptographic Engineering_ , vol. 18 january 2023.
  * [20] M. Christofi, B. Chetali, L. Goubin, and D. Vigilant, “Formal verification of a CRT-RSA implementation against fault attacks,” _Journal of Cryptographic Engineering_ , vol. 3, no. 3, pp. 157–167, 2013.
  * [21] P. Rauzy and S. Guilley, “A Formal Proof of Countermeasures Against Fault Injection Attacks on CRT-RSA,” _Journal of Cryptographic Engineering_ , vol. 4, no. 3, pp. 173–185, 2014.
  * [22] L. Goubet, K. Heydemann, E. Encrenaz, and R. De Keulenaer, “Efficient Design and Evaluation of Countermeasures against Fault Attack with Formal Verification,” in _14th Smart Card Research and Advanced Application Conference, CARDIS_ , Nov. 2015.
  * [23] K. Heydemann, J.-F. Lalande, and P. Berthomé, “Formally verified software countermeasures for control-flow integrity of smart card c code,” _Computers & Security_, vol. 85, pp. 202–224, 2019.
  * [24] T. CCRA Management Committee. (2012, Sep.) Common Criteria for Information Technology Security Evaluation. [Online]. Available: http://www.commoncriteriaportal.org/files/ccfiles/CCPART3V3.1R4.pdf
  * [25] “Application of Attack Potential to Smartcards and Similar Devices,” Joint Interpretation Library, Tech. Rep. Version 3.0, June 2020.
  * [26] L. Dureuil, M.-L. Potet, P. d. Choudens, C. Dumas, and J. Clédière, “From code review to fault injection attacks: Filling the gap using fault model inference,” in _14th Smart Card Research and Advanced Application Conference, CARDIS15_. LNCS, 2015.
  * [27] N. Theissing, D. Merli, M. Smola, F. Stumpf, and G. Sigl, “Comprehensive analysis of software countermeasures against fault attacks,” in _2013 Design, Automation & Test in Europe Conference & Exhibition (DATE)_. IEEE, 2013, pp. 404–409.
  * [28] N. Moro, K. Heydemann, A. Dehbaoui, B. Robisson, and E. Encrenaz, “Experimental evaluation of two software countermeasures against fault attacks,” in _IEEE International Symposium on Hardware-Oriented Security and Trust, HOST 2014_.
  * [29] F. Kirchner, N. Kosmatov, V. Prevosto, J. Signoles, and B. Yakobowski, “Frama-c: A software analysis perspective,” _Formal Asp. Comput._ , vol. 27, no. 3, pp. 573–609, 2015.
  * [30] M. Abadi, M. Budiu, Ú. Erlingsson, and J. Ligatti, “Control-flow integrity principles, implementations, and applications,” _ACM Transactions on Information and System Security (TISSEC)_ , vol. 13, no. 1, pp. 1–40, 2009.
  * [31] S. Ducousso, S. Bardin, and M.-L. Potet, “Adversarial reachability for program-level security analysis,” in _European Symposium on Programming (ESOP)_ , 2023.

##  Appendix 0.A Frama-C schemes

###  0.A.1 Mutation schemes

####  0.A.1.1 Data load mutation.

We can mutate a load with the right value, introducing in that a fault with no
dangerous impact.

⬇

1/*@ ensures assigns \nothing ;@*/

2int choose (int value);

3/*@ assigns \nothing ;

4 @ behavior nominal :

5 @ assumes fault==0 ;

6 @ ensures \result==value;

7 @ behavior faulted :

8 @ assumes fault!=0 ;

9 @ ensures \true ; @*/

10int mutation_scheme_dl(int value, int fault)

11 { int x = value; if(fault) x = choose(value); return x; }

Listing 7: Data load mutation scheme

####  0.A.1.2 Else following Then mutation.

We encode the flow by 0x10 when we cross the block Then, by 0x01 when we cross
the block Else and by 0x11 when we cross the block Then followed by the block
Else.

⬇

1 /*@ behavior nominal :

2 @ assumes fault==0 ;

3 @ ensures ((C!=0 ==> \result == 0x10)) ;

4 @ ensures ((C==0 ==> \result == 0x01)) ;

5 @ behavior faulted :

6 @ assumes fault!=0 ;

7 @ ensures ((C!=0 ==> \result == 0x11)) ;

8 @ ensures ((C==0 ==> \result == 0x01)) ;@*/

9 int mutation_scheme_then_else(int C, int fault)

10 { int FLOW = 0x00;

11 if(C) goto then_br; else goto else_br;

12 then_br : FLOW|=0x10; if(!fault) goto fin;

13 else_br : FLOW|=0x01;

14 fin : return FLOW; }

Listing 8: Else following Then mutation scheme

###  0.A.2 Countermeasure schemes

⬇

1/*@ ensures \result==value;

2 @ assigns \nothing ; @*/

3void LoadDup(int res, int value)

4 {int cm_var = value; if(res != cm_var) atk_detected();}

Listing 9: Load duplication countermeasure scheme

###  0.A.3 Protected schemes

⬇

1/*@ behavior nominal :

2 @ ensures \result==value; @*/

3int protected_DL_LD(int value, int fault){

4 int res = mutation_scheme_dl(value,fault);

5 LoadDup(res,value); return(res); }

Listing 10: Load modification protected by load duplication

⬇

1/*@ assigns RTS;

2 @ behavior nominal :

3 @ ensures (C!=0 ==> \result==br_then) ;

4 @ ensures (C==0 ==> \result==br_else) ; @*/

5int protected_EfT_SIG(int C, int br_then, int br_else, int fault){

6 sign(C, br_then, br_else);

7 if((C && !fault) || (!C && fault)

8 { check(br_then); return br_then; }

9 else { check(br_else); return br_else; } }

Listing 11: TI protected by Signature

⬇

1 /*@ requires id_true!=id_false ;

2 @ assigns RTS ;

3 @ behavior nominal :

4 @ ensures ((C!=0 ==> \result == 0x10)) ;

5 @ ensures ((C==0 ==> \result == 0x01)) ;@*/

6 int protected_ThenElse_CMSign(int C, int id_true, int id_false, int fault)

7 { int FLOW = 0x00;

8 sign(C,id_true,id_false);

9 if(C) goto then_br; else goto else_br;

10 then_br : check(id_true); FLOW|=0X10; if(!fault) goto fin;

11 else_br : {check(id_false); FLOW|=0X01;}

12 fin : return FLOW; }

Listing 12: Else following Then protected by Signature

⬇

1 /*@ behavior nominal :

2 @ ensures ((C!=0 && fault==0 ==> \result == THEN)) ;

3 @ ensures ((C==0 ==> \result == ELSE)) ;@*/

4 int protected_ThenElse_CM_Testdup(int C, int fault)

5 { int FLOW = 0x00;

6 if(C) { CM_TestDup(C); goto then_br; } else { CM_TestDup(!C); goto else_br;
}

7 then_br : FLOW|=0x10; if(!fault) goto fin;

8 else_br : FLOW|=0x01;

9 fin : return FLOW; }

Listing 13: Else following Then protected by Test duplication