#  Scalable Security Investment Methods for Voltage Stability of Power Systems

Lu An, Pratishtha Shukla, Aranya Chakrabortty,
and Alexandra Duel-Hallen

###### Abstract

We develop investment approaches to secure electric power systems against load
attacks that may cause voltage instability. The attacker attempts to alter the
reactive power setpoints of the loads covertly and intelligently to reduce the
voltage stability margin of the grid. The defender, or the system operator,
aims to compensate for this reduction by retuning the reactive power dispatch
of control devices such as shunt capacitor banks. The question is: how much
financial investment should the attacker and the defender plan for to succeed
in their respective objectives? To address this question, we formulate a cost-
based Stackelberg game, where the defender is aware of the attacker’s budget,
and a robust-defense sequential algorithm for the realistic case when the
defender is not fully informed about the attacker’s resources. We demonstrate
that these methods operate reliably under time-varying load uncertainties. To
provide scalability to large-scale power system models, we develop a genetic
algorithm where both players evolve their candidate solutions in opposite
directions simultaneously. Finally, the proposed methods are validated using
IEEE prototype models, demonstrating that reliable and robust defense is
feasible unless the defender’s resources are severely limited relative to the
attacker’s resources.

###### Index Terms:

Stackelberg game, voltage stability, load attacks, security investment, robust
defense, power systems

##  I Introduction

Over the past decade, significant research has been performed on cyber-
security of power systems [1] with applications to state estimation, volt/VAr
control, automatic generation control, manipulation of electricity pricing,
and so on. In this paper we address an equally important and yet less-visited
cyber-attack scenario in power systems - namely, covert attacks on loads
causing degradation of voltage stability. Load attacks have been reported in
references, such as [2, 3], but mostly from the point of view of detection and
control. Our objective, in contrast, is to formulate an investment strategy
that power system operators can adopt to secure the grid when an attacker
tries to drive it to voltage collapse by manipulating a chosen set of load
setpoints. This manipulation can be done in a covert way for each individual
load, so that the user does not feel any difference in consumption, but when
hundreds of such loads are tweaked simultaneously, the cumulative effect can
still result in severe degradation of voltage stability [4]. Finding such an
investment strategy is an important part of power system planning as it
enables the system operator to foresee the financial costs that one would need
to protect the grid in face of such load attacks.

We use ideas from game theory to formulate this investment strategy. Game-
theoretic methods have been widely applied to study security of wide ranges of
cyber-physical systems, including power systems. For example, both cooperative
and noncooperative games were proposed for load balancing and voltage
stability in [5, 6]. Most game-theoretic investment approaches employ repeated
games [7] where the players often update their investment strategies in
response to the opponents’ actions. This, however, may not be practical when
long-term, fixed security investment is desired. To address this issue, in our
recent paper [8], we developed a cost-based Stackerberg game (CBSG) to
strategically allocate the players’ security investment resources. The cost-
based Stackelberg equilibrium (CBSE) of this game not only optimizes the
players’ payoffs but also saves their costs.

However, the security-investment game in [8] was developed under idealistic
assumption of fixed power system model and complete knowledge of the
opponent’s resources. In practice, the defender is often unable to optimize
its strategy as in [8] since it acts first and its knowledge of the opponent’s
budget might be incomplete while the attacker’s follow-up action does not
require such knowledge. In this paper, we develop a robust-defense sequential
method and demonstrate that it provides reliable protection when the defender
assumes a lower bound on the attacker’s cost per load. Moreover, the CBSG in
[8] was developed and evaluated assuming fixed values of constant-power loads
only. In this paper, we consider a more generic case of time-varying loads,
which is what most constant-power loads in the grid will be in reality.
Furthermore, from a computational point of view, the algorithm in [8] requires
traversal search of both players’ action spaces, which are exponentially
expanded with the number of target loads, making it poorly scalable. To
address scalability of the CBSG, we develop an iterative, evolutionary,
bidirectional, genetic algorithm (GA)-based method to find a CBSE. This
algorithm improves upon previously investigated evolutionary methods for
finding an SE [9, 10, 11, 12] by utilizing parallel evolution for both
players. We demonstrate the proposed methods on IEEE 9-bus and 39-bus power
system models and show that reliable and robust defense is feasible unless the
defender’s resources are severely limited relative to the attacker’s
resources.

The rest of the paper is organized as follows. Section II presents the power
system. In Section III, the cost-based Stackelberg game is formulated. A
bidirectional evolutionary algorithm is introduced in Section IV. A robust-
defense method for the scenarios where the defender is not fully aware of the
attacker’s resources is proposed in Section V. Numerical results for IEEE
prototype models are contained in Section VI, and Section VII concludes the
paper.

##  II System Model

We consider a power system with $G\geq 1$ generators and $K\geq 1$ loads,
where the load buses are indexed as the first $K$ buses, followed by $G$
generator buses. Let us denote the steady-state voltage magnitudes at the load
buses as ${{\bm{V}}_{L}}=[V_{1},\cdots,V_{K}]\in{\mathbb{R}^{K}}$ and at the
generator buses as
${{\bm{V}}_{G}}=[V_{K+1},\cdots,V_{K+G}]\in{\mathbb{R}^{G}}$. The admittance
matrix of the network is denoted as ${\bm{Y}}={\bm{G}}+j{\bm{B}}$. We
partition the susceptance matrix ${\bm{B}}\in\mathbb{R}^{(K+G)\times(K+G)}$
into four block matrices as

${\bm{B}}=\left({\begin{array}[]{*{20}{c}}{{{\bm{B}}_{LL}}}&{{{\bm{B}}_{LG}}}\\\
{{{\bm{B}}_{GL}}}&{{{\bm{B}}_{GG}}}\end{array}}\right)$ (1)

where ${{\bm{B}}_{LL}}$ contains the interconnections among loads and
${{\bm{B}}_{LG}}={{\bm{B}}_{GL}}^{T}$ represents the interconnections between
loads and generators. Following the derivations in [4], one can then define
the open-circuit load voltage vector as
${\bm{V}}_{L}^{*}=-{\bm{B}}_{LL}^{-1}{{\bm{B}}_{LG}}{{\bm{V}}_{G}}$, and,
subsequently, the symmetric stiffness matrix as

${{\bm{Q}}_{cirt}}\triangleq\frac{1}{4}{\rm{diag}}({\bm{V}}_{L}^{*})\cdot{{\bm{B}}_{LL}}\cdot{\rm{diag}}({\bm{V}}_{L}^{*}),$
(2)

where $\rm{diag}(\cdot)$ denotes the diagonal matrix.

Let ${{\bm{Q}}_{L}}=[Q_{1},\cdots,Q_{K}]\in\mathbb{R}^{K}$ denote the
$K$-dimensional real vector that represents the reactive power setpoints at
the load buses. Using (2), the voltage instability index of the system is
defined as

$\Delta=||{\bm{Q}}_{cirt}^{-1}{{\bm{Q}}_{L}}|{|_{\infty}}.$ (3)

The $k^{\text{th}}$ entry of the matrix-vector product
${\bm{Q}}_{cirt}^{-1}{{\bm{Q}}_{L}}$ captures the stability stress on load
$k$, with $||\cdot||_{\infty}$ identifying the maximally stressed node.
According to Theorem 1 in [4], the power flow equation will have a unique,
stable solution if $\Delta<1$. Equivalently, $\Delta\geq 1$ indicates that at
least one load bus in the model $i$ is overly stressed and can be responsible
for a voltage collapse. We refer to $1-\Delta$ as the voltage stability margin
[13]. The larger the value of $\Delta$, the narrower the stability margin is
and the closer the power system is to a voltage collapse. Finally, let
${{\bm{Q}}_{L}^{n}}$ denote the system’s nominal reactive power setpoint
vector. Then the nominal voltage stability index ${\Delta^{n}}$ is computed as
the value of $\Delta$ (3) using ${{\bm{Q}}_{L}^{n}}$.

As proven in [4, Supp. 6] and [8], the voltage instability index $\Delta$
increases as the reactive power demands of the loads grow. The attacker can
increase the reactive power demands at appropriately chosen load buses by
adding an incremental vector
${{\bm{q}}_{a}}=[q_{a}^{1},\cdots,q_{a}^{K}]\in\mathbb{R}^{K}$ to
${{\bm{Q}}_{L}^{n}}$, thus driving ${\Delta}$ towards 1. Moreover, the
attacker can make such load attacks covert by designing the entries of
${{\bm{q}}_{a}}$ small enough that they maintain the load bus voltages to be
within their usual allowable range of 0.9 per unit (pu) to 1.1 pu while sill
pushing ${\Delta}$ towards 1. To prepare for possible future attacks, the
operator, or the defender, can switch on pre-installed voltage control
devices, such as shunt capacitors and power electronic converters, to
compensate for the potential increase in consumption in advance [14]. In
reality, not all of the load buses have control devices installed. Denote the
set of $N$ load buses that have control devices installed as

$\mathscr{L}_{ctrl}=\\{l_{n}:n=\\{1,\cdots,N\\}\\},$ (4)

where $l_{n}$ is the load bus index and $N\leq K$. Thus, the $K$-dimensional
reactive power compensation vector for all loads can be denoted as
${{\bm{q}}_{d}}=[q_{d}^{1},\cdots,q_{d}^{K}]\in\mathbb{R}^{K}$ where
$q_{d}^{k}=0,\forall k\notin\mathscr{L}_{ctrl}$, i.e. those buses without
control devices cannot be compensated directly. When an attack happens, the
overall reactive power balance becomes
${{\bm{Q}}_{L}}={{{\bm{Q}}_{L}^{n}}+{\bm{q}}_{a}-{{\bm{q}}_{d}}}$. The goal of
the defender is to strategically compensate for the attacker’s actions and to
avoid the voltage collapse by maintaining $\Delta$ as close as possible to the
nominal $\Delta^{n}$.

##  III Cost-Based Stackelberg Game

In the cost-based Stackelberg game (CBSG) [8], the actions of the attacker,
$\bm{a}=[a_{k}:k\in\\{1,\cdots,K\\}]\in\mathbb{R}^{K}$, correspond to a finite
number of discrete investment levels into the $K$ loads where a higher value
of each element $a_{k}$ indicates a greater chance of successful attack for
the $k^{\text{th}}$ load. Similarly, the actions of the defender are
represented as $\bm{d}=[d_{k}:k\in\\{1,\cdots,K\\}]\in\mathbb{R}^{K}$ where
$d_{k}$ indicates the investment level into the control device for the
compensation of the $k^{\text{th}}$ load (see (4)). Given an investment pair
(${\bm{a}}$, ${\bm{d}}$), the utilities, or payoffs, of the attacker and the
defender are termed ${U^{a}}({\bm{a}},{\bm{d}})$ and
${U^{d}}({\bm{a}},{\bm{d}})$, respectively, expressed in terms of the
instability index $\Delta$. The attacker aims to increase the instability
index of any system realization (thus degrading the system performance) while
the defender aims to reduce it. The defender is the leader, who establishes
its investment profile ${\bm{d}}$ first. The attacker follows by choosing its
action

${{\bm{a}}}=g({\bm{d}})=\mathop{\arg\max}\limits_{{\bm{a}}}{U^{a}}({{\bm{a}}},{{\bm{d}}}),$
(5)

a best response to ${\bm{d}}$, which maximizes the attacker’s utility given
the defender’s strategy ${\bm{d}}$. On the other hand, the defender chooses a
strategy ${{\bm{d}}^{*}}$ that maximizes its utility given the attacker’s best
responses $g({\bm{d}})$ to all its actions. Moreover, ${{\bm{d}}^{*}}$ has the
lowest cost among all the latter strategies. Once ${{\bm{d}}^{*}}$ is found,
the attacker chooses a best response (5) to ${{\bm{d}}^{*}}$ with the lowest
cost. A resulting CBSE [8] (${{\bm{a}}^{*}}$, ${{\bm{d}}^{*}}$) optimizes the
utility of each player in an SG under the players’ cost constraints while
saving the players’ costs.

###  III-A Players’ Actions and Costs

Denote the attacker’s action space as $\Gamma^{a}$ and its actions as
$\bm{a}=[a_{k}:k\in\\{1,\cdots,K\\}]\in\mathbb{R}^{K}$, where
$a_{k}\in\\{0,1/(L_{a}-1),\allowbreak 2/(L_{a}-1),\cdots,1\\}$ is a discrete
level of investment into load $k$, and $L_{a}$ denotes the number of
attacker’s investment levels. The value of $a_{k}$ denotes the probability of
successfully hacking into load $k$, which is determined by attacker’s
investment level, i.e., the amount of resources allocated to hacking this
load. Thus, for any attack action ${\bm{a}}$, there are $2^{K}$ possible
outcomes. Define the $j^{\text{th}}$ outcome of attack at all loads by a
binary $K$-tuple
$\bm{O}^{j}=[o_{1}^{j},\cdots,o_{k}^{j},\cdots,o_{K}^{j}],\forall
j=1,\cdots,2^{K}$, where $o_{k}^{j}=1$ if attack at load $k$ is successful and
$o_{k}^{j}=0$ if it fails. Given an attacker’s action vector ${\bm{a}}$, the
probability of outcome $\bm{O}^{j}$ is given by

$P_{{\bm{a}}}(\bm{O}^{j})=\prod\limits_{k:{\forall
o_{k}^{j}}=1}{{a_{k}}}\prod\limits_{k:{\forall
o_{k}^{j}}=0}{\left({1-{a_{k}}}\right)}.$ (6)

When the attacker successfully hacks into load $k$, the nominal reactive power
demand $Q_{k}$ of this load will be increased by $q_{a}^{k}$, where $Q_{k}$ is
the $k^{\text{th}}$ element of ${{\bm{Q}}_{L}}$ in (3). Thus, for outcome
$\bm{O}^{j}$, the incremental reactive power demand is represented by a
$1\times K$ vector given by

${\bm{q}}_{a}^{j}=\bm{O}^{j}\odot{{\bm{q}}_{a}},$ (7)

where ${{\bm{q}}_{a}}=[q_{a}^{1},\cdots,q_{a}^{k},\cdots,q_{a}^{K}]$ and
$\odot$ indicates element-wise multiplication.

Assuming that the attack on load $k$ at full effort level (i.e. when
$a_{k}=1$) has cost $\gamma_{a}\geq 0$, the total cost of the attacker is
represented as

$C_{a}=\gamma_{a}{||{{\bm{a}}}||_{1}}\leq 1,$ (8)

where $||\cdot||_{1}$ denotes the $\ell_{1}$-norm. Without loss of generality,
we assume that the total cost of the attacker is bounded by 1, and the scalar
$\gamma_{a}$ is the scaled cost per load.

Moreover, considering that the voltage at every load bus is mandated to be
within an operating range of 0.9 to 1.1 pu, the attacker must be covert in the
sense that it cannot change the demand at any target bus $k$ beyond a limit
$q_{a}^{k,\max}$ as that may violate this voltage range, leading to the attack
being caught by the operator. This covertness constraint is, therefore,
modeled as

$q_{a}^{k}\leq q_{a}^{k,\max},\forall k.$ (9)

Note that $q_{a}^{k,\max}$ will be different for different $k$ due to physical
variabilities of the loads.

Next, we define the defender’s actions as
$\bm{d}=[d_{k}:k\in\\{1,\cdots,K\\}]\in\mathbb{R}^{K}$, where
$d_{k}\in\\{0,1/(L_{d}-1),2/(L_{d}-1),\cdots,1\\},\forall
k\in\mathscr{L}_{ctrl}$, denotes the defender’s investment level on the
control device of load $k$, $L_{d}$ is the number of defender’s investment
levels, $\mathscr{L}_{ctrl}$ is the set of load buses with control devices,
and $d_{k}\triangleq 0,\forall k\notin\mathscr{L}_{ctrl}$. Let us assume the
maximum reactive power that the defender is able to compensate on load
$k\in\mathscr{L}_{ctrl}$ is $q_{d}^{k,\max}$ when the level $d_{k}=1$ where
$q_{d}^{k,\max}$ is selected so that the voltage at that load bus does not
exceed 1.1 pu. For the loads with control devices (i.e.
$k\in\mathscr{L}_{ctrl}$), given investment level $d_{k}$, the defender’s
compensation is $q_{d}^{k}=d_{k}q_{d}^{k,\max}$. On the other hand, for those
loads without control devices, the defender’s compensation is always zero,
i.e. $q_{d}^{k}\triangleq 0,\forall k\notin\mathscr{L}_{ctrl}$. The reactive
power demand compensation for all loads is specified by the $1\times K$ vector

${{\bm{q}}_{d}}=[q_{d}^{1},\cdots,q_{d}^{k},\cdots,q_{d}^{K}].$ (10)

Assuming full protection (i.e. $d_{k}=1$) for load $k\in\mathscr{L}_{ctrl}$
costs $\gamma_{d}\geq 0$, the defender’s total investment cost, i.e., its
budget constraint is given by

$C_{d}=\gamma_{d}{||{{\bm{d}}}||_{1}}\leq 1.$ (11)

Similarly, the scalar $\gamma_{d}$ represents the scaled investment cost per
protected load.

###  III-B Players’ Utility Functions and Cost-Based Stackelberg Equilibrium

The attacker aims to increase the voltage index $\Delta$ (3) but not exceed
$\Delta=1$ since the latter results in system voltage collapse and any
additional investment wastes the attacker’s resources. Moreover, to save its
cost, the defender invests only to compensate for the attacker’s action, i.e.,
it aims to reduce $\Delta$ while maintaining $\Delta\geq\Delta^{n}$. Given the
attacker’s and defender’s actions ${\bm{a}}$ and ${\bm{d}}$, the reactive
power demand vector for the $j^{\text{th}}$ outcome $\bm{O}^{j}$ is computed
as

${{\bm{Q}}_{L}^{j}}={{{\bm{Q}}_{L}^{n}}+{\bm{q}}_{a}^{j}-{{\bm{q}}_{d}}}$ (12)

The performance loss for the $j^{\text{th}}$ outcome $\bm{O}^{j}$ is given by

$L_{j}({\bm{d}})=Clip\left({{{\left\|{{\bm{Q}}_{cirt}^{-1}{\bm{Q}}_{L}^{j}}\right\|}_{\infty}}};(\Delta^{n},1)\right)$
(13)

where

$Clip(x;({\Delta^{n}},1))=\left\\{{\begin{array}[]{*{20}{c}}{{\Delta^{n}}}&{x\leq{\Delta^{n}}}\\\
x&{{\Delta^{n}}<x<1}\\\ 1&{x\geq 1}\end{array}}\right..$ (14)

Given a strategy pair $({{\bm{a}}},{{\bm{d}}})$, the attacker’s utility is
represented as the expectation of performance loss (13) over all outcomes

$\displaystyle U^{a}({\bm{a}},{\bm{d}})$ $\displaystyle={\rm
E}\left({{L_{j}}({\bm{d}})}\right)=\sum\limits_{j}^{{2^{K}}}{{P_{\bm{a}}}({{\bm{O}}^{j}}){L_{j}}({\bm{d}})},$
(15) s.t. $\displaystyle{\gamma_{a}}{\left\|{\bm{a}}\right\|_{1}}\leq
1;q_{a}^{k}\leq q_{a}^{k,\max},\forall k.$

Similarly, the defender’s utility is given by

$\displaystyle U^{d}({\bm{a}},{\bm{d}})$ $\displaystyle={\rm
E}\left(-{{L_{j}}({\bm{d}})}\right)=-\sum\limits_{j}^{{2^{K}}}{{P_{\bm{a}}}({{\bm{O}}^{j}}){L_{j}}({\bm{d}})},$
s.t. $\displaystyle{\gamma_{d}}{\left\|{\bm{d}}\right\|_{1}}\leq 1.$ (16)

From (15) and (III-B), the proposed game is zero-sum.

Finally, we make the following realistic assumptions: (i) the attacker is able
to cause voltage collapse when it has unlimited resources and the defender is
inactive, and (ii) the defender is able to compensate fully for the attacks
when both players have unlimited budgets.

Since multiple SEs are possible in an SG, in [8], we modified the traditional
Backward Induction (BI) method to select an SE that saves both players’ costs.
The resulting cost-based BI (CBBI) algorithm produces a CBSE
$(\bm{a}_{o}^{*},\bm{d}_{o}^{*})$. Theorem 1 in [8] and [15, Appx.A-B]
demonstrate that a CBSE exists and describe the proposed game’s properties.

###  III-C Evaluating Robustness of the CBSG to Time-Varying Loads

We assumed above that the system model is given by the nominal system model.
However in practice, the reactive power setpoint vector ${{\bm{Q}}_{L}}$ is
time-variant and is uncertain a priori. This uncertainty is due to the
fluctuation of real-time power consumption.

While the CBSE $(\bm{a}_{o}^{*},\bm{d}_{o}^{*})$ is optimal for the utilities
(15)$\sim$(III-B) designed for the nominal model, it is not necessarily the
optimal investment strategy when the actual model $i$ occurs. In the latter
case, the actual payoffs of the players are given by
$U_{i}^{a}({\bm{a}}_{o}^{*},{\bm{d}}_{o}^{*})=-U_{i}^{d}({\bm{a}}_{o}^{*},{\bm{d}}_{o}^{*})$
which can be obtained from (15)$\sim$(III-B) by substituting the nominal
reactive power vector ${{\bm{Q}}_{L}^{n}}$ of the nominal model by the
$i^{\text{th}}$ model’s nominal reactive power vector ${\bm{Q}}_{L}^{n,i}$ in
(12).

The difference of the utilities of the nominal model and uncertain model $i$
at CBSE is defined as

${\mu_{i}}\%=\left|{\frac{{U^{a}({\bm{a}}_{o}^{*},{\bm{d}}_{o}^{*})-U_{i}^{a}({\bm{a}}_{o}^{*},{\bm{d}}_{o}^{*})}}{{U^{a}({\bm{a}}_{o}^{*},{\bm{d}}_{o}^{*})}}}\right|\times
100\%,$ (17)

where the mismatch of the nominal model is ${\mu_{nom}}=0$. The metric (17)
will be used in Sec.VI to evaluate robustness of the proposed CBSG to time-
variant loads.

##  IV A Bidirectional Evolutionary Method for Computing a CBSE

The CBBI algorithm developed in [8] is referred to as a traversal searching
method. Since the sizes of the attacker’s and the defender’s action spaces are
bounded by $L_{a}^{K}$ and $L_{d}^{N}$, respectively, this algorithm has the
computational complexity of ${\mathcal{O}}\left({L_{a}^{K}L_{d}^{N}}\right)$,
which becomes very large as the system size ($K$ and $N$) and the players’
numbers of investment levels ($L_{a}$ and $L_{d}$) grow. To reduce the
computational complexity, we employ the following bidirectional parallel
evolutionary GA-based (BPEGA) method (Algorithm 1).

Result: One Strategy Pair $({\bm{a}_{o}^{*}},{\bm{d}_{o}^{*}})$

Parameter Initialization: Population sizes $S_{a}$ and $S_{d}$, crossover
probability $P_{c}$, mutation rate $P_{m}$, maximum number of generations $T$.

Current generation $t=0$;

Step 1. Population Initialization: Randomly selected feasible initial
populations for both players
$POP^{0}_{a}=\\{{{\bm{a}}}_{1},\cdots,{{\bm{a}}}_{S_{a}}\\}$ and
$POP^{0}_{d}=\\{{{\bm{d}}}_{1},\cdots,{{\bm{d}}}_{S_{d}}\\}$, where
$\forall\bm{a}\in POP^{0}_{a}$ satisfies (8) and $\forall\bm{d}\in
POP^{0}_{d}$ satisfies (11);

while  _the termination criteria are not satisfied,_ do

Step 2. Evaluation: The defender and attacker compute
${U^{d}}({\bm{a}},{\bm{d}})$ and ${U^{a}}({\bm{a}},{\bm{d}})$ for all
${{\bm{d}}}\in POP^{t}_{d}$ and ${{\bm{a}}}\in POP^{t}_{a}$ and evaluate all
individuals in the current generation to compute their fitness values based on
(19) and (21);

for  _Attacker and Defender_ do

Step 3. Selection: Select $S_{a}/2$ (or $S_{d}/2$) pairs of parents
$tmp_{P}^{a}$ (or $tmp_{P}^{d}$) using the Roulette Wheel selection method [9,
16];

Step 4. Reproduction: Apply crossover with probability $P_{c}$ and mutation
operation with rate $P_{m}$ [9] to generate $S_{a}$ (or $S_{d}$) children
$tmp_{c}^{a}$ (or $tmp_{c}^{d}$);

Step 5. Check feasibility: For each individual in $tmp_{c}^{a}$ (or
$tmp_{c}^{d}$), check if it is a feasible solution to (8) (or (11)). Include
all feasible children in the set $tmp_{c,f}^{a}$ (or $tmp_{c,f}^{d}$);

Step 6. Combine and sort: Combine the current generation $POP^{t}_{a}$ (or
$POP^{t}_{d}$) with the set of feasible children $tmp_{c,f}^{a}$ (or
$tmp_{c,f}^{d}$). Sort by the fitness value in the descending order. Sort the
individuals with the same fitness values by their investment cost in ascending
order. The $S_{a}$ (or $S_{d}$) individuals with the highest ranking are
selected as the next generation $POP^{t+1}_{a}$ (or $POP^{t+1}_{d}$);

end for

$t\leftarrow t+1$

end while

Step 7. Apply the CBBI algorithm [8] to the final generation $POP^{T}_{a}$ and
$POP^{T}_{d}$ to determine $({\bm{a}_{o}^{*}},{\bm{d}_{o}^{*}})$;

Algorithm 1 Bidirectional Parallel Evolutionary Genetic Algorithm

In Algorithm 1, the population sizes of the each generation of the attacker
and defender are represented by even non-negative integers $S_{a}$ and
$S_{d}$, respectively. In Step 2 of Algorithm 1, in each generation, both
players determine the fitness values of the current population where a higher
fitness value indicates a better performance according to a certain
optimization objective. For the defender’s strategy candidate ${{\bm{d}}}\in
POP^{t}_{d}$, an attacker’s best response within its population $POP^{t}_{a}$
is given by

${g_{tmp}}({\bm{d}})=\mathop{\arg\max}\limits_{{\bm{a}}\in
POP^{t}_{a}}U^{a}({\bm{a}},{\bm{d}}).$ (18)

Thus, the fitness value of each ${{\bm{d}}}\in POP^{t}_{d}$ is given by

$fit_{d}(\bm{d})=U^{d}({g_{tmp}}({\bm{d}}),{\bm{d}}),\forall{{\bm{d}}}\in
POP^{t}_{d}.$ (19)

Suppose that $\bm{d}^{\prime}\in POP^{t}_{d}$ has the highest fitness value
within the current population

$\bm{d}^{\prime}=\mathop{\arg\max}\limits_{{\bm{d}}\in
POP^{t}_{d}}fit_{d}(\bm{d}).$ (20)

For each value of $\bm{d}^{\prime}$ that satisfies (20), the attacker assigns
the fitness value to all ${{\bm{a}}}\in POP^{t}_{a}$ as

$fit_{a}(\bm{a})=U^{a}(\bm{a},{\bm{d}^{\prime}}),\forall{{\bm{a}}}\in
POP^{t}_{a}.$ (21)

In Step 6, if several individuals have the same fitness value and cost, they
are sorted according to the seniority of being selected, with the individuals
who were selected in an earlier generation placed ahead of those selected
later. Since the “combine and sort” process guarantees that the individual
with the highest fitness value among the members of the current generation and
of the feasible children set is selected for the next generation, the proposed
BPEGA algorithm is an elitist GA [17, 18].

Given a preset maximum iteration number $T$, the iteration will stop when
either of the following two conditions is satisfied:

  1. 1.

For each player, all individuals in the current population are identical, i.e.
${\bm{a}}_{i}={\bm{a}}_{j},\forall{\bm{a}}_{i},{\bm{a}}_{j}\in POP^{t}_{a}$
and ${\bm{d}}_{i}={\bm{d}}_{j},\forall{\bm{d}}_{i},{\bm{d}}_{j}\in
POP^{t}_{d}$.

  2. 2.

The iteration number has reached $T$.

Finally, we discuss the convergence and complexity of Algorithm 1. A GA
converges when a sequence of objective function evaluations approaches the
maximum of the objective function as the number of iterations $T$ tends to
infinity [19, 20, 11, 21, 22, 23, 24]. Since the proposed BPEGA employs
parallel evolution of both players, the convergence result of [12], which
assumes only the defender’s evolution, is not applicable to Algorithm 1.

######  Proposition 1.

Assume a crossover probability $P_{c}>0$ and a mutation rate $P_{m}>0$. As the
number of iterations $T$ tends to infinity, both players’ expected utilities
generated by the BPEGA (Algorithm 1) converge to the utilities at any SE and
the strategy pair selected by the BPEGA has the lowest players’ costs among
all SEs of the CBSG.

###### Proof.

First, since the action spaces of both players are finite, from Thm. 1(a) in
[8], an SE exists in the proposed game. Moreover, according to Thm. 1(b) [8],
all SEs provide the same pair of payoffs in a zero-sum game.

From Theorem 4.1 in [24], due to the crossover and mutation process, any
feasible solution appears at least once in some generation with probability 1
when the set of feasible solutions is finite and the maximum number of
iterations $T$ tends to infinity. Thus, since the action spaces of both
players in the proposed game are finite, any feasible strategy pair
$({\bm{a}},{\bm{d}})$ appears at least once in some generation with
probability 1 when $T\rightarrow\infty$.

Denote the set of all SEs as
$\mathscr{E}=\\{({\bm{a}}^{*}_{i},{\bm{d}}^{*}_{i}):i=1,\cdots,S\\}$. Suppose
$({\bm{a}}^{*}_{o},{\bm{d}}^{*}_{o})$ is an SE with the lowest players’ costs
(i.e. it satisfies (16) and (18) in [8]) and appear first in the evolution
process among all lowest-cost SEs. Its fitness value (19) for the defender
must be the highest among the members of that generation since all SEs have
the same utility for the defender. Moreover, since ${\bm{d}}^{*}_{o}$ has the
smallest cost among all ${\bm{d}}^{*}_{i}$, it must have the highest ranking
among the members of that generation due to the cost-sorting property in Step
6 of Algorithm 1. Thus, ${\bm{d}}^{*}_{o}$ will be included in the next
generation. By a similar argument, ${\bm{a}}^{*}_{o}$ will also be retained in
the following generation.

By the argument above, the SE pair $({\bm{a}}^{*}_{o},{\bm{d}}^{*}_{o})$ will
occur in all remaining generations of BPEGA. Thus, it will be included in the
generations $POP^{T}_{a}$ and $POP^{T}_{d}$ as $T\rightarrow\infty$.

Finally, according to the Step 7 of the BPEGA algorithm,
$({\bm{a}}^{*}_{o},{\bm{d}}^{*}_{o})$ or another SE with the lowest players’
costs will be produced by the BPEGA method. ∎

The computational complexity of the BPEGA is ${\mathcal{O}}(TS_{a}S_{d})$. The
latter is significantly lower than the complexity of the CBBI algorithm, which
is on the order of ${\mathcal{O}}\left({L_{a}^{K}L_{d}^{N}}\right)$ [8], when
the system size is large and $TS_{a}S_{d}\ll{L_{a}^{K}L_{d}^{N}}$. In
practice, power systems with different system sizes might require different
$T$ values to achieve convergence as discussed in Sec.VI.

##  V Robust Defense against Load Attacks

While [8] assumes that each player knows the opponent’s budget, in practice,
such information might be incomplete. Note that the attacker does not require
the the knowledge of the defender’s cost per load $\gamma_{d}$ to determine
its best response in the CBBI algorithm [8] or BPEGA Algirhtm 1 since it acts
as the follower and thus observes the defender’s strategy before acting.
However, the defender, or the system operator, relies on the knowledge of
$\gamma_{a}$ in the CBSG in [8]. Thus, the defender’s actions must be modified
when it does not have complete information about the attacker’s resources. In
the robust-defense (RD) sequential method described below, the defender
employs a lower bound $\gamma_{a}^{est}$ on the attacker’s actual cost
$\gamma_{a}$, i.e. $0\leq\gamma_{a}^{est}\leq\gamma_{a}$, to compute its
investment strategy.

Step 1: (a) For each defender’s action ${\bm{d}}$ that satisfies (11), the
defender estimates the set of attacker’s best responses
$\mathcal{G}(\gamma_{a}^{est},{\bm{d}})$, where
${g}(\gamma_{a}^{est},{{\bm{d}}})\in\mathcal{G}(\gamma_{a}^{est},{\bm{d}})$ if

$\displaystyle{g}(\gamma_{a}^{est},{{\bm{d}}})$
$\displaystyle=\mathop{\arg\max}\limits_{{\bm{a}}}{U^{a}}({{\bm{a}}},{{\bm{d}}}),\hfill$
(22) s.t. $\displaystyle\gamma_{a}^{est}{||{{\bm{a}}}||_{1}}\leq
1,\;q_{a}^{k}\leq q_{a}^{k,\max},\forall k.$

(b) For each ${\bm{d}}$, the smallest-cost estimated best response is found

$g_{o}(\gamma_{a}^{est},{\bm{d}})=\mathop{\arg\min}\limits_{{g}(\gamma_{a}^{est},{\bm{d}})\in\mathcal{G}(\gamma_{a}^{est},{\bm{d}})}||{g}(\gamma_{a}^{est},{\bm{d}})||_{1}.$
(23)

Step 2: (a) The defender determines the set of its investment strategies
$\mathcal{D}_{est}$ where ${{\bm{d}}_{RD}^{*}}\in\mathcal{D}_{est}$ if

$\displaystyle{{\bm{d}}_{RD}^{*}}$
$\displaystyle=\mathop{\arg\max}\limits_{\bm{d}}{U^{d}}(g_{o}(\gamma_{a}^{est},{\bm{d}}),{\bm{d}}),\hfill$
(24) s.t. $\displaystyle{\gamma_{d}}||{{\bm{d}}}||_{1}\leq 1.$

(b) A strategy in (24) with the smallest cost is chosen

${{\bm{d}}_{RD}^{o}}=\mathop{\arg\min}\limits_{{{{\bm{d}}_{RD}^{*}}}\in\mathcal{D}_{est}}||{{\bm{d}}_{RD}^{*}}||_{1}.$
(25)

Step 3: (a) By observing the defender’s action ${{\bm{d}}_{RD}^{o}}$, the
attacker finds its actual set of best responses
$\mathcal{G}(\gamma_{a},{{\bm{d}}_{RD}^{o}})$, where
${{{\bm{a}}_{RD}^{*}}}\in\mathcal{G}(\gamma_{a},{{\bm{d}}_{RD}^{o}})$ if

$\displaystyle{{{\bm{a}}_{RD}^{*}}}$
$\displaystyle=\mathop{\arg\max}\limits_{{\bm{a}}}{U^{a}}({{\bm{a}}},{{\bm{d}}_{RD}^{o}}),\hfill$
(26) s.t. $\displaystyle\gamma_{a}{||{{\bm{a}}}||_{1}}\leq 1;q_{a}^{k}\leq
q_{a}^{k,\max},\forall k,$

(b) If multiple solutions exist in
$\mathcal{G}(\gamma_{a},{{\bm{d}}_{RD}^{o}})$, the attacker chooses a strategy
with the smallest cost

${{\bm{a}}_{RD}^{o}}=\mathop{\arg\min}\limits_{{{{\bm{a}}_{RD}^{*}}}\in\mathcal{G}(\gamma_{a},{{\bm{d}}_{RD}^{o}})}||{{\bm{a}}_{RD}^{*}}||_{1}.$
(27)

A strategy pair $({{\bm{a}}_{RD}^{o}},{{\bm{d}}_{RD}^{o}})$ is a cost-based RD
method solution.

After an RD solution is found, the actual payoff of the defender differs from
its estimate
${U^{d}}(g_{o}(\gamma_{a}^{est},{{\bm{d}}_{RD}^{o}}),{{\bm{d}}_{RD}^{o}})$,
and is given by ${U^{d}}({{\bm{a}}_{RD}^{o}},{{\bm{d}}_{RD}^{o}})$. Similarly
to the proof of [25, Theorem 4.1], we can show that the latter payoff does not
exceed the defender’s payoff $U^{d}({\bm{a}}_{o}^{*},{\bm{d}}_{o}^{*})$ at a
CBSE of the CBSG in Sec.III. This result is expected since in the latter
idealistic game the defender is aware of the attacker’s cost while in the RD
method the defender is uncertain about this cost.

Since $\gamma_{a}^{est}\leq\gamma_{a}$, the actual utility of the defender is
no less than its estimated utility according to Theorem 1(c) in [8], i.e.
${U^{d}}({{\bm{a}}_{RD}^{o}},{{\bm{d}}_{RD}^{o}})\geq{U^{d}}(g_{o}(\gamma_{a}^{est},{{\bm{d}}_{RD}^{o}}),{{\bm{d}}_{RD}^{o}})$.
Thus, assuming the worst-case attack scenario provides robust defense when the
defender is uncertain about the attacker’s actual cost information. Finally,
as $\gamma_{a}^{est}$ grows, the defender’s knowledge of the attacker’s cost
$\gamma_{a}$ improves. Using Theorem 1(c) in [8], we can show that
${U^{d}}({{\bm{a}}_{RD}^{o}},{{\bm{d}}_{RD}^{o}})$ increases with
$\gamma_{a}^{est}$ and approaches ${U^{d}({\bm{a}}_{o}^{*},{\bm{d}}_{o}^{*})}$
as $\gamma_{a}^{est}$ tends to the actual $\gamma_{a}$ value.

To evaluate the defender’s utility loss due to its uncertainty about the
attacker’s budget, we compute the mismatch (loss) of actual defender’s utility
using the RD method relative to that at a CBSE of the ideal CBSG. For each set
of $\gamma_{a}^{est}$, $\gamma_{a}$, and $\gamma_{d}$ values, this mismatch is
computed as

${\mu_{RD}}\%=\left|{\frac{{{U^{d}}({{\bm{a}}_{RD}^{o}},{{\bm{d}}_{RD}^{o}})-U^{d}({\bm{a}}_{o}^{*},{\bm{d}}_{o}^{*})}}{{U^{d}({\bm{a}}_{o}^{*},{\bm{d}}_{o}^{*})}}}\right|\times
100\%$ (28)

Finally, the evolutionary Algorithm 1 can be easily modified peform the RD
method and shown to converge to an RD solution. Moreover, as for the CBSG, the
RD method is developed assuming the nominal model, but can be applied to any
uncertain model $i$ and evaluated using a metric similar to (17). The details
are omitted for brevity.

##  VI Numerical Results

In Sec.VI.A, we evaluate the proposed methods for the IEEE 9-bus system where
the traversal algorithm in [8] is computationally tractable. Thus, we are able
to validate computationally efficient BPEGA method proposed in Sec.IV by
comparing its performance with the traversal algorithm. Then in Sec.VI.B, we
employ GA methods to analyze performance of the IEEE 39-bus system.

###  VI-A IEEE 9-bus System

The IEEE 9-bus system has 6 load buses, which are all potential targets for
the attacker. We assume that the set of buses with control devices installed
is $\mathscr{L}_{ctrl}=\\{4,5,6,8\\}$, which means that the defender can only
directly compensate to bus $k,\forall k\in\mathscr{L}_{ctrl}$. In the
simulation, $q_{a}^{k,\max}$ is determined by the covertness constraint (9),
and we set $q_{d}^{k,\max}=2$ pu, $\forall k$. It was verified that these
compensations do not violate the $[0.9,1.1]$ pu voltage range for any bus. The
$M$ load-uncertain models are created by adding independent zero-mean Gaussian
random variables to the components of the nominal reactive power setpoint
vector $\bm{Q}_{L}^{n}$, i.e., the $i^{\text{th}}$ model’s nominal reactive
power setpoint vector is given by
$\bm{Q}_{L}^{n,i}=\bm{Q}_{L}^{n}(1+\bm{\epsilon}^{i})$, where
$\bm{\epsilon}^{i}\sim N(0,{\sigma}^{2})$, for $i=\\{1,\cdots,M\\}$, and the
standard deviation $\sigma=0.1$ [26]. Note that the number of models $M=20$,
which satisfies the criteria for the sample size given the margin of error
(MOE) of 0.05 and the confidence interval (CI) of $95\%$ [27].

Figure 1: Attacker’s expected utility at CBSE vs. $\gamma_{a}$ and
$\gamma_{d}$ for $L_{a}=L_{d}=3$, IEEE 9-bus system

Fig. 1 shows the attacker’s expected utility
$U^{a}({\bm{a}_{o}^{*}},{\bm{d}_{o}^{*}})$ (15) at a CBSE for varying
$\gamma_{a}$ and $\gamma_{d}$ assuming a fixed nominal model. These trends are
consistent with Theorem 1 in [8]. In particular, each player’s utility
improves as that player’s cost decreases while the opponent’s cost is fixed.
Moreover, multiple SEs exist in the regions shown in Fig. 1, and the proposed
CBBI method finds a cost-efficient CBSE without sacrificing the players’
payoffs. Furthermore, when $\gamma_{a}\leq 0.15$ while $\gamma_{d}\geq 1.05$,
the defender’s cost per load greatly exceeds that of the attacker’s, so the
attacker is able to increase its reactive power demand to achieve
$\Delta_{0}=1$ (i.e. voltage collapse) while the defender cannot compensate
due to its limited resources and thus does not act in this region to save its
cost (11). On the other hand, when the defender’s cost is small
($\gamma_{d}\leq 0.3$), implying the defender has sufficient resources to
compensate for any level of attack, the attacker does not act, and the
resulting $U^{a}({\bm{a}_{o}^{*}},{\bm{d}_{o}^{*}})=\Delta_{0}^{n}$ (i.e. the
defender maintains the nominal voltage index). In addition, we found that both
players target “important” loads with the greatest impact on $\Delta_{0}$, but
the attacker tends to avoid the loads protected by the defender [8]. Finally,
it was shown in [15, Chap.4] that choosing $L_{a}=L_{d}=3$ provides excellent
performance-complexity trade-off for both players, so these parameters are
employed throughout the rest of this paper.

Figure 2: Boxplot of the utility difference $\mu_{i}\%$ (17) over the randomly
generated set of 20 models and 5 cost pairs ($\gamma_{a}$, $\gamma_{d}$):
(0.1, 0.1), (0.1, 1.5), (0.75, 0.75), (1.5, 0.1), (1.5, 1.5); $L_{a}=L_{d}=3$;
the IEEE 9-bus system Figure 3: Convergence of the attacker’s utility in the
BPEGA Algorithm 1 with $L_{a}=L_{d}=3$; the IEEE 9-bus system

Fig. 2 represents the utility difference (17) statistics for the CBSG employed
over the set of $M=20$ uncertain models for five different cost pairs. We
observe that most uncertain models experience modest utility differences from
the CBSE shown in Fig. 1. Specifically, the mean utility difference fluctuates
from $0.46\%$ to $3.83\%$ for the five cost pairs in Fig. 2, demonstrating
robustness of CBSG to load uncertainty in the IEEE 9-bus power system.

Next, we validate convergence of BPEGA Algorithm 1. We set the attacker’s
population size $S_{a}=30$ and the defender’s population size $S_{d}=20$. The
crossover probability $P_{c}=0.85$, and the mutation rate $P_{m}=0.05$. The
maximum number of generations is set to $T=30$. These initialization
parameters are selected experimentally and reflect a trade-off between
convergence and computational complexity for the IEEE 9-bus system. For the
nominal system model, Fig. 3 shows that the BPEGA Algorithm 1 converges to a
CBSE obtained by the traversal CBBI algorithm in fewer than 15 iterations,
thus confirming Proposition 1. Similar results were obtained for other cost
pairs, demonstrating fast convergence of the BPEGA Algorithm 1 [15].

Figure 4: The mismatch ${\mu_{RD}}\%$ (28) for varying $\gamma_{a}^{est}$
given $\gamma_{a}=\\{0.45,0.75,1.05\\}$, and $\gamma_{d}=\\{0.75,1.5\\}$;
$L_{a}=L_{d}=3$; the IEEE 9-bus system Figure 5: Boxplot of the mismatch
${\mu_{RD}}\%$ (28) of the RD solution for $\gamma_{a}^{est}=0$ over the cost
pairs: $\gamma_{a}=[0:0.075:1.5]$, $\gamma_{d}=\\{0.45,0.75,1.5\\}$;
$L_{a}=L_{d}=3$; the IEEE 9-bus system

Finally, we compare the performance of the RD method (Sec.V) and the ideal
CBSG (Sec.III) for the nominal-model IEEE 9-bus system. Fig. 4 shows the
mismatch ${\mu_{RD}}\%$ (28) for varying estimated costs $\gamma_{a}^{est}$
given selected $\gamma_{a}$ and $\gamma_{d}$ values. We observe that the
mismatch reduces as the worst-case cost estimate $\gamma_{a}^{est}$ approaches
the actual $\gamma_{a}$ value. We also found that the mismatch is zero for all
$\gamma_{a}^{est}$ values in many cases, e.g. when
$(\gamma_{a},\gamma_{d})=(0.75,0.45)$ or $(0.75,1.05)$.

Fig. 5 shows that when the most powerful attacker is assumed
($\gamma_{a}^{est}=0$) while the actual $\gamma_{a}$ values are in the range
$0\leq\gamma_{a}\leq 1.5$ and $\gamma_{d}\in\\{0.45,0.75,1.5\\}$, the mismatch
(28) for most cost pairs was less than 1% with median of $\mu_{RD}=0$ and 75%
percentile = 0.575, indicating that the RD method is robust to the defender’s
uncertainty about the attacker’s resources. On the other hand, we found that
the defender’s cost of the RD solution is at least as large as the defender’s
cost of a CBSE of the ideal CBSG. When the actual $\gamma_{a}$ is large while
$\gamma_{d}$ is relatively small, e.g. $\gamma_{a}\geq 1.05$ and
$\gamma_{d}\leq 0.225$, the overpayment by the defender using the RD method
compared to the ideal CBSG can be as high as $300\%$. This is due to the fact
that in the former the defender overestimates the attacker’s budget while in
the latter the defender is aware of the attackers’ resources and thus invests
more strategically. However, the defender’s cost loss of the RD solution
reduces as $\gamma_{d}$ increases and $\gamma_{a}^{est}$ approaches the actual
$\gamma_{a}$ value. Furthermore, the attacker’s investment cost is the same in
both games.

###  VI-B IEEE 39-bus system

Next, we apply the proposed methods to the IEEE 39-bus system, which contains
29 loads. We assume that the set of buses with control devices installed is
$\mathscr{L}_{ctrl}=\\{5,6,7,8,10,\allowbreak 11,13\\}$. This set includes
both players’ five most important loads [8]. The $M=20$ load-uncertain models
were created using the same method for the IEEE 9-bus system (Sec VI.A).

Figure 6: Attacker’s utility at a CBSE found by the BPEGA algorithm with
$L_{a}=L_{d}=3$; the IEEE 39-bus system Figure 7: Boxplot of the utility
difference $\mu_{i}\%$ (17) over a randomly generated set of 20 models and 5
cost pairs ($\gamma_{a}$, $\gamma_{d}$): (0.1, 0.1), (0.1, 1.5), (0.75, 0.75),
(1.5, 0.1), (1.5, 1.5); $L_{a}=L_{d}=3$; the IEEE 39-bus system

For levels of investment $L_{a}=L_{d}=3$, the complexities of the CBBI
algorithm and the BPEGA Algorithm 1 are ${\mathcal{O}}\left({3^{29}\times
3^{7}}\right)$ and ${\mathcal{O}}(TS_{a}S_{d})={\mathcal{O}}(30\times 30\times
20)={\mathcal{O}}(18000)$, respectively. Since the former complexity is too
high, we apply the BPEGA Algorithm 1 to compute a CBSE for IEEE 39-bus system.
Fig. 6 shows the attacker’s utilities at a CBSE computed by Algorithm 1. We
observe the same performance trends as in Fig. 1. Note that voltage collapse
occurs only when the attacker has very small cost $\gamma_{a}$ while the
defender’s cost $\gamma_{d}\gg\gamma_{a}$. We conclude that voltage collapse
can be successfully prevented in both IEEE 9-bus and 39-bus systems unless the
defender’s security resources are disproportionately limited relative to the
attacker’s budget.

Figure 8: Boxplot of the mismatch ${\mu_{RD}}\%$ (28) of the RD solution for
$\gamma_{a}^{est}=0$ over the cost pairs: $\gamma_{a}=[0:0.075:1.5]$,
$\gamma_{d}=\\{0.45,0.75,1.5\\}$; $L_{a}=L_{d}=3$; the IEEE 39-bus system

Fig. 7 shows the utility difference (17) statistics for the IEEE 39-bus system
over $M=20$ randomly generated uncertain models for five different cost pairs.
We found that most uncertain models experience slight utility differences that
range from $0\%$ to $1.836\%$, confirming the robustness of proposed CBSG to
load uncertainty. Fig. 8 shows the boxplot for the mismatch (28) of the
defender’s utilities of the RD solution for selected defender’s costs when the
most powerful attacker is assumed ($\gamma_{a}^{est}=0$) and the actual
$\gamma_{a}$ values are in the range $0\leq\gamma_{a}\leq 1.5$. We observe
that the mismatch for most cost pairs is modest, confirming that the RD
solution is robust to imperfect knowledge of the attacker’s budget by the
defender. Moreover, as in IEEE 9-bus system, to prevent attacks, the defender
overestimates the attacker’s budget in the RD method. The resulting
overpayment can be as high as $367\%$ when $\gamma_{a}\geq 1.05$ and
$\gamma_{d}\leq 0.15$ in the IEEE 39-bus system.

Finally, for both IEEE 9-bus and 39-bus systems, we found that the RD method
is robust to model uncertainty, with statistics similar to those in Fig. 2 and
Fig. 7.

##  VII Conclusion

We developed a cost-based Stackelberg game for finding long-term investment
strategies that can secure electric power systems from load attacks and
associated voltage collapse. This game allocates the resources of the defender
and the attacker strategically over the loads and control devices under the
players’ budget constraints. To address the scalability of the CBSG to large
power systems, a bidirectional, parallel, evolutionary generic algorithm
(BPEGA) was developed. Finally, we proposed a robust-defense (RD) method to
address realistic scenarios where the defender lacks full information about
the attacker’s resources. It was demonstrated that robust, successful defense
is feasible for both load and/or information-uncertain scenarios unless the
defender is much more resource-limited than the attacker.

## References

  * [1] S. M. Dibaji, M. Pirani, D. B. Flamholz, A. M. Annaswamy, K. H. Johansson, and A. Chakrabortty, “A systems and control perspective of cps security,” _Annual Reviews in Control_ , vol. 47, pp. 394 – 411, 2019.
  * [2] A. Teixeira, G. Dán, H. Sandberg, R. Berthier, R. B. Bobba, and A. Valdes, “Security of smart distribution grids: Data integrity attacks on integrated volt/var control and countermeasures,” in _2014 American Control Conference_ , June 2014, pp. 4372–4378.
  * [3] S. Amini, F. Pasqualetti, and H. Mohsenian-Rad, “Dynamic load altering attacks against power system stability: Attack models and protection schemes,” _IEEE Transactions on Smart Grid_ , vol. 9, no. 4, pp. 2862–2872, July 2018.
  * [4] J. W. Simpson-Porco, F. Dörfler, and F. Bullo, “Voltage collapse in complex power grids,” _Nature communications_ , vol. 7, p. 10790, 2016.
  * [5] C. Avraam, J. Rines, A. Sarker, F. Paganini, and E. Mallada, “Voltage collapse stabilization: A game theory viewpoint,” _ArXiv_ , vol. abs/1810.10133, 2018\.
  * [6] M. A. Arif, M. Ndoye, G. V. Murphy, and K. Aganah, “A cooperative game theory algorithm for distributed reactive power reserve optimization and voltage profile improvement,” in _2017 North American Power Symposium (NAPS)_ , Sep. 2017, pp. 1–6.
  * [7] S. Etesami and T. Basar, “Dynamic Games in Cyber-Physical Security: An Overview,” _Dynamic Games and Applications_ , pp. 1–30, 2019.
  * [8] L. An, A. Chakrabortty, and A. Duel-Hallen, “A stackelberg security investment game for voltage stability of power systems,” in _2020 IEEE 59th Conference on Decision and Control (CDC)_ , 2020.
  * [9] B. Liu, “Stackelberg-nash equilibrium for multilevel programming with multiple followers using genetic algorithms,” _Computers & Mathematics with Applications_, vol. 36, no. 7, pp. 79–89, 1998.
  * [10] W. Liu and S. Chawla, “A game theoretical model for adversarial learning,” in _2009 IEEE International Conference on Data Mining Workshops_ , 2009, pp. 25–30.
  * [11] E. D’Amato, E. Daniele, L. Mallozzi, and G. Petrone, “Equilibrium strategies via ga to stackelberg games under multiple follower’s best reply,” _International Journal of Intelligent Systems_ , vol. 27, no. 2, pp. 74–85, 2012.
  * [12] T. Vallée and T. Başar, “Off-line computation of stackelberg solutions with the genetic algorithm,” _Computational Economics_ , vol. 13, no. 3, pp. 201–209, 1999.
  * [13] T. Van Cutsem and C. Vournas, _Voltage stability of electric power systems_. Springer Science & Business Media, 2007.
  * [14] D. Mondal, A. Chakrabarti, and A. Sengupta, _Power system small signal stability analysis and control_. Academic Press, 2020.
  * [15] L. An, “Game-theoretic methods for cost allocation and security in Smart Grid,” 2020, Ph.D. thesis. [Online]. Available: https://repository.lib.ncsu.edu/handle/1840.20/38411
  * [16] D. Whitley, “A genetic algorithm tutorial,” _Statistics and computing_ , vol. 4, no. 2, pp. 65–85, 1994.
  * [17] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist multiobjective genetic algorithm: Nsga-ii,” _IEEE transactions on evolutionary computation_ , vol. 6, no. 2, pp. 182–197, 2002.
  * [18] Chang Wook Ahn and R. S. Ramakrishna, “Elitism-based compact genetic algorithms,” _IEEE Transactions on Evolutionary Computation_ , vol. 7, no. 4, pp. 367–385, 2003.
  * [19] J. P. Pedroso Ramos Dos Santos, “Niche search and evolutionary optimisation,” Université catholique de Louvain, Center for Operations Research and Econometrics (CORE), Tech. Rep., 1996.
  * [20] J. P. Pedroso _et al._ , “Numerical solution of nash and stackelberg equilibria: an evolutionary approach,” in _Proceedings of SEAL_ , vol. 96, 1996, pp. 151–160.
  * [21] T. Bäck, G. Rudolph, and H.-P. Schwefel, “Evolutionary programming and evolution strategies: Similarities and differences,” in _In Proceedings of the Second Annual Conference on Evolutionary Programming_. Citeseer, 1993.
  * [22] D. Bhandari, C. Murthy, and S. K. Pal, “Genetic algorithm with elitist model and its convergence,” _International journal of pattern recognition and artificial intelligence_ , vol. 10, no. 06, pp. 731–747, 1996.
  * [23] L. Jun-hua and L. Ming, “An analysis on convergence and convergence rate estimate of elitist genetic algorithms in noisy environments,” _Optik_ , vol. 124, no. 24, pp. 6780–6785, 2013.
  * [24] R. F. Hartl and R. Belew, “A global convergence proof for a class of genetic algorithms,” _University of Technology, Vienna_ , 1990.
  * [25] P. Shukla, “Game-theoretic investment planning for cyber-security of Network Control Systems,” 2021, Ph.D. thesis. [Online]. Available: https://www.lib.ncsu.edu/resolver/1840.20/39267
  * [26] R. Bo and F. Li, “Probabilistic lmp forecasting considering load uncertainty,” _IEEE Transactions on Power Systems_ , vol. 24, no. 3, pp. 1279–1289, 2009.
  * [27] X. S. Liu, B. Loudermilk, and T. Simpson, “Introduction to sample size choice for confidence intervals based on t statistics,” _Measurement in Physical Education and Exercise Science_ , vol. 18, no. 2, pp. 91–100, 2014.