# Improving Robustness with Image Filtering

Matteo Terzi, Mattia Carletti, Gian Antonio Susto
University of Padova
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Adversarial robustness is one of the most challenging problems in Deep
Learning and Computer Vision research. All the state-of-the-art techniques
require a time-consuming procedure that creates cleverly perturbed images. Due
to its cost, many solutions have been proposed to avoid Adversarial Training.
However, all these attempts proved ineffective as the attacker manages to
exploit spurious correlations among pixels to trigger brittle features
implicitly learned by the model. This paper first introduces a new image
filtering scheme called Image-Graph Extractor (IGE) that extracts the
fundamental nodes of an image and their connections through a graph structure.
By leveraging the IGE representation, we build a new defense method, Filtering
As a Defense, that does not allow the attacker to entangle pixels to create
malicious patterns. Moreover, we show that data augmentation with filtered
images effectively improves the model’s robustness to data corruption. We
validate our techniques on CIFAR-10, CIFAR-100, and ImageNet.

##  1 Introduction

The present work was inspired by visual inspection of many adversarial example
images.

Figure 1: Micro-patterns created by $\ell_{2}$-bounded adversarial attacks on
CIFAR-10 images.

As can be appreciated in Figure 1, where adversarial examples related to
CIFAR-10 are reported, the attack seeks to create spurious micro-patterns
that, although perceptually negligible to humans, dramatically impact the
behavior of Deep Neural Networks (DNNs) [26]. These artifacts give rise to
many tailored ‘virtual edges’, which leverage structural properties of the DNN
architecture (specifically, the sum-aggregation operation in convolutional
layers and the increasing receptive field size in deeper layers) to excite
specific internal model biases. As a result, the model predictions can be
easily subverted, causing input samples to be misclassified with high
confidence.

Indeed, current DNNs are strongly dependent on a fragile balance of features
that, if broken, can lead to a significant drop in generalization power [28,
11].

The fundamental strength of the micro-patterns introduced by adversarial
attacks lies in their simultaneous local and non-local nature. Locality stems
from the fact that the perturbation can be very precise in spatial location
and directly impacts only strictly neighboring pixels. Non-locality, instead,
emerges from the possibility to leverage the correlation among spatially
distant pixels. In view of this, the attacker is allowed to be very precise in
choosing the optimal (relative) spatial locations so that the malign effects
of perturbations can conveniently accumulate across layers.

In recent years, many defenses against adversarial examples have been
proposed. However, the vast majority of the proposed approaches proved to be
ineffective under stronger attacks or adaptive ones, as their false sense of
robustness turned out to be related to different forms of obfuscated gradients
[1, 27].

The most reliable method to truly enforce robustness in DNNs is represented by
Adversarial Training (AT) [16], which consists in training DNNs on adversarial
samples rather than natural ones. In these settings, the training process is
induced to find a configuration of parameters so that the model is robust to
adversarial examples. The spurious textures characterizing adversarial
examples let the model infer that a peculiar configuration of a few distant
pixels is, in principle, not correlated with the task or that a particular
local texture is not predictive. As this process takes place during training,
the model can adapt its parameters accordingly.

As mentioned above, many issues related to the correlation of spatially
distant pixels are due to the model architecture itself rather than the
training procedure or the training data. Therefore, major enhancements in
adversarial robustness should come from novel architecture layouts taking into
account the structural weaknesses discussed above. However, significant
improvements can still be achieved by acting on the input space.

Based on the above intuitions, this paper presents a novel image filtering
framework aimed at improving robustness against adversarial attacks without
resorting to AT. While it is not the first contribution introducing techniques
based on input manipulation to prevent adversarial attacks [9, 15, 7], this
work differs significantly from prior literature in the fundamental concepts
at the basis of its functioning. The main difference between our method and
others relying on input transformations is that we explicitly remove textures
from input images while preserving the informative features. This is achieved
through an iterative filtering procedure based on simple rules that do not
require additional training routines. Details and analysis of our proposed
approach are provided in Section 4 and Section 5.

The image filtering strategy proposed in this paper can be exploited as a data
augmentation, which we show being beneficial also for robustness against
common corruptions (e.g., shot noise, pixelate, glass blur). Additionally, our
framework allows for high flexibility in the choice of its key components, as
long as its underpinning principles are not violated.

Our contributions can be summarized as follows:

  * •

We introduce a novel image filtering framework, dubbed Image-Graph Extractor
(IGE), whose main objective is to remove textures while preserving the image’s
semantic content. Our framework is based on an implicit graph representation
of images, the Image-Graph (IG).

  * •

We propose an effective defense against bounded adversarial attacks, called
Filtering As a Defense (FAD), leveraging the textures suppression effect of
IGE.

  * •

We exploit filtered images produced through the IGE filtering framework as a
data augmentation strategy. We show that models trained with our data
augmentation strategy are more robust against common corruptions such as shot
noise, pixelate, glass blur.

While IGE has been primarily conceived and designed to enforce robustness
through explicit textures suppression in the input image, the intermediate
graph representation necessary for the iterative filtering procedure might
prove useful in many other applications. In fact, the IG associated with the
input image enables the emergence of structure in an unstructured domain. The
extracted structure, if adequately exploited, might boost performance in
downstream tasks. This will be the subject of further investigations in future
research.

The paper is organized as follows. In Section 2 we review existing literature
related to our work. Section 3 lays out the necessary notation and preliminary
concepts. In Section 4 we introduce and analyze our IGE framework, while
Section 5 describes the FAD defense. In Section 6 we present the results of
our experiments. In Section 7 we discuss limitations and potential
improvements of our framework. In Section 8 we summarize our findings and
point to some possible research directions for future work.

##  2 Related Work

Since the discovery of the existence of adversarial examples [26], countless
attempts to make image classifiers robust to adversarial perturbations have
appeared in the literature. As anticipated in Section 1, current state-of-the-
art methods to enforce robustness directly leverage adversarial examples
during training [8, 14, 12, 16]. A major limitation affecting AT is the high
computational cost as adversarial examples are typically crafted through
iterative optimization routines. For this reason, significant efforts have
been made to improve the robustness of models without resorting to AT.
Proposed solutions cover a wide range of techniques based on curvature
regularization [17], robust optimization to improve local stability [23], the
use of additional unlabeled data [4], local linearization [21], Parseval
networks [5], defensive distillation [19], model ensembles [18], channel-wise
activations suppressing [2], feature denoising [29], self-supervised learning
for adversarial purification [25], and input manipulations [9, 7, 15]. All the
listed techniques, except those based on input manipulations, require training
the model or an auxiliary module from scratch. Our method, instead, can be
used in combination with pretrained models.

In [15] the authors analyze the effect of image rescaling on adversarial
examples. [7] explores the possibility to improve robustness through JPG
(re)compression, based on the intuition that adversarial perturbations are
unlikely to leave an image in the space of JPG images. In [9], the authors
assess the effectiveness of input transformations based on image cropping and
rescaling, bit-depth reduction, JPEG compression, total variance minimization,
and image quilting. The main objective is to remove the adversarial
perturbations from images, while preserving sufficient information to
correctly classify them. Unlike the mentioned input-based techniques that only
implicitly (or not at all) induce mitigation of the bias towards textures, our
method makes this effect explicit by directly removing textures from the input
image.

##  3 Preliminaries and Notation

We introduce here the notation used throughout this paper. We focus on image
classification problems where a classifier $f_{\boldsymbol{\theta}}(\cdot)$ is
encoded by a DNN parameterized by
$\boldsymbol{\theta}\in\mathbb{R}^{|\boldsymbol{\theta}|}$. The parameters
vector $\boldsymbol{\theta}$ is optimized by minimizing a suitable loss
function $\mathcal{L}(\mathbf{x},y;\boldsymbol{\theta})$ on samples
$(\mathbf{x},y)$ drawn from the underlying training data distribution
$\mathcal{D}$. A training sample consists of an RGB (normalized) image
$\mathbf{x}\in\mathcal{X}\subset[0,1]^{3\times h\times w}$ and the
corresponding true label $y\in\\{1,\dots,K\\}$ (as common practice, we assume
$h=w$). We consider the cross-entropy loss optimized using Stochastic Gradient
Descent (SGD) [3], which updates the parameters $\boldsymbol{\theta}$ with a
noisy estimate of the gradient computed from a mini-batch of samples. The
predicted label corresponding to the input image $\mathbf{x}$ is denoted by
$\hat{y}=f_{\boldsymbol{\theta}}(\mathbf{x})$.

#### PGD

Given an input image $\mathbf{x}$, Projected Gradient Descent (PGD) [16] is a
gradient-based method for finding an adversarial example $\mathbf{x}^{\prime}$
that satisfies the condition
$\|\mathbf{x}^{\prime}-\mathbf{x}\|_{p}\leq\epsilon$, for given norm
$\|\cdot\|_{p}$ and perturbation budget $\epsilon>0$. Let $B$ denote the
$\ell_{p}$-ball of radius $\epsilon$ centered at $\mathbf{x}$. The attack is
initialized at a random point $\mathbf{x}_{0}\in B$, and repeatedly applies
(for a pre-specified number of iterations) the following update rule:

$\displaystyle\mathbf{x}_{k+1}$
$\displaystyle=\Pi_{B}(\mathbf{x}_{k}+\alpha\cdot\mathbf{g})$
$\displaystyle\quad\text{for }\mathbf{g}$
$\displaystyle=\operatorname*{arg\,max}_{\|\mathbf{u}\|_{p}\leq
1}\mathbf{u}^{\top}\nabla_{\mathbf{x}_{k}}\mathcal{L}(\mathbf{x}_{k},y)\;.$

Here, $\mathcal{L}(\mathbf{x},y)$ is a loss function (e.g., cross-entropy),
$\alpha$ is a step-size, $\Pi_{B}$ projects an input onto $B$, and
$\mathbf{g}$ is the gradient, that represents the steepest ascent direction
for a given $\ell_{p}$-norm.

##  4 Image Graph Extractor

In this Section we introduce the IGE framework. For the sake of readability,
we first highlight the guiding principles at its core (Section 4.1) and
describe thoroughly its key components, i.e., the merging rule (Section 4.2)
and the iterative filtering procedure (Section 4.3). All the design choices
characterizing the specific instance of the IGE method used for our
experiments are discussed in more detail in Section 4.4. In order to satisfy
parallel and efficient image processing we developed IGE in CUDA/C++ and
NVIDIA Thrust. Implementation details are given in Appendix A.

###  4.1 Guiding Principles

The information contained in an image depends on the effective resolution $r$
we see at: when ‘zooming out’ the image (i.e., decreasing $r$), pixels that
are distinct at higher resolutions are no more distinguishable and merge
together forming patches with larger size. Conversely, finer details emerge
when ‘zooming in’ the image (i.e., increasing $r$) and patches split into
multiple sub-patches with smaller sizes. Based on these observations, an image
$\mathbf{x}$ at a given resolution $r$ can be represented as a graph, where
nodes correspond to patches of similar pixels and edges encode information
about connectivity of the different patches. The size of each node amounts to
the number of pixels the corresponding patch is composed of. As a result, each
image can be partitioned into a collection of patches according to an
underlying graph representation.

###  4.2 Merging Rule

The objective of IGE is to emulate the perceptual effects described in Section
4.1. While the produced filtered images reveal complex elaborations that might
be hard to model, IGE is driven by a simple merging rule, which we describe
hereinafter.

Let $\mathbf{x}^{r}$ be an image at resolution $r$ and let
$\mathcal{G}^{r}=(\mathcal{V}^{r},\mathcal{E}^{r})$ be the corresponding IG,
where $\mathcal{V}^{r}$ is the set of nodes and $\mathcal{E}^{r}$ the set of
edges at resolution $r$. For simplicity, we assume edges to be binary-valued,
with $e_{i,j}^{r}=1$ indicating that $v_{i}^{r}$ and $v_{j}^{r}$ are
neighboring nodes ($e_{i,j}^{r}\in\mathcal{E}^{r}$ and
$v_{i}^{r},v_{j}^{r}\in\mathcal{V}^{r}$). In the image domain, this translates
into the fact that patches corresponding to $v_{i}^{r}$ and $v_{j}^{r}$ are
neighbors (according to some specified criterion). Let $s_{i}^{r}$ and
$s_{j}^{r}$ be the size of nodes $v_{i}^{r}$ and $v_{j}^{r}$, respectively. We
define the color associated with node $v_{i}^{r}$ (dubbed $c_{i}^{r}$) as the
average value of pixels in $v_{i}^{r}$. Given an opportune color distance
$d_{c}$, we define the adjusted distance $d_{a}$ as

$d_{a}(c_{i}^{r},c_{j}^{r};s_{i}^{r},s_{j}^{r},r)=\phi(d_{c}(c_{i}^{r},c_{j}^{r}),s_{i}^{r},s_{j}^{r},r)$
(1)

for some function $\phi(\cdot)$. Notice that $d_{a}$ is function of the color
distance $d_{c}(c_{i}^{r},c_{j}^{r})$, the size of nodes $s_{i}^{r}$ and
$s_{j}^{r}$, and the resolution $r$. Neighboring nodes $v_{i}^{r}$ and
$v_{j}^{r}$ are merged if

$d_{a}(c_{i}^{r},c_{j}^{r};s_{i}^{r},s_{j}^{r},r)<\tau(d_{0},r)$ (2)

where $\tau(d_{0},r)$ is the threshold (function of the resolution $r$) that
determines if two nodes are in the same patch at a given resolution $r$. The
parameter $d_{0}$ represents the minimum color distance perceived at maximum
resolution $r_{0}$, corresponding to the resolution of the original
(unfiltered) image $\mathbf{x}=\mathbf{x}^{r_{0}}$. We assume by convention
that $r_{0}=1$. In order to be consistent with the guiding principles in
Section 4.1, $\tau$ must be a monotonically decreasing function with respect
to $r$, that is $\dfrac{\partial\tau(d_{0},r)}{\partial r}<0$. Indeed, as we
zoom out the image ($r$ decreases), pixels that are more and more distant from
each other in color space become indistinguishable, resulting in a less strict
threshold.

The adjustment of the color distance $d_{c}$ modeled by Equation 1 takes into
account the size of nodes. The rationale of the proposed procedure can be
better explained by an example: a black 1-pixel node surrounded by white
pixels becomes perceptually indistinguishable from its neighbors at a far
larger resolution if compared to a black 5-pixels node.

In summary, the core of the IGE framework is represented by the merging rule
described in this Section. The key ingredients defining the rule are: the
criterion used to determine neighboring patches, the color distance $d_{c}$,
the adjusted distance $d_{a}$ (specifically, the function $\phi$), the minimum
color distance perceived at maximum resolution $d_{0}$, and the threshold
function $\tau$. The choice of the latters is subject to the following
constraints:

  * C1.

If all the variables but $r$ are kept fixed, it is required that
$\frac{\partial\tau(r)}{\partial r}<0$ and/or $\frac{\partial
d_{a}(r)}{\partial r}>0$.

  * C2.

If all the variables but $s_{i}^{r}$ are kept fixed, it is required that
$\frac{\partial d_{a}(s_{i}^{r})}{\partial s_{i}^{r}}>0$.

  * C3.

If all the variables but $d_{c}$ are kept fixed, $\displaystyle\frac{\partial
d_{a}}{\partial d_{c}}>0$.

The IGE framework is general in that it allows flexibility in the choice of
the key ingredients as long as the constraints listed above are satisfied. In
Section 4.4 are given the specific design choices adopted in this work.

###  4.3 Iterative Filtering

The iterative filtering procedure consists in repeatedly performing merging
operations, after a preliminary phase for initialization. The functioning of
the whole IGE framework is summarized in Algorithm 1.

Let $r^{*}$ be the target resolution and $r_{0}=1$ the initial resolution
corresponding to the original image $\mathbf{x}$, as mentioned in Section 4.2.
In our framework, the target resolution $r^{*}\in\left]0,r_{0}\right[$,
together with the filtering step size $\Delta_{r}$, defines the number of
filtering steps $N_{r}=\frac{1-r^{*}}{\Delta_{r}}$ to be performed.

The initialization phase is aimed at extracting the IG $\mathcal{G}^{r_{0}}$
associated with the original image $\mathbf{x}=\mathbf{x}^{r_{0}}$. We
represent this operation with the generic function $\mathtt{img2graph}$.

The filtering routine can be seen as an anti-causal discrete dynamical system
$\mathcal{G}^{r-\Delta_{r}}=m(\mathcal{G}^{r})$, where $m$ is the map
obtaining $\mathcal{G}^{r-\Delta_{r}}$ from $\mathcal{G}^{r}$. Specifically,
$m$ is composed of the following elementary operations:

  1. 1.

Merge nodes according to Equation 2 (we denote this operation with the
function $\mathtt{merge}$).

  2. 2.

Update the color value of each node by averaging the values of all pixels in
the node (we denote this operation with the function $\mathtt{avg\\_color}$).

The map $m$ is applied iteratively $N_{r}$ times until the target resolution
$r^{*}$ is reached. The final step consists in reconstructing the filtered
image at the target resolution from the final IG $\mathcal{G}^{r^{*}}$. We
represent this operation with the generic function $\mathtt{graph2img}$.

A potential issue that might arise is the co-existence, for a specific node,
of multiple candidate nodes for the merging operation. Although not optimal
from a computational point of view, one trivial solution would be to choose
the candidate node with the smallest adjusted distance. In this regard, the
filtering step size $\Delta_{r}$ controls the goodness of the filtering
process. Small values of $\Delta_{r}$ help in mitigating the problem of
multiple candidate nodes for merging. Since $N_{r}\propto\Delta_{r}^{-1}$,
also this strategy should call for thoughtful considerations about
computational cost. In light of this, we set $\Delta_{r}$ to the biggest value
that guarantees the required quality of the filtered images.

Notice that the IG associated with the filtered image at the target resolution
$r^{*}$ can be extracted with no extra computations as it comes as an
intermediate by-product of the filtering procedure. The IG can be enriched by
considering additional node attributes besides node color (e.g., shape) and
encoding additional information about connectivity as edge attributes. A
simplified example of IG is depicted in Figure 2.

Figure 2: A simplified example of (part of) an IG associated with an image
from the ImageNet dataset. For graphical purposes, only main nodes are
displayed.

Input: original image $\mathbf{x}$

Output: filtered image $\mathbf{x}^{r^{*}}$

Parameters : $r^{*}$, $d_{c}$, $d_{a}$, $d_{0}$, $\tau$

$\mathcal{G}^{r}=\mathtt{img2graph}(\mathbf{x}^{r_{0}})$

// init

$r=r_{0}-\Delta_{r}$

// update res

1 while  _$r >r^{*}$_ do

2 $\mathcal{G}^{r}=\mathtt{merge}(\mathcal{G}^{r};d_{c},d_{a},d_{0},\tau)$

3 $\mathcal{G}^{r}=\mathtt{avg\\_color}(\mathcal{G}^{r})$

$r=r-\Delta_{r}$

// update res

4

$\mathbf{x}^{r^{*}}=\mathtt{graph2img}(\mathcal{G}^{r})$

// rec filt img

Algorithm 1 IGE pseudocode.

###  4.4 Design Choices

As stated in Section 4.2, to define an instantiation of the IGE method we need
to specify: (i) the criterion used to define neighboring patches (i.e., the
minimum number of neighboring pixels), (ii) the color distance $d_{c}$, (iii)
the adjusted distance $d_{a}$ (specifically, we should define $\phi$), (iv)
the minimum color distance perceived at maximum resolution $d_{0}$ and (v) the
merging threshold function $\tau$.

As for (i), we consider two distinct patches as neighbors if their connection
is at least one pixel. While simple, this choice could cause merges that are
not optimal. A possible improvement may come by defining a different
threshold.

In respect of the color distance $d_{c}$, we tested two formulations:
CIEDE2000 [24] and Euclidean. While CIEDE2000 offers more human-aligned color
perception, we did not appreciated significant differences. Thus, we opted for
the Euclidean distance. Given two RGB vectors
$p=\left[p_{R},p_{G},p_{B}\right]^{\top}$ and
$q=\left[q_{R},q_{G},q_{B}\right]^{\top}$ their Euclidean distance is given by
$d_{c}(p,q)=\sqrt{(p_{R}-q_{R})^{2}+(p_{G}-q_{G})^{2}+(p_{B}-q_{B})^{2}}$.

For the adjusted distance $d_{a}$, we adopted the following formulation

$\displaystyle d_{a}$
$\displaystyle=d_{c}(c_{i}^{r},c_{j}^{r})\cdot\tilde{\phi}(s_{i}^{r},s_{j}^{r},r),$
(3) $\displaystyle\tilde{\phi}(s_{i}^{r},s_{j}^{r},r)$
$\displaystyle={\Big{(}1+e^{(-\alpha(\min\\{s_{i}^{r},s_{j}^{r}\\}-\frac{\beta}{r}))}}\Big{)}^{-1/r}$
(4)

where the color distance $d_{c}$ and adjustment $\tilde{\phi}$ terms are
decoupled. We set $\alpha=0.04$ and $\beta=10$.

The term $\tilde{\phi}$ has the objective of reducing the perceived color
distance when the resolution is low and/or the size of nodes is small. The
size adjustment is represented by the term $\min\\{s_{i}^{r},s_{j}^{r}\\}$,
meaning that the decision depends on the smallest node. This is aimed to
reproduce the behavior of vanishing nodes as we zoom out the image. However,
this rule may be refined by also considering the relative size, that is
$s_{i}^{r}/s_{j}^{r}$. The term $\frac{\beta}{r}$ is helpful to incentivize
the early merge of small nodes. We want to remark that we tested several rules
$\phi$. Interestingly, the quality of results is stable with respect to the
choice of $\phi$ as soon as it satisfies the constraints of the IGE framework.

Regarding the threshold function $\tau$ we investigated two different forms:

  * T1

: $\tau(d_{0},r)=\frac{d_{0}}{r}$;

  * T2

: $\tau(d_{0},r)=d_{0}\frac{1-(r-r_{m})}{r_{m}}$, where $r_{m}=0.1$.

T1 is not linear. Thus, at low resolutions a small change in $r$ can have a
visible impact on filtering. We opted for T2 even if also T1 provided good
results. Moreover, we would like to remark that there is no ‘correct’ rule.
The form controls how we establish to perceive the zooming and image
structures. We set $d_{0}=0.03$.

##  5 Filtering As a Defense

In this Section we describe FAD, our proposed defense against adversarial
attacks leveraging the IGE iterative filtering procedure. We first discuss the
motivation in Section 5.1 and then introduce the full defense protocol in
Section 5.2.

Figure 3: Block diagram describing the FAD defense.

###  5.1 Motivation

We want to provide an intuitive argument about the effectiveness of FAD. Let
$\mathbf{x}$ be an input image of $n=h\times w$ pixels and let
$\mathbf{x}^{r^{*}}$ its filtered counterpart at a given target resolution
$r^{*}$. We define $p$ as the effective number of pixels in the filtered image
$\mathbf{x}^{r^{*}}$, that is, the number of patches the filtered image is
composed of. In the context of the IGE framework, $p$ is equivalent to the
number of nodes in the IG $\mathcal{G}^{r^{*}}$ associated with the filtered
image $\mathbf{x}^{r^{*}}$. Intuitively, we may think the filtered image as a
$\sqrt{p}\times\sqrt{p}$ image. As $p\leq n$ (and potentially $p\ll n$), it is
evident that the total number of directions that an attacker can exploit
decreases significantly if compared to the original input image.

As discussed in Section 1, in order to be effective the attacker must have the
possibility to create local micro-patterns characterized by precise relative
spatial locations. Said micro-patterns, correlated together across layers,
cause the classification error. In standard settings (i.e., when dealing with
unfiltered images), the attacker has the freedom to perturb the image along
almost any direction not too far from the original one. In visual terms, this
amounts to the fact that the adversarial example is perceptually very similar
to the original image. When considering filtered images, this assumption is
not valid anymore and the actual number of effective directions the attacker
can focus on is limited. This fact can be intuitively explained by comparing
the underlying structures of original and filtered images. For original
images, the structure is represented by a rigid uniform grid: if we assume
$8$-connectivity, each pixel’s neighborhood has fixed size equal to $8$. The
effect of perturbing a single pixel directly impacts (in terms of created
‘virtual edges’) only the 8 pixel’s neighbors. For filtered images, instead,
the structure is represented by a non-uniform grid defined by the IG
associated with the image. This allows patches (corresponding to nodes in the
IG) to potentially border with many other patches. In the worst case scenario,
from the perspective of the attacker, a single patch could border with all
other patches. In this case, the perturbation of a single patch would impact
all other patches (in terms of created ‘virtual edges’), leading to a loss in
precision for the attacker.

###  5.2 Defense Protocol

The IGE filtering procedure outlined in Section 4 causes a shift in the input
image statistics that results in distribution shifts of batch norm layers’
inputs. Thus, especially in complex datasets like ImageNet, models are less
accurate when evaluated on filtered images as population statistics estimated
on training data do not match statistics of test data.

To overcome this mismatch, the first phase of the protocol is represented by
fine-tuning the pretrained model to align to filtered images. Since the
semantic content of input images is preserved, the fine-tuning process
requires only a few epochs to converge.

The second phase is the defense itself, which consists in running the IGE
iterative filtering routine as a pre-processing step as soon as an input image
is queried. Figure 3 reports the described FAD defense. Notice that the FAD
defense module inherits the parameters defining the specific instantiation of
the IGE filtering method.

##  6 Results

In this Section we present three sets of results: (i) the visual inspection of
filtered images, (ii) FAD under attack, and (iii) data augmentation with
filtered images. Additional analyses are presented in the Appendices B and C.

###  6.1 Filtering

In Figure 4 we show the progression of the iterative filtering process, i.e.,
the resulting filtered images as the target resolution varies. When reducing
the target resolution, the number of nodes progressively decreases. However,
the semantic content of the image remains preserved. In Figure 5 we show few
examples of filtered images with fixed target resolution $r^{*}=0.6$. The
final number of nodes depends on the complexity of the original image. On
average, at target resolution $r^{*}=0.6$, images have roughly $[150,1000]$
nodes.

Figure 4: Example of the effect of the IGE filtering when decreasing the
target resolution (from left to right).

(a) Original Images.

(b) Filtered Images.

Figure 5: Examples of filtered ImageNet images with $r^{*}=0.6$ and
$\Delta_{r}=0.1$.

###  6.2 Robustness Evaluation

As discussed by [27], one should develop adaptive attacks to test the
robustness of a defense. In this context, the Backward Pass Differentiable
Approximation (BPDA) is a gradient approximation technique [1] for dealing
with non-differentiable components. Let $\tilde{f}$ be the composition
$\tilde{f}(x)=f\circ h(x)$ where $h$ is non-differentiable (or hard to
differentiate). In our case $h$ is the FAD module and $f$ represents the fine-
tuned model. To approximate the total gradient $\nabla_{x}\tilde{f}(x)$, BPDA
needs to find a differentiable function $g(x)$ such that $g(x)\approx h(x)$.
Then, when computing the gradient $\nabla_{x}\tilde{f}(x)$, the forward pass
is a standard forward pass through $\tilde{f}$ (including the forward pass
through $h$), while in the backward pass $h$ is replaced by $g$. In our case,
a reasonable choice for $g$ is the identity function $g(x)=x$ as the FAD
module is a simple ‘denoising’ layer.

A vanilla application of BPDA may not be effective as one step of gradient
might not allow the attacker to craft adversarial patches. Hence, we apply
BPDA (as an outer loop) with the only difference of an inner attack (addressed
to $f$) where multiple PGD steps are applied.

###  6.3 Experimental Settings

We validate FAD with the datasets CIFAR-10 [13], CIFAR-100 [13] and ImageNet
[6]. As discussed in Section 5.2, filtered images exhibit different statistics
if compared to their original counterparts. For this reason, the first phase
of the FAD protocol is a fine-tuning of the model. While the optimal solution
would be to fine-tune the model with the same resolution used in the FAD
module, we decided to fine-tune the model for only one resolution $r^{*}=0.6$.
We do this as it provides a good balance between simplicity and accuracy. For
all the datasets, we employ ResNet-18 [10] architectures trained for 150
epochs, initial learning rate $\eta=0.1$ and a drop of $\eta$ by a factor $10$
every 50 epochs.

As regards BPDA, we evaluate FAD with $50$ outer steps and $20$ inner steps.
Attacks are bounded in $\ell_{2}$-norm. Due to computational constraints, we
tested on $30\%$ of the test sets with shuffling.

|  | Accuracy |  | Loss
---|---|---|---|---
|  | $\epsilon_{test}$ |  | $\epsilon_{test}$
| $r^{*}$ | 0 | 1 | 2 | 3 | 4 |  | 0 | 1 | 2 | 3 | 4
CIFAR-10 | 0.4 | 78.45 | 70.15 | 55.37 | 35.09 | 21.16 |  | 0.68 | 1.03 | 1.74 | 3.19 | 4.56
0.5 | 82.32 | 74.38 | 52.67 | 27.31 | 11.36 |  | 0.54 | 0.85 | 1.91 | 4.17 | 6.48
0.6 | 85.06 | 73.18 | 49.80 | 18.26 | 4.39 |  | 0.48 | 0.91 | 2.16 | 5.56 | 9.44
0.7 | 86.07 | 73.63 | 44.50 | 8.66 | 0.68 |  | 0.44 | 0.91 | 2.68 | 8.14 | 13.95
0.8 | 89.55 | 74.48 | 33.40 | 1.82 | 0.00 |  | 0.33 | 0.90 | 3.87 | 12.57 | 20.18
CIFAR-100 | 0.4 | 41.08 | 34.99 | 25.91 | 19.04 | 15.01 |  | 2.90 | 3.47 | 4.49 | 5.63 | 6.56
0.5 | 47.88 | 38.74 | 27.25 | 16.50 | 12.79 |  | 2.47 | 3.17 | 4.59 | 6.35 | 7.58
0.6 | 52.60 | 41.24 | 25.52 | 13.93 | 7.49 |  | 2.14 | 3.09 | 4.97 | 7.40 | 9.80
0.7 | 59.54 | 43.23 | 23.40 | 8.72 | 2.99 |  | 1.80 | 3.03 | 5.45 | 9.54 | 13.82
0.8 | 63.74 | 45.31 | 19.17 | 3.32 | 0.23 |  | 1.57 | 2.88 | 6.88 | 14.71 | 23.82

|  | Accuracy |  | Loss
---|---|---|---|---
|  | $\epsilon_{test}$ |  | $\epsilon_{test}$
| $r^{*}$ | 0 | 3 | 5 | 10 |  | 0 | 3 | 5 | 10
ImageNet | 0.4 | 31.51 | 26.44 | 22.79 | 14.10 |  | 3.71 | 4.14 | 4.55 | 5.89
0.5 | 37.94 | 31.90 | 23.61 | 12.81 |  | 3.14 | 3.71 | 4.31 | 6.01
0.6 | 43.58 | 31.86 | 26.77 | 12.16 |  | 2.68 | 3.39 | 3.96 | 6.42
0.7 | 49.54 | 36.92 | 28.62 | 9.14 |  | 2.28 | 3.02 | 3.84 | 7.16
| 0.8 | 56.61 | 41.86 | 28.84 | 5.92 |  | 1.88 | 2.68 | 3.72 | 8.53

Table 1: Robust accuracy (in $\%$) and loss for CIFAR-10 , CIFAR-100 and
ImageNet under BPDA $\ell_{2}$-attack at different resolution scales. Attacked
models are fine-tuned with $r^{*}=0.6$.

###  6.4 FAD

Table 3 presents the results when attacking FAD with BPDA for CIFAR-10,
CIFAR-100 and ImageNet, respectively. As the resolution increases, the
attacker prevails the defense, and the robust accuracy decreases. The same
holds for $\epsilon$. For values of $\epsilon$ bigger than the reported ones,
accuracy quickly approaches zero. These behaviors are adherent to what one
should expect. By reducing $r^{*}$, fewer textures are present, and the
attacker is less effective. As regards the attack strength, when $\epsilon$
increases, the attacker succeeds in creating new relatively small but strong
patches or in removing patches that impact the shape (see Figure 6).

Figure 6: Bottom: Filtered Images. Top: Adversarial Images. Attacks remove
important patches or add small ones.

| $f_{nat}$ |  | $f_{aug}$
---|---|---|---
Corruption | Level 1 | Level 2 | Level 3 | Level 4 | Level 5 |  | Level 1 | Level 2 | Level 3 | Level 4 | Level 5
$\mathtt{gaussian\\_noise}$ | 78.8 | 59.45 | 40.54 | 33.52 | 28.78 |  | 86.05 | 76.05 | 61.71 | 54.64 | 47.79
$\mathtt{shot\\_noise}$ | 86.81 | 77.78 | 55.88 | 47.29 | 35.05 |  | 89.17 | 84.03 | 69.15 | 63.04 | 52.39
$\mathtt{impulse\\_noise}$ | 85.39 | 74.64 | 64.77 | 42.31 | 26.59 |  | 84.34 | 73.92 | 63.44 | 43.09 | 28.72
$\mathtt{glass\\_blur}$ | 53.34 | 53.26 | 59.27 | 41.63 | 47.08 |  | 85.27 | 85.56 | 86.34 | 76.37 | 78.08
$\mathtt{defocus\\_blur}$ | 94.91 | 93.91 | 90.22 | 83.19 | 62.77 |  | 92.95 | 92.92 | 92.59 | 91.29 | 88.35
$\mathtt{motion\\_blur}$ | 92.02 | 88.82 | 83.09 | 84.12 | 77.5 |  | 91.14 | 90.37 | 88.51 | 88.71 | 85.29
$\mathtt{zoom\\_blur}$ | 94.25 | 92.71 | 90.53 | 85.05 | 77.86 |  | 92.87 | 93.09 | 92.87 | 92.23 | 91.28
$\mathtt{fog}$ | 94.77 | 93.98 | 93.16 | 91.07 | 79.07 |  | 92.68 | 91.51 | 89.19 | 84.52 | 69.08
$\mathtt{frost}$ | 93.41 | 91.46 | 88.59 | 87.35 | 83.55 |  | 92.0 | 91.0 | 89.03 | 88.44 | 85.49
$\mathtt{snow}$ | 90.82 | 82.02 | 77.04 | 74.55 | 67.71 |  | 91.7 | 88.82 | 83.03 | 78.65 | 77.42
$\mathtt{contrast}$ | 94.73 | 93.19 | 91.73 | 87.8 | 56.06 |  | 92.85 | 90.99 | 88.68 | 83.82 | 53.87
$\mathtt{brightness}$ | 94.9 | 94.72 | 94.33 | 93.95 | 92.55 |  | 92.99 | 92.81 | 92.49 | 91.91 | 90.4
$\mathtt{jpeg\\_compression}$ | 87.27 | 82.08 | 79.91 | 77.25 | 73.96 |  | 88.89 | 86.19 | 85.18 | 83.83 | 82.0
$\mathtt{pixelate}$ | 92.24 | 87.91 | 82.65 | 64.62 | 44.4 |  | 91.86 | 91.49 | 90.82 | 89.94 | 86.69
$\mathtt{elastic\\_transform}$ | 91.07 | 90.7 | 87.57 | 81.37 | 76.54 |  | 90.24 | 90.84 | 90.9 | 88.64 | 85.07

Table 2: CIFAR-10. Accuracy (in $\%$) under data corruptions and different
levels of severity. (Left) Natural model. (Right) Augmented model.

###  6.5 Data Augmentation with Filtered Images

As IGE removes textures, we may expect more robustness to corruptions. To test
this hypothesis, we trained a ResNet-18 model $f_{aug}$ on CIFAR-10 by
augmenting the training set of natural images $\mathbf{x}$ with their
corresponding filtered counterparts $\mathbf{x}^{r^{*}}$ ($r^{*}=0.6$). The
resulting augmented dataset is denoted
$\mathcal{D}_{aug}=\\{\mathcal{D},\mathcal{D}^{r^{*}}\\}$. The test set used
for the evaluation is the original one (i.e., formed only by natural images).

As we may expect, the accuracy of $f_{aug}$ drops in relation to that of
$f_{nat}$ (from roughly $95\%$ to $93.3\%$). However, if we look at the
response of the two models under data corruption, $f_{aug}$ is almost always
more accurate than $f_{nat}$. In some cases, e.g., glass blur, the gap is
remarkable. The results are reported in Table 2. The code for applying data
corruptions has been adapted from [22].

##  7 Discussion

In the following, we list the major limitations of IGE. Regarding the
implementation, although images are pre-processed using a GPU, it is still not
ready for real-time applications. However, we pinpoint several improvements.
The first is to increase computational performance by efficiently using the
GPU architecture (e.g., threads and warping). The second direction regards the
design of tailored merging rules to reduce the number of steps required.

We want to remark that we did not spend much effort tuning parameters and
merging rules. Indeed, our main interest is to show that the model gains
robustness by injecting the local stability of pixels. We believe that the
design of better merging rules will help improving accuracy and robustness.
Moreover, we expect further improvements by training models on a tailored
resolution instead of using a unique model for all the resolutions.

##  8 Conclusions

This paper introduces a GPU-based image filtering framework, Image-Graph
Extractor (IGE). Its outputs are the filtered image and the corresponding
Image-Graph (IG). One of the several benefits of IGE is that it effectively
enforces robustness to bounded perturbations by removing brittle textures from
images.

Although the main focus is on the use of IGE for adversarial robustness, its
flexible IG representation opens to several directions for future research.
The first one concerns the use of the IG representation to cast image
classification as a graph classification problem. This would allow to easily
inject in the model user-defined data invariances (e.g., to the scale of
objects). Another research direction consists in defining a specialized data
augmentation strategy where nodes can be cleverly perturbed so as to remove
common shortcut features such as the background color.

## References

  * [1] Anish Athalye, Nicholas Carlini, and David Wagner.  Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples.  arXiv:1802.00420, 2018.
  * [2] Yang Bai, Yuyuan Zeng, Yong Jiang, Shu-Tao Xia, Xingjun Ma, and Yisen Wang.  Improving adversarial robustness via channel-wise activation suppressing.  arXiv preprint arXiv:2103.08307, 2021.
  * [3] Léon Bottou, Frank E Curtis, and Jorge Nocedal.  Optimization methods for large-scale machine learning.  Siam Review, 60(2):223–311, 2018.
  * [4] Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John C Duchi.  Unlabeled data improves adversarial robustness.  arXiv preprint arXiv:1905.13736, 2019.
  * [5] Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier.  Parseval networks: Improving robustness to adversarial examples.  In International Conference on Machine Learning, pages 854–863. PMLR, 2017.
  * [6] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 248–255. Ieee, 2009.
  * [7] Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M Roy.  A study of the effect of jpg compression on adversarial images.  arXiv preprint arXiv:1608.00853, 2016.
  * [8] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  arXiv:1412.6572, 2014.
  * [9] Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens Van Der Maaten.  Countering adversarial images using input transformations.  arXiv preprint arXiv:1711.00117, 2017.
  * [10] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Identity mappings in deep residual networks.  arXiv:1603.05027, 2016.
  * [11] Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry.  Adversarial examples are not bugs, they are features.  In Advances in Neural Information Processing Systems, pages 125–136, 2019.
  * [12] Harini Kannan, Alexey Kurakin, and Ian Goodfellow.  Adversarial logit pairing.  arXiv preprint arXiv:1803.06373, 2018.
  * [13] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.  Cifar-10 and cifar-100 datasets.  URl: https://www. cs. toronto. edu/kriz/cifar. html, 6, 2009.
  * [14] Alexey Kurakin, Ian Goodfellow, and Samy Bengio.  Adversarial machine learning at scale.  arXiv:1611.01236, 2016.
  * [15] Jiajun Lu, Hussein Sibai, Evan Fabry, and David Forsyth.  No need to worry about adversarial examples in object detection in autonomous vehicles.  arXiv preprint arXiv:1707.03501, 2017.
  * [16] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.  Towards deep learning models resistant to adversarial attacks.  arXiv:1706.06083, 2017.
  * [17] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Jonathan Uesato, and Pascal Frossard.  Robustness via curvature regularization, and vice versa.  11 2018.
  * [18] Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu.  Improving adversarial robustness via promoting ensemble diversity.  In International Conference on Machine Learning, pages 4970–4979. PMLR, 2019.
  * [19] Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami.  Distillation as a defense to adversarial perturbations against deep neural networks.  In 2016 IEEE Symposium on Security and Privacy (SP), pages 582–597. IEEE, 2016.
  * [20] Daniel Peter Playne and Ken Hawick.  A new algorithm for parallel connected-component labelling on gpus.  IEEE Transactions on Parallel and Distributed Systems, 29(6):1217–1230, 2018.
  * [21] Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, and Pushmeet Kohli.  Adversarial robustness through local linearization.  arXiv preprint arXiv:1907.02610, 2019.
  * [22] Hadi Salman, Andrew Ilyas, Logan Engstrom, Sai Vemprala, Aleksander Madry, and Ashish Kapoor.  Unadversarial examples: Designing objects for robust vision.  arXiv preprint arXiv:2012.12235, 2020.
  * [23] Uri Shaham, Yutaro Yamada, and Sahand Negahban.  Understanding adversarial training: Increasing local stability of neural nets through robust optimization.  arXiv preprint arXiv:1511.05432, 2015.
  * [24] Gaurav Sharma, Wencheng Wu, and Edul N Dalal.  The ciede2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations.  Color Research & Application: Endorsed by Inter-Society Color Council, The Colour Group (Great Britain), Canadian Society for Color, Color Science Association of Japan, Dutch Society for the Study of Color, The Swedish Colour Centre Foundation, Colour Society of Australia, Centre Français de la Couleur, 30(1):21–30, 2005.
  * [25] Changhao Shi, Chester Holtz, and Gal Mishne.  Online adversarial purification based on self-supervised learning.  In International Conference on Learning Representations, 2020.
  * [26] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.  Intriguing properties of neural networks.  arXiv:1312.6199, 2013.
  * [27] Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry.  On adaptive attacks to adversarial example defenses.  arXiv preprint arXiv:2002.08347, 2020.
  * [28] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry.  Robustness may be at odds with accuracy.  In International Conference on Learning Representations, 2019.
  * [29] Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan L Yuille, and Kaiming He.  Feature denoising for improving adversarial robustness.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 501–509, 2019.

##  Appendix A Implementation Details

In order to satisfy parallel and efficient image processing we developed IGE
in CUDA/C++ and NVIDIA Thrust. The first piece of filtering is the extraction
of connected-components. We use the GPU-based CCL (Connected Component
Labeling) algorithm adapted from [20]. The output of CCL is the $h\times w$
label matrix, where each location contains the identifier of the node/patch.
After running CCL, we run a CUDA-kernel called edge which takes in input the
label matrix edge <<< grid, block, block.x*block.y>>> and outputs the graph
(node, edge) in dataframe-like standard. That is, each connection consists of
a row with source, destination and attributes. Attributes are color distance,
number of connected pixels, etc. After the first phase, the actual filtering
procedure starts. At each step, the merge function is implemented in two
phases: the first function has the objective to define the candidates for
merging, while the second function actually merge nodes. After the merge, the
filtered image is reconstructed with the function $\mathtt{avg\\_color}$ that
takes in input the label matrix and the original images and return the
filtered images by averaging color for each node. Moreover, the graph can be
optionally saved on csv files and elaborated with the NVIDIA library libcudf
(https://github.com/rapidsai/cudf/).

The python wrapper has been developed using pybind11. For future releases, we
plan to develop a Torch extensions.

##  Appendix B IGE

With high-dimensional images, we have found beneficial to apply a de-noising
pre-processing that consists of a partial removal of textures by the
minimization of the following:

$\operatorname*{arg\,min}_{\theta}L_{TV}(\theta)+L_{surf}(\theta)+\lambda\|x-\theta\|^{2}_{2}$

where $x$ is the raw image, $L_{TV}$ is the total variation loss and
$L_{surf}$ is the surface loss. The objective of $L_{surf}$ loss is to
minimize the number of changes in color pixel. In fact, total variation can be
low even when the number of changes in pixels is very high. This happens for
example when changes in color are minimal (e.g. 1 or 2 out of 255). The change
of pixels’ color is defined with the step function $\mathbf{1}(\cdot)$, which
is not differentiable. We approximate the step function with the
differentiable surrogate:

$\dfrac{1}{1+e^{-{\alpha x}}}$

where $\alpha\gg 1$.

In Figure 7 are displayed filtered images at decreasing values of the target
resolution $r^{*}$. Moreover, in Figures 8 and 9 we present an example of
computed graphs from different resolutions $r^{*}$.

Figure 7: Examples of the effect of the IGE filtering when decreasing the
target resolution. The values of the target resolutions are (from left to
right): $1$, $0.7$, $0.45$, $0.35$.

Figure 8: Example of Graph Image for a $1000\times 1000$ image. (Left)
Original image. (Right) Graph over the image with $r^{*}=0.5$. (Bottom) Image
Graph. The width of edges is proportional to the fraction of perimeter shared
by the source node with the target node.

Figure 9: Example of Graph Image for a $1000\times 1000$ image. (Left)
Original image. (Right) Graph over the image with $r^{*}=0.6$. (Bottom) Image
Graph. The width of edges is proportional to the fraction of perimeter shared
by the source node with the target node.

##  Appendix C FAD

Models are fine-tuned for $10$ epochs with learning rate $\eta=0.001$ and
fixed resolution $r^{*}=0.6$. For ImageNet, we sample $300$ filtered images
per class from the training set. In Table 3 and Table 4 are reported the
results when attacking FAD with BPDA with attacks bounded in $\ell_{2}$-norm
and $\ell_{\infty}$-norm, respectively. As regards BPDA, we evaluate FAD with
$50$ outer steps and $20$ inner steps. We did not take into account other
white-box attacks (such as C&W) since we consider PGD with BPDA to be the
strongest attack we could evaluate against.

|  | Accuracy |  | Loss
---|---|---|---|---
|  | $\epsilon_{test}$ |  | $\epsilon_{test}$
| $r^{*}$ | 0 | 1 | 2 | 3 | 4 |  | 0 | 1 | 2 | 3 | 4
CIFAR-10 | 0.4 | 78.45 | 70.15 | 55.37 | 35.09 | 21.16 |  | 0.68 | 1.03 | 1.74 | 3.19 | 4.56
0.5 | 82.32 | 74.38 | 52.67 | 27.31 | 11.36 |  | 0.54 | 0.85 | 1.91 | 4.17 | 6.48
0.6 | 85.06 | 73.18 | 49.80 | 18.26 | 4.39 |  | 0.48 | 0.91 | 2.16 | 5.56 | 9.44
0.7 | 86.07 | 73.63 | 44.50 | 8.66 | 0.68 |  | 0.44 | 0.91 | 2.68 | 8.14 | 13.95
0.8 | 89.55 | 74.48 | 33.40 | 1.82 | 0.00 |  | 0.33 | 0.90 | 3.87 | 12.57 | 20.18
CIFAR-100 | 0.4 | 41.08 | 34.99 | 25.91 | 19.04 | 15.01 |  | 2.90 | 3.47 | 4.49 | 5.63 | 6.56
0.5 | 47.88 | 38.74 | 27.25 | 16.50 | 12.79 |  | 2.47 | 3.17 | 4.59 | 6.35 | 7.58
0.6 | 52.60 | 41.24 | 25.52 | 13.93 | 7.49 |  | 2.14 | 3.09 | 4.97 | 7.40 | 9.80
0.7 | 59.54 | 43.23 | 23.40 | 8.72 | 2.99 |  | 1.80 | 3.03 | 5.45 | 9.54 | 13.82
0.8 | 63.74 | 45.31 | 19.17 | 3.32 | 0.23 |  | 1.57 | 2.88 | 6.88 | 14.71 | 23.82

|  | Accuracy |  | Loss
---|---|---|---|---
|  | $\epsilon_{test}$ |  | $\epsilon_{test}$
| $r^{*}$ | 0 | 3 | 5 | 10 |  | 0 | 3 | 5 | 10
ImageNet | 0.4 | 31.51 | 26.44 | 22.79 | 14.10 |  | 3.71 | 4.14 | 4.55 | 5.89
0.5 | 37.94 | 31.90 | 23.61 | 12.81 |  | 3.14 | 3.71 | 4.31 | 6.01
0.6 | 43.58 | 31.86 | 26.77 | 12.16 |  | 2.68 | 3.39 | 3.96 | 6.42
0.7 | 49.54 | 36.92 | 28.62 | 9.14 |  | 2.28 | 3.02 | 3.84 | 7.16
| 0.8 | 56.61 | 41.86 | 28.84 | 5.92 |  | 1.88 | 2.68 | 3.72 | 8.53

Table 3: Robust accuracy (in $\%$) and loss for CIFAR-10 , CIFAR-100 and
ImageNet under BPDA $\ell_{2}$-attack at different resolution scales. Attacked
models are fine-tuned with $r^{*}=0.6$.

|  | Accuracy |  | Loss
---|---|---|---|---
|  | $\epsilon_{test}$ |  | $\epsilon_{test}$
| $r^{*}$ | $0/255$ | $8/255$ | $16/255$ | $24/255$ | $32/255$ |  | $0/255$ | $8/255$ | $16/255$ | $24/255$ | $32/255$
CIFAR-10 | 0.4 | 78.65 | 71.68 | 59.47 | 46.16 | 33.79 |  | 0.68 | 0.93 | 1.45 | 2.16 | 3.15
0.5 | 83.79 | 74.02 | 61.78 | 43.72 | 26.79 |  | 0.52 | 0.88 | 1.43 | 2.45 | 3.81
0.6 | 84.90 | 74.12 | 59.70 | 40.40 | 18.62 |  | 0.47 | 0.88 | 1.53 | 2.80 | 5.19
0.7 | 86.91 | 74.28 | 56.51 | 28.74 | 2.12 |  | 0.39 | 0.90 | 1.77 | 4.04 | 11.73
0.8 | 88.57 | 70.87 | 44.86 | 1.17 | 0.00 |  | 0.37 | 1.06 | 2.72 | 13.60 | 25.87
CIFAR-100 | 0.4 | 43.46 | 34.96 | 29.43 | 23.21 | 16.34 |  | 2.79 | 3.41 | 4.07 | 4.84 | 5.78
0.5 | 48.57 | 41.34 | 30.24 | 23.18 | 15.85 |  | 2.41 | 3.08 | 4.02 | 5.04 | 6.29
0.6 | 53.12 | 39.26 | 30.63 | 20.25 | 12.70 |  | 2.13 | 3.15 | 4.13 | 5.57 | 7.38
0.7 | 59.90 | 43.36 | 29.07 | 17.58 | 5.89 |  | 1.81 | 2.93 | 4.48 | 6.60 | 11.39
0.8 | 63.22 | 43.95 | 26.43 | 5.08 | 0.33 |  | 1.55 | 3.03 | 5.38 | 13.80 | 28.71
ImageNet | 0.4 | 31.36 | 22.09 | 10.97 | 5.37 | 2.75 |  | 3.71 | 4.68 | 6.85 | 9.14 | 11.65
0.5 | 37.67 | 24.29 | 10.60 | 4.53 | 2.21 |  | 3.15 | 4.48 | 7.14 | 10.16 | 13.50
0.6 | 43.36 | 25.10 | 10.46 | 4.38 | 1.22 |  | 2.68 | 4.29 | 7.50 | 10.99 | 15.46
0.7 | 49.65 | 26.83 | 8.73 | 2.55 | 0.39 |  | 2.28 | 4.20 | 7.95 | 12.68 | 23.46
0.8 | 56.34 | 25.95 | 5.75 | 0.39 | 0.00 |  | 1.90 | 4.37 | 9.71 | 22.82 | 57.72

Table 4: Robust accuracy (in $\%$) and loss for CIFAR-10 , CIFAR-100 and
ImageNet under BPDA $\ell_{\infty}$-attack at different resolution scales.
Attacked models are fine-tuned with $r^{*}=0.6$.


gobble

Mathematical formulation of the
modelsupplemental:material_mathematical_formulation The model used in our
study is based on PyPSA-Eur, which relies on a linear programme to minimise
the total annualised cost of the entire energy system in an optimal fashion.
Formally, this minimisation is represented by the following objective
function:

$\min_{G_{n,s},E_{n,s},F_{l},g_{n,s,t}}\left[\sum_{n,s}c_{n,s}G_{n,s}+\sum_{n,s}\hat{c}_{n,s}E_{n,s}+\sum_{l}c_{l}F_{l}+\sum_{n,s,t}o_{n,s,t}g_{n,s,t}\right]$
(1)

where, for technology $s$ in node $n$, $c_{n,s}$ is the annualised cost for
generator power capacity $G_{n,s}$, $\hat{c}_{n,s}$ is the annualised cost for
storage energy capacity $E_{n,s}$, $c_{l}$ is the fixed annualised cost for
capacity $F_{l}$ of link $l$, and $o_{n,s,t}$ are the marginal costs of
generation and storage dispatch $g_{n,s,t}$ at time step $t$. In addition, the
linear programme consists of several constraints. The most relevant
constraints imposed on our model are succinctly described next.
At every hour, the demand for electricity must be met by the supply, which can
be expressed as follows:

$\sum_{s}g_{n,s,t}+\sum_{l}\alpha_{n,l,t}f_{l,t}=d_{n,t}\hskip
4.26773pt\leftrightarrow\hskip 4.26773pt\lambda_{n,t}\hskip 14.22636pt\forall
n,t$ (2)

where the sum of generation and storage dispatch $g_{n,s,t}$ of technology
$s$, added to the sum of power flow $f_{l,t}$ in link $l$ with a direction and
efficiency $\alpha_{n,l,t}$, equals demand $d_{n,t}$ in every node $n$ at
every time step $t$. The dual value $\lambda_{n,t}$ of this constraint
represents the electricity shadow price for node $n$ at time step $t$.
By default, a model based on PyPSA-Eur is constrained with a global cap
(limit) on CO2 emissions. To pursue our study, we extended PyPSA-Eur with new
logic to allow for two additional types of constraints on CO2 emissions: local
(national) and nodal. In detail, the global constraint applies at a
continental level (Europe), requiring that all the nodes of the model comply
with a specific cap on CO2 emissions collectively. The local (national)
constraint applies at a country level, requiring that (only) the nodes of a
given country comply with a specific cap on CO2 emissions assigned to the
country collectively, whereas the nodal constraint applies to each node,
requiring each one to comply with a specific cap on CO2 emissions assigned to
it individually. The constraint representing a model based on a global CO2
emissions cap can be formulated as follows:

$\sum_{n,s,t}\varepsilon_{s}\frac{g_{n,s,t}}{\eta_{s}}\leq CAP_{CO_{2}}\hskip
4.26773pt\leftrightarrow\hskip 4.26773pt\mu_{CO_{2}}$ (3)

where the sum of CO2 emissions $\varepsilon_{s}$ in tonnes per each
MWh${}_{\text}{th}$ produced by technology $s$ with efficiency $\eta_{s}$ in
all nodes $n$ of the model must be equal to or lower than CO2 limit
$CAP_{CO_{2}}$. The dual value $\mu_{CO_{2}}$ of this constraint represents
the CO2 shadow price for the entire Europe.
The constraint representing a model based on local (national) CO2 emissions
caps can be formulated as follows:

$\sum_{n,s,t}\varepsilon_{s}\frac{g_{n,s,t}}{\eta_{s}}\leq
CAP_{CO_{2},k}\hskip 4.26773pt\leftrightarrow\hskip
4.26773pt\mu_{CO_{2},k}\hskip 14.22636ptn\in k\hskip 14.22636pt\forall k$ (4)

where, for every country $k$, the sum of CO2 emissions $\varepsilon_{s}$ in
tonnes per each MWh${}_{\text}{th}$ produced by technology $s$ with efficiency
$\eta_{s}$ in all nodes $n$ belonging to $k$ must be equal to or lower than
CO2 limit $CAP_{CO_{2},k}$. The dual value $\mu_{CO_{2},k}$ of this constraint
represents the CO2 shadow price for country $k$.
At last, the constraint representing a model based on nodal CO2 emissions caps
can be formulated as follows:

$\sum_{s,t}\varepsilon_{s}\frac{g_{n,s,t}}{\eta_{s}}\leq CAP_{CO_{2},n}\hskip
4.26773pt\leftrightarrow\hskip 4.26773pt\mu_{CO_{2},n}\hskip 14.22636pt\forall
n$ (5)

where, for every node $n$, CO2 emissions $\varepsilon_{s}$ in tonnes per each
MWh${}_{\text}{th}$ produced by technology $s$ with efficiency $\eta_{s}$ in
$n$ must be equal to or lower than CO2 limit $CAP_{CO_{2},n}$. The dual value
$\mu_{CO_{2},n}$ of this constraint represents the CO2 shadow price for node
$n$.

##  1 Figures

[height = 12.1cm, angle = 90]./figures/figure_S1.png

Figure : The European energy system spatial layout spanning across 33
countries in both the global and local net-zero CO2 emissions scenarios. It is
modelled with a 90 nodes resolution (indicated by red dots) and 370 regions
representing renewable resources, with each region associated with one of the
nodes. The (blue) lines connecting nodes represent the electricity grid
topology, which includes both existing power lines and new ones planned for
construction in Europe as per ENTSO-E’s TYNDP 2018.

[][width = 0.72]./figures/figure_S2a.png [][width =
0.72]./figures/figure_S2b.png

Figure : The European methane gas network topology in (a) the global net-zero
CO2 emissions scenario and (b) the local net-zero CO2 emissions scenario. Each
pipeline has a specific capacity and is organised according to the SciGRID_gas
project, which forms the basis for transporting methane gas in the model. The
model is allowed to expand the capacity of pipelines if it reduces the total
system cost.

[][width = 0.72]./figures/figure_S3a.png [][width =
0.72]./figures/figure_S3b.png

Figure : The European H2 network topology in (a) the global net-zero CO2
emissions scenario and (b) the local net-zero CO2 emissions scenario. The
model is allowed to build new H2 pipelines if it reduces the total system
cost. These have optimal capacities and are organised according to the
electricity grid and methane gas network topologies.

[][width = 0.72]./figures/figure_S4a.png [][width =
0.72]./figures/figure_S4b.png

Figure : The European CO2 network topology in (a) the global net-zero CO2
emissions scenario and (b) the local net-zero CO2 emissions scenario. The
model is allowed to build new CO2 pipelines if it reduces the total system
cost. These have optimal capacities and are organised according to the
electricity grid topology.

[][width = 0.96]./figures/figure_S5a.png [][width =
0.96]./figures/figure_S5b.png

Figure : Sensitivity analysis of the model under different temporal
resolutions in (a) the global net-zero CO2 emissions scenario and (b) the
local net-zero CO2 emissions scenario. Due to computational limitations, both
sensitivity analyses were conducted with an 80-node spatial resolution of the
model instead of the 90-node spatial resolution used in our study. The red and
blue bars show the variations of the objective function and the aggregated
amount of CO2 captured, converted, and sequestered underground, respectively,
in comparison to the baseline of one-hour temporal resolution.

[width = 0.96]./figures/figure_S6.png

Figure : In the global net-zero CO2 emissions scenario, countries that emit
more CO2 than they capture are represented by red bars, while countries that
capture more CO2 than they emit are represented by green bars. The percentages
on top of the red bars indicate how much CO2 net emitter countries do not
capture, compared to their own emission values. On the other hand, the
percentages at the bottom of the green bars indicate how much additional CO2
net absorber countries capture, compared to their own emission values.

[width = 0.96]./figures/figure_S7.png

Figure : Exogenous CO2 emissions per country and sector in both the global and
local net-zero CO2 emissions scenarios. CO2 emissions total 634 Mt/a, with 153
Mt/a coming from process emissions, 23 Mt/a from oil for agriculture
machinery, 135 Mt/a from methanol for shipping, 177 Mt/a from naphtha for
industry, and 146 Mt/a from kerosene for aviation.

[][width = 0.485]./figures/figure_S8a.png [][width =
0.485]./figures/figure_S8b.png [][width = 0.96]./figures/figure_S8c.png

Figure : CO2 underground sequestration usage across Europe in (a) the global
net-zero CO2 emissions scenario and (b) the local net-zero CO2 emissions
scenario, and (c) CO2 underground sequestration potential per country and
usage in the global and local net-zero CO2 emissions scenarios. Despite the
varying levels of sequestration usage amongst countries in the two scenarios,
the conservative sequestration potential limit of 200 MtCO2/a is fully
exhausted in both scenarios.

[][width = 0.8]./figures/figure_S9a.png [][width =
0.8]./figures/figure_S9b.png

Figure : (a) Onshore wind capacity factors across Europe and (b) Solar
photovoltaic capacity factors across Europe. The capacity factors for wind are
predominantly higher in northern Europe, whereas the capacity factors for
solar are predominantly higher in southern Europe.

[][width = 0.8]./figures/figure_S10a.png [][width =
0.96]./figures/figure_S10b.png

Figure : Solid biomass potential (a) per node and (b) per country. Europe is
assumed to have an aggregated potential of 1038 TWh per year of solid biomass.
This assumption is based on the medium bioenergy availability profile of the
ENSPRESO database, which the model uses in both the global and local net-zero
CO2 emissions scenarios.

[][width = 0.96]./figures/figure_S11a.png [][width =
0.96]./figures/figure_S11b.png

Figure : Total system cost sensitivity analysis under different CO2 emissions
limits and respective technology configuration in (a) the global scenario and
(b) the local scenario. Until reaching a 35% limit in CO2 emissions (relative
to 1990 levels), the total system cost remains constant since the CO2
constraint is not binding. After that point, the cost increases non-linearly
in both scenarios until Europe is fully decarbonised.

[][width = 0.96]./figures/figure_S12a.png [][width =
0.485]./figures/figure_S12b.png [][width = 0.485]./figures/figure_S12c.png

Figure : CO2 shadow price sensitivity analysis under different CO2 emissions
limits (a) for Europe (global scenario) and all the modelled countries (local
scenario), (b) for Europe (global scenario) and the interior countries (local
scenario), and (c) for Europe (global scenario) and the exterior countries
(local scenario). In a continent fully decarbonised under local constraints,
countries that used to emit more CO2 than they absorb annually in the global
scenario have local CO2 shadow prices higher than Europe’s global CO2 shadow
price. Conversely, countries that used to absorb more CO2 than they emit in
the global scenario have local CO2 shadow prices lower than Europe’s global
CO2 shadow price.

[][width = 0.8]./figures/figure_S13a.png [][width =
0.8]./figures/figure_S13b.png

Figure : (a) CO2 shadow price per country across Europe in the local net-zero
CO2 emissions scenario and (b) Product of CO2 shadow price and CO2 emissions
per country across Europe in the local net-zero CO2 emissions scenario. See
Figure 1 for the amount of CO2 emissions per country. As a reference, the CO2
shadow price for the whole of Europe is set at 540 € per tonne of CO2 in the
global net-zero CO2 emissions scenario.

[][width = 0.96]./figures/figure_S14a.png [][width =
0.96]./figures/figure_S14b.png

Figure : (a) Total system cost and technology configuration in the global net-
zero CO2 emissions scenario without a CO2 network and (b) Cost variation per
technology between the global net-zero CO2 emissions scenario with a CO2
network and the global net-zero CO2 emissions scenario without a CO2 network.
The total cost increases by 0.3% when modelling a system without a CO2 network
compared to its counterpart equipped with such a network. Other than the
differences in deployment levels, the technology configuration is the same in
both types of systems.

[][width = 0.96]./figures/figure_S15a.png [][width =
0.96]./figures/figure_S15b.png

Figure : (a) Total system cost and technology configuration in the local net-
zero CO2 emissions scenario without a CO2 network and (b) Cost variation per
technology between the local net-zero CO2 emissions scenario with a CO2
network and the local net-zero CO2 emissions scenario without a CO2 network.
The total cost increases by 0.6% when modelling a system without a CO2 network
compared to its counterpart equipped with such a network. Other than the
differences in deployment levels, the technology configuration is the same in
both types of systems.

[][width = 0.8]./figures/figure_S16a.png [][width =
0.96]./figures/figure_S16b.png

Figure : (a) Electricity production in the global net-zero CO2 emissions
scenario and (b) Electricity production variation per country and technology
between the global net-zero CO2 emissions scenario and the local net-zero CO2
emissions scenario. Major variations occur in electricity production from
renewable sources. While net CO2 emitter countries increase solar and wind to
produce electricity, net CO2 absorber countries, on the other hand, decrease
these. This is because the former countries capture much more CO2 under local
constraints using DAC and convert it into methanol, both processes requiring
significant amounts of power, while the latter countries relax these (since
they have to handle less captured CO2).

[][width = 0.96]./figures/figure_S17a.png [][width =
0.96]./figures/figure_S17b.png

Figure : CO2 capture sensitivity analysis under different CO2 emissions limits
in (a) the global scenario and (b) the local scenario. In the local scenario,
CO2 captured from DAC and solid biomass used in the industry are adopted for
less stringent CO2 targets in the CO2 reduction path compared to the global
scenario. In a fully decarbonised Europe, these two technologies combined
account for 60% and 67% of the total CO2 captured in the global and local
scenarios, respectively.

[][width = 0.96]./figures/figure_S18a.png [][width =
0.96]./figures/figure_S18b.png

Figure : CO2 conversion and sequestration sensitivity analysis under different
CO2 emissions limits in (a) the global scenario and (b) the local scenario. In
both scenarios, Fischer-Tropsch and methanolisation play an important role in
a fully decarbonised Europe by converting 70% of all captured CO2 into
synthetic oil and methanol. The remaining captured CO2 is sequestered
underground to enable negative emissions. As expected, the CO2 conversion and
sequestration profile mimics the CO2 capture profile in both scenarios.

[][width = 0.8]./figures/figure_S19a.png [][width =
0.96]./figures/figure_S19b.png

Figure : (a) Oil, methanol, methane gas, solid biomass, H2, heat, and
electricity demand in the global net-zero CO2 emissions scenario and (b) Oil,
methanol, methane gas, solid biomass, H2, heat, and electricity demand
variation per country between the global net-zero CO2 emissions scenario and
the local net-zero CO2 emissions scenario. In the local scenario, the interior
countries significantly increase the demand for H2, heat, and electricity due
to the growth in manufacturing synthetic products, along with the deployment
of DAC and air heat pumps.

[][width = 0.8]./figures/figure_S20a.png [][width =
0.96]./figures/figure_S20b.png

Figure : (a) Net synthetic oil producers and consumers in the global net-zero
CO2 emissions scenario and (b) Net synthetic oil variation per country in the
global net-zero CO2 emissions scenario. Countries that produce more synthetic
oil than they consume are represented by green bars, while countries that
consume more synthetic oil than they produce are represented by red bars. The
percentages on top of the green bars indicate how much additional synthetic
oil net producer countries manufacture, beyond what is needed to meet their
own demand. On the other hand, the percentages at the bottom of the red bars
indicate how much synthetic oil net consumer countries do not manufacture to
meet their own demand, relying instead on the former countries.

[][width = 0.8]./figures/figure_S21a.png [][width =
0.96]./figures/figure_S21b.png

Figure : (a) Net synthetic methanol producers and consumers in the global net-
zero CO2 emissions scenario and (b) Net synthetic methanol variation per
country in the global net-zero CO2 emissions scenario. Countries that produce
more synthetic methanol than they consume are represented by green bars, while
countries that consume more synthetic methanol than they produce are
represented by red bars. The percentages on top of the green bars indicate how
much additional synthetic methanol net producer countries manufacture, beyond
what is needed to meet their own demand. On the other hand, the percentages at
the bottom of the red bars indicate how much synthetic methanol net consumer
countries do not manufacture to meet their own demand, relying instead on the
former countries.

[][width = 0.485]./figures/figure_S22a.png [][width =
0.485]./figures/figure_S22b.png

Figure : Temporal CO2 capture from exogenously fixed sources across all of
Europe in (a) the global net-zero CO2 emissions scenario and (b) the local
net-zero CO2 emissions scenario.

[][width = 0.485]./figures/figure_S23a.png [][width =
0.485]./figures/figure_S23b.png

Figure : Temporal CO2 capture from endogenous sources across all of Europe in
(a) the global net-zero CO2 emissions scenario and (b) the local net-zero CO2
emissions scenario. Except for the absence of CO2 captured from gas-based CHP
units and SMR, which are not cost-competitive, the other two processes of
capturing CO2 exhibit a seasonal pattern. CO2 captured from solid biomass-
based CHP units increases in winter due to higher heat demand and decreases in
summer. In contrast, CO2 captured from DAC decreases in winter and increases
in summer due to the availability of low-cost electricity.

[][width = 0.485]./figures/figure_S24a.png [][width =
0.485]./figures/figure_S24b.png

Figure : Temporal CO2 conversion and sequestration across all of Europe in (a)
the global net-zero CO2 emissions scenario and (b) the local net-zero CO2
emissions scenario. Although less prominent in Fischer-Tropsch, all CO2
conversion processes exhibit a seasonal pattern. These processes convert more
CO2 during summer due to lower electricity and heating prices in this season
and convert less during winter. Sequestration of CO2 underground also follows
a seasonal pattern, albeit inverted from the one governing the CO2 conversion
processes. During winter, more CO2 is sequestered underground, while less is
sequestered during summer due to its utilisation in conversion processes.

[][width = 0.8]./figures/figure_S25a.png [][width =
0.8]./figures/figure_S25b.png

Figure : CO2 flow with a 1000 Mt/a underground sequestration potential in (a)
the global net-zero CO2 emissions scenario and (b) the local net-zero CO2
emissions scenario. The main factor determining the directions and
destinations of CO2 flows is the assumed potential for sequestering CO2
underground. Increasing the potential assumption to 1000 Mt/a (from 200 Mt/a)
across Europe clearly shows CO2 flows converging towards countries with
significant underground sequestration potential.

[] [width = 0.8]./figures/figure_S26a.png [][width =
0.8]./figures/figure_S26b.png

Figure : H2 flow in (a) the global net-zero CO2 emissions scenario and (b) the
local net-zero CO2 emissions scenario. The total amount of H2 flowing amongst
and within countries increases by 6% under local constraints relative to the
global constraint. Most of this increase is to provide the necessary H2 for
the interior countries and Switzerland to meet their industrial needs and
produce more synthetic oil and methanol under the local scenario.

[][width = 0.8]./figures/figure_S27a.png [][width =
0.8]./figures/figure_S27b.png

Figure : Electricity flow in (a) the global net-zero CO2 emissions scenario
and (b) the local net-zero CO2 emissions scenario. In both scenarios, most of
the electricity flows are confined within individual countries. Regions within
the same country supply electricity to each other to power DAC and
methanolisation, as well as H2 production, with limited electricity exchange
occurring between countries.

[][width = 0.8]./figures/figure_S28a.png [][width =
0.96]./figures/figure_S28b.png

Figure : (a) Wind, solar, and H2 electrolysis installed capacity in the global
net-zero CO2 emissions scenario and (b) Wind, solar, and H2 electrolysis
installed capacity variation per country between the global net-zero CO2
emissions scenario and the local net-zero CO2 emissions scenario.

[][width = 0.8]./figures/figure_S29a.png [][width =
0.96]./figures/figure_S29b.png

Figure : (a) H2 production in the global net-zero CO2 emissions scenario and
(b) H2 production variation between the global net-zero CO2 emissions scenario
and the local net-zero CO2 emissions scenario. Thanks to high renewable energy
capacity factors and their relative proximity to the interior countries and
Switzerland, Spain, Denmark, and Great Britain significantly increase H2
production to satisfy a growing demand for H2 in the former countries when
under local constraints.

[][width = 0.96]./figures/figure_S30a.png [][width =
0.96]./figures/figure_S30b.png

Figure : Temporal H2 production across all of Europe in (a) the global net-
zero CO2 emissions scenario and (b) the local net-zero CO2 emissions scenario.
In both scenarios, the production of H2 varies seasonally, with higher levels
during the summer when there is an abundance of cost-effective energy to power
H2 electrolysis compared to winter.

##  2 Tables

[H]p4.7cm p2.65cm p2.65cm Characteristics assumptions about the main
technologies used in PyPSA-Eur are based on the Energy System Technology Data
for the year 2030. The comprehensive list of technologies is available at
https://github.com/PyPSA/technology-data.
Technology & Parameter Value
2* Battery storage Investment 142 €/kWh
Lifetime 25 years
5* Biogas Investment 1539.62 €/kW
Lifetime 20 years
Efficiency 1 per unit
FOM 12.84%/year
Capture rate 0.98 per unit
4* Biomass boiler Investment 649.3 €/kWth
Lifetime 20 years
Efficiency 0.86 per unit
FOM 7.49%/year
3* CO2 pipeline Investment 2000 €/tCO2/h/km
Lifetime 50 years
FOM 0.9%/year
1* CO2 sequestration VOM 10 €/tCO2
5* Central air heat pump Investment 856.25 €/kWth
Lifetime 25 years
Efficiency 3.6 per unit
FOM 0.23%/year
VOM 2.51 €/MWhth
5* Central gas boiler Investment 50 €/kWth
Lifetime 25 years
Efficiency 1.04 per unit
FOM 3.8%/year
VOM 1 €/MWhth
5* Central ground heat pump Investment 507.6 €/kWth
Lifetime 25 years
Efficiency 1.73 per unit
FOM 0.39%/year
VOM 1.25 €/MWhth
5* Central resistive heater Investment 60 €/kWth
Lifetime 20 years
Efficiency 0.99 per unit
FOM 1.7%/year
VOM 1 €/MWhth
3* Central solar thermal Investment 140000 €/1000m2
Lifetime 20 years
FOM 1.4%/year
3* Central water tank storage Investment 0.54 €/kWh
Lifetime 25 years
FOM 0.55%/year
4* Decentral air heat pump Investment 850 €/kWth
Lifetime 18 years
Efficiency 3.6 per unit
FOM 3%/year
4* Decentral gas boiler Investment 296.82 €/kWth
Lifetime 20 years
Efficiency 0.98 per unit
FOM 6.69%/year
4* Decentral ground heat pump Investment 1400 €/kWth
Lifetime 20 years
Efficiency 3.9 per unit
FOM 1.82%/year
4* Decentral resistive heater Investment 100 €/kWth
Lifetime 20 years
Efficiency 0.9 per unit
FOM 2%/year
3* Decentral solar thermal Investment 270000 €/1000m2
Lifetime 20 years
FOM 1.3%/year
3* Decentral water tank storage Investment 18.38 €/kWh
Lifetime 20 years
FOM 1%/year
6* Direct air capture Investment 6000000 €/tCO2/h
Lifetime 20 years
FOM 4.95%/year
Electricity input 0.32 MWh/tCO2
Heat input 2 MWh/tCO2
Heat output 1 MWh/tCO2
3* Electricity distribution grid Investment 500 €/kW
Lifetime 40 years
FOM 2%/year
5* Fischer-Tropsch Investment 650711.26 €/MWFT
Lifetime 20 years
Efficiency 0.8 per unit
FOM 3%/year
Capture rate 0.98 per unit
5* Gas CHP Investment 560 €/kW
Lifetime 25 years
Efficiency 0.41 per unit
FOM 3.32%/year
VOM 4.2 €/MWh
10* Gas CHP CC Investment 560 €/kW
Lifetime 25 years
Efficiency 0.41 per unit
FOM 3.32%/year
VOM 4.2 €/MWh
Capture rate 0.9 per unit
CO2 intensity 0.2 tCO2/MWhth
Electricity input 0.02 MWh/tCO2
Heat input 0.72 MWh/tCO2
Heat output 0.72 MWh/tCO2
5* Gas for industry CC Investment 2600000 €/tCO2/h
Lifetime 25 years
FOM 3%/year
Capture rate 0.9 per unit
CO2 intensity 0.2 tCO2/MWhth
3* Gas storage Investment 0.03 €/kWh
Lifetime 100 years
FOM 3.59%/year
4* H2 electrolysis Investment 450 €/kWe
Lifetime 30 years
Efficiency 0.68 per unit
FOM 2%/year
4* H2 fuel cell Investment 1100 €/kWe
Lifetime 10 years
Efficiency 0.5 per unit
FOM 5%/year
3* H2 pipeline Investment 267 €/MW/km
Lifetime 40 years
FOM 3%/year
3* H2 storage tank type 1 + compr. Investment 44.91 €/kWh
Lifetime 30 years
FOM 1.11%/year
3* H2 storage underground Investment 2 €/kWh
Lifetime 100 years
FOM 0%/year
VOM 0 €/MWh
3* HVAC overhead Investment 432.97 €/MW/km
Lifetime 40 years
FOM 2%/year
3* HVDC overhead Investment 432.97 €/MW/km
Lifetime 40 years
FOM 2%/year
3* HVDC submarine Investment 471.16 €/MW/km
Lifetime 40 years
FOM 0.35%/year
4* Hydroelectricity Investment 2208.16 €/kWel
Lifetime 80 years
Efficiency 0.9 per unit
FOM 1%/year
5* Methanation (Sabatier) Investment 628.6 €/MW${}_{CH_{4}}$
Lifetime 20 years
Efficiency 0.8 per unit
FOM 3%/year
Capture rate 0.98 per unit
3* Methane gas pipeline Investment 79 €/MW/km
Lifetime 50 years
FOM 1.5%/year
3* Methanolisation Investment 650711.26 €/MWMeOH
Lifetime 20 years
FOM 3%/year
5* OCGT Investment 435.24 €/kW
Lifetime 25 years
Efficiency 0.41 per unit
FOM 1.78%/year
VOM 4.5 €/MWh
4* Offshore wind Investment 1523.55 €/kWe
Lifetime 30 years
FOM 2.32%/year
VOM 0.02 €/MWhel
4* Onshore wind Investment 1035.56 €/kW
Lifetime 30 years
FOM 1.22%/year
VOM 1.35 €/MWh
4* Process emissions CC Investment 2600000 €/tCO2/h
Lifetime 25 years
FOM 3%/year
Capture rate 0.9 per unit
4* SMR Investment 493470.4 €/MW${}_{CH_{4}}$
Lifetime 30 years
Efficiency 0.76 per unit
FOM 5%/year
5* SMR CC Investment 572425.66 €/MW${}_{CH_{4}}$
Lifetime 30 years
Efficiency 0.69 per unit
FOM 5%/year
Capture rate 0.9 per unit
4* Solar Investment 492.11 €/kWe
Lifetime 40 years
FOM 1.95%/year
VOM 0.01 €/MWhel
4* Solid biomass Investment 2209 €/kWel
Lifetime 30 years
Efficiency 0.47 per unit
FOM 4.53%/year
5* Solid biomass CHP Investment 3349.49 €/kWe
Lifetime 25 years
Efficiency 0.27 per unit
FOM 2.87%/year
VOM 4.58 €/MWhe
8* Solid biomass CHP CC Investment 2700000 €/tCO2/h
Lifetime 25 years
FOM 3%/year
Capture rate 0.9 per unit
CO2 intensity 0.2 tCO2/MWhth
Electricity input 0.02 MWh/tCO2
Heat input 0.72 MWh/tCO2
Heat output 0.72 MWh/tCO2
5* Solid biomass for industry CC Investment 2600000 €/tCO2/h
Lifetime 25 years
FOM 3%/year
Capture rate 0.9 per unit
CO2 intensity 0.37 tCO2/MWhth

Table : Solid biomass transportation cost per country. The costs are based on the JRC European TIMES Energy System Model (JRC-EU-TIMES) and are available at https://data.jrc.ec.europa.eu/collection/id-00287. Country |  ISO |  Cost [€/km/MWh]
---|---|---
Albania |  AL |  0.05521
Austria |  AT |  0.13646
Bosnia and Herzegovina |  BA |  0.05938
Belgium |  BE |  0.14063
Bulgaria |  BG |  0.06354
Switzerland |  CH |  0.17708
Czech Republic |  CZ |  0.08750
Germany |  DE |  0.13333
Denmark |  DK |  0.19063
Estonia |  EE |  0.08646
Spain |  ES |  0.11979
Finland |  FI |  0.14896
France |  FR |  0.14271
Great Britain |  GB |  0.14375
Greece |  GR |  0.11146
Croatia |  HR |  0.08021
Hungary |  HU |  0.07292
Ireland |  IE |  0.13333
Italy |  IT |  0.13229
Lithuania |  LT |  0.07396
Luxembourg |  LU |  0.14583
Latvia |  LV |  0.08125
Montenegro |  ME |  0.06042
North Macedonia |  MK |  0.05104
The Netherlands |  NL |  0.14583
Norway |  NO |  0.16146
Poland |  PL |  0.07813
Portugal |  PT |  0.09271
Romania |  RO |  0.06250
Serbia |  RS |  0.05729
Sweden |  SE |  0.16146
Slovenia |  SI |  0.09792
Slovakia |  SK |  0.08333
|  |

Table : CO2 underground sequestration potential per country. The potentials are based on the European CO2 storage database (CO2StoP) and are available at https://energy.ec.europa.eu/publications/assessment-co2-storage-potential-europe-co2stop_en. The model limits the aggregated CO2 sequestration potential across Europe to 2.1 Gt and only considers the offshore underground potential. Country |  ISO |  Potential [MtCO2]
---|---|---
Albania |  AL |  0
Austria |  AT |  0
Bosnia and Herzegovina |  BA |  0
Belgium |  BE |  0
Bulgaria |  BG |  0.21
Switzerland |  CH |  0
Czech Republic |  CZ |  0
Germany |  DE |  41.09
Denmark |  DK |  603.56
Estonia |  EE |  0.65
Spain |  ES |  7.27
Finland |  FI |  0
France |  FR |  0
Great Britain |  GB |  1000
Greece |  GR |  121.97
Croatia |  HR |  0
Hungary |  HU |  0
Ireland |  IE |  18.89
Italy |  IT |  33.82
Lithuania |  LT |  0.36
Luxembourg |  LU |  0
Latvia |  LV |  33.37
Montenegro |  ME |  0
North Macedonia |  MK |  0
The Netherlands |  NL |  4.66
Norway |  NO |  13.43
Poland |  PL |  0.56
Portugal |  PT |  212.23
Romania |  RO |  0
Serbia |  RS |  0
Sweden |  SE |  8.23
Slovenia |  SI |  0
Slovakia |  SK |  0
|  |

Table : CO2 shadow price per country in the local net-zero CO2 emissions scenario. The prices are the result of solving the model and are referred to as dual values associated with each constraint in a linear programming problem. As a reference, the CO2 shadow price for all of Europe is set at 540 € per tonne of CO2 in the global net-zero CO2 emissions scenario. Country |  ISO |  Price [€/tCO2]
---|---|---
Albania |  AL |  455
Austria |  AT |  474
Bosnia and Herzegovina |  BA |  454
Belgium |  BE |  584
Bulgaria |  BG |  458
Switzerland |  CH |  571
Czech Republic |  CZ |  476
Germany |  DE |  569
Denmark |  DK |  535
Estonia |  EE |  412
Spain |  ES |  534
Finland |  FI |  435
France |  FR |  548
Great Britain |  GB |  542
Greece |  GR |  538
Croatia |  HR |  479
Hungary |  HU |  480
Ireland |  IE |  530
Italy |  IT |  536
Lithuania |  LT |  405
Luxembourg |  LU |  579
Latvia |  LV |  402
Montenegro |  ME |  456
North Macedonia |  MK |  436
The Netherlands |  NL |  575
Norway |  NO |  544
Poland |  PL |  461
Portugal |  PT |  446
Romania |  RO |  460
Serbia |  RS |  467
Sweden |  SE |  436
Slovenia |  SI |  477
Slovakia |  SK |  426
|  |

Table : Conversion from MWh of energy to tonnes of CO2 content based on stoichiometric reactions and thermal content of energy carriers. A factor of 3.6 is used to convert MWh to gigajoules (GJ). The molecular mass of CO2 is 44.01 g/mol. Solid biomass is assumed to be composed of 50% carbon (C). Energy carrier |  GJ per tonne of energy carrier
(a) |  Material molecular mass [g/mol]
(b) |  Tonnes of CO2 per MWh of energy carrier
3.6 / (a) * 44.01 / (b)
---|---|---|---
Solid biomass |  18 |  12.01 (C) |  0.3664
Methane gas |  50 |  16.04 (CH4) |  0.1976
Oil |  44 |  14 (C12H23) |  0.2572
Methanol |  22.7 |  32.04 (CH3OH) |  0.2178
|  |  |