# Relational Quantum Mechanics and the PBR Theorem: A Peaceful Coexistence

Andrea Oldofredi Contact Information: Université de Lausanne, Section de
Philosophie, Lausanne, Switzerland. E-mail<EMAIL_ADDRESS>Claudio
Calosi Contact Information: Université de Geneve, Section de Philosophie,
Geneve, Switzerland. E-mail<EMAIL_ADDRESS>
###### Abstract

According to Relational Quantum Mechanics (RQM) the wave function $\psi$ is
considered neither a concrete physical item evolving in spacetime, nor an
object representing the absolute state of a certain quantum system. In this
interpretative framework, $\psi$ is defined as a computational device encoding
observers’ information; hence, RQM offers a somewhat epistemic view of the
wave function. This perspective seems to be at odds with the PBR theorem, a
formal result excluding that wave functions represent knowledge of an
underlying reality described by some ontic state. In this paper we argue that
RQM is not affected by the conclusions of PBR’s argument; consequently, the
alleged inconsistency can be dissolved. To do that, we will thoroughly discuss
the very foundations of the PBR theorem, i.e. Harrigan and Spekkens’
categorization of ontological models, showing that their implicit assumptions
about the nature of the ontic state are incompatible with the main tenets of
RQM. Then, we will ask whether it is possible to derive a _relational_ PBR-
type result, answering in the negative. This conclusion shows some limitations
of this theorem not yet discussed in the literature.

_Keywords_ : Relational Quantum Mechanics; PBR Theorem; Ontic States;
Epistemic States;

_Accepted for publication in Foundations of Physics_

###### Contents

  1. 1 Introduction
  2. 2 Setting the Stage: RQM and the PBR Theorem
    1. 2.1 RQM in a Nutshell
    2. 2.2 Harrigan and Spekkens’ Distinction: $\psi$-Ontic and $\psi$-Epistemic Models
    3. 2.3 The PBR Theorem
  3. 3 The Peaceful Coexistence
  4. 4 A Relational PBR Theorem?
  5. 5 Conclusions

##  1 Introduction

In his seminal essay on Relational Quantum Mechanics (RQM), Carlo Rovelli
claims that “the experimental evidence at the basis of quantum mechanics
forces us to accept that distinct observers give different descriptions of the
same events” (Rovelli, (1996), p. 1638). Such a claim, which entails a
radically novel perspective on the interpretation of Quantum Mechanics (QM),
finds its roots in Einstein’s idea behind special relativity, i.e. that
physical observations depends on the specific reference frame in which
interactions occur. Furthermore, these insights on the meaning of non-
relativistic quantum theory are also motivated by considerations coming from
quantum gravity and general relativity, theories where, according to Rovelli,
the relational features of spacetime clearly emerge.

Interestingly, Rovelli claims that RQM is the most natural result of the
unification of these ingredients and their application to the formalism of
standard quantum theory taken at face value. In the light of this
interpretative framework, the notion of an observer-independent state of
physical systems is abolished in favor of a perspectival, observer-dependent
description of reality (cf. Rovelli, (1996, 2016, 2018), Laudisa and Rovelli,
(2019)). In RQM various observers may provide diverse descriptions of the same
sequence of physical events, and such descriptions are equally correct and
non-contradictory, despite their being different. As a consequence, the values
of measurable properties are _relative to_ specific observers.

Rovelli’s theory has remarkable foundational implications. We mention a few:

  * •

A novel meaning of the wave function collapse, simply considered an update of
the information possessed by a certain observer relatively to the value of a
measured observable. In RQM nothing physical is actually collapsing in
measurements interactions, and this is why the tension between the unitary
Schrödinger dynamics and the collapse postulate is dissolved;

  * •

A new treatment of Wigner’s friend scenarios, both old and new. In RQM the
descriptions of a quantum measurement provided by Wigner’s friend—who actually
measured a certain observable on a quantum system obtaining a definite
result—and by Wigner himself—who has no clue about the outcome—are equally
correct and non-contradictory (Frauchiger and Renner, (2018); Bong et al.,
(2020));

  * •

The commitment to a subjective interpretation of quantum probabilities (cf.
Smerlak and Rovelli, (2007)), without being a subjective version of QM as e.g.
Quantum Bayesianism, since RQM provides an ontic representation of physical
systems, as recently argued by e.g. Oldofredi and Lopez, (2020).

This paper focuses on yet another aspect of RQM, namely its interpretation of
the wavefunction. According to Rovelli, the wavefunction is considered neither
a concrete physical item dynamically evolving in spacetime (or in
configuration space), nor an object representing the absolute state of a
certain quantum system (cf. Rovelli, (1996, 2016)). In this context, $\psi$ is
defined as a useful computational device encoding the information available to
a particular observer about a specific system. Hence, it is generally claimed
that RQM offers an epistemic view of the wavefunction. This perspective about
the nature of the quantum state, however, seems to be at odds with a formal
result obtained by Matthew Pusey, Jonathan Barrett and Terry Rudolph—known as
the PBR theorem—according to which all ontic models reproducing the
predictions and the statistics of the Born rule must be $\psi$-ontic (cf.
Pusey et al., (2012)). Alternatively stated, as Leifer, (2014) pointed out,
such a theorem excludes the possibility that the wavefunction represents the
knowledge of an underlying reality described by some ontic state (usually
denoted $\lambda$). Hence, one would be led to conclude that RQM is in plain
contradiction with the PRB theorem.

In this respect, Laudisa and Rovelli, (2019) claim that

> [T]he wave function, and more in general the quantum state $\psi$, are
> interpreted realistically in several presentations of quantum theory. From
> the perspective of RQM, this is precisely what generates the confusion about
> quantum theory (Rovelli, 2018). RQM circumvents the theorems for the reality
> of the wave function (Leifer, (2014); Pusey et al., (2012)) because it is
> not a strongly realist theory […] which is an implicit assumption of these
> theorems.

Although we agree that in RQM the wave function is not taken to be a real,
concrete object in itself, it should be noted that Rovelli’s theory can be
properly considered a realist formulation of quantum theory, as argued by
several philosophers in the recent years (cf. Dorato, (2016), Candiotto,
(2017), Oldofredi, (2020), and Calosi and Mariani, (2020)). Hence, we believe
that Laudisa and Rovelli’s answer to the issue raised by the PBR theorem
can–and should—be developed and refined. Indeed, we think that there is a
deeper and more precise reason why RQM and the PRB theorem are not mutually
inconsistent. The main aim of the paper is exactly to investigate such a
reason. We are going to show why RQM is not affected by the conclusions of
PBR’s argument and, consequently, this alleged inconsistency can be dissolved.

In order to do this, we will thoroughly discuss the foundations of the PBR
theorem, i.e. Harrigan and Spekkens’ categorization of ontological models (cf.
Harrigan and Spekkens, (2010)). More precisely, it will be argued that their
implicit assumptions about the nature of the ontic state $\lambda$ are
incompatible with the basic tenets of RQM. Conforming to this classification,
$\lambda$ is considered an observer-independent representation of the state of
a certain quantum system. However, the relational character of Rovelli’s
theory requires that, in order to define correctly what ontic states are, one
has to use completely different criteria w.r.t. those employed by Harrigan and
Spekkens. According to the main tenets of RQM, in fact, _both_ $\psi$ and
$\lambda$ must be relational, meaning that

  * •

$\lambda$ represents the ontic state of quantum systems relatively to a
certain observer,

  * •

$\psi$ stores information that a particular observer has relatively to a given
system.

Thus, in the following sections we will carefully explain which assumptions
RQM makes about $\lambda$, and how they diverge from those employed by
Harrigan and Spekkens. In particular, it will be argued that Harrigan and
Spekkens’ approach does not have the necessary formal and ontological
resources to be correctly applied to RQM. This has a remarkable consequence—or
so we contend: given that PBR theorem relies on such a classification of
quantum models, but the latter fails to capture some basic tenets of RQM, one
can safely conclude that Rovelli’s theory does not lie within the scope of the
theorem, avoiding any formal contradiction with it. In addition, we will ask
whether it is possible to derive a PBR-type result in the context of RQM. We
will answer in the negative. This conclusion also shows some limitations of
the PBR-theorem that, to the best of our knowledge, have not been discussed in
literature.

The structure of the paper is the following: in Section 2 we introduce the
essential elements of our discussion, providing a brief description of RQM,
Harrigan and Spekkens’ categorization of ontological models, and the PBR
theorem (readers familiar with the literature may skip this part, though our
reconstruction of the PBR-theorem is somewhat unorthodox). In Section 3 we
will argue that RQM and the PRB theorem can peacefully coexist, whereas in
Section 4 a relational PBR-type result will be discussed taking into account
several scenarios involving different observers. Section 5 concludes the
paper.

##  2 Setting the Stage: RQM and the PBR Theorem

###  2.1 RQM in a Nutshell

RQM was first proposed in Rovelli, (1996). In its original formulation it
consisted of two parts:

  * •

A re-interpretation of the usual quantum formalism;

  * •

A derivation of this formalism from basic, general principles.

The focus of this paper is restricted to the first part. We follow closely
Rovelli, (1996) here. Rovelli explicitly understands RQM as a “generalization”
of the standard Copenhagen interpretation of quantum mechanics. This
generalization is twofold. First, there is no privileged notion of an
“observer”—thus every quantum system is on a par:

> [R]elational QM is Copenhagen quantum mechanics made democratic by bringing
> all systems onto the same footing (Rovelli, (2018), p. 11).

Thus, the notion of an “observer” does not

> [M]ake any reference to conscious, animate, or computing, or in any other
> manner special, system (Rovelli, (1996), p. 1641).

In the light of this, any physical system can count as an “observer”. Second,
there is no privileged notion of “measurement”—thus every physical interaction
is on a par:

> [M]easurement is an interaction like any other (Rovelli, (2018), p. 5).

There are four “basic tenets” of the Copenaghen interpretation that, when
suitably adjusted, RQM is supposed to retain:

Eigenfunction-Eigenvalue Link (EEL). A physical system $s$ has a definite
value $v$ of an observable $O$ iff the state of $s$ is an eigenstate of $O$
that belongs to $v$.

Schrödinger Dynamics. The state of $s$ evolves according to the Schrödinger
equation

$\displaystyle H(t)|\psi(x,t)\rangle=i\hbar\frac{\partial}{\partial
t}|\psi(x,t)\rangle.$ (1)

“Collapse” Postulate. At the time of measurement the state of $s$ collapses
into one of the eigenstates of $O$ with probability given by the Born rule.

Born rule. If an observable quantity $O$ is measured:

  * •

the result will be one of the eigenvalues $o_{i}$ of $O$;

  * •

the probability $P$ to obtain eigenvalue $o_{i}$ is given by
$\langle\psi|P_{i}|\psi\rangle$.

As we will see in due course, caution is due to interpret the Collapse
Postulate. As of now, we think RQM is best appreciated by focusing on what
Rovelli calls “The Third Person” (or “third system”) problem. Consider an
observable $O$ with two eigenvectors $|+\rangle$ and $|-\rangle$, with
eigenvalues $+1$ and $-1$ respectively. Suppose we have a physical system
$s_{1}$, that, at time $t_{1}$, is in a superposition of $O$-states:

$|\psi\rangle_{s_{1}}=(c_{1}|+\rangle+c_{2}|-\rangle)_{s_{1}}$ (2)

A second system $s_{2}$ interacts with $s_{1}$ in what the Copenaghen
interpretation will call a “measurement” of $O$, and finds $s_{1}$ to have
value $O=+1$. According to the “Collapse” Postulate” we have:

$\displaystyle t_{1}\rightarrow t_{2}$ (3)
$\displaystyle(c_{1}|+\rangle+c_{2}|-\rangle)_{s_{1}}\rightarrow|+\rangle_{s_{1}}$

The result of the quantum interaction between $s_{1}$ and $s_{2}$ is that
$s_{1}$ acquires a definite value property, namely $O=+1$—this much follows
from EEL. Consider now a system $s_{3}$ that does not interact with either
$s_{1}$ or $s_{2}$. Its description of the quantum situation encoded in (3)
will only refer to the Schrödinger Dynamics. Thus, we get:

$\displaystyle t_{1}\rightarrow t_{2}$ (4)
$\displaystyle((c_{1}|+\rangle+c_{2}|-\rangle\otimes|init\rangle)_{s_{12}}\rightarrow(c_{1}|+\rangle\otimes|plus\rangle+c_{2}|-\rangle\otimes|minus\rangle)_{s_{12}}$

where, as conventions, $s_{12}$ is the quantum system comprising $s_{1}$ and
$s_{2}$ and states $|+\rangle$ and $|plus\rangle$ are correlated in the
obvious way.111Clearly, the same goes for $|-\rangle$ and $|minus\rangle$.

It is immediately clear that the descriptions of the very same events given by
$s_{2}$ and $s_{3}$ are very different: according to $s_{2}$, $s_{1}$ is in an
eigenstate of $O$. Thus, $s_{1}$ has the definite value property $O=+1$. This
is not the case for $s_{3}$. According to $s_{3}$, $s_{1}$ is not in an
eigenstate of $O$, and therefore does not have any definite value property of
$O$.222Different metaphysical readings of this situation have been proposed in
the literature. Dorato, (2016) argues in favor of a dispositionalist account,
whereas Calosi and Mariani, (2020) argue for an indeterminate account. This is
what Rovelli calls Main Observation:

Main Observation. In QM different systems may give different accounts of the
same sequence of events.

RQM can be understood as that interpretation of quantum mechanics that, by
contrast with more familiar ones, holds that both descriptions (3) and (4) are
correct. But, how can that be? These descriptions are clearly different, as we
saw already. One possibility is simply that, both are correct, and yet one (or
both) is only a partial description description of quantum phenomena. Rovelli
rejects such a possibility explicitly. Quantum formalism, at least in its
usual applications, is complete:

Completeness. QM provides a complete and self-consistent scheme of description
of the physical world.

There seems to be one other possibility. Descriptions (3) and (4) are
different, correct, and complete descriptions of the relevant quantum events
because they are correct relatively to different relativization targets. That
is to say, description (3) is a correct and complete description of quantum
events at $t_{1}\rightarrow t_{2}$ relative to $s_{2}$, whereas description
(4) is a correct and complete description of the very same quantum events
relative to $s_{3}$. This is in effect the basic tenet of RQM:

Basic Tenet of RQM. RQM dictates a relativization of states and observables of
physical systems to other physical systems.

In Rovelli’s own words:

> [Q]uantum mechanics is a theory about the physical description of physical
> systems relative to other systems, and this is a complete description of the
> world (Rovelli, (1996), p. 1650).

> [T]he actual value of all physical quantities of any system is only
> meaningful in relation to another system (Rovelli, (2018), p. 6, italics in
> the original).

We could at this point, slightly change the formalism in order to incorporate
relativization explicitly in the formalism itself. For instance we could write
$O=v_{1}(s_{i}/s_{j})$ for “$s_{i}$ has value $v_{1}$ of observable $O$
relative to $s_{j}$”. More importantly for our purposes, we suggest to write:

$|\psi\rangle_{s_{i}/s_{j}}$ (5)

for “$s_{1}$ is in state $\psi$ relative to $s_{j}$”, and

$\lambda_{s_{i}/s_{j}}$ (6)

for “$s_{i}$ has an ontic state $\lambda$ relative to $s_{j}$”—the rationale
behind this direct incorporation in the formalism will be clear in due course.
If so, the quantum phenomena we started from, i.e., the ones in the “Third
Person Problem”, will be more perspicuously represented—at $t_{2}$—by:

$\displaystyle|\psi\rangle_{s_{1}/s_{2}}=|+\rangle_{s_{1}}$ (7)
$\displaystyle|\psi\rangle_{s_{12}/s_{3}}=(c_{1}|+\rangle\otimes|plus\rangle+c_{2}|-\rangle\otimes|minus\rangle)_{s_{12}}$

Before we move on to the next section we should pause to consider a crucial
issue. Rovelli himself presents RQM using quantum states, and frequently uses
talk of “relativization of states”. This clearly does not entail that he
recognizes a substantive ontological reading of such quantum states. In
effect, Rovelli is explicitly skeptical about such an heavy-weight ontological
reading. He writes:

> [T]he conceptual step was to introduce the notion of “wavefunction” $\psi$,
> soon to be evolved in the notion of “quantum state” $\psi$, endowing it with
> heavy ontological weight. This conceptual step was wrong, and dramatically
> misleading. We are still paying the price for the confusion it generated
> (Rovelli, (2018), p. 2, italics added).

This will play a crucial role. As of now, it is unclear what the (alleged)
“heavy ontological weight” of the quantum state amounts to. Harrigan and
Spekkens, (2010) provides a clear framework to define rigorously such a notion
by classifying them either as ontic or epistemic. To this we now turn.

###  2.2 Harrigan and Spekkens’ Distinction: $\psi$-Ontic and $\psi$-Epistemic
Models

In order to illustrate the distinction between ontic and epistemic states, let
us consider a straightforward example of a classical particle (cf. Leifer,
(2014)). In classical mechanics one assigns a precise position $q$ and a
momentum $p$ to it, and the pair $(q,p)$ constitutes the phase space point of
the object under consideration. This point describes the physical state of the
particle, i.e. its ontic state, which exists objectively and independently of
any observer.333Its future positions and momenta, moreover, are univocally
determined via Hamilton’s equations, meaning that at any time the ontic state
of the particle will be described by $(q,p)$. In addition, every other
observable property of the relevant physical system is a function of $(q,p)$.
However, if one does not have precise information about the position and
momentum of this physical system, then one can represent the uncertain
knowledge about its ontic state by a probability density $f(q,p)$ over phase
space. Interestingly, $f(q,p)$—the epistemic state—represents mere knowledge,
and does not describe any inherent property of the particle. Different
observers may not have equal knowledge of the state of our particle. If so, it
is possible to associate several epistemic states to the same ontic state. In
effect, such epistemic states—i.e. such probability distributions—may overlap
in the particle’s phase space. By contrast, the particle’s ontic state is
uniquely determined by the point $(q,p)$.

Harrigan and Spekkens’ categorization of ontological quantum models employs
the ontic/epistemic distinction in order to determine whether the quantum
state $\psi$ has to be interpreted as representing some underlying reality or
just observers’ knowledge of particular systems (cf. Harrigan and Spekkens,
(2010)). This constitutes the very theoretical heart of the PBR theorem. The
taxonomy is framed within operational quantum theory, an approach to QM where
primitive notions consist exclusively in preparations procedures, i.e.
instructions about how to prepare physical systems in certain states, and
measurements performed on such states. Harrigan and Spekkens claim that

> [I]n an operational formulation of quantum theory, every preparation $P$ is
> associated with a density operator $\rho$ on Hilbert space, and every
> measurement $M$ is associated with a positive operator valued measure (POVM)
> $\\{E_{k}\\}$. (In special cases, these may be associated with vectors in
> Hilbert space and Hermitian operators respectively.) The probability of
> obtaining outcome $k$ is given by the generalized Born rule,
> $p(k|M,P)=\textrm{Tr}(\rho E_{k})$ (Harrigan and Spekkens, (2010), p. 128).

The main aim of such operational models, they continue, is to provide
probabilities $p(k|M,P)$ of outcomes $k$ for some measurement $M$ performed on
prepared systems, given a set of preparation instructions $P$. When a
measurement is carried out, a measuring device will “reveal something about
those properties” (_ibid._). In this respect, Harrigan and Spekkens assume
that a complete specification of the properties of a given individual physical
system under scrutiny is provided by its ontic state, denoted by
$\lambda$.444As in the classical case discussed above, the ontic state
$\lambda$ belongs to an ontic state space $\Lambda$. More precisely, $\lambda$
provides a complete specification of the preparation procedures that are
performed on a particular quantum system, and hence, it yields a complete
description of its measurable properties.555It is worth noting, however,
Harrigan and Spekkens use the term “ontological” in a much weaker sense with
respect to standard philosophical jargon: they simply mean that $\lambda$
refers to something real in the world, leaving many details regarding the
metaphysics of quantum objects completely unspecified. Furthermore, they
underline that although an observer knows exactly the preparation procedures
prior the performance of a certain measurement, she may not have complete
knowledge about the ontic state $\lambda$ of the system under investigation.
Thus, it follows that while the future outcomes $k$ of a certain measurement
are determined by $\lambda$—and therefore the probability to obtain them is
given by $p(k|\lambda,M)$—the epistemic state of the experimenter is
represented only by $p(\lambda|P)$. Notably, an observer that has incomplete
information about $\lambda$ assigns “non-sharp” probability distributions over
the ontic state space $\Lambda$, i.e. multiple probability distributions can
be assigned to the very same ontic state, exactly as in the classical case
discussed a few lines above.

Against this background, Harrigan and Spekkens go on to provide conditions to
classify ontological quantum models as $\psi-$ontic or $\psi-$epistemic.
Briefly and roughly, a model is defined $\psi-$ontic if the ontic state
$\lambda$ can be consistently described by a unique pure state. Consequently,
in $\psi-$ontic models different quantum states correspond to disjoint
probability distributions over the space of ontic states $\Lambda$. More
precisely, a model is said to be $\psi-$ontic if “for any pair of preparation
procedures, $P_{\psi}$ and $P_{\phi}$, associated with distinct quantum states
$\psi$ and $\phi$, we have $p(\lambda|P_{\psi})p(\lambda|P_{\phi})=0$ for all
$\lambda$” (Harrigan and Spekkens, (2010), pp. 131-132). That is, observers’
epistemic states associated with different quantum states do not overlap in
$\Lambda$. Conversely, a model is defined $\psi-$epistemic if there exists
ontic states consistent with more than one pure state; in such epistemic
models, thus, there are quantum states that correspond to overlapping
probability distributions in $\Lambda$. In this operational context, this
implies that agents’ epistemic states may overlap, i.e. there exist
preparation procedures $P_{\psi},P_{\phi}$ such that
$p(\lambda|P_{\psi})p(\lambda|P_{\phi})\neq 0$. Hence, the ontic state
$\lambda$ can be consistently represented by both quantum states $\psi$ and
$\phi$:

> [I]n a $\psi-$epistemic model, multiple distinct quantum states are
> consistent with the same state of reality—the ontic state $\lambda$ does not
> encode $\psi$” (_ibid._ , p. 132).

In the case of $\psi-$epistemic models, then, the quantum state refers to
observers’ incomplete knowledge of reality, it is not a description of reality
itself.

On top of that, Harrigan and Spekkens divide quantum models in
$\psi-$_complete_ and $\psi-$_incomplete_. In $\psi-$complete models, the pure
quantum state encodes every information about the represented physical system.
Moreover, in such models there is a one-to-one relation between reality and
its complete description provided by $\psi$. If one knows the pure quantum
state of a certain system under consideration, one has a complete knowledge of
its ontic state. Therefore, $\psi-$complete models are also $\psi-$ontic.
Examples of such models are given by standard QM, Everett’s relative-state
formulation (cf. Everett, (1957)), the Many-World interpretation (cf. Wallace,
(2012)), and Wave-Function Realism (cf. Albert, (2013)).

If $\psi$ does not represent reality completely, then a quantum model is
$\psi-$incomplete. Notably, $\psi-$incomplete models may be either
$\psi-$supplemented or $\psi-$epistemic. In the former case, the description
of a physical system provided by $\psi$ is supplemented by some additional (or
hidden) variables, whose value is generally unknown. In hidden variables
models, trivially, the quantum state provides partial or incomplete knowledge
of the system. Well-known examples of hidden variables models are Bohmian
mechanics (cf. Dürr et al., (2013)), Bohm’s pilot-wave theory (cf. Bohm,
(1952)), and Nelson’s stochastic mechanics (cf. Nelson, (1966)). Notably, also
the class of $\psi-$supplemented models is $\psi-$ontic. Finally, in the case
of $\psi-$epistemic models, $\psi$ represents agents’ incomplete knowledge of
reality, and not reality itself. Typical examples of such a kind of models is
given by QBism (cf. Caves et al., (2002)) and RQM.666From these distinctions
some conclusions can be derived. Firstly, there cannot be models which are
simultaneously $\psi-$complete _and_ $\psi-$epistemic. Thus, if a model is
$\psi-$complete, it must be $\psi-$ontic (cf. Lemma 6, Harrigan and Spekkens,
(2010), p. 133). Conversely, if a model is $\psi-$epistemic, it cannot be
$\psi-$ontic, since it describes only observers’ knowledge and not any
underlying physical reality. Secondly, if a model is $\psi-$incomplete, then
it can either be $\psi-$ontic, as in the case of hidden variable models, or
$\psi-$epistemic.

In the rest of the paper we will critically analyze some implicit assumptions
in Harrigan and Spekkens’ categorization. In particular we will argue that
some requirements concerning the nature of $\lambda$ are in tension, if not
inconsistent, with basic ontological tenets of RQM. As we pointed out already,
this will have notable consequences for the relation between RQM and the PBR
theorem, a formal result that we are now going to introduce.

###  2.3 The PBR Theorem

The PBR theorem was heralded as the single most important result in quantum
foundations after the Bell-theorem.777This judgement was given by Anthony
Valentini, cf. Reich, (2011). It allegedly establishes that an epistemic
reading of the quantum state cannot recover quantum predictions. Pusey,
Barrett and Rudolph phrase the result in the form of a no-go theorem:

> [T]his Article presents a no-go theorem: if the quantum state merely
> represents information about the real physical state of a system, then
> experimental predictions are obtained that contradict those of quantum
> theory (Pusey et al., (2012), p. 475).

We present here a streamlined reconstruction of the result. The argument
depends on different assumptions, but we will mention just a few:

Preparation Independence. It is possible to prepare different physical systems
in such a way that their ontic states are independent, i.e. uncorrelated.

Harrigans and Spekkens Definition of Epistemic State. Quantum states
$|\psi\rangle$ and $|\phi\rangle$ associated with preparation procedures
$P_{\psi}$ and $P_{\phi}$ are epistemic states iff
$p(\lambda|P_{\psi})p(\lambda|P_{\phi})\neq 0$.888We saw in the previous
subsection this is really not the definition of an epistemic state, but rather
it is implied by the definition. The reason we use this will be clear in a
minute.

Measurement Response. If two quantum systems are prepared in such a way as to
fulfill Preparation Independence, measurements respond solely to the
properties of the system that is being measured.

We already discussed Harrigans and Spekkens Definition of Epistemic State. As
for Measurement Response, let us quote directly from the original PBR-paper:

> [T]he outcome of the measurement can only depend on the physical states of
> the two systems at the time of measurement (Pusey et al., (2012), p. 476).

As we will see, this will play a crucial role for our discussion. As of now,
let us go back to proof of the PBR theorem.

Let $s_{1}$ be a quantum system prepared with two different procedures
$P_{\psi}$ and $P_{\phi}$. Let $\langle\psi|\phi\rangle=\frac{1}{\sqrt{2}}$.
Consider a two-dimensional Hilbert space $\mathcal{H}$ spanned by
$|\psi\rangle=|\uparrow\rangle$ and
$|\phi\rangle=|+\rangle=\frac{1}{\sqrt{2}}(|\uparrow\rangle+|\downarrow\rangle)$.
Suppose now $|\uparrow\rangle$ and $|+\rangle$ are epistemic states. Then the
ontic state of $s_{1}$ should be compatible with both and we can write:

$\displaystyle p(\lambda_{1}|P_{\uparrow})\neq 0,$ (8) $\displaystyle
p(\lambda_{1}|P_{+})\neq 0.$

Prepare another system $s_{2}$ in such a way that the ontic states of the two
systems are uncorrelated—as guaranteed by Preparation Independence. Then, by
the same argument:

$\displaystyle p(\lambda_{2}|P_{\uparrow})\neq 0,$ (9) $\displaystyle
p(\lambda_{2}|P_{+})\neq 0.$

Equations (8) and (9) are meant to capture that the ontic states of $s_{1}$
and $s_{2}$, $\lambda_{1}$ and $\lambda_{2}$ respectively, are compatible with
both $|\uparrow\rangle$ and $|+\rangle$. Then, the complex system $s_{12}$ is
compatible with the following tensor product states:

$\displaystyle|\omega_{1}\rangle_{12}=|\uparrow\rangle_{1}\otimes|\uparrow\rangle_{2}$
(10)
$\displaystyle|\omega_{2}\rangle_{12}=|\uparrow\rangle_{1}\otimes|+\rangle_{2}$
$\displaystyle|\omega_{3}\rangle_{12}=|+\rangle_{1}\otimes|\uparrow\rangle_{2}$
$\displaystyle|\omega_{4}\rangle_{12}=|+\rangle_{1}\otimes|+\rangle_{2}$

If the ontic state $\lambda_{12}$ is compatible with all of them, we can
write:

$\displaystyle p(\lambda_{12}|P_{\omega_{1}})\neq 0$ (11) $\displaystyle
p(\lambda_{12}|P_{\omega_{2}})\neq 0$ $\displaystyle
p(\lambda_{12}|P_{\omega_{3}})\neq 0$ $\displaystyle
p(\lambda_{12}|P_{\omega_{4}})\neq 0$

Set $|-\rangle=\frac{1}{\sqrt{2}}(|\uparrow\rangle-|\downarrow\rangle)$. Now
we perform a measurement on $s_{12}$ that projects onto the following states:

$\displaystyle|\chi_{1}\rangle_{12}=\frac{1}{\sqrt{2}}(|\uparrow\rangle_{1}\otimes|\downarrow\rangle_{2}+|\downarrow\rangle_{1}\otimes|\uparrow\rangle_{2})$
(12)
$\displaystyle|\chi_{2}\rangle_{12}=\frac{1}{\sqrt{2}}(|\uparrow\rangle_{1}\otimes|-\rangle_{2}+|\downarrow\rangle_{1}\otimes|+\rangle_{2})$
$\displaystyle|\chi_{3}\rangle_{12}=\frac{1}{\sqrt{2}}(|+\rangle_{1}\otimes|\downarrow\rangle_{2}+|-\rangle_{1}\otimes|\uparrow\rangle_{2})$
$\displaystyle|\chi_{4}\rangle_{12}=\frac{1}{\sqrt{2}}(|+\rangle_{1}\otimes|-\rangle_{2}+|-\rangle_{1}\otimes|+\rangle_{2})$

This measurement responds solely to the properties of $s_{12}$, given
Measurement Response. Now, algebraic manipulation shows that

$\langle\omega_{1}|\chi_{1}\rangle=\langle\omega_{2}|\chi_{2}\rangle=\langle\omega_{3}|\chi_{3}\rangle=\langle\omega_{4}|\chi_{4}\rangle=0$
(13)

That is to say that for every measurement direction in (12) we can find a
state of $s_{12}$ in (10) that is orthogonal to it. In this case, standard
quantum predictions yield that:

$\displaystyle p(\lambda_{12}|P_{\omega_{1}})=0$ (14) $\displaystyle
p(\lambda_{12}|P_{\omega_{2}})=0$ $\displaystyle
p(\lambda_{12}|P_{\omega_{3}})=0$ $\displaystyle
p(\lambda_{12}|P_{\omega_{4}})=0$

contradicting (11). Thus quantum states are not epistemic states. Or so the
argument goes.

##  3 The Peaceful Coexistence

Given the formal result above, it seems to follow that RQM contradicts the
predictions of standard quantum mechanics. The argument seems straightforward:
according to Rovelli, $\psi$ is to be interpreted epistemically. And the PBR
theorem (allegedly) establishes that an epistemic interpretation of quantum
states cannot reproduce standard quantum predictions. However, it is well-
known that RQM is in perfect empirical agreement with respect to the
predictions and statistics of quantum theory, the PBR argument
notwithstanding. How can this be? How can we dissolve the tension between
these two apparently incoherent claims? In what follows we answer to these
questions. 999To be clear: it is not the aim of the paper to provide a
thorough defense of RQM. Rather, it is to show that there the PBR-theorem does
not proivde an argument against it.

In order to show how RQM can escape the consequences of PBR theorem, we need
to focus on the implicit assumptions made by Harrigan and Spekkens concerning
the nature of the ontic state $\lambda$. In Section 2.2 we saw that
ontological models are framed within operational quantum theory, and that
their basic ingredients are (i) properties of individual physical systems
associated with experimental procedures, and (ii) measurements performed on
such systems revealing the value of some such particular properties. In
effect, Harrigan and Spekkens assume that the ontic state $\lambda$ of a
quantum system provides a _complete_ description of its _inherent_ properties
(Harrigan and Spekkens, (2010), p. 128)—i.e. attributes that objects
instantiate in virtue of the way themselves are, independently of any relation
they stand in with external observers and/or contexts (cf. Lewis, (1986), p.
61). Moreover, $\lambda$ provides an _objective_ representation of quantum
systems, since the ontic state of a quantum object refers to an underlying
physical reality whose features do not depend upon any particular observer.
Hence, according to Harrigan and Spekkens, we can uniquely characterize a
certain system through the specification of its instantiated properties by
specifying a list of preparation procedures. In the light of this, $\lambda$
is taken to be an observer-independent, complete description of physical
systems.101010This is in line with the analysis in e.g. Oldofredi and Lopez,
(2020).

For these reasons, in $\psi$-complete models—as e.g. standard QM—the authors
claim that:

  * •

variations of $\lambda$ cause variations in the wave function $\psi$;

  * •

there is a one-to-one correspondence (a bijective map) between ontic states of
physical systems and pure quantum states representing them, since there is a
unique correct description of the various quantum systems under consideration.

From these premises, it follows straightforwardly that if a certain $\lambda$
can be described by multiple $\psi$-s, then these $\psi$s are taken to
represent the incomplete knowledge of the ontic state possessed by different
observers. Hence, it is easy to understand why overlapping probability
distributions over $\Lambda$ are defined as epistemic states: these represent
the lack of information (ignorance) of a certain observer about the complete
specification of the properties of a given system. Thus, if multiple wave
functions can be ascribed to the same $\lambda$, they equally fail to describe
some property in the ontic state of the object at hand.

These assumptions about the nature of $\lambda$ are clearly violated in the
context of RQM. Recall the Third Person scenario presented in Section 2.1. In
that case, it is clear upon inspection that the ontic state of a particular
system strictly depends on the perspective of a given observer, since physical
systems are represented by relational states. In effect, according to the
observer $s_{2}$, the system $s_{1}$ is in the $|+\rangle$ state, and thus, it
has a definite value for the observable $O$. By contrast, according to
$s_{3}$, $s_{1}$ system is in a superposition of states, and hence it does not
possess any definite value for $O$. However, from the Main Observation, we
know that in RQM both $s_{2}$ and $s_{3}$ provide correct descriptions of a
given sequence of physical events. This immediately entails that in RQM the
ontic state of physical systems (and thereby their properties) depends on the
perspective of a specific observer, in plain contrast with Harrigan and
Spekkens’ entire farmework.

Now, if the relational nature of quantum systems entails that their ontic
states depend on the specification of a certain perspective, then different
observers can ascribe diverse wave functions to the very same $\lambda$.
Against this background, however, these $\psi$s do not represent epistemic
limitations, such as, for instance, incomplete knowledge. Rather, in RQM they
simply reflect the perspectival nature of physical systems. Alternatively,
given that physical systems in RQM are described by relational properties,
which by definition require the specification of a certain observer and/or
context to be meaningful, different observers may assign different ontic
states to the same target object, as in the already mentioned example of the
Third Person problem—and as we are about to see again in a different context.
Such relational states, however, should be not interpreted epistemically, but
ontologically: they reflect that reality (and thus the ontic states of quantum
objects) is observer-dependent, or better, perspectival, contrary to what
Harrigan and Spekkens simply assume. It is thus clear that RQM employs
completely different criteria to define what ontic states are with respect to
those assumed by Harrigan and Spekkens. In particular, according to Rovelli’s
theory it is possible to assign several $\psi$s to the same observed system
without necessarily be committed to a $\psi$-epistemic view as defined by
their classification. This entails that the wavefunction must be relational as
well: $\psi$ can no longer be associated with a perspective-independent ontic
state, but rather with a relational $\lambda$, i.e. an underlying perspectival
reality. Thus, we conclude, according to RQM, _both_ $\psi$ and $\lambda$ must
be relational, that is:

  * •

$\lambda$ represents a quantum system always relatively to a certain observer,

  * •

$\psi$ stores information that a particular observer has relatively to a given
system.

To sum up: given that in RQM an absolute representation of physical systems is
unavailable by construction, there cannot be a one-to-one correspondence
between $\lambda$ and $\psi$. Strictly speaking, there is neither an absolute,
i.e. observer-independent, $\lambda$, nor an absolute $\psi$. RQM allows to
assign more than one $\psi$ to the very same individual physical system.
Remarkably, such $\psi$s are not interpreted as a manifestation of the
ignorance of the various observers, but are a simple consequence of the
relational ontology of the theory. As a matter of fact, one could see some
irony here. Suppose we were to insist that there must be a one-to-one
correspondence between $\lambda$ and $\psi$. Then, given that different
observers assign different $\lambda$-s to the same quantum systems, they do
assign different $\psi$-s as well.

Even if RQM supports a somewhat epistemic take on the quantum wavefunction,
this differs substantially from Harrigan and Spekkens definition of
$\psi$-epistemic models. In particular, although in RQM it is possible to
ascribe multiple wave functions to the very same system, it is not the case
that these descriptions fail to capture certain intrinsic, observer-
independent properties instantiated by a certain physical system. There are no
such properties to be captured to begin with! Therefore, since the assumptions
employed to define what ontic states are in RQM differ completely with respect
to those at play in Harrigan and Spekkens, (2010), we conclude that this
classification does not have the correct metaphysical and formal resources to
deal with relational ontic states and relational $\psi$s. Thus, Harrigan and
Spekkens’ classification cannot straightforwardly be applied to RQM. This fact
has a remarkable implication for our discussion: given that the PBR theorem
crucially relies on Harrigan and Spekkens’ classification of quantum
ontological models, as per the Harrigans and Spekkens Definition of Epistemic
States assumption, but the latter cannot be used to evaluate RQM, one can
safely conclude that RQM does not lie within the scope of the theorem. This
means that when Pusey, Barrett and Rudolph refer to $\psi$-epistemic model,
they are not referring to RQM. There is no contradiction between the epistemic
interpretation of $\psi$ in RQM and the PBR result.

From such a conclusion, we can also infer a more general lesson, i.e. that it
is not necessarily true that the PBR theorem excludes every type of
$\psi$-epistemic model, or better, not every $\psi$-epistemic model
contradicts the predictions and statistics of standard QM, _contra_ the main
claim of the PBR argument.

##  4 A Relational PBR Theorem?

In the light of the above, a natural question arises: could we find a
relational counterpart of the PBR-theorem? That is, could we give an analogous
result once the relativization of states, observables and ontic states
characterizing RQM is taken explicitly into account? In this section we
provide an argument that answers the question in the negative.

From our reconstruction, the PBR theorem crucially depends on three
assumptions. Our argument in Section 3 can be read as a challenge to the
second, namely to the Harrigan and Spekkens Definition of Epistemic State
assumption. In particular, it can be read as challenging the exhaustiveness of
Harrigan and Spekkens’s classification. This is because we argued that their
categorization importantly leaves out RQM—or more precisely, that it cannot be
consistently used to evaluate RQM.111111This assumption can be challenged on
other grounds as well, as done for instance in Hance et al., (2021). Hance et
al. argue that wavefunctions can represent simultaneously reality _and_
knowledge, contrary to Harrigan and Spekkens definitions which are mutually
exclusive. Moreover, the paper shows on the one hand that such exclusiveness
is simply presupposed, and on the other that it is neither necessary, nor
sufficient to capture the relation existing between the ontic state $\lambda$
and $\psi$. The Preparation independence assumption has been the focus of a
great deal of scrutiny as well.121212For critical discussions about the
validity of this premise cf. Mansfield, (2016), Schlossauer and Fine, (2012,
2013, 2014) and references therein. Interestingly, dropping this assumption
altogether, Lewis et al., (2012) showed with an explicit model that it is
mathematically possible to interpret quantum states as agent’s information
concerning an underlying physical state. In addition, Mansfield, (2014) shows
that the PBR theorem “breaks down” in models that relax Preparation
independence. To our knowledge, however, Measurement Response has not yet been
properly discussed. We now turn to argue that RQM sheds interesting new light
on this assumption as well. In particular, as we are going to show, it seems
that its use in the context of RQM undermines one crucial step in the
derivation of the PBR-theorem.

As we saw in Section 2.1, according to the main tenet of RQM, states and
observables must always be relativized to other physical systems functioning
as observers. We suggested to incorporate this relativization directly into
the formalism; going back to the inner workings of the PBR-theorem this
results in the following. An observer $s$, which is a physical system like any
other, prepares two quantum systems $s_{1}$ and $s_{2}$ which can be both
compatible with the epistemic states $|\psi\rangle$ and $|+\rangle$
encountered in Section 2.3, implementing two different protocols $P_{\psi}$
and $P_{\phi}$ and using two copies of the same preparation device. We have to
recall here that in RQM every quantum state is epistemic, since it refers to
the knowledge that a given observer has about a certain system. Moreover,
although $s$ knows the instructions to prepare the systems $s_{1},s_{2}$, the
agent “may nonetheless have incomplete knowledge of $\lambda$” (Harrigan and
Spekkens, (2010), p. 128). As a consequence of RQM, after the preparations
protocols have been implemented, $s_{1}$ and $s_{2}$ acquire a particular
ontic state relative to $s$, i.e. $\lambda_{1/s}$, and $\lambda_{2/s}$
respectively—where, for the sake of simplicity, we assumed that relativization
to the same physical system is consistent.131313To this regard, we have
reasons to believe that our assumption is consistent. Indeed, discussing the
third person scenario we saw a similar situation; an observer $s_{2}$
interacted with a system $s_{1}$ which before the interaction was in a
superposition of $O$-states, and found $s_{1}$ to have a definite value for
the observable $O$. In the case discussed in the present section, an observer
$s$ prepares two systems interacting directly with them, thus, they will
acquire a certain state relative to $s$. Here we make a standard maneuver in
operational quantum mechanics, namely to consider preparations as
measurements. Hence, equations (8) and (9) should be replaced with their
relational counterparts:

$\displaystyle p(\lambda_{1/s}|P_{\uparrow/s})\neq 0,$ (15) $\displaystyle
p(\lambda_{1/s}|P_{+/s})\neq 0$

and

$\displaystyle p(\lambda_{2/s}|P_{\uparrow/s})\neq 0,$ (16) $\displaystyle
p(\lambda_{2/s}|P_{+/s})\neq 0.$

Interestingly, in this scenario Preparation Independence is trivially met
since both systems are prepared at the same site by the same agent $s$ who can
freely choose among the possible protocols how to prepare $s_{1}$ and $s_{2}$.
After their preparation, suppose that the two systems are then brought
together to form a complex system $s_{12}$ as required in the proof of the PBR
theorem. Referring to this, it is worth noting that Pusey et al., (2012)
implicitly assume that the composite system $s_{12}$ is simply given by the
product of the independent physical states of the individual systems forming
it. While this not a mereologically trivial fact, we accept it for the sake of
the argument; then, the ontic state of the complex system $s_{12}$ relative to
the agent $s$ is given by $\lambda_{12/s}$.141414Cf. Schlossauer and Fine,
(2013) for critical remarks concerning this mereological issue. Moreover, let
us suppose that the ontic state $\lambda_{12/s}$ is compatible with all
possible tensor product states of (10), so as to get the relativized
counterpart of (11), namely (17):

$\displaystyle p(\lambda_{12/s}|P_{\omega_{1/s}})\neq 0$ (17) $\displaystyle
p(\lambda_{12/s}|P_{\omega_{2/s}})\neq 0$ $\displaystyle
p(\lambda_{12/s}|P_{\omega_{3/s}})\neq 0$ $\displaystyle
p(\lambda_{12/s}|P_{\omega_{4/s}})\neq 0.$

Now the complex system $s_{12}$ is measured. But note that, according to
Measurement Response, the measurement system $s^{*}$—which is another observer
with respect to $s$—will respond solely to the properties of $s_{12}$. The
crucial question in a relational context becomes: The properties of $s_{12}$
relative to what other system? To answer this question, we have to take into
account two crucial facts: on the one hand, any system is on a par in RQM,
and, on the other hand, physical objects do not possess inherent absolute
properties _per se_ , independently of any observer. Hence, it is reasonable
to assume that the measurement system will respond to the properties of
$s_{12}$ relative to $s^{*}$, represented by the relativized ontic state
$\lambda_{12}/s^{*}$. In this respect, it is crucial to note that there is no
guarantee that, in RQM, $\lambda_{12/s}=\lambda_{12/s^{*}}$. On the contrary,
in general they will differ. Thus, no contradiction will arise. In effect,
suppose that the relativized counterpart of (14), that is (18) below, holds
true:

$\displaystyle p(\lambda_{12/s^{*}}|P_{\omega_{1/s^{*}}})=0$ (18)
$\displaystyle p(\lambda_{12/s^{*}}|P_{\omega_{2/s^{*}}})=0$ $\displaystyle
p(\lambda_{12/s^{*}}|P_{\omega_{3/s^{*}}})=0$ $\displaystyle
p(\lambda_{12/s^{*}}|P_{\omega_{4/s^{*}}})=0$

Still, (17) and (18) constitute no contradiction because of the difference in
their relativization targets. Thus, we can safely claim that one cannot derive
the conclusion of the PBR theorem in this relational context. At this
juncture, one may claim that we over-simplified the relevant details of the
PBR theorem, by relativizing states of $s_{1}$ and $s_{2}$ to the same
observer.

Let us discuss another experimental situation involving two observers, say
Alice and Bob, located at two different sites. We will show that neither in
this case one can derive a relational PBR-like result.151515The generalization
of our conclusion to $N$ observers is trivial, and it is left as an exercise
to the reader. According to this scenario, Alice’s task is to prepare the
quantum system $s_{1}$ in one of the possible epistemic states $|+\rangle$ or
$|\uparrow\rangle$, as required by the PBR argument (cf. Section 2.3). Bob has
the same assignment for the quantum system $s_{2}$. Let us assume for the sake
of argument that Preparation Independence holds. Then, from this experimental
setting, the ontic state of $s_{1}$ relative to Alice, i.e. $\lambda_{1/A}$
should be compatible with the following probabilities:

$\displaystyle p(\lambda_{1/A}|P_{\uparrow/A})\neq 0,$ (19) $\displaystyle
p(\lambda_{1/A}|P_{+/A})\neq 0.$

Similarly for Bob:

$\displaystyle p(\lambda_{2/B}|P_{\uparrow/B})\neq 0,$ (20) $\displaystyle
p(\lambda_{2/B}|P_{+/B})\neq 0.$

It is worth noting that in virtue of Preparation Independence, Alice’s
preparation of $s_{1}$ is not influenced by (i.e. is independent of) Bob’s
preparation of the system $s_{2}$, and vice versa. Interestingly, Alice does
not know how Bob prepared $s_{2}$. Therefore she has no information about the
final state he obtained after the implementation of the preparation protocols.
Thus, according to RQM, from Alice’s perspective the system $s_{2}$ will have
an ontic state $\lambda_{2/A}$ which differs from $\lambda_{2/B}$. More
precisely, given the limited knowledge that Alice has about Bob’s
preparations—she only knows that he is preparing $s_{2}$ and that this system
is compatible with the state $|\uparrow\rangle$ or $|+\rangle$—it follows that

$\displaystyle\lambda_{2/A}=\frac{1}{\sqrt{2}}|\uparrow\rangle\otimes|up\rangle_{Bob}+\frac{1}{\sqrt{2}}|+\rangle\otimes|plus\rangle_{Bob}.$
(21)

Hence, from her perspective the physical state of $s_{2}$ is represented by a
superposition of the $|\uparrow\rangle$ and $|+\rangle$ states each entangled
with Bob’s state after the preparation, something allowed—and indeed
required—by RQM, as we saw in Section 2.1 while discussing the Third Person
problem. Indeed, the state in (21) has notable similarities with (4). As we
pointed out several times already, Alice’s description of $\lambda_{2}$ is
correct even though it is not equivalent to Bob’s, who does not obtain a
superposition of states in virtue of his interaction with the system $s_{2}$
via the preparation device. An analogous argument holds for Bob, who will
assign to the system $s_{1}$ the following ontic state:

$\displaystyle\lambda_{1/B}=\frac{1}{\sqrt{2}}|\uparrow\rangle\otimes|up\rangle_{Alice}+\frac{1}{\sqrt{2}}|+\rangle\otimes|plus\rangle_{Alice}$
(22)

which is clearly different from the $\lambda_{1/A}$ obtained by Alice after
her preparations.

This fact is relevant for our discussion—and more generally for the PBR-like
argument we are exploring—since Alice and Bob have to send their prepared
systems $s_{1}$ and $s_{2}$ to a measuring device $s^{*}$, the third observer
of this scenario. As in the previous case, $s^{*}$ will interact only with the
composite system $s_{12}$ formed by the product of the individual states of
$s_{1}$ and $s_{2}$. Remarkably, since $\lambda_{1/A}\neq\lambda_{1/B}$ and
$\lambda_{2/A}\neq\lambda_{2/B}$, it follows that Alice and Bob will provide
different relational descriptions of the ontic state of the complex system
$s_{12}$. In effect, given (21) and (22), we expect that
$\lambda_{{12}/A}\neq\lambda_{{12}/B}$. In particular, for Alice the ontic
state $\lambda_{{12}}$ can be represented by these two possible options:

$\displaystyle\lambda_{{12}/A}=|\uparrow\rangle_{A}\bigg{(}\frac{1}{\sqrt{2}}|\uparrow\rangle\otimes|up\rangle_{Bob}+\frac{1}{\sqrt{2}}|+\rangle\otimes|plus\rangle_{Bob}\bigg{)}=|\omega_{1}\rangle_{A},$
(23)
$\displaystyle\lambda_{{12}/A}=|+\rangle_{A}\bigg{(}\frac{1}{\sqrt{2}}|\uparrow\rangle\otimes|up\rangle_{Bob}+\frac{1}{\sqrt{2}}|+\rangle\otimes|plus\rangle_{Bob}\bigg{)}=|\omega_{2}\rangle_{A},$

while for Bob the ontic state $\lambda_{{12}}$ of $s_{12}$ is given by the
following states:

$\displaystyle\lambda_{{12}/B}=|\uparrow\rangle_{B}\bigg{(}\frac{1}{\sqrt{2}}|\uparrow\rangle\otimes|up\rangle_{Alice}+\frac{1}{\sqrt{2}}|+\rangle\otimes|plus\rangle_{Alice}\bigg{)}=|\omega_{3}\rangle_{B},$
(24)
$\displaystyle\lambda_{{12}/B}=|+\rangle_{B}\bigg{(}\frac{1}{\sqrt{2}}|\uparrow\rangle\otimes|up\rangle_{Alice}+\frac{1}{\sqrt{2}}|+\rangle\otimes|plus\rangle_{Alice}\bigg{)}=|\omega_{4}\rangle_{B}.$

It is straightforward to see that these descriptions are incompatible with the
products states of (10). However, from Alice’s perspective we have that:

$\displaystyle p(\lambda_{{12}/A}|P_{\omega_{1/A}})\neq 0,$ (25)
$\displaystyle p(\lambda_{{12}/A}|P_{\omega_{2/A}})\neq 0.$

Once again, similarly for Bob:

$\displaystyle p(\lambda_{{12}/B}|P_{\omega_{3/B}})\neq 0,$ (26)
$\displaystyle p(\lambda_{{12}/B}|P_{\omega_{4/B}})\neq 0.$

Interestingly, the probabilities in (25) will be zero according to Bob’s
perspective. By construction Bob cannot find any superposition-state for
$s_{2}$. Similarly, the probabilities in (26) will be zero for Alice. These
results simply follow from the relativization of states in RQM. Thus, there is
no contradiction between Alice’s and Bob’s perspectives.

Finally, if Alice and Bob send their prepared systems $s_{1}$ and $s_{2}$ to
the measurement device $s^{*}$, the latter will interact with a composite
system $s_{12/s^{*}}$ which will not contain any superposition of states
contrary to the cases represented in (23) and (24), so that the observer
$s^{*}$ will not find the complex system $s_{12/s^{*}}$ in any of the states
$|\omega_{1}\rangle_{A},|\omega_{2}\rangle_{A},|\omega_{3}\rangle_{B},|\omega_{4}\rangle_{B}$.
This fact is due once again to the relativization of states, from which it
follows that

$\displaystyle\lambda_{{12}/s^{*}}\neq\lambda_{{12}/A},$ (27)
$\displaystyle\lambda_{{12}/s^{*}}\neq\lambda_{{12}/B}.$

In turn, given the ontic states defined in (23), (24), the inequality in (27),
and from the main tenet of RQM, it follows that:

$\displaystyle p(\lambda_{{12}/A}|P_{\omega_{1/A}})\neq
p(\lambda_{12/s^{*}}|P_{\omega_{1/s^{*}}}),$ (28) $\displaystyle
p(\lambda_{{12}/A}|P_{\omega_{2/A}})\neq
p(\lambda_{12/s^{*}}|P_{\omega_{2/s^{*}}}),$ $\displaystyle
p(\lambda_{{12}/B}|P_{\omega_{3/A}})\neq
p(\lambda_{12/s^{*}}|P_{\omega_{3/s^{*}}}),$ $\displaystyle
p(\lambda_{{12}/B}|P_{\omega_{4/A}})\neq
p(\lambda_{12/s^{*}}|P_{\omega_{4/s^{*}}}).$

Therefore, the measuring device $s^{*}$ will interact with a composite
physical system having an ontic state completely different w.r.t. those
described by Alice and Bob. As in the previous case, then, the relativization
of states implies that the three observers involved in our scenario—namely
Alice, Bob and $s^{*}$—will assign different ontic states to the composite
system $s_{12}$, so that no contradiction can arise among these different
perspectives and standard quantum predictions. The conclusion stands: RQM is
neither vulnerable to the original PBR-theorem, nor to its relativized
counterpart.

##  5 Conclusions

In this paper we discussed throughly the relation between RQM and the PBR-
theorem. This is because there is a prima-facie significant tension between
the two. RQM suggests an epistemic take on the quantum wavefunction, whereas
the PBR theorem allegedly establishes that no epistemic interpretation is
empirically equivalent to standard quantum mechanics. This seems to threaten
the empirical adequacy of RQM. We argued there is no such threath. This sheds
new light on both RQM and PBR. As for the former, it forces us to accept that
there is an epistemic reading of the quantum wavefunction that is both (i)
empirically adequate, and (ii) is not captured by extant characterization of
quantum epistemic states. As for the latter, it shows that there are
limitations to the very applicability of the theorem and its conclusion that
have largely gone unnoticed in the literature.161616For insightful discussions
on previous drafts of the paper we would like to thank [Redacted].

## References

  * Albert, (2013) Albert, D. Z. (2013).  Wave function realism.  In Ney, A. and Albert, D. Z., editors, The Wave Function: Essays on the Metaphysics of Quantum Mechanics, pages 52–57. Oxford University Press.
  * Bohm, (1952) Bohm, D. (1952).  A suggested interpretation of the quantum theory in terms of “hidden” variables. I.  Physical Review, 85(2):166 –179.
  * Bong et al., (2020) Bong, K.-W., Utreras-Alarcón, A., Ghafari, F., Liang, Y.-C., Tischler, N., Cavalcanti, E., Pryde, G., and Wiseman, H. (2020).  A strong no-go theorem on the Wigner’s friend paradox.  Nature Communications, 16:1199–2005.
  * Calosi and Mariani, (2020) Calosi, C. and Mariani, C. (2020).  Quantum relational indeterminacy.  Studies in History and Philosophy of Science Part B: Studies in History and Philosophy of Modern Physics, 71:158–169.
  * Candiotto, (2017) Candiotto, L. (2017).  The Reality of Relations.  Giornale di Metafisica, 2:537–551.
  * Caves et al., (2002) Caves, C., Fuchs, C., and Schack, R. (2002).  Quantum probabilities as Bayesian probabilities.  Physical Review A, 65:022305.
  * Dorato, (2016) Dorato, M. (2016).  Rovelli’s Relational Quantum Mechanics, Anti-Monism, and Quantum Becoming.  In Marmodoro, A. and Yates, D., editors, The Metaphysics of Relations, pages 235–262. Oxford University Press.
  * Dürr et al., (2013) Dürr, D., Goldstein, S., and Zanghì, N. (2013).  Quantum physics without quantum philosophy.  Berlin: Springer.
  * Everett, (1957) Everett, H. (1957).  “Relative state” formulation of quantum mechanics.  Reviews of Modern Physics, 29(3):454–462.
  * Frauchiger and Renner, (2018) Frauchiger, D. and Renner, R. (2018).  Quantum Theory Cannot Consistently Describe the Use of Itself.  Nature Communications, 9:1–10.
  * Hance et al., (2021) Hance, J., Rarity, J., and Ladyman, J. (2021).  Wavefucntions can Simultaneously Represent Knowledge and Reality.  arXiv:2101.06436v1, pages 1–6.
  * Harrigan and Spekkens, (2010) Harrigan, N. and Spekkens, R. (2010).  Einstein, Incompleteness, and the Epistemic View of Quantum States.  Foundations of Physics, 40:125–157.
  * Laudisa and Rovelli, (2019) Laudisa, F. and Rovelli, C. (2019).  Relational Quantum Mechanics.  Stanford Encyclopedia of Philosophy.
  * Leifer, (2014) Leifer, M. (2014).  Is the quantum state real? An extended review of $\psi$-ontology theorems.  Quanta, 3(1):67–155.
  * Lewis, (1986) Lewis, D. (1986).  Philosophical papers, volume 2.  Oxford: Oxford University Press.
  * Lewis et al., (2012) Lewis, P., Jennings, D., Barrett, J., and Rudolph, T. (2012).  Disticnt Quantum State Can Be Compatible with a Single State of Reality.  Physical Review Letters, 109:150404.
  * Mansfield, (2014) Mansfield, S. (2014).  Reflections on the PBR Theorem: Reality Criteria & Preparation Independence.  In Coecke, B., Hasuo, I., and Panangaden, P., editors, Proceeding of the 11th workshop on Quantum Physics and Logic 2014, pages 102–112. EPTCS 172.
  * Mansfield, (2016) Mansfield, S. (2016).  Reality of the quantum state: Towards a stronger $\psi$-ontology theorem.  Physical Review A, 94:042124.
  * Nelson, (1966) Nelson, E. (1966).  Derivation of the schrödinger equation from newtonian mechanics.  Physical Review, 150(4):1079 – 1085.
  * Oldofredi, (2020) Oldofredi, A. (2020).  The Bundle Theory Approach to Relational Quantum Mechanics.  Foundations of Physics, 51(1):1–22.
  * Oldofredi and Lopez, (2020) Oldofredi, A. and Lopez, C. (2020).  On the Classification between $\psi-$Ontic and $\psi-$Epistemic Ontological Models.  Foundations of Physics, 50(11):1315–1345.
  * Pusey et al., (2012) Pusey, M. F., Barrett, J., and Rudolph, T. (2012).  On the reality of the quantum state.  Nature Physics, 6(8):475–478.
  * Reich, (2011) Reich, E. (2011).  Quantum theorem shakes foundations.  Nature.
  * Rovelli, (1996) Rovelli, C. (1996).  Relational Quantum Mechanics.  International Journal of Theoretical Physics, 35(8):1637–1678.
  * Rovelli, (2016) Rovelli, C. (2016).  An Argument Against the Realistic Interpretation of the Wave Function.  Foundations of Physics, 46:1229 – 1237.
  * Rovelli, (2018) Rovelli, C. (2018).  Space is blue and birds fly through it.  Philosophical Transactions of the Royal Society A, Physical and Engineering Sciences, 376(2123):2017.0312.
  * Schlossauer and Fine, (2012) Schlossauer, M. and Fine, A. (2012).  Implications of the Pusey-Barrett-Rudolph Quantum No-Go Theorem.  Physical Review Letters, 108:260404.
  * Schlossauer and Fine, (2013) Schlossauer, M. and Fine, A. (2013).  Is the Pusey-Barrett-Rudolph Theorem Compatble with Quantum Nonseparability?  arXiv:1306.5805v1, pages 1–4.
  * Schlossauer and Fine, (2014) Schlossauer, M. and Fine, A. (2014).  No-go theorem for the composition of quantum system.  Physical Review Letters, 112(7):070407.
  * Smerlak and Rovelli, (2007) Smerlak, M. and Rovelli, C. (2007).  Relational EPR.  Foundations of Physics, 37:427–445.
  * Wallace, (2012) Wallace, D. (2012).  The emergent multiverse. Quantum theory according to the Everett interpretation.  Oxford: Oxford University Press.


# Superpixel-Based Building Damage Detection from Post-earthquake Very High
Resolution Imagery Using Deep Neural Networks

Jun Wang, Zhoujing Li, Yixuan Qiao, Qiming Qin, Peng Gao, Guotong Xie J. Wang,
Y. Qiao, P. Gao and G. Xie are with PingAn Technology, Beijing, PR China
e-mail: (wangjun.cau@foxmail.com).Z. Li is with Agricultural Information
Institute, Chinese Academy of Agricultural Sciences, Beijing, PR China.Q. Qin
is with Institute of Remote Sensing and Geographic Information System, Peking
University, Beijing, 100871 PR China.This work was finished in 2016 and
revised in 2021.

###### Abstract

Building damage detection after natural disasters like earthquakes is crucial
for initiating effective emergency response actions. Remotely sensed very high
spatial resolution (VHR) imagery can provide vital information due to their
ability to map the affected buildings with high geometric precision. Many
approaches have been developed to detect damaged buildings due to earthquakes.
However, little attention has been paid to exploiting rich features
represented in VHR images using Deep Neural Networks (DNN). This paper
presents a novel superpixel based approach combining DNN and a modified
segmentation method, to detect damaged buildings from VHR imagery. Firstly, a
modified Fast Scanning and Adaptive Merging method is extended to create
initial over-segmentation. Secondly, the segments are merged based on the
Region Adjacent Graph (RAG), considered an improved semantic similarity
criterion composed of Local Binary Patterns (LBP) texture, spectral, and shape
features. Thirdly, a pre-trained DNN using Stacked Denoising Auto-Encoders
called SDAE-DNN is presented, to exploit the rich semantic features for
building damage detection. Deep-layer feature abstraction of SDAE-DNN could
boost detection accuracy through learning more intrinsic and discriminative
features, which outperformed other methods using state-of-the-art alternative
classifiers. We demonstrate the feasibility and effectiveness of our method
using a subset of WorldView-2 imagery, in the complex urban areas of
Bhaktapur, Nepal, which was affected by the Nepal Earthquake of April 25,
2015.

###### Index Terms:

Deep neural networks, Very high resolution (VHR), Image segmentation, Damage
detection.

##  I Introduction

Rapid damage assessment after the earthquake is crucial for initiating
effective emergency response actions. Remote sensing can quickly provide
regional information about the affected areas with high thematic precision
[1]. However, accurate and reliable damage detection remains a daunting
challenge due to the complicated background interference, shooting angle,
shadows, and solar illumination conditions. In reality, Building damage
detection from very high resolution (VHR) remotely sensed imagery is one of
the most challenging topics in emergency response and disaster management.

Valuable pre-earthquake data are always not guaranteed shortly after an event.
Consequently, building damage detection using only post-earthquake VHR images
plays a significant role in rapid damage assessment during the ﬁrst response
[1, 2]. More precisely, post-earthquake VHR image classification based on
various image features is the most commonly adopted strategy [2]. Along with
image classification moving from pixel to object, Blaschke et al. [3] have
elucidated the shortcomings of the pure per-pixel classification approach,
which leads to splitting meaningful objects into separated pixels without
explicit semantic features. Whereas, in terms of the per-superpixel strategy,
Geographic Object-Based Image Analysis (GEOBIA) [3] provides a general
architecture that is capable of segmenting images and extracting features over
objects of interest, it avoids the intrinsic drawbacks of pixel-based
methodology. In general, superpixel segmentation methods can be categorized
into two types according to the features used: edge-based methods and region-
based methods [3]. Edge-based methods take boundary as the primary vision clue
for partition image contents, without considering regional features. On the
other hand, region-based methods split or merge neighboring pixels according
to their regional semantic homogeneity and heterogeneity. However, most
region-based segmentations like Fractal Net Evolution Approach (FNEA) [4]
embedded in eCognition suffered from inconsistent segmentation for utilizing
merely spectral and shape features, but neglected the key visual cues like
texture. Therefore, to address the lack of exploration on rich semantic
features like texture, we need to incorporate textural semantics over the
techniques mentioned above to obtain more reasonable segmentation.

Figure 1: Flowchart of the proposed building damage detection method.

Based on the per-superpixel strategy, Liu et al. [5] used simple linear
iterative clustering (SLIC) [6] to generate superpixels for hyperspectral
classification, but SLIC had notable drawbacks in short of shape matching.
Moreover, it’s worth noting that they stated in [5] that the segmentation was
only treated as spatial constraints, but not involved in feature
representation stage to exploit the rich regional semantic features. More
importantly, besides the image segmentation results, the subsequent feature
representation of semantic objects plays an utmost critical role in detection.
To the best of our knowledge, the detection or classification tasks contain
two critical steps: feature design by prior human knowledge and machine
learning by classifiers. However, most of the existing building damage
detection methods utilize conventional classifiers, such as Support Vector
Machine (SVM) [7], Random Forests (RF) [8], Extreme Learning Machine (ELM) [9]
and Multi-Layer Perceptron (MLP) [10], these methods always encounter the main
weakness arising from insufficient feature learning [7], and suffer from
tedious feature designing and selection. Recent research has shown that the
performance of machine learning methods heavily depends on feature
representation and learning [11]. In this case, designing or finding good
features is confirmed to be the critical problem of developing a proper
classifier-based strategy. Alternatively, unlike conventional frameworks, DNN
or deep learning architecture could manipulate the hand-craft feature design
process into a machine self-learning manner [12]. Consequently, feature
learning and pattern discrimination can be achieved by DNN itself jointly.

Given this, the main contributions of this paper are twofold. First, a
texture-enhanced segmentation method is proposed to generate more meaningful
superpixels. Second, we proposed a DNN-based framework to combine the benefit
of superpixel segmentation and deep feature learning architecture.

##  II BUILDING DAMAGE DETECTION WITH DNN

Based on the approaches mentioned above, the intuition of our paper is to
develop a superpixel-based method for building damage detection to address the
issue of semantic feature exploration. A framework combining image
segmentation and deep neural networks is proposed for building damage
detection from post-earthquake VHR optical imagery. The flowchart of the
proposed approach is summarized in Fig. 1, which is structured as follows:

1) The VHR imagery is initially partitioned by modifying the Fast Scanning and
Adaptive Merging (FSAM) algorithm [13], Meanwhile, the semantic features of
each superpixel include Local Binary Pattern (LBP) texture [15] are
calculated.

2) Then, a region merging criterion integrating texture feature is proposed,
more perceptually meaningful superpixels are generated based on Region
Adjacent Graph (RAG). Then various image features of merged superpixels are
extracted to represent the semantic information of regions.

3) On the basis of feature extraction on previously generated superpixels, a
pre-trained DNN using Stacked Denoising AutoEncoders [14] called SDAE-DNN was
presented to classify and detect superpixels that indicate affected buildings,
the results are evaluated and compared with state-of-the-art alternative
methods in the 2015 Nepal earthquake.

###  II-A Initial partition using modified FSAM

Firstly, the VHR imagery was over-segmented by modifying the Fast Scanning and
Adaptive Merging (FSAM) algorithm [7]. FSAM was proved to be an efficient
initial segmentation algorithm with good shape connectivity and shape matching
[7]. Nevertheless, FSAM was originally designed for only single-band images,
which paid less attention to excavating the rich spectral information of
multi-spectral VHR images. Therefore, spectral angle mapping (SAM) method was
applied to customize for VHR multi-band imagery in our study, so as to make
full use of the rich spectral features. The homogeneity between region
$R_{obj1}$ and $R_{obj2}$ is calculated as:

$\displaystyle
h_{SAM}\left(R_{obj1},R_{obj2}\right)=cos^{-1}\left(\frac{\theta_{obj1}\cdot\theta_{obj2}}{\left\|\theta_{obj1}\right\|_{2}\left\|\theta_{obj2}\right\|_{2}}\right)$
(1)
$\displaystyle=\cos^{-1}{\left(\frac{\sum_{b=1}^{B}{\theta_{obj1}\cdot\theta_{obj2}}}{\left(\sum_{b=1}^{B}{\theta_{obj1}}^{2}\right)^{1/2}\left(\sum_{b=1}^{B}{\theta_{obj2}}^{2}\right)^{1/2}}\right)}$

where $\theta_{obj1}$ and $\theta_{obj2}$ represent the spectral vector of
adjacent obj1 and obj2, respectively. $B$ represents the total band number of
the input image.

###  II-B Region Merging based on RAG

In this section, an improved region merging criterion incorporating texture
information was proposed to group the pixels into perceptually meaningful
geospatial regions. Regions of the initial partition are represented as nodes
of a corresponding RAG, as illustrated in Fig. 2. The weights of the edges
represent the heterogeneity between the adjacent pair of regions. A minimum
heterogeneity criterion [4] was applied to RAG to find the optimal
segmentation. During the merging, texture clustering can refine the real edges
of the texture region. More precisely, the semantic heterogeneity consisted of
spectral, textual and geometric information. It was defined as:

$\displaystyle h=\omega_{spec}\ast h_{spec}+\omega_{texture}\ast
h_{texture}+\omega_{shape}\ast h_{shape}$ (2)

where $h_{\text{spec}}$, $h_{\text{texture}}$ and $h_{\text{shape}}$
represented the heterogeneity derived from spectral, texture and shape feature
respectively. $W_{\text{spec}}$, $W_{\text{texture}}$ and $W_{\text{shape}}$
indicated the weights of spectral, texture and shape feature respectively, and
$W_{\text{spec}}$, $W_{\text{texture}}$, $W_{\text{shape}}$ $\in[0,1]$,
$W_{\text{spec}}$ \+ $W_{\text{texture}}$ \+ $W_{\text{shape}}$ = 1. Spectral/
texture/shape weights were set to 0.7/0.2/0.1, because generally spectral
information plays a more primary role in segmentation. The shape heterogeneity
$h_{\text{shape}}$ describes the difference of compact degree
$h_{\text{compact}}$ and smooth degree $h_{\text{smooth}}$ before and after
adjacent region merging, the details of $h_{\text{compact}}$ and
$h_{\text{shape}}$ are clearly described in [4]. Smoothness/compactness
weights for shape feature were set to 0.5/0.5 because we did not want to favor
either compact or non-compact superpixels. As the experimental image has 8
bands, the spectral heterogeneity was also calculated using SAM defined in
Equation (1).

Figure 2: Region merging based on RAG. Left: image region merge. Right:
Corresponding Region Adjacent Graph (RAG).

Besides spectral and shape features, a theoretically simple but effective
texture descriptor named local binary pattern (LBP) has been proposed and
widely used in computer vision [15]. Therefore, in our study, the joint
distribution of LBP was used to characterize textural features over adjacent
regions generated by the previous modified FSAM algorithm. LBP labels the
pixels of an image by thresholding the neighbourhood of each pixel and
considers the result as a binary number. The obtained values are summed as the
LBP value of the current texture region. It is deﬁned as:

$\displaystyle{LBP}_{P,R}$ $\displaystyle=\
\sum\nolimits_{p=0}^{P-1}s\left(g_{p}-g_{c}\right)2^{p},$ (3) $\displaystyle
s(x)$ $\displaystyle=\left\\{\begin{matrix}0,x<0\\\ 1,x\geq 0\\\
\end{matrix}\right.$

where $P$ is the number of neighboring pixels on a circle of radius $R$,
$g_{p}$ and $g_{c}$ denote the image intensity values of the $p_{\text{th}}$
neighbor and the center pixel, respectively; and $s(x)$ is a binary ﬂag
denoting the relation of $g_{p}$ and $g_{c}$, i.e., $x=g_{p}-g_{c}$.

The initial modified FSAM segmentation and minimum heterogeneity region
merging results are illustrated in Fig. 3. The following elementary features
were calculated for each superpixel: 1) Spectral feature (mean values and
variance for each band); 2) Shape feature (area, shape index, length/width,
rectangular ﬁt, roundness and density), which are calculated to distinguish
between buildings and roads; 3) Texture feature (The near-infrared reflectance
(NIR) band was used for grey-level co-occurrence matrix (GLCM) contrast,
correlation, and entropy (all directions), considering that the NIR band is
one of the most informative bands in VHR images for urban land cover
classification). Besides, 4) mean normalized difference vegetation index
(NDVI) of each segment was also calculated to handle vegetations and non-
vegetations.

Figure 3: The Initial superpixel segmentation result using modified FSAM and
region merge results at scale 20.

###  II-C SDAE-DNN pre-train strategy

After obtaining the superpixels and their elementary features mentioned above,
the DNN classifier was trained and refined feature representation to detect
building damage information (see Fig. 1). Typical deep neural network
architectures include deep belief networks (DBNs) [11], SAEs [12], and stacked
denoising AEs (SDAEs) [14]. Correspondingly, the layer-wise training models
have a bunch of alternatives such as restricted Boltzmann machine (RBM) [11],
auto-encoder (AE) [12], and denoising AE (DAE) [14]. Moreover, with respect to
a supervised fine-tuned criterion, the SDAE deep architecture systematically
yielded better generalization performance than DBN and SAE in several computer
vision tasks [14]. Hence, in this paper, SDAE was introduced as the
corresponding deep architecture for building damage detection from VHR
imagery.

Figure 4: The Denoising Auto-Encoder (DAE) architecture.

In the proposed work, the whole deep learning process of SDAE has 2 phases:
unsupervised pre-training phase and supervised fine-tuning phase. We aim to
conduct classification and exploit feature abstraction and learning
simultaneously. Firstly, SDAE was constructed by stacking the modules of DAE.
DAE acts as an extension of classical AE. A typical AE contains two parts:
encoder and decoder. DAE reconstructs the input into a partially corrupted
version, the key difference is that now $z$ is the reconstruction of $x^{’}$
rather than the original input $x$ in AE. The difference between the AE and
the DAE is illustrated in Fig. 4, where the flow of the AE is represented as
dotted lines and the flow of the DAE is represented as solid lines. The
encoder corresponds to a nonlinear mapping function
$y={f_{\theta}}(x)=s(Wx+b)$ which maps the input x from its original feature
space into a hidden representation y. where W is an n*n weight matrix and b is
a bias vector. $Tanh$ function is used as the active function $f(x)$. Then,
the hidden representation y is mapped back to the input space through a
similar decoding transformation:
$z={g_{{\theta}^{{}^{\prime}}}}(y)=s({W^{{}^{\prime}}}y+{b^{{}^{\prime}}})$.
The parameters of this model are optimized by minimizing the average
reconstruction error between x and z as:

$\displaystyle\theta^{\ast},\theta^{\prime\ast}=\mathop{\arg\min}_{\theta,\theta^{\prime}}{\frac{1}{n}\sum\nolimits_{i=1}^{n}L\left(x_{i},\
z_{i}\right)}$ (4)

$\displaystyle\theta^{\ast},\theta^{\prime\ast}=\mathop{\arg\min}_{\theta,\theta^{\prime}}{\frac{1}{n}\sum\nolimits_{i=1}^{n}L\left(x_{i},g_{\theta^{\prime}}\left(f_{\theta}\left(x_{i}\right)\right)\right)}$
(5)

where the loss function $L(x,z)$ is defined as reconstruction cross-entropy,
i.e.,

$\displaystyle L\ {\left(x,\
z\right)=-\sum\nolimits_{k=1}^{n}\left\\{x_{k}log{z_{k}}+\left(1-x_{k}\right)log\left(1-z_{k}\right)\right\\}}$
(6)

The DAE was considered as a regularized AE. In order to discover robust
features and prevent it from simply learning the identity, the AE is trained
to reconstruct the input $x$ from a corrupt $x^{{}^{\prime}}$,
$x^{{}^{\prime}}={d_{\gamma}}(x)$, where ${d_{\gamma}}(x)$ denotes the
corruption function. The original input is corrupted by randomly setting part
of the inputs to be zero according to the corruption rate ${\gamma}$ (in our
study ${\gamma}=0.3$). The target function of the DAE becomes:

$\displaystyle\theta^{\ast},\theta^{\prime\ast}=\mathop{\arg\min}_{\theta,\theta^{\prime}}{\frac{1}{n}\sum\nolimits_{i=1}^{n}L\left(x_{i},g_{\theta^{\prime}}\left(f_{\theta}\left(d\gamma\left(x_{i}\right)\right)\right)\right)}$
(7)

The layers of SDAE are pre-trained sequentially from top to bottom. The
learning target of each layer is to get the optimal weight by minimizing the
target cost function defined in Equation (6). After pre-training, a logistic
regression layer is added to the top layer of the SDAE. Then, the whole
network is fine-tuned as an MLP using backpropagation [11].

SDAE encodes the corrupted input into a representation, then decodes the
representation back to the input space, seeking to minimize the average
reconstruction error between the output of the decoder and the original input.
In this manner, SDAE is able to get more robust features from the hidden layer
and capture the joint distribution among a subset of the inputs, consequently,
it can be treated as a practical feature learner and classifier.

##  III EXPERIMENTS AND RESULTS

Bhaktapur, the third-largest city of Nepal, was struck on 25 April 2015 by the
Magnitude 7.8 2015 Nepal earthquake. The above methodology was implemented on
a 4144×2072 pixels pan-sharpened multi-spectral WorldView-2 VHR imagery of
Bhaktapur with 0.5 m resolution. The results are generated on a PC equipped
with an Intel Core2 2.8 GHz processor and an NVIDIA GeForce GT 620 GPU card,
implemented in Python and C++. As shown in Fig. 5, the ground truth data
overlaid on original input images was shown in Fig. 5(a), the yellow points
indicate the affected buildings, which were delineated and verified manually
by visual inspection, based on the damage assessment map produced by UNOSAT
(United Nations Operational Satellite Applications). Then, the detection
result of SDAE-DNN was demonstrated in 5(b). Besides, the damage detection
results by SVM, RF, ELM and MLP are listed in Fig. 5(c)-5(f), respectively. It
should be noted that the motivation of this paper is not to investigate the
performance of different machine learning algorithms, but to construct a novel
superpixel-based deep learning framework for the requirements of building
damage detection in VHR images.

Moreover, in terms of hyperparameter setting, ﬁve-fold cross-validation is
used to perform grid search and seek optimal parameters for each method. The
proposed superpixel-based SDAE-DNN was implemented with 5 hidden layers, and
the number of neurons in each layer is selected from (20, 50, 200, 800), the
learning rate is 0.001. Then the comparison is conducted with a series of
state-of-the-art algorithms, e.g., SVM, RF, ELM and MLP, which are commonly
used in remote sensing image classification. More precisely, SVM is run with
linear kernels and the ranges for optimization of penalty coefﬁcient are (0.1,
1 10, 100, 800). The parameter of RF mainly refers to the number of trees. In
this study, a set of values (20, 50, 200, 800) are deﬁned for parameter
selection. For ELM, the number of neurons in the hidden layer ranges from (20,
50, 200, 800). For MLP, the standard one hidden-layer MLP is implemented with
hidden neurons selected from (20, 50, 200, 800).

Figure 5: The ground truth data and comparison of our detection results with
four alternative algorithms. (a) The ground truth data, the yellow points
indicate the damaged building pixels. (b) Detection results of proposed SDAE-
DNN method . (c) Detection results of SVM method. (d) Detection results of RF
method. (e) Detection results of ELM method. (f) Detection results of MLP
method. (Red demonstrates the damaged buildings, yellow represents undamaged
buildings.)

Then, after we obtain the optimal parameters that achieve the best performance
for the aforementioned methods, another five-fold stratified cross-validation
is used to get the average accuracy of 5 cross-validation loops, where four
folds are used for training and the remaining unseen fold is reserved to
measure accuracy. All the accuracy evaluation of the building damage detection
results is presented in Table I based on four statistics, i.e., true positive
rate (TP), false positive rate (FP), true negative rate (TN), and false
negative rate (FN). The metrics used for evaluation are precision, recall,
f1-score, cross-validation accuracy and Kappa coefficient of classification.

$\displaystyle
Precision=\frac{|TP|}{|TP|+|FP|},Recall=\frac{|TP|}{\left|TP\right|+|FN|}$ (8)

$\displaystyle f_{1}=\frac{2\ast precision\ast recall}{precision+recall}$ (9)

It can be observed that our method delivers the best cross-validation accuracy
of 0.877, compared with other approaches. Moreover, it yields nearly the
highest precision, recall, f1-score. It is demonstrated to be effective and
competitive in dealing with the issue of building damage detection based on a
superpixel-based deep learning architecture.

Method | Kappa | Precision | Recall | F1-score |  cross validation accuracy
---|---|---|---|---|---
SVM | 0.911 | 0.924 | 0.953 | 0.938 | 0.843
RF | 0.905 | 0.919 | 0.893 | 0.905 | 0.827
MLP | 0.813 | 0.883 | 0.794 | 0.836 | 0.776
ELM | 0.889 | 0.878 | 0.906 | 0.892 | 0.810
Proposed | 0.934 | 0.951 | 0.948 | 0.949 | 0.877
TABLE I: ACCURACY COMPARISON OF PROPOSED METHOD.

Table II summarizes the computing time of mentioned methods for the VHR image.
For time efficiency comparison, the time does not include the segmentation
phase, since it is the same for all the methods. It is observed that the
proposed SDAE-DNN framework causes increased training complexity, but it is
worth noting that the deep architecture yields competitive and reasonable
predicting time, even slightly better than SVM.

Image ID | SVM | RF | MLP | ELM | Proposed
---|---|---|---|---|---
Training time (s) | 13.89 | 2.72 | 16.85 | 8.90 | 195.13
Predicting time (s) | 19.33 | 5.50 | 3.39 | 11.64 | 12.07
TABLE II: COMPARISON OF THE EFFICIENCY WITH PREVIOUS WORKS.

To our knowledge, the proposed framework is the first attempt to apply deep
learning to detect damaged buildings in VHR imagery. Experimental results
demonstrated that the SDAE-DNN framework not only boosted damage detection
precision, but also offered a possible route to avoid the common disadvantages
of time-consuming feature designing and selection in building damage
detection, which outperforms conventional machine learning-based frameworks.
Considering that the training samples are delineated manually by time-
consuming visual inspection, future work would refer to a more intelligent
sampling strategy. Fortunately, the active learning [16] technique can be
considered to optimize the sampling strategy and save the cost of manual
labelling. Moreover, the classical edge detection-based building extraction
methods [17,18] and image matching-based methods [19] should also be
considered to cooperate with the deep learning techniques.

##  IV CONCLUSION

In this paper, we concern the issue of feature exploration and learning for
building damage detection in VHR imagery, a superpixel segmentation based
strategy integrating the state-of-the-art SDAE-DNN architecture is proposed.
Firstly, a modified region-merge segmentation method binding LBP texture is
proposed, to generate meaningful superpixels better account for the spatial
constraints, which could better handle the intrinsic consistency of semantics
of geo-spatial objects. More importantly, the main contribution is that the
proposed SDAE-DNN deep architecture can learn discriminate features
hierarchically. The experiment results reveal that the proposed approach can
achieve competitive building damage detection performance. The proposed
framework would play a significant role in damage assessment, and contribute
to better rescue management and post-earthquake reconstruction missions.

## Acknowledgment

This work was supported by the ”National Science and Technology Major Project
of China (11-Y20A05-9001-15/16)”.

## References

  * [1] D. Brunner, G. Lemoine, and L. Bruzzone, “Earthquake damage assessment of buildings using VHR optical and SAR imagery,” IEEE Trans. Geosci. Remote Sens., vol. 48, no. 5, pp. 2403-2420, 2010.
  * [2] L. Dong, and J. Shan, “A comprehensive review of earthquake-induced building damage detection with remote sensing techniques,” ISPRS J. Photogramm. Remote Sens., vol. 84, no. 0, pp. 85-99, 2013.
  * [3] T. Blaschke, G. J. Hay and M. Kelly, “Geographic Object-Based Image Analysis – Towards a new paradigm,” ISPRS J. Photogramm. Remote Sens., vol. 87, no. 0, pp. 180-191, 2014.
  * [4] M. Baatz and A. Sch, “Multiresolution segmentation: An optimization approach for high quality multi-scale image segmentation,” in Proc. Angewandte Geographische Informationsverarbeitung XII. Beiträge zum AGIT-Symp, Salzburg, Karlsruhe, Germany, 2000, pp. 12–23.
  * [5] Y. Liu, G. Cao and Q. Sun, “Hyperspectral classification via deep networks and superpixel segmentation,” Int. J. Remote Sens., vol. 36, no. 13, pp. 3459-3482 , 2015.
  * [6] R. Achanta, A. Shaji and K. Smith, “SLIC Superpixels Compared to State-of-the-Art Superpixel Methods,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 11, pp. 2274-2281, 2012.
  * [7] K. Khoshelham, S. O. Elberink, and S. Xu, “Segment-Based Classification of Damaged Building Roofs in Aerial Laser Scanning Data,” IEEE Geosci. Remote Sens. Lett., vol. 10, pp. 1258-1262, 2013.
  * [8] J. Thomas, A. Kareem, and K. W. Bowyer, “Automated Poststorm Damage Classification of Low-Rise Building Roofing Systems Using High-Resolution Aerial Imagery,” IEEE Trans. Geosci. Remote Sens., vol. 52, pp. 3851-3861, 2014.
  * [9] H. Alhichri, Y. Bazi and N. Alajlan, “A hierarchical learning paradigm for semi-supervised classification of remote sensing images,” in Proc. IEEE Int. Geosci. Remote Sens. Symp.(IGARSS), pp. 4388-4391, 2015.
  * [10] D. Dubois and R. Lepage, “Fast and Efficient Evaluation of Building Damage From Very High Resolution Optical Satellite Images,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., pp. 4167-4176, 2014.
  * [11] G. E. Hinton, S. Osindero, and Y. W. Teh, “A Fast Learning Algorithm for Deep Belief Nets,” Neural Comput., vol. 18, pp. 1527-54, 2006.
  * [12] Y. Bengio, A. Courville, and P. Vincent, “Representation Learning: A Review and New Perspectives,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 8, pp. 1798-1828, 2013.
  * [13] J. Ding, C. Kuo, and W. Hong, “An efficient image segmentation technique by fast scanning and adaptive merging,” CVGIP, Aug, 2009.
  * [14] P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol, “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion,” J. Mach. Learn. Res., vol. 11, pp. 3371–3408, 2010.
  * [15] T. Ojala, M. Pietikäinen, and T. Mäenpää, “Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, pp. 971-987, 2002.
  * [16] Tuia D., F. Ratle, F. Pacifici, M. F. Kanevski, and W. J. Emery, ”Active Learning Methods for Remote Sensing Image Classification,” IEEE Trans. Geosci. Remote Sens., vol. 47, pp. 2218-2232, 2009.
  * [17] X. Ye, M. Liu, J. Wang, Q. Qin, H. Ren, J. Wang, and J. Hui, ”Building-based damage detection from postquake image using multiple-feature analysis,” IEEE Geosci. Remote Sens. Lett., vol. 14, pp. 499-503, 2017.
  * [18] J. Wang, Q. Qin, X. Ye, J. Wang, X. Qin, X. Yang. ”A Survey of Building Extraction Methods from Optical High Resolution Remote Sensing Imagery,” Remote Sensing Technology and Application. 2016 vol. 14, pp. 653-62, 2016.
  * [19] X. Yang, J. Wang, X. Qin, J. Wang, X. Ye, and Q. Qin ”Fast urban aerial image matching based on rectangular building extraction,” IEEE Geosci. Remote Sens. Magaz., vol. 3, no. 4, pp. 21-27, 2015.