# Whose projection postulate?

Anthony Sudbery1
Department of Mathematics, University of York,
Heslington, York, England YO10 5DD
1<EMAIL_ADDRESS>
(23 February 2024)

###### Abstract

The projection postulate is a description of the effect on a quantum system,
assumed to be in a pure state, of a measurement of an observable with a
discrete spectrum, in nonrelativistic quantum mechanics. It is often called
“von Neumann’s projection postulate” or “the Lüders rule”. This paper is an
examination of the versions of this postulate due to Dirac, von Neumann and
Lüders. It is shown that Dirac, in 1930, proposed what is now generally known
as the projection postulate. Von Neumann, in 1932, gave a different theory
which only applies in special and rather unusual cases. Lüders, in 1951,
rejected this theory and presented one which is the same as Dirac’s.
Treatments of observables with continuous spectra by both Dirac and von
Neumann are criticised, and the possibility of a generalised version of the
projection postulate for this case is considered. The paper concludes with a
discussion of the status of the projection postulate (in its various forms) as
a separate postulate (independent of the other postulates of quantum
mechanics) and as a separate form of time development (in addition to the
time-dependent Schrödinger equation).

##  1 Introduction

The projection postulate of non-relativistic quantum mechanics, in its
simplest form (which will be elaborated later in this paper), is the following
statement:

> P1 Consider a quantum system $S$ whose pure states belong to a Hilbert space
> $\mathcal{H}$, and an observable $A$ represented by a self-adjoint operator
> $\hat{A}$ on $\mathcal{H}$ which has a discrete spectrum. If $A$ is measured
> when $S$ is in a pure state $|\psi\rangle$ and the value $\alpha$ is found,
> then after the measurement $S$ is in the pure state
> $\Pi_{\alpha}|\psi\rangle$ where $\Pi_{\alpha}$ is the operator on
> $\mathcal{H}$ of orthogonal projection onto the subspace of eigenstates of
> $\hat{A}$ with eigenvalue $\alpha$.

From the earliest adumbrations of quantum theory by Bohr and Heisenberg, it
was recognised that a central feature of the theory was that observation had
an inescapable effect on a physical system; after Schrödinger’s formulation of
the theory in terms of wave functions, this came to be known as the “collapse
of the wave function”. The projection postulate makes this vague idea precise.
Versions of it were published by Dirac in 1930, by von Neumann in 1932, and by
Lüders in 1951. In the following sections (2, 3 and 4) we will examine each of
these and consider their status as “postulates” as opposed to _theorems_ which
can be deduced from a definition of “measurement” together with the other
postulates of quantum mechanics.

The above statement does not give a full description of the effect of all
possible types of measurement. In order to be applicable to all possible
measurements in a full physical theory, it needs to be extended to mixed
states, to observables with continuous spectra and to measurements whose
results have error bars. These extensions will be discussed in Section 5. The
penultimate section is a discussion of the status of the projection postulate
in the foundations of quantum mechanics, with a brief mention of some of the
literature on establishing the validity of the projection postulate and on
formulating a version of it in a relativistic theory.

##  2 Dirac

Paul Dirac’s version of the projection postulate is stated on p. 49 of the
first edition of his book _The Principles of Quantum Mechanics_ [4]. He bases
it on the theorem that any state can be expanded as a linear combination of
eigenstates of $\hat{A}$,

$\psi=\sum_{\alpha}c_{\alpha}\psi_{\alpha}$ (2.1)

where $\psi$ and all the eigenvectors $\psi_{\alpha}$ are normalised (in 1930
Dirac had not yet invented his bra-ket notation). Dirac proposes to use this
expansion to obtain “a mathematical form for the condition that an observation
is made with the minimum of disturbance”. Given the expansion, he invites us
to “suppose the observation to be made in such a way that the state of the
system afterwards is always one of those that occur in the expansion of the
initial $\psi$ in terms of eigen-$\psi$’s of $A$, _i.e._ one of the
$\psi_{\alpha}$’s in (2.1)”. These states are precisely the projections of
$\psi$ onto the eigenspaces of $\hat{A}$, so this “supposition” is the
projection postulate as stated in the introduction.

To what extent is Dirac proposing an independent postulate?

Dirac takes account of something that is often ignored in more recent
discussions of measurements and the projection postulate: there may well be
many different ways of measuring a given quantity, and it is likely that they
will have different effects on the measured system. It is unreasonable to
suppose that there is a single formula for the effect of a measurement of $A$,
such as the generally accepted projection postulate. However, one can ask for
the least possible disturbance caused by a measurement, and this is what Dirac
does. He takes this minimum effect to be given by the projection onto an
eigenstate, writing “This …may conveniently be defined to be the one that
causes the minimum of disturbance to the system”. This is indeed postulation.
But it need not be; it is clear that, given that the state vector after
measurement lies in the eigensubspace with the given eigenvalue, the
eigenvector that is closest to the state before measurement, in the metric
induced by the inner product of Hilbert space, is just the projection of that
state onto the eigenspace.

There is a further condition that needs to be added to the statement of the
projection postulate before it can be regarded as following from the other
postulates of quatum mechanics. Dirac argues that the state vector of the
system after the measurement of $A$ must be an eigenvector of $\hat{A}$, since
it is certain that a repetition of the measurement will give the same result.
But this assumes that the system is still present and available to be measured
after the first measurement (indeed, this assumption underlies the very idea
of a mathematical form for the disturbance to the system). This is not always
true; there are measurements which destroy the system, or convert it into a
system with a quite different state space – for example, absorption of a
photon by an atom. Measurements which can be repeated in the way required for
Dirac’s argument to be valid are known as “measurements of the first kind”.
Thus the full statement of the projection postulate should be

> P2 Consider a quantum system $S$ whose state is the Hilbert space
> $\mathcal{H}$, and an observable $A$ represented by an self-adjoint operator
> $\hat{A}$ on $\mathcal{H}$ which has a discrete spectrum. If $A$ is measured
> by a measurement of the first kind when $S$ is in a pure state
> $|\psi\rangle$ and the value $\alpha$ is found, and if this measurement
> causes the minimum possible change to the state of the system, then after
> the measurement $S$ is in the pure state $\Pi_{\alpha}|\psi\rangle$ where
> $\Pi_{\alpha}$ is the operator on $\mathcal{H}$ of orthogonal projection
> onto the subspace of eigenstates of $\hat{A}$ with eigenvalue $\alpha$.

As has been pointed out by Isham ([7] p. 35), this is not a postulate but a
theorem. In order to establish this, it is necessary to consider the status of
the proposition that any state can be expanded as a linear combination of
eigenstates. This is indeed a theorem in the mathematical theory of Hilbert
and von Neumann if the spectrum of $\hat{A}$ is discrete, but Dirac does not
appeal to the mathematical literature; he attempts to prove it himself, but
acknowledges that his proof is incomplete. In later editions of his book he
simply assumed that this expansion exists, regarding it as a necessary
condition for an operator to represent a physical observable. However, a proof
meeting Dirac’s standard of rigour can be given from the physical postulates
of quantum mechanics ([12] p. 51).

Moreover, Dirac does not consider it necessary to restrict his argument to
observables with discrete spectra, considering that his invention of the delta
function is adequate to deal with continuous spectra. This was trenchantly
criticised by von Neumann, as we will see in the next section, and we will
respect von Neumann’s strictures.

We note that if the actual result is not examined, the measurement still has a
disturbing effect on the system. In this _non-selective_ case the post-
measurement state must be considered to be the mixed state

$\rho^{\prime}=\sum_{\alpha}P_{\alpha}|\psi\rangle\langle\psi|P_{\alpha}.$

This need not be considered as a peculiar effect of measurements. It can be
derived from a description of the measurement as a physical process of the
measured system and the measuring apparatus, proceeding according to the
general unitary development of quantum systems: the state of the measured
system is a mixed state obtained by tracing over the states of the measuring
apparatus. So this version of the projection postulate does not result from
assigning any special physical significance to the process of measurement. In
general, it can be stated as

> P3 If the pre-measurement state is a (mixed, in general) state $\rho$, and
> if the measurement always produces a definite (though unknown) result
> $\alpha$, then the post-measurement state will be
>
> $\rho^{\prime}=\sum_{\alpha}P_{\alpha}\rho P_{\alpha}.$ (2.2)

##  3 Von Neumann

In 1927 John von Neumann published three papers [16, 18, 17] on the
foundations of quantum mechanics, in reaction to the transformation theory of
Jordan, Dirac and Schrödinger, whose work depended on “improper” functions
such as the $\delta$-function whose properties von Neumann regarded as
“absurd”. He therefore set out to give a mathematically rigorous account of
this theory, not, as he later wrote ([19] p. ix), by giving “a mathematically
rigorous refinement and explanation of Dirac’s ideas”, but by following “a
procedure differing from the very beginning , namely, the reliance on the
Hilbert theory of operators” . However, these 1927 papers do not contain a
precise account of the effects of measurement on a quantum system, but only a
statement equivalent to “collapse of the wave function”. The precise statement
can be found in his 1932 book _Mathematical Foundations of Quantum Mechanics_
[19]. The main burden of this book is to expound the mathematically rigorous
theory in the earlier papers, in particular giving a satisfactory treatment of
operators with a continuous spectrum. Nevertheless, in his discussion of the
effects of measurement von Neumann considers only observables with discrete
spectra, arguing that a measurement of an observable with a continuous
spectrum cannot give a single number as a result, but can only identify an
interval in which the result lies; the measurement is therefore equivalent to
a measurement of an observable with a discrete spectrum. If the spectrum is
non-degenerate von Neumann argues, in the same way as Dirac, that the state of
the system after the measurement must be the eigenstate with the actual result
of the measurement as eigenvalue.

If some of the eigenvalues are degenerate, however, von Neumann postulates a
different form for the post-measurement state. Suppose the observable $A$ has
an eigenvalue $\alpha$ whose eigenspace $\mathcal{S}_{\alpha}$ has finite
dimension $n_{\alpha}$ greater than $1$. Von Neumann always considers the
general case of mixed states. Let $\rho$ be the statistical operator before
the measurement of $A$, $\rho^{\prime}$ the statistical operator after the
measurement; then $\rho^{\prime}$ can be expanded as

$\rho^{\prime}=\sum_{\alpha}\sum_{i}r_{\alpha i}|\psi_{\alpha
i}\rangle\langle\psi_{\alpha i}|$

where $\\{|\psi_{\alpha 1}\rangle,\ldots|\psi_{\alpha n_{\alpha}}\rangle\\}$
is an orthonormal basis of $\mathcal{S}_{\alpha}$, and $0\leq r_{\alpha i}\leq
1$, while $\sum_{\alpha i}r_{\alpha i}=1$. Von Neumann assumes that the
measurement actually results in one of the states in this basis, so that
$r_{\alpha i}$ is the probability that the measurement results in the state
$|\psi_{\alpha i}\rangle$. But he is embarrassed by the fact that this post-
measurement state $\rho^{\prime}$ appears to depend on the choice of the basis
$|\psi_{ai}\rangle$ and thus on more than the pre-measurement state and the
quantity $A$ being measured. He therefore looks for cases when $\rho^{\prime}$
is independent of the choice of basis of $\mathcal{S}_{\alpha}$, and finds
that this is true when $r_{\alpha i}=c_{\alpha}$ is independent of $i$, so
that

$\rho^{\prime}=\sum_{\alpha}c_{\alpha}P_{\alpha}\quad\text{where}\quad
c_{\alpha}=\operatorname{tr}[P_{\alpha}\rho^{\prime}]/n_{\alpha}.$ (3.1)

He declares this to be the post-measurement state in this special case. Thus
in this case the measurement leaves the system in one of the eigenspaces
$\mathcal{S}_{\alpha}$, but there is total ignorance as to its position in
this subspace. He notes that this requires the pre-measurement state $\rho$ to
satisfy

$P_{\alpha}\rho P_{\alpha}=c_{\alpha}P_{\alpha}$

which is “clearly restricting $\rho$ sharply”; in fact it requires the pre-
measurement state to be one of total ignorance as to position in the
eigenspace $\mathcal{S}_{\alpha}$. It is then not surprising that the post-
measurement state also exhibits such ignorance. If this condition is not
satisfied, von Neumann does not attempt to specify the post-measurement state,
saying that “different arrangements of measurement can actually transform $U$
into different $U^{\prime}$” (von Neumann uses $U$ where we use $\rho$). This
is the same point that Dirac had made, but von Neumann makes a different
choice of assumption to get a definite post-measurement state.

##  4 Lüders

Gerhart Lüders misread von Neumann, claiming that von Neumann thought that the
post-measurement state is always given by (3.1), with an orthonormal basis
$\\{|\psi_{\alpha i}\rangle\\}$ determined by the measurement apparatus111I
was guilty of the same misreading in 1986, when I wrote “von Neumann stated a
postulate about the results of measurement later than Dirac, and he got it
wrong” [15].. Lüders validly argues that this would be absurd; as an extreme
case, measurement of an observable represented by a multiple of the identity
operator can be done without affecting the system at all, but what he
attributes to von Neumann would be an extreme change of state, resulting in a
totally mixed state, whatever the pre-measurement state. However, von
Neumann’s caveat means that this post-measurement state applies only if the
pre-measurement state is already totally mixed.

Lüders’s proposed amendment is precisely the projection postulate as stated in
Section 1. It is now often called “the Lüders rule”, but as we have seen, it
had already been stated by Dirac. Lüders claims that it “shows exactly what is
meant by the expression ‘reduction of the wave function”’, for which the only
reference he gives is Pauli’s _Handbuch der Physik_ treatise [8]. We must
assume that he was not familiar with Dirac’s book.

##  5 Continuous spectra

None of the three writers considered in this paper gave a satisfactory
treatment of the measurement of observables with continuous spectra. Dirac
regards continuous spectra as being covered by his theory of
$\delta$-functions, which he considers makes it possible to treat a continuous
spectrum in the same way as a discrete spectrum; he makes only a vague gesture
in this direction. Von Neumann states that a measurement of a quantity with a
continuous spectrum never yields an exact number as a value of the quantity,
but only an interval in which the value lies, a model which he attributes to
Wigner. He therefore regards the measurement as equivalent to a measurement of
the quantity which specifies in which interval the value lies, a quantity that
has a discrete spectrum. Lüders, in order to “simplify the treatment”, assumes
that all relevant operators have pure point spectra.

It is, of course, true that the statement of the result of a measurement of a
continuous quantity is always accompanied by error bars. But von Neumann’s
assumption that this is a specification of an interval, one of a pre-assigned
set of intervals, does not seem to be a universally valid representation of
the meaning of error bars. For a start, von Neumann’s identification of this
as a measurement of an observable whose eigenvalues correspond to set
intervals is surely not always correct. It may be appropriate when the error
is due to finite instrumental resolution, as for example an instrument with a
scale marked in multiples of a given unit; but in other cases the experimenter
does not select one of a set of intervals that are given before the
experiment. If the result is given as $\alpha\pm\epsilon$, the interval
$[\alpha-\epsilon,\alpha+\epsilon]$ has been determined by the experiment, not
given beforehand; and it seems wrong to interpret the experimenter’s result
$\alpha\pm\epsilon$ as a statement that the true value of $A$ certainly lies
in the interval $[\alpha-\epsilon,\alpha+\epsilon]$ and not outside it. It is
surely much more realistic to regard the stated result $\alpha\pm\epsilon$ as
an indication that the value of $A$ is not certain, but is given by a
probability distribution with mean $\alpha$ and variance $\epsilon$ (perhaps a
normal distribution). This seems to be the only possible interpretation if the
result is given, as it often is, in the form $\alpha^{+\delta}_{-\epsilon}$.

Thus the post-measurement state is a mixed state. The corresponding situation
in the case of a discrete spectrum would be an experimental result in the form
of a probability distribution $w(\alpha)$ of a discrete variable $\alpha$, and
the projection postulate, with a mixed pre-measurement state $\rho$, would
give the post-measurement state

$\rho^{\prime}=\sum_{\alpha}w(\alpha)\frac{P_{\alpha}\rho
P_{\alpha}}{\operatorname{tr}[P_{\alpha}\rho P_{\alpha}]}.$

It is not clear what the corresponding expression should be for a general
spectrum. However, for a non-selective projection the postulate (2.2) post-
measurement state the postulate P3 can be re-expressed without explicitly
assuming that the observable has a discrete spectrum, as follows.

> P4 (conjectural) Suppose the observable $A$ is measured when the system is
> in the state $\rho$. The effect of the measurement is to change the state to
> $\Pi(\rho)$ where $\Pi$ is the orthogonal projection onto the commutant of
> $\hat{A}$.

“Orthogonal” refers to the Hilbert-Schmidt inner product
$(A,B)=\operatorname{tr}(AB^{\dagger})$.

A simple calculation shows that if $\hat{A}$ has a discrete spectrum,
$\Pi(\rho)$ is a positive operator with trace $1$ if $\rho$ is. In general,
however, precise necessary conditions for this, if any, remain to be
determined. Unlike the discrete case, this cannot in general be regarded as a
theorem resulting from assumptions on the nature of a measurement, in
particular repeatability, because an observable capable of repeatable
measurement necessarily has a discrete spectrum ([3] Chap. IV).

##  6 Status of the projection postulate

We have seen how the projection postulate, in its original form referring to
an observable with a discrete spectrum, can be regarded as a theorem stating
that the projected post-measurement state necessarily results from certain
assumptions about the measurement. This leaves open the question of how this
is to be achieved: what physical process will lead to this post-measurement
state? A common argument is that it cannot be a unitary evolution, such as is
taken to govern physical processes apart from measurements, because unitary
evolution cannot take a pure state to a mixed state: the rank of the
statistical operator $\rho$ is invariant under unitary transformations
$\rho\rightarrow U\rho U^{-1}$. Von Neumann, indeed, asserts that there are
two kinds of process in the physical world: Process 1, which takes place when
a measurement is made, and Process 2, governed by the time-dependent
Schrödinger equation, which occurs when the system is not disturbed by
measurement. Yet a measurement is a normal process involving physical objects;
why should it be governed by different laws from the rest of the physical
world? This unsatisfactory situation is the _measurement problem_ of
nonrelativistic quantum mechanics. How is it to be resolved?

Dirac does not discuss this problem in any edition of his book. John
Polkinghorne, in a reminiscence of Dirac, comments “Physically, Dirac was
astonishingly uninterested in the great unresolved interpretative issue of the
act of measurement in quantum theory: the collapse of the wave packet is
simply attributed to the unanalysed concept of ‘disturbance”’[10]. In teaching
quantum mechanics, Dirac maintained this attitude; if a student raised the
question — and we did — he would just brush it aside. This attitude remains
common; see, for example, Asher Peres’s witty refusal to consider the problem
([9] p. 442).

Von Neumann, on the other hand, took this question seriously. In the final
section of his book he constructs a model of the measuring process in which
the measuring apparatus is taken to be a quantum system. This leads to a chain
of systems that must be considered: the measuring apparatus, for example the
mercury column in a thermometer – light quanta reflected off the mercury
column – the retina of an observer’s eye, receiving the light – the optic
nerve of the observer – their brain. At each stage the quantum state of the
whole composite system remains in a pure state, and the projection cannot be
understood. Finally, von Neumann considers the consciousness of the observer,
correlated with their brain by a principle of psycho-physical parallelism, and
finds himself forced to the conclusion that it is the non-physical phenomenon
of consciousness that is responsible for the collapse.

However, the contradiction between unitary evolution and the development of a
mixed state does not arise if one considers only a subsystem of a whole system
which is evolving unitarily. Evolution of the state of a subsystem from a pure
state to a mixed state is compatible with unitary evolution of the combined
system if the unitary operator entangles the states of the subsystem and the
rest of the whole system. Let $\mathcal{S}$ be the state space of the system
$S$ which is undergoing measurement, $\mathcal{R}$ the state space of the
measuring apparatus and the rest of the universe. If $|\Psi\rangle$ is a pure
state of the whole system, an element of $\mathcal{S}\otimes\mathcal{R}$, then
the state of $S$ after the measurement is the mixed state given by the partial
trace $\operatorname{tr}_{\mathcal{R}}|\Psi\rangle\langle\Psi|$, which need
not have rank $1$. To pursue von Neumann’s analysis into a chain of observing
systems, we can take $\mathcal{S}$ to be the whole of this chain up to the
brain of the observer, and $\mathcal{B}$ to be the brain of the observer. If
one believes in psycho-physical _identity_ rather than psycho-physical
parallelism, there is no need to follow von Neumann into a non-physical realm
of consciousness; we have already reached consciousness. Presumably there are
operators representing the contents of consciousness, and a basis of
eigenstates of the brain representing possible states of consciousness,
including possible observations of the measurement. If one follows the
understanding of the quantum state due to Schrödinger, Everett and Wheeler
([11, 6, 20]), the fact that we always see a single result of a measurement is
then a consequence of the fact that each of us can only be aware of — indeed,
can only exist in — one of these eigenstates of consciousness; which one is
seen is governed by the probabilities in the relative state, which must be
seen as relative to the observer.

Thus there are three possible answers to the question “What is the status of
the projection postulate?”:

A1 The projection postulate describes a real physical process, peculiar to the
context of measurement, in which the Schrödinger equation does not apply (von
Neumann’s Process 1). Instead, a new non-unitary evolution is supposed to
occur.

This raises the measurement problem in its most acute form. It has been
strongly argued by John Bell [2] that this cannot be accepted: the concept of
“measurement” is not defined in fundamental physical terms, and cannot have a
place in the fundamental laws of physics.

A2 The projection postulate, in the non-selective form (2.2), is a summary of
the results of normal unitary evolution, applied to a subsystem of the full
system in which the Schrödinger equation applies, in a situation in which this
unitary evolution converts a separable pure state of the subsystem and the
measuring apparatus into an entangled state.

The question of whether and how real measurements do actually conform to a
projection postulate in this way has been investigated by many authors, for
example Beige and Hegerfeldt [1].

This answers Bell’s criticism, but it evades the question which is raised by
the form P1 of the projection postulate: it does not address the clear fact
that the result of a measurement, in the discrete case, is a single eigenvalue
of the measured observable, not a mixture of states with different
eigenvalues. There is a third interpretation which meets this objection.

A3 The projection postulate refers to the experience of an observer who is a
physical subsystem of the world, and whose experience is described by a one of
a set of eigenvectors in the state space of that subsystem. The changing
experience of the observer is governed stochastically by the state vector of
the universe according to a generalisation of the guidance equation of Bohm
and Bell [14]. If the observer has a memory, the Bell-Bohm evolution results
in the memory of an apparent history with repeated projections according to
Dirac’s postulate, as is demonstrated in a simple model in [14]. This restores
the indeterminism of von Neumann’s Process 1, but removes its dependence on
the context of measurement.

It is a matter of perspective whether the universal state vector, or the
observer’s experience eigenstate, is regarded as a description of “reality”
[13].

Being non-relativistic, the theory considered here, however interpreted,
cannot be regarded as a possibly absolutely true theory of physical reality.
For theories of measurements in relativistic quantum theory, see [5]. A
difficulty arises in comparing these with the nonrelativistic projection
postulate, since quantum field theory is most naturally considered in the
Heisenberg picture, while the nonrelativistic projection postulate, at least
in the context of von Neumann’s Process 1, is most naturally considered in the
Schrödinger picture and a Heisenberg picture may not be possible. The relation
between the relativistic and non-relativistic theories of measurement calls
for a separate discussion.

##  7 Conclusion

The answer to the question in the title is “Dirac’s”. However, the idea, often
associated with the projection postulate, that it describes an unpredictable
change in physical reality which happens only when a measurement is performed,
interrupting the deterministic evolution described by the Schrödinger
equation, is due to von Neumann. In this paper we have briefly indicated two
ways in which the projection postulate can take a place in an understanding of
quantum theory which the state vector is taken to be an objective description
of physical reality, evolving according to the time-dependent Schrödinger
equation without collapsing.

## Acknowledgements

I am grateful to Fred Muller for making von Neumann’s 1927 papers available to
me, to Chris Fewster and Eli Hawkins for reading and making valuable comments
on earlier drafts of this paper, and to all three of them for helpful
discussions.

## References

  * [1] A. Beige and G. C. Hegerfeldt.  Projection postulate and atomic quantum Zeno effect.  Phys. Rev. A, 53:53, 1996.  arXiv:quant-ph/9512012.
  * [2] J. S. Bell.  Against “measurement”.  Physics World, 3(8):33–40, August 1990.
  * [3] Paul Busch, Marian Grabowski, and Pekka J. Lahti.  Operational quantum physics.  Springer, 1997.
  * [4] P. A. M. Dirac.  The Principles of Quantum Mechanics.  Oxford University Press, 1930.
  * [5] Christopher J. Fewster and Rainer Verch.  Measurement in quantum field theory.  Encyclopedia of Mathematical Physics, to appear.  arXiv:2304.13356.
  * [6] Hugh Everett III.  “Relative state” formulation of quantum mechanics.  Rev. Mod. Phys., 29:141–153, 1957.
  * [7] C. J. Isham.  Lectures on Quantum Theory.  Imperial College Press, 1995.
  * [8] Wolfgang Pauli.  General Principles of Quantum Mechanics.  Springer-Verlag, 1980.
  * [9] Asher Peres.  Quantum Theory: Concepts and Methods.  Kluwer, 1993.
  * [10] J. C. Polkinghorne.  Dirac and the interpretation of quantum mechanics.  In Behram N. Kursunoglu and Eugene P. Wigner, editors, Paul Adrien Maurice Dirac: reminiscences about a great physicist, pages 228–230. Cambridge University Press, 1987.
  * [11] Erwin Schrödinger.  July 1952 colloquium.  In M. Bitbol, editor, The Interpretation of Quantum Mechanics: Dublin Seminars (1949-1955) and other unpublished essays, pages 19–37. Ox Bow Press, 1996.
  * [12] Anthony Sudbery.  Quantum Mechanics and the Particles of Nature.  Cambridge University Press, 1986.
  * [13] Anthony Sudbery.  Einstein and Tagore, Newton and Blake, Everett and Bohr: the dual nature of reality.  In P. Ghose, editor, The nature of reality: the perennial debate. Routledge, 2016.  arXiv:1205.1479.
  * [14] Anthony Sudbery.  Histories without collapse.  Int. J. Theor. Phys., 61:39, 2022.  arXiv:2012.13430.
  * [15] Tony Sudbery.  Continuous state reduction.  In R. Penrose and C. J. Isham, editors, Quantum concepts in space and time, pages 65–83. Oxford University Press, 1986.
  * [16] J. von Neumann.  Mathematische Begründung der Quantenmechanik.  Nachrichten von der Gesellschaft der Wissenschaften zu Göttingen, pages 1–57, 1927.
  * [17] J. von Neumann.  Thermodynamik quantenmechanischer Gesamtheiten.  Nachrichten von der Gesellschaft der Wissenschaften zu Göttingen, pages 273–291, 1927.
  * [18] J. von Neumann.  Wahrscheinlichkeitstheoretischer Aufbau der Quantenmechanik.  Nachrichten von der Gesellschaft der Wissenschaften zu Göttingen, pages 245–272, 1927.
  * [19] J. von Neumann.  Mathematical Foundations of Quantum Mechanics.  Princeton University Press, 1955.  translated by Robert T. Beyer. Originally published 1932.
  * [20] J. A. Wheeler.  Assessment of Everett’s “relative state” formulation of quantum theory.  Rev. Mod. Phys., 29:463–465, 1957.


# Arena-Bench: A Benchmarking Suite for Obstacle Avoidance Approaches in
Highly Dynamic Environments

Linh Kästner1, Teham Bhuiyan1, Tuan Anh Le1, Elias Treis1, Johannes Cox1,
Boris Meinardus1,
Jacek Kmiecik1, Reyk Carstens1, Duc Pichel1, Bassel Fatloun1, Niloufar
Khorsandi1 and Jens Lambrecht1 Manuscript received: February, 24, 2022;
Revised May, 14, 2022; Accepted June, 12, 2022.This letter was recommended for
publication by Editor Tamim Asfour upon evaluation of the Associate Editor and
Reviewers’ comments.1The authors are with the Chair Industry Grade Networks
and Clouds, Faculty of Electrical Engineering, and Computer Science, Berlin
Institute of Technology, Berlin, Germany<EMAIL_ADDRESS>Object
Identifier (DOI): see top of this page.

###### Abstract

The ability to autonomously navigate safely, especially within dynamic
environments, is paramount for mobile robotics. In recent years, DRL
approaches have shown superior performance in dynamic obstacle avoidance.
However, these learning-based approaches are often developed in specially
designed simulation environments and are hard to test against conventional
planning approaches. Furthermore, the integration and deployment of these
approaches into real robotic platforms are not yet completely solved. In this
paper, we present Arena-bench, a benchmark suite to train, test, and evaluate
navigation planners on different robotic platforms within 3D environments. It
provides tools to design and generate highly dynamic evaluation worlds,
scenarios, and tasks for autonomous navigation and is fully integrated into
the robot operating system. To demonstrate the functionalities of our suite,
we trained a DRL agent on our platform and compared it against a variety of
existing different model-based and learning-based navigation approaches on a
variety of relevant metrics. Finally, we deployed the approaches towards real
robots and demonstrated the reproducibility of the results. The code is
publicly available at github.com/ignc-research/arena-bench.

###### Index Terms:

Software Tools for Benchmarking and Reproducibility; Motion and Path Planning;
Collision Avoidance; Reinforcement Learning

##  I Introduction

Mobile robots are increasingly being employed for various use cases such as
last-mile delivery, healthcare services, or operation in hazardous
environments [1]. Safe and reliable navigation in these highly complex and
dynamic environments is essential for the operation of mobile robotics. In
recent years, Deep Reinforcement Learning (DRL) has accomplished remarkable
results for dynamic obstacle avoidance due to its ability to swiftly react to
unexpected changes [2],[3],[4]. A common barrier is that most of the research
work evaluated their approaches on specifically designed simulation
environments or test setups, making a general comparison against existing
approaches difficult [5]. Furthermore, deployment and integration of DRL into
real robotic platforms is still an open frontier due to safety reasons [6],
[7]. Thus, a benchmark to properly assess those approaches in realistic
scenarios and against existing algorithms is not only an essential step
towards the deployment of DRL into real robots but also assists in the
development and validation of learning-based approaches on mobile robots.
Existing benchmarks for robot navigation algorithms mostly focus on static
environments, but few exist that cover both dynamic and static ones. Moreover,
existing benchmarks for navigation in dynamic environments often contain and
compare only a small number of planners [8], [9], [10]. On that account, we
propose Arena-bench, a benchmark suite consisting of tools to train, test, and
evaluate navigation algorithms for dynamic obstacle avoidance on different
robotic systems. This benchmark provides an intuitive interface to design and
create dynamic scenarios within 2D and 3D simulators based on Flatland and
Gazebo, respectively. To realistically simulate the environments, we
integrated an extended version of Pedsim, which utilizes the social force
model of Helbing et al. [11]. The benchmark is completely integrated into the
robot operating system (ROS) and includes a variety of classic and state-of-
the-art learning-based planners. Arena-bench further provides tools to
evaluate all planners on a variety of navigational metrics ranging from
navigational safety and robustness to path quality and efficiency. The
platform also provides a set of different robots with various robot kinematics
to test the approaches.

Figure 1: Arena-bench is a benchmark suite that enables to train and evaluate
navigation approaches in realistic dynamic environments. It provides tools to
develop navigation approaches, design and generate scenarios, and evaluation
tasks on a variety of robotic platforms. The results can be plotted on a
variety of different navigational metrics.

The main contributions of this work are the following:

  * •

Proposal of a benchmark suite to develop navigation approaches and design and
generate realistic, dynamic scenarios. The Pedsim library is utilized to
realistically model dynamic obstacles.

  * •

Integration of a complete training pipeline to train DRL agents on different
robots. The user can integrate new robots and train DRL algorithms on an
efficient 2D simulator or on a more realistic 3D simulator.

  * •

Extensive evaluation of several planners developed using this work and
provision of tools to evaluate the results on a variety of relevant
navigational metrics.

##  II Related works

The fundamentals of this work are based on arena-rosnav proposed by Kästner et
al. [12], which is a 2D simulator platform for testing, training, and
benchmarking robotic navigation approaches in highly dynamic 2D environments.
The results were previously presented in [12], where model-based, as well as
learning-based methods, namely TEB [13], DWA [14], MPC [15], CADRL [16], and
their DRL agent were evaluated and compared based on different performance
metrics.
A similar benchmark suite named Bench-mr was presented in a paper by Heiden et
al. [17]. It provides the utility to test, evaluate, and compare different
motion-planning techniques in complex environment scenarios. Although the
focus was placed on wheeled mobile robots, all notions of dynamic obstacles
were disregarded. Thus, only static situations were considered. Other
benchmarks that focus on static environments are presented in the works of
Althoff et al. [18] or Moll et al. [19]. Nonetheless, path planning is a vital
aspect of autonomous navigation, not only in static but also in dynamic
environments.
Rano et al. [10] proposed a methodology used to evaluate the performance of
different approaches, given that specific scenarios can vary from each other.
Consequently, this methodology attempts to quantify several aspects of an
actual environment, such as the amount of space occupied by obstacles and the
space available for the robot to maneuver between different static
obstructions. Based on these metrics, scenarios were classified and evaluated
legitimately compared to previous methodologies.
Robobench, introduced by Weisz et al. [9] is another venture of a similar
notion, trying to create uniform grounds for benchmarking in the field of
robotics. They present a platform suitable for an array of tasks, such as
navigation, manipulation, and observation. Different types of robots can be
evaluated using an appropriate simulation engine and planning framework. The
authors went a step beyond by taking the idea of virtualization and enabling
users to save their simulations in a runnable state. Such containers can then
be easily shared between users, making it easier for others to replicate
results. Albeit functioning, with exemplary missions present to test out, the
project did not grow much beyond its proof of concept.
A different aspect of robotic navigation, namely Social Robot Navigation, was
explored by Holtz et al. [20]. The researchers developed a framework for
benchmarking navigation approaches in dynamic human environments. This
framework contributes an easily extensible 2D robot and human behavior
simulator, an integration with OpenAI Gym interface [21] for learning
purposes, as well as an example benchmark scenario. Several learning policies
were trained and evaluated in a novel environment. However, the authors only
provided a comparison of learning-based approaches.

Figure 2: Our proposed benchmark suite consists of multiple modules for
designing and generating different scenarios and evaluating different
navigation approaches on specifically designed tasks. The suite includes a
variety of different robot kinematics, including car-like, holonomic, and
differential drive. The benchmark suite consists of a 2D and 3D simulation
engine based on Flatland and Gazebo to conduct the evaluation runs. Finally,
the evaluation class provides a variety of different metrics to visualize the
test runs.

Portugal et al. [22] introduced _patrolling_sim_ , a ROS and Stage-based [23]
framework for simulation, benchmarking of multi-robot patrolling algorithms.
The framework was packaged into an intuitive GUI, where the configuration of
the simulation can be tuned to the user’s needs. Additional tools were set up
to integrate new algorithms, benchmark approaches, and compare the results. In
particular, a node was established, which monitors the exchange of information
in the system. Subsequently, the gathered data can be used to evaluate in
terms of several performance metrics.
In a recent paper by Wen et al., [8], a similar ROS and Gazebo-based
benchmarking platform under the name MRPB 1.0 was proposed. It allows
researchers to compare their way of local planning against several existing
state-of-the-art approaches. For the sake of this goal, diverse, dynamic, and
complex scenarios were created. Dynamic obstacles were taken care of using
Gazebo’s actor principle by describing the desired looped trajectory. This
way, several realistic, dynamic environments, e.g. an office and a shopping
mall, were developed. Finally, two well-known local planners, DWA [14], and
TEB [13] were evaluated on performance metrics such as smoothness, efficiency,
and safety.
Tsoi et al. [24] presented a simulation framework for social navigation is
suggested by combining the features of ROS with state-of-the-art graphics and
physics simulation of Unity. To facilitate the social navigation aspect of the
platform, human models were introduced, with starting and goal position and
behavior predicted according to the crowd flow presented in [25]. Two
different environments were supplied, a replica of an actual lab and a larger
city scene. The evaluation toolkit allows for a systematical evaluation of
navigation approaches by taking care of creating reproducible tasks and
environments while collecting several performance metrics.
All aforementioned benchmarks either focus on comparing classic model-based
[8] or on learning-based approaches [20]. On the other hand, our benchmark is
not limited to one paradigm and aims to provide a cross-discipline benchmark,
enabling us to develop and evaluate both classic and learning-based navigation
approaches directly on different kinds of mobile robots. We emphasize the
development and deployment of DRL-based planners and provide a training
pipeline to train these approaches. Moreover, arena-benchmark provides an
extensive set of tools to create and design unique evaluation tasks and
scenarios.

##  III Methodology

Our proposed benchmark consists of multiple modules that enable the user to
design and generate evaluation scenarios, train DRL-based navigation
algorithms, integrate ROS-based planners, and plot the results a variety of
different navigational metrics.

###  III-A System Design

The complete system design is illustrated in Figure 2. The first module in the
benchmark suite is called arena-tools, which includes an extensive toolset to
design and generate maps, scenarios, and tasks. The specific components are
described in the next section. The user can choose between a manual task
generation mode consisting of generating their own map, specific scenarios,
and an automatic generation of random tasks. Using these tasks, the navigation
approaches can be tested and benchmarked using the simulator module. Our
framework is built on top of the 2D simulator Flatland and the 3D simulator
Gazebo. Moreover, we provide the possibility to train DRL agents in both
simulation environments. The resulting planners are cross-compatible across
both simulators and utilize ROS as the middleware. Finally, Arena-bench
provides an automatic evaluation class to qualitatively and quantitatively
evaluate all planning algorithms on up to 14 different navigation metrics
ranging from navigational safety to trajectory quality and efficiency.

###  III-B Dynamic Obstacles with Pedsim

The dynamic obstacles are spawned and controlled using this Pedsim library
which, includes social states and calculates trajectories [11]. We integrated
Pedsim into our 2D and 3D simulator for more realistic behavior and extended
it with more behavior patterns and social states such as running, group
talking, group listening, or waiting. Furthermore, we added a new service for
changing the obstacles of the scene randomly and made use of the ability to
spawn and re-spawn human agents during simulation. An exemplary scenario with
multiple pedestrians is depicted in Fig. 1.

###  III-C Map Generator

Figure 3: Arena-tools. We provide a variety of tools within our benchmark to
generate or load existing maps, scenarios, and tasks. The detailed
documentation of arena-tools can be found in the GitHub documentation. With
the map editor, the user has the choice of loading an existing map or
generating random maps with increasing difficulty. The user can choose between
indoor and outdoor maps. The generated ROS maps can be transformed into a 3D
Gazebo world. The scenario editor lets the user construct and design specific
scenarios with specific start and endpoints as well as define the obstacle
behavior. This includes their movements patterns, social states, and
velocities. Using the task editor, the user can select parameters like
planning algorithm, robot model, and task mode.

Part of arena-tools is the map editor, which generates 2D and 3D worlds. The
user can either manually create a scenario with dynamic and static obstacles
or select parameters for automatic scenario generation. Both can be done with
our arena-tools application, which provides a variety of functionalities for
scenario generation. Thus, even specific scenarios from the end-user can be
constructed, which is crucial for real-world validations. Exemplary maps are
illustrated in Fig. 4. We integrated the random map generation approach of
Heiden et al. [17] and extended it for generating 3D gazebo environments.
Moreover, we included the option to create increasing levels of difficulty by
generating increasing amount and size of obstacles for outdoor maps and
decreasing corridor size for indoor maps. The user has the option to randomly
set obstacle positions and corridors after each episode (random mode) or use a
specific curriculum (staged mode).

###  III-D Task Generator

Once scenarios are generated, tasks within these scenarios can be defined
using the task editor. Thereby, the user defines configuration files, which
indicate the task mode, the number of runs, used planners, and the robot
model. Fig. 3 illustrates the editor with all parameters to set. In total, the
task generator has three modes: Random, Scenario, and Staged, which will be
described in the following sections.
The Random Mode creates a random scenario for the used Gazebo world in each
run. Different arbitrary locations are used for spawning the robot, as well as
setting its desired goal position. We ensure to exclude setting these
positions to the same location. The number of dynamic obstacles is specified
before starting the simulation and fixed for the whole duration. The actors’
starting and goal position is randomly generated based on the provided map.
This information is then sent to the Pedsim simulator when creating the
desired Pedsim obstacles. The randomness of this mode makes it the preferred
choice for quantitatively benchmarking different navigation approaches
assuming that a large number of random scenarios are created and evaluated to
ensure statistically significant evaluations.
The Scenario Mode is a reasonable choice for both qualitative and quantitative
evaluations. The idea is to limit the randomness without sacrificing
complexity to deliver reproducible scenarios, which can be tested and
evaluated using separate planning methods. Consequently, each run of this task
mode should provide the same environment except for the dynamic behavior of
the robot and human agents, which can not be accounted for. We achieve this
through arena-tools [26], where a scenario editor was developed. Existing maps
of the environment can be used to then specify the robot’s spawning location
and goal position. Furthermore, we can add an arbitrary number of dynamic
obstacles, describe their start/goal spot, as well as their waypoints, which
should be included in their overall trajectory. The scenario editor generates
a configuration file describing the specifics of a scenario. The data is read
by the task_manager, converted into a compatible format for our platform, and
subsequently used in the simulation.
Staged Mode The staged mode is the preferred choice to automate a long list of
evaluation scenarios. Here, the user specifies several levels with increasing
difficulty and can set the threshold of when to reach this level. Thus, an
evaluation curriculum is generated, which makes it an appropriate choice,
especially for quantitative evaluations of planners. The stages can also be
designed using arena-tools and are encoded into a configuration file.

###  III-E Navigation Stack and Planning Approaches

Our system is fully integrated into ROS, which enables the usage of existing
ROS functionalities. In our evaluation, a classic A-star approach by Hart et
al. [27] was utilized as a global planner. The local planning approaches are
distinguished between model-based and learning-based approaches. While the
model-based approaches work directly with the ROS move base interface, the
learning-based approaches are trained using OpenAI Gym and require an
additional entity to down-sample the global path due to the myopic nature of
DRL approaches [5]. Therefore, we utilize an intermediate waypoint generator
of our previous work [28] to mitigate the myopic nature of DRL agents by
providing a shorter goal horizon and avoiding local minima. The training
pipeline and system design with the intermediate planner as an interconnection
entity was presented in our previous work. The complete integration of classic
move base planners and learning-based planners into ROS, enables them to be
deployed and compared side-by-side, which makes direct comparison feasible.
Currently, three classic planning approaches MPC [15], TEB [13], DWA [14], and
four learning-based approaches, NAVREP [5], Gring [29], as well as our trained
approach called ROSNAV are integrated. The ROSNAV planner was trained using
the 2D simulator of our arena-bench suite for efficient training. It takes the
structure and reward system of the agent presented by our previous work [12]
and was retrained with an adjusted action and observation space to match the
specific robots. We utilize the following continuous action space $A$ for the
different robots: Turtlebot3 (TB3) - $v^{TB3}$, Jackal (JKL) - $v^{JKL}$, and
Robotino (RTO) - $v^{RTO}$:

$\displaystyle a=\\{v_{lin}$ $\displaystyle,v_{ang}\\}$ (1) $\displaystyle
v_{lin}^{TB3}\in[0,0.22]m/s,\quad$ $\displaystyle
v_{ang}^{TB3}\in[-2.7,2.7]m/s$ (2) $\displaystyle
v_{lin}^{JKL}\in[-2.0,2.0]m/s,\quad$ $\displaystyle
v_{ang}^{JKL}\in[-4.0,4.0]m/s$ (3) $\displaystyle
v_{lin,x,y}^{RTO}\in[-2.78,2.78]m/s,\quad$ $\displaystyle
v_{ang}^{RTO}\in[-1.0,1.0]m/s$ (4)

A complete list of robot parameters can be found on the arena-bench GitHub
wiki page. Furthermore, we included our All-in-One planner (AIO) presented in
our previous work [30], which is a trained DRL-based control switch, which can
choose between different planners based on specific situations.

Figure 4: Experiments in simulation. Quantitative results of all planners on
three robots over three worlds within the 3D simulation over the number of
obstacles. The small warehouse world was integrated from a third-party
package, and the outdoor and indoor environments generated using arena-tools.
Collision rates, path length, and movement jerks are plotted to represent
navigational safety, efficiency, and smoothness, respectively.

##  IV Evaluations

To demonstrate the functionalities of our benchmark, we conducted experiments
on several different maps, which were either imported from existing open
source projects like the AWS-robomaker project or created using the arena-
tools functionalities. For our evaluations, we included the small warehouse
world from AWS-robomaker (20x20m) and an indoor and outdoor world created with
our randomized world generator (15x15m). Each world contains two scenarios
with 5 and 10 pedestrians with an average speed of 0.3m/s. In each scenario,
the obstacle velocities are set to 0.3 m/s. As a global planner, A-Star is
used for all approaches. For the DRL-based planners, we utilized the Spatial
Horizon waypoint generator of Kästner et al. [28] using the time and location
horizon of $t_{lim}=4s$ and $d_{ahead}=2m$. For the model-based approaches, we
utilize the ROS move-base interface. Localization is acquired using the
Adaptive Monte Carlo (AMCL) module. For each planner, we conduct 15 test runs
on each scenario. During the test runs, our platform will record the necessary
data automatically and provide it to our evaluation class, which is able to
generate qualitative and quantitative plots on a variety of metrics listed in
Table I. It includes metrics to evaluate navigational efficiency, robustness,
safety, and smoothness. Furthermore, some metrics are subdivided in sub-
metrics such as the average, minimum, maximum and normalized values, which
could give additional insight.

TABLE I: Evaluation metrics

Metric | Unit | Explanation
---|---|---
Success Rate2 | % | Runs with ¡ 2 collisions
Collision1,2 | - | Total number of collisions
Time to reach goal2 | [$s$] | Time required to reach the goal
Path Length1,2 | [$m$] | Path length in m
Velocity (avg.)2 | [$\frac{m}{s}$] | Velocity of the robot
Acceleration (avg.)2 | [$\frac{m}{s^{2}}$] | Acceleration of the robot
Movement Jerk2 | [$\frac{m}{s^{3}}$] | Derivation of Acceleration
Curvature(avg.,max.,min.,norm.)2 | [$m$] | Degree of trajectory changes
Angle over length2 | $[\frac{rad}{m}]$ | Curvature overthe path-length
Roughness2 | - | Quantifies trajectory smoothness
Clearing Dist.(avg.,max.,min.,norm.)2 | $[m]$ | Distance kept to obstacles

1Quantitative metric, 2Qualitative metric

Figure 5: Experiments on the real robot. Upper row: Robotics laboratory where
the real experiments were conducted and generated 3D world. The goals are
illustrated as colored dots. The two different start positions are indicted by
stars of different colors (yellow and orange). Lower row: qualitative plots of
the real test runs. Goals are illustrated as colored circles to better
visualize the fluctuation of the goal position due to non-optimal localization
on the real robot. The runs on the real robot are bold, whereas simulation
runs are dashed. The obstacle trajectories are illustrated in black. Out of
the 15 different runs conducted, one exemplary trajectory is selected.

###  IV-A Simulation Results

To represent navigational safety, efficiency, and smoothness, the aggregated
collision rates, path lengths in meters, and movement jerks of all planners
for three different robot platforms on each map are depicted in Fig. 4
respectively. Thereby, a collision is counted when the laser scan detects a
value smaller than the robot radius. The movement jerk expresses the rate at
which the robot changes its acceleration with respect to time. To represent
all common robotic kinematic platforms, we deployed all planners except for
the GRING planner on the Jackal (Ackerman Drive), the Turtlebot3 (TB3)
(Differential Drive), and the Robotino (RTO) (Holonomic) robot. The GRING
planner was only deployed on the Jackal and Turtlebot3 because it was only
trained for wheeled robots, and deployment on the Robotino resulted in flawed
behavior.
Navigational Safety: It is observed that the AIO planner excels in terms of
navigational safety, accomplishing the lowest collision rates in all scenarios
with 5 and 10 obstacles. Our ROSNAV planner also accomplishes competitive
results in terms of navigational safety with low collision rates. Only for the
Robotino, ROSNAV produced collisions in the indoor and outdoor map. On all
other robot platforms and maps, no collisions were produced. Based on the
presented results, the classic move-base planners TEB, DWA, and MPC planners
follow up in terms of navigational safety with competitive results over all
robots and maps. In general, they also produce low collisions and high success
rates over all scenarios and on all robots. However, only DWA on the Turlebot3
produces high collision rates. Aggravating factors are the slow maximum
velocity of the Turlebot3 and computationally more demanding calculation times
for DWA, which result in slower reaction times to incoming obstacles. The
performance of the other learning-based approaches GRING and NAVREP is
competitive in scenarios with 5 obstacles but drops significantly for
scenarios with 10 obstacles. In situations with a high amount of dynamic
obstacles, both planners are not able to react in time.
Navigational Efficiency: With regards to navigation efficiency, differences
between the robot platforms can be observed. The AIO planner produces high
path lengths for the Jackal and Turlebot3. However, for the Robotino, AIO is
significantly more efficient, accomplishing one of the lowest path lengths
compared to all planners. Whereas ROSNAV results are mediocre for the Jackal
and Turlebot3, it outperforms all other planners on the Robotino, similar to
our AIO planner. This indicates that our ROSNAV planner works best on
holonomic platforms due to the more flexible set of movements the robot can
exercise. The move-base planners TEB and DWA produce similar results with
competitive results over all robots, maps, and scenarios. However, the MPC
planner produces competitive results only on the Jackal and Turlebot3 but
performs worst on the holonomic Robotino. Since MPC was specifically designed
for car-like robots, these results are expected. GRING and NAVREP perform
worst with regards to efficiency producing high path lengths in most scenarios
and all robots.

Navigational Smoothness: In terms of navigational smoothness, ROSNAV produces
mediocre results with higher movement jerk values compared to the classic
model-based planners DWA, MPC, and TEB, which all produce competitive results.
Especially the MPC planner accomplishes the lowest movement jerk values on all
robots and scenarios. Whereas on the Jackal and Robotino robots, the movement
jerk of MPC is close to zero, the jerk is higher on the Turlebot3. Similar
observations can be made for most of the other planners, which produce higher
jerks for the Turlebot3. This is due to the low maximum velocity of the robot,
which results in abrupt velocity changes that cause higher values. The
findings also show that the movement jerk is directly correlated with the
maximum velocity of the robot. Thus, the Robotino produces a significantly
higher jerk compared to the Turlebot3 robot. Nevertheless, the MPC planner is
still able to accomplish a very low movement jerk on the Robotino and
outperforms all other planners in terms of navigational smoothness.

Figure 6: Quantitative plots for the real world tests. (Upper row):
collisions, success rate, time traveled and path length indicate navigational
efficiency and robustness, (Lower row): Angel over lengths, clearing
distances, and movement jerks indicate navigational safety and smoothness. The
metrics are plotted over an increasing number of obstacles.

###  IV-B Test on the Real Robot

To validate our results and provide a proof of concept, we transferred the
algorithms towards real robots and conducted extensive experiments on the
Turtlebot3 within our robotics laboratory. In parallel, we used arena-tools to
generate a 3D world based on a 2D map of the laboratory (16x7m), which was
recorded using SLAM. Subsequently, we generated the same scenarios using our
scenario editor to ensure consistent evaluations across simulation and
reality. Finally, we conducted 15 runs per planner in simulation and reality
on each scenario. For the localization of the robot, we used the AMCL package
in simulation and reality. It is to be noted that the real-world experiments
are subject to inaccuracies due to inconsistent trajectories, velocities, and
behavior of the human pedestrians. To mitigate those disturbances, we included
trajectory marks and visual cues to mimic the scenarios as close as possible.
However, a bias can not be completely avoided within real-world experiments.
Qualitative Evaluations: The qualitative results for both simulation and real
runs are illustrated side-by-side in Fig. 5. It is observed that the
trajectories of the real robot on all runs are similar to the simulation and
that our simulation results can be reproduced within real environments. As
expected, it is observed that the movement jerk is higher on real robots
compared to the simulation, where the paths are more smooth. Especially for
TEB and DWA, this can be observed. For TEB the typically stop and backwards
movements once an obstacle is perceived is clearly visible e.g. in in Fig 5 a)
[x: 12, y:1], c) [x: 13, y:1.8], or d) [x: 15, y:1]. These backward movements
are more pronounced on the real robot compared to the simulation, with the
real robot driving a significantly longer path backward when perceiving an
incoming obstacle, which leads to longer overall paths. This effect is also
visible on the DWA planner, albeit not as distinct as on the TEB planner. Our
ROSNAV planner behaves similar to the simulation and reaches the goal without
collisions. Contrarily, our ROSNAV planner performs a stop and, if necessary
small backward movement or immediately turns away from incoming obstacles. It
can perceive incoming obstacles from approximately 1 m and react timely to
avoid them. The performance of all planners is visualized in action in our
supplementary video. Another difference between simulation and reality is the
non-optimal localization. Therefore, it is observed that the robot reaches the
goal at slightly different goal positions, whereas the simulation results can
keep a consistent goal position. However, this gap is only slight and can be
mitigated by using a more accurate localizer, which uses multiple sensor
information.
Quantitative Evaluations: Additional quantitative evaluations to demonstrate
these findings are illustrated in Fig. 6, where 8 different metrics are
plotted. In addition to the previously introduced metrics presented in the
simulation results, we plotted the mean and max clearing distance in meters to
assess navigational safety, the angle over length, and the success rate. The
plots validate the simulation-to-reality difference. This is most obvious with
the path length and time to reach the goal of the TEB planner. On the real
robot, the TEB planner often drives a long distance backward whenever an
obstacle is perceived, which is exercised in simulation but to a much smaller
extent. This increases the path length and time to reach the goal, which is
significantly higher compared to ROSNAV and DWA. Regarding collision and
success rates, our ROSNAV planner behaves similar to the simulation and can
reproduce the results almost perfectly. However, the ROSNAV planner produces
higher movement jerk values compared to the simulation, which was already
visible in the qualitative plots. Nevertheless, this was also observed for DWA
and TEB and expected due to fluctuating real-world factors like battery status
and motor strength. In terms of path lengths, time traveled, and clearing
distances in scenarios with 5 obstacles the simulation-to-reality gap is less
distinct, with only slight differences between simulation and reality, which
demonstrates the reproducibility of our experiments on real robots on these
scenarios. However, in scenarios with 10 obstacles, the planners on the real
robots required less time to reach the goal compared to the same scenarios in
simulation. This could be due to the more cautious human behavior as well as
non-consistent velocities of the humans within the real-world experiments.
This also explains the large discrepancy between simulation and reality
regarding the better performance of real robots in terms of collision and
success rates.
Discussion: The experiments run on the real robot provided additional insight
into the sim-2-real gap, which is valuable to validate reproducibility on real
robots, optimize planners, or address aspects of the simulation. Despite the
gap between simulation and reality on a number of metrics, the differences
were expected and due to known factors like non-perfect localization, sensor
noise, fluctuating real-world factors like battery and motor status, non-
optimal floor areas, or human behavior. However, the real-world evaluations
could still validate the comparability of all planners in terms of different
relevant metrics. Since the evaluations took place in Gazebo, which
realistically simulates robot kinematics as well as sensor data, transfer
towards real robotic platforms can be achieved without major issues assuming
the observation and action space is consistent. Using the platform to compare
different robotic systems and navigational metrics against each other provides
valuable information to be considered and assessed when choosing an
appropriate robot for specific scenarios. The findings of the quantitative
evaluations assisted in the assessment of each planner, scenario, and robot.
Whereas classic model-based planners like MPC, TEB, or DWA excel with regard
to navigational smoothness, they struggle in highly dynamic environments,
where learning-based approaches are a better choice. Furthermore, planners
like MPC accomplish better results on wheeled robots like the Jackal, while
our learning-based DRL agents can take advantage of the more flexible
holonomic robot platform. A downside of our approach is that it needs to be
trained on each robotic platform again. On the other hand, model-based
approaches like MPC and TEB require finetuning of hyperparameters for each
robot, which is tedious and not always trivial.

##  V Conclusion

In this paper, we proposed Arena-bench, a benchmarking suite to train, test,
and compare navigation approaches within ROS in highly dynamic simulation- and
real-world environments. Therefore, we introduced a set of tools to design and
generate worlds, scenarios, and tasks. Our platform enables the development
and training of DRL approaches as well as the integration of classic planners
for different robots. To demonstrate the functionalities, we included three
different robot types, trained DRL agents on them, and included three
conventional planners. Subsequently, we created a variety of test scenarios
and conducted extensive evaluations of all approaches with a wide range of
alternative navigation approaches. Finally, we transfer the approaches towards
real robots and conduct field experiments to assess the sim-to-real gap. The
platform provides an important tool not only to assist researchers in
developing and evaluating navigation approaches in dynamic environments but
also in deploying DRL-based planners towards real robots. In future works, we
aspire to do more extensive evaluations on real robotic systems and compare
the impact of different robot kinematics on the performance of the navigation
approaches. Furthermore, we aim to elucidate the impact of different
simulation environments for training and testing.

## References

  * [1] M. B. Alatise and G. P. Hancke, “A review on challenges of autonomous mobile robot and sensor fusion methods,” _IEEE Access_ , vol. 8, pp. 39 830–39 846, 2020.
  * [2] H.-T. L. Chiang, A. Faust, M. Fiser, and A. Francis, “Learning navigation behaviors end-to-end with autorl,” _IEEE Robotics and Automation Letters_ , vol. 4, no. 2, pp. 2007–2014, 2019.
  * [3] Y. F. Chen, M. Everett, M. Liu, and J. P. How, “Socially aware motion planning with deep reinforcement learning,” in _2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2017, pp. 1343–1350.
  * [4] A. Faust, K. Oslund, O. Ramirez, A. Francis, L. Tapia, M. Fiser, and J. Davidson, “Prm-rl: Long-range robotic navigation tasks by combining reinforcement learning and sampling-based planning,” in _2018 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2018, pp. 5113–5120.
  * [5] D. Dugas, J. Nieto, R. Siegwart, and J. J. Chung, “Navrep: Unsupervised representations for reinforcement learning of robot navigation in dynamic human environments,” in _2021 IEEE International Conference on Robotics and Automation (ICRA)_ , 2021, pp. 7829–7835.
  * [6] B. R. Kiran, I. Sobh, V. Talpaert, P. Mannion, A. A. Al Sallab, S. Yogamani, and P. Pérez, “Deep reinforcement learning for autonomous driving: A survey,” _IEEE Transactions on Intelligent Transportation Systems_ , 2021\.
  * [7] Z. Zhu and H. Zhao, “A survey of deep rl and il for autonomous driving policy learning,” _arXiv preprint arXiv:2101.01993_ , 2021.
  * [8] J. Wen, X. Zhang, Q. Bi, Z. Pan, Y. Feng, J. Yuan, and Y. Fang, “Mrpb 1.0: A unified benchmark for the evaluation of mobile robot local planning approaches,” in _2021 IEEE International Conference on Robotics and Automation (ICRA)_ , 2021, pp. 8238–8244.
  * [9] J. Weisz, Y. Huang, F. Lier, S. Sethumadhavan, and P. Allen, “Robobench: Towards sustainable robotics system benchmarking,” in _2016 IEEE International Conference on Robotics and Automation (ICRA)_ , 2016, pp. 3383–3389.
  * [10] I. Rañó and J. Minguez, “Steps towards the automatic evaluation of robot obstacle avoidance algorithms,” in _Proc. of workshop of benchmarking in robotics, in the IEEE/RSJ int. conf. on intelligent robots and systems (IROS)_ , vol. 88, 2006, pp. 90–91.
  * [11] D. Helbing and P. Molnar, “Social force model for pedestrian dynamics,” _Physical review E_ , vol. 51, no. 5, p. 4282, 1995.
  * [12] L. Kästner, T. Buiyan, L. Jiao, T. A. Le, X. Zhao, Z. Shen, and J. Lambrecht, “Arena-rosnav: Towards deployment of deep-reinforcement-learning-based obstacle avoidance into conventional autonomous navigation systems,” in _2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, pp. 6456–6463.
  * [13] C. Rösmann, F. Hoffmann, and T. Bertram, “Timed-elastic-bands for time-optimal point-to-point nonlinear model predictive control,” in _2015 European Control Conference (ECC)_ , 2015, pp. 3352–3357.
  * [14] D. Fox, W. Burgard, and S. Thrun, “The dynamic window approach to collision avoidance,” _Robotics & Automation Magazine, IEEE_, vol. 4, pp. 23 – 33, 04 1997.
  * [15] C. Rosmann, A. Makarow, F. Hoffmann, and T. Bertram, “Time-optimal nonlinear model predictive control with minimal control interventions,” in _2017 IEEE Conference on Control Technology and Applications (CCTA)_ , 2017, pp. 19–24.
  * [16] M. Everett, Y. F. Chen, and J. P. How, “Motion planning among dynamic, decision-making agents with deep reinforcement learning,” in _2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2018, pp. 3052–3059.
  * [17] E. Heiden, L. Palmieri, L. Bruns, K. O. Arras, G. S. Sukhatme, and S. Koenig, “Bench-mr: A motion planning benchmark for wheeled mobile robots,” _IEEE Robotics and Automation Letters_ , vol. 6, no. 3, pp. 4536–4543, 2021\.
  * [18] M. Althoff, M. Koschi, and S. Manzinger, “Commonroad: Composable benchmarks for motion planning on roads,” in _2017 IEEE Intelligent Vehicles Symposium (IV)_. IEEE, 2017, pp. 719–726.
  * [19] M. Moll, I. A. Sucan, and L. E. Kavraki, “Benchmarking motion planning algorithms: An extensible infrastructure for analysis and visualization,” _IEEE Robotics & Automation Magazine_, vol. 22, no. 3, pp. 96–102, 2015\.
  * [20] J. Holtz and J. Biswas, “Socialgym: A framework for benchmarking social robot navigation,” _arXiv preprint arXiv:2109.11011_ , 2021.
  * [21] G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” 2016.
  * [22] D. Portugal, L. Iocchi, and A. Farinelli, “A ros-based framework for simulation and benchmarking of multi-robot patrolling algorithms,” in _Robot Operating System (ROS)_. Springer, 2019, pp. 3–28.
  * [23] B. P. Gerkey, R. T. Vaughan, and A. Howard, “The player/stage project: Tools for multi-robot and distributed sensor systems,” in _In Proceedings of the 11th International Conference on Advanced Robotics_ , 2003, pp. 317–323.
  * [24] N. Tsoi, M. Hussein, J. Espinoza, X. Ruiz, and M. Vázquez, “Sean: Social environment for autonomous navigation,” in _Proceedings of the 8th International Conference on Human-Agent Interaction_ , 2020, pp. 281–283.
  * [25] S. S. Sohn, H. Zhou, S. Moon, S. Yoon, V. Pavlovic, and M. Kapadia, “Laying the foundations of deep long-term crowd flow prediction,” in _European Conference on Computer Vision_. Springer, 2020, pp. 711–728.
  * [26] (2021) Arena-tools. [Online]. Available: https://github.com/ignc-research/arena-tools
  * [27] P. E. Hart, N. J. Nilsson, and B. Raphael, “A formal basis for the heuristic determination of minimum cost paths,” _IEEE transactions on Systems Science and Cybernetics_ , vol. 4, no. 2, pp. 100–107, 1968.
  * [28] L. Kästner, X. Zhao, T. Buiyan, J. Li, Z. Shen, J. Lambrecht, and C. Marx, “Connecting deep-reinforcement-learning-based obstacle avoidance with conventional global planners using waypoint generators,” in _2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2021, pp. 1213–1220.
  * [29] R. Guldenring, M. Görner, N. Hendrich, N. J. Jacobsen, and J. Zhang, “Learning local planners for human-aware navigation in indoor environments,” in _2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2020, pp. 6053–6060.
  * [30] L. Kästner, J. Cox, T. Bhuiyan, and J. Lambrecht, “All-in-one: A drl-based control switch combining state-of-the-art navigation planners,” _arXiv preprint arXiv:2109.11636_ , 2021.