#  pystorms: A simulation sandbox for the development and evaluation of
stormwater control algorithms

Sara P. Rimer  0000-0002-3035-7698 Argonne National LaboratoryDecision and
Infrastructure Sciences Division9700 Cass AvenueLemontIL60439USA
<EMAIL_ADDRESS>,  Abhiram Mullapudi  0000-0001-8141-3621 University of
MichiganCivil and Environmental Engineering Department2350 Hayward AvenueAnn
ArborMI48105USA<EMAIL_ADDRESS>,  Sara C. Troutman
0000-0002-6809-7959 University of MichiganCivil and Environmental Engineering
Department2350 Hayward AvenueAnn ArborMI48105USA<EMAIL_ADDRESS>,
Gregory Ewing  0000-0002-0106-7712 University of IowaDepartment of Civil and
Environmental Engineering300 South Riverside DriveIowa CityIA52245USA gregory-
<EMAIL_ADDRESS>,  Benjamin D. Bowes  0000-0001-8349-4787 University of
VirginiaDepartment of Engineering Systems and Environment151 Engineer’s
WayCharlottesvilleVA22904USA<EMAIL_ADDRESS>,  Aaron A. Akin
0000-0001-7033-9307 University of TennesseeDepartment of Civil and
Environmental Engineering851 Neyland DriveKnoxvilleTN37996USA
<EMAIL_ADDRESS>,  Jeffrey Sadler  0000-0001-8776-4844 University of
VirginiaDepartment of Engineering Systems and Environment151 Engineer’s
WayCharlottesvilleVA22904USA<EMAIL_ADDRESS>,  Ruben Kertesz
0000-0001-7936-8235 Xylem, Inc.121 South Niles Avenue, 32South BendIN46617USA
<EMAIL_ADDRESS>,  Bryant McDonnell  0000-0002-6250-2220 Xylem,
Inc.121 South Niles Avenue, 32South BendIN46617USA<EMAIL_ADDRESS>,  Luis Montestruque  0000-0001-5494-585X Xylem, Inc.121 South Niles Avenue,
32South BendIN46617USA<EMAIL_ADDRESS>,  Jon Hathaway
0000-0003-1666-8550 University of TennesseeDepartment of Civil and
Environmental Engineering851 Neyland DriveKnoxvilleTN37996USA<EMAIL_ADDRESS>,  Jonathan L. Goodall  0000-0002-1112-4522 University of VirginiaDepartment
of Engineering Systems and Environment151 Engineer’s
WayCharlottesvilleVA22904USA<EMAIL_ADDRESS>and  Branko Kerkez
0000-0002-8041-5366 University of MichiganCivil and Environmental Engineering
Department2350 Hayward AvenueAnn ArborMI48105USA<EMAIL_ADDRESS>
###### Abstract.

Recent accessibility of affordable sensing technologies, microcontrollers, and
wireless communication technology has made it possible for stormwater systems
to be retrofitted with an assortment of sensors and actuators. These smart
stormwater systems have enabled the real-time sensing of their surrounding
environmental dynamics, and subsequently, provide the basis for autonomous and
adaptive operational control strategies. Additionally, these systems allow for
inexpensive and minimally-invasive stormwater control interventions (e.g.
hydraulic valve operated by cellularly-connected actuator) in lieu of new
construction. However promising this area of smart stormwater control, there
still remain barriers — for experts and novices alike — to access a set of
shared tools and methods for investigating, developing, and contributing to
it. In an effort to make smart stormwater control research more methodical,
efficient, and accessible, we present pystorms, a Python-based simulation
sandbox that facilitates the quantitative evaluation and comparison of control
strategies. The pystorms simulation sandbox comes with (i) a collection of
real world-inspired stormwater control scenarios on which any number of
control strategies can be applied and tested via (ii) an accompanying Python
programming interface coupled with a stormwater simulator. For the first time,
pystorms enables rigorous and efficient evaluation of smart stormwater control
methodologies across a diverse set of watersheds with only a few lines of
code. We present the details of pystorms here and demonstrate its usage by
applying and evaluating two stormwater control strategies.

Stormwater systems, Intelligent infrastructure, Adaptive control, Quantitative
evaluation, Simulation sandbox, Open-source software

††copyright: none

##  1\. Introduction

The advent of smart cities is poised to transform the management of our built
environment (Harrison and Donnelly, 2011; Batty et al., 2012; Chourabi et al.,
2012; Kitchin, 2014; Bibri and Krogstie, 2017; Eggimann et al., 2017).
Specific to stormwater, a new generation of smart and connected stormwater
systems aims to reduce flooding and improve water quality management by
autonomously sensing watershed parameters and controlling hydraulic components
across complete watersheds. These smart systems can provide an alternative to
costly concrete-and-steel construction by squeezing even more performance out
of existing stormwater infrastructure. Early ideas of controlling distributed
stormwater systems in real-time date back to the 1970s (Trotta et al., 1977).
The concept has, however, only recently gained widespread attention—in large
part due to the affordability of internet-connected sensors, the increased
capacity of data services, the broader emergence of other autonomous systems
such as self-driving cars and robots, and the increasing prevalence of climate
change stressors such as changing rainfall patterns and sea level rise.
Relative to other fields of autonomy, however, smart water systems are still
in the early stages of adoption. Thus, developing and implementing smart water
systems presents an exciting opportunity for researchers and practitioners to
propose new visions, standards, and technologies.

The intelligence of smart stormwater systems broadly refers to the acquisition
(i.e. “sensing”) and processing of data into decisions and actions (i.e.
“control strategies”) that are then used to guide the operation of gates,
valves, pumps, and other actuators within a watershed or drainage network.
Ultimately, the logic embedded via these control rules determines how water is
moved around the collection system to meet specific performance objectives,
such as the reduction of flooding or improvement of water quality. Developing
this control logic poses a major research frontier (Kerkez et al., 2016) and
will require the engagement of groups and individuals from a wide range of
experience and expertise.

Yet, entering this research field is presently precluded by a number of
practical barriers. For new groups to make early contributions, they must have
access to simulation testbeds, real-world inspired case studies, and
appropriate control objectives. Due to privacy and safety concerns, access to
network models and management details is difficult to come by without personal
connections to those who manage local watersheds or municipal water systems.
Thus, it can be difficult for new groups to obtain the necessary details of
how real-world stormwater systems actually operate—leaving them to evaluate
their control algorithms solely on idealized “toy problems.” When access to
such details is available, developing computational simulations that are true
to real-world systems and objectives requires significant effort and
expertise. Still, even when all of these other barriers are addressed and
promising control algorithms have been proposed, they have usually all been
evaluated on highly specific case studies and simulators, making it difficult
to evaluate the extent of their success when applied to additional networks.
In an effort to address these limitations, the contribution of this paper is
pystorms, an open-source Python package comprised of:

  1. (i)

A collection of real world-inspired smart stormwater control scenarios that
facilitate the quantitative evaluation and comparison of control strategies.

  2. (ii)

A programming interface and a stormwater simulator to provide a stand alone
package for developing stormwater control strategies.

Our aspiration is for pystorms to emerge as a community-driven resource that
fosters accessibility and collaboration amongst researchers and practitioners,
both novices and experts alike.

##  2\. Background

Figure 1. pystorms abstracts the control of stormwater systems as scenarios,
each characterized by an overlaying control system and an underlying
stormwater network. The overlaying control system (in orange) encapsulates
what can be considered the stormwater control system’s virtual elements — that
is, the components of the stormwater control system that are changeable and/or
readable by an implemented control algorithm — and include its Control
Objectives, Observable States, and Controllable Assets. The underlying
stormwater network (in blue) represents the computational implementation of
the hydraulic and hydrological elements of the stormwater control system,
namely its Network Topology and Event Driver. Table 1 presents a detailed
description of these elements.

###  2.1. Control of Stormwater Systems

A stormwater control problem can be defined as finding a strategy to
manipulate the flow of stormwater to achieve a desired water quantity or
quality objective. Traditionally, stormwater control has relied on _passive_
solutions, in which control is achieved via large-scale and expensive
construction. Instead, the emergence of microcontrollers, wireless
communication technologies, and low-cost sensors has allowed for _active_
solutions, in which existing infrastructure can be retrofitted with low-cost
wireless valves, pumps, and other actuators, installed at strategic locations
throughout a stormwater network, and utilized adaptively, in _real-time_.
Consequently, stormwater infrastructure can now be instantly redesigned to
respond to its dynamic environment.

While real-time stormwater control engineering solutions were documented at
least a decade earlier (Trotta et al., 1977), research-oriented discussion of
these implementations did not occur until 1989 (Schilling, 1989). Furthermore,
while limited implementation of smart stormwater control began at the end of
the 20th-century, the 21st-century has seen far more extensive and systematic
successes, as described in the foundational reviews of Schütze et al. (2004)
and Vanrolleghem et al. (2005). Some notable smart stormwater control
implementations include Ocampo-Martinez (2010); Gaborit et al. (2013); Vezzaro
and Grum (2014); García et al. (2015); Gaborit et al. (2016); Mullapudi et al.
(2017); Montestruque (2018); Sadler et al. (2019); Persaud et al. (2019);
Bowes et al. (2021). These references contain instances of smart stormwater
control from single control assets to watershed-scale implementations. For
more comprehensive reviews of stormwater control implementations, we direct
the reader to some recently published survey articles on the topic (Yuan et
al., 2019; Lund et al., 2018; Shishegar et al., 2018; van Daal et al., 2017;
García et al., 2015).

####  2.1.1. Simulating Stormwater Systems

Due to safety concerns and the variability of storm events, it is often
infeasible to test various control strategies on actual stormwater networks. A
more practical approach is to first estimate the outcomes of different control
decisions _in-silico_ , before deciding to port them to real systems. These
simulations can be carried out using a computational stormwater model. The
components of such a model usually include a (i) runoff module and (ii) a
routing module, and are driven by (iii) precipitation events (e.g. rain,
snow). The runoff module converts precipitation into overland runoff; the
overland runoff then undergoes hydrological processes (e.g. infiltration,
evaporation) and is hydraulically transported to the stormwater collection
system.

Over the years, several different software applications have been developed
for modeling and simulating stormwater networks. The different software
applications all function in a similar manner: they compute the dynamics of
stormwater as it moves through its local watershed. The US-EPA’s Stormwater
Management Model (SWMM) (Rossman, 2015), MIKE URBAN+ from the MIKE Powered by
DHI software suite111mikepoweredbydhi.com, and the Model for Urban Stormwater
Improvement Conceptualisation (MUSIC) by eWater222ewater.org.au are examples
of popular stormwater software applications. The theory and computational
details of these models are summarized in Rossman and Huber (2015), Rossman
(2017), and Rossman and Huber (2016). While these models provide powerful
simulation adroitness, they confine real-time control to limited rule-based
approaches, making them difficult to use in the study of smart stormwater
systems.

####  2.1.2. Implementing Adaptive Control

Here, we aim to maintain the idea of control in its broadest but most
straightforward meaning: after receiving some sort of _cue_ (from a sensor or
state estimate), an _action_ is taken (for instance opening or closing a
valve) with the aim of achieving a desired outcome (such as reducing flooding
or improving water quality). When we implement _control_ , we are deciding on
the course of action for optimizing our system to meet some specified
objective. A stormwater control strategy seeks to formalize this process of
deciding a set of actions. We define the computational process of implementing
this strategy as the _stormwater control algorithm_.

Suppose a stormwater system with only one valve was installed, and that valve
can either be completely opened or closed every hour. Deciding on a pattern
for the complete opening and closing of the valve, over the period of a few
hours to even a few days, presents a multitude of permutations and is a non-
trivial undertaking. By expanding the task to allow for the valve to opened at
any number of percent increments between 0–100%, the combination of actions
that can be implemented becomes even more expansive, making simple rule-based
or “if-else” algorithms either ineffective or bafflingly large. This
complexity is particularly true for more sizable drainage systems, where tens
to hundreds of valves need to be operated.

Hence, finding the “best” control actions is difficult and—by
nature—subjective. Furthermore, most algorithms are often only evaluated on a
single case study, and only in comparison to an uncontrolled solution. While
valuable, this has the effect of making most algorithms appear very effective,
since the only baseline that exists otherwise is the unoptimized, “as-built”
static system.

###  2.2. The Need for a Simulation Sandbox

For new researchers and practitioners in the field of smart stormwater
control, the barrier to entry is significant. Newcomers must spend
considerable time searching for real-world case studies, synthesizing relevant
control objectives, developing algorithms, and building the capacity to carry
out simulation in order to test new ideas. To grow this research field and
support its broader scientific context, there is a need to better enable the
cross-comparison of smart stormwater control strategies, their algorithms, and
the case study instances for when they are used. While there have been some
prior efforts to benchmark specific stormwater networks in order to evaluate
adaptive control strategies (Schütze et al., 2017; Borsányi et al., 2008),
there is still a shortage of stormwater control case studies with diverse
control objectives.

Other research domains provide compelling examples of community-driven
simulation tool chains that have been developed for similar cross-comparison
needs. For example, the ARPA-E GRID DATA program has enabled an active
research community in the energy sector by providing open source case studies
and benchmarking tools of power system networks (Advanced Research Projects
Agency–Energy, U.S. Department of Energy, 2016). In a similar vein, the water
distribution community has created their own active cross-comparison tools
(Walski et al., 1987; Ostfeld et al., 2008; Marchi et al., 2014). pystorms is
our effort to develop a similar set of research tools for the stormwater
control community. Additionally, we have also been inspired by the streamlined
control algorithm testing of the OpenAI Gym toolkit (Brockman et al., 2016).
As such, we contend that there is a need for a similar, more “out-of-the-box”
software toolset with an unambiguous programming interface that allows
stormwater control researchers to get started more quickly. To that end, we
have formulated pystorms as a _simulation sandbox_ , in which we systematize a
collection of stormwater control testbed examples, and foster the
experimentation and testing of new control strategies.

Table 1. A pystorms scenario is comprised of five distinct components: its Network Topology, an Event Driver, a set of Controllable Assets, a set of Observable States, and the Control Objectives. Network Topology |  A _network_ is the physical system of conduits (e.g. pipes, culverts), storage elements (e.g. retention and detention basins), and any other subcatchment infrastructure (e.g. green infrastructure, wetlands) that collect, convey, and/or treat stormwater runoff.
---|---
Event Driver |  The _event driver_ consists of any inputs or “disturbances” to the network that govern the generation and flow of runoff. Most often, an event driver is the precipitation generating runoff in the watershed, but can also include wastewater flows, tidal fluctuations, or other phenomena.
Controllable Assets |  The _controllable assets_ are the subset of the network topology that are equipped with valves, pumps, or any other flow control infrastructure that can be actuated to manipulate stormwater flow.
Observable States |  The _observable states_ are the collection of states in the network that can be accessed by the users during a simulation. In the real world, these states are measured by a set of sensors installed at the corresponding network locations.
Control Objectives |  The overall goal or set of goals (e.g. preventing flooding, reducing erosion, minimizing overflows, improving water quality) of manipulating the behavior of a stormwater network using controllable assets during a simulation. A control strategy’s ability to achieve a particular objective is quantified via a corresponding performance metric.

##  3\. pystorms

To achieve the objectives of curating an open repository of smart stormwater
control testbed examples, and reducing the learning curve for newcomers
testing new control strategies, pystorms is underpinned by two distinct
features. First, it provides a collection of diverse stormwater control
scenarios, which are drawn from real-world urban watersheds to encompass
diverse features appertaining to stormwater systems (Section 3.1). Second,
these scenarios are coupled with a streamlined Python programming interface
(Section 3.2) that explicates the computational backend of a corresponding
stormwater control simulator (Section 3.3). Together, these features provide
researchers with a standalone software package that focuses its usage on the
development and testing of stormwater control algorithms.

###  3.1. Scenarios

Table 2. pystorms includes a curated collection of real world-inspired stormwater scenarios. Scenario |  Network Topology |  Control Objectives
---|---|---
theta |  $2\,\mathrm{km^{2}}$ idealized separated stormwater network |  Maintain the flows at the outlet below a threshold and avoid flooding (2 storage basin outlets)
alpha |  $0.12\,\mathrm{km^{2}}$ residential combined sewer network |  Minimize total combined sewer overflow volume (5 weirs at interceptor connections)
beta |  $1.3\,\mathrm{km^{2}}$ separated stormwater network with a tidally-influenced receiving river |  Minimize flooding (1 storage basin outlet, 1 pump, 1 inline storage dam)
gamma |  $4\,\mathrm{km^{2}}$ highly urban separated stormwater network |  Maintain channel flows below threshold and avoid flooding (11 detention pond outlets)
delta |  $2.5\,\mathrm{km^{2}}$ combined sewer network in which the stormwater ponds also serve as waterfront |  Maintain water levels within upper and lower thresholds for water quality and aesthetic objectives (4 storage basin outlets; 1 infiltration basin inlet)
epsilon |  $67\,\mathrm{km^{2}}$ highly urban combined sewer network |  Maintain sewer network outlet total suspended solids (TSS) load below threshold and avoid flooding (11 in-line storage dams)
zeta |  $1.8\,\mathrm{km^{2}}$ combined and separated sewer network (based on the Astlingen benchmarking network (Schütze et al., 2017; Sun et al., 2020)) |  Maximize flow to downstream wastewater treatment plant and minimize total combined sewer overflow volume (4 storage basin outlets)

pystorms abstracts smart stormwater systems into _scenarios_. Each scenario
captures a combination of elements that comprise a stormwater control problem.
A fully defined scenario includes a network topology (the system or watershed
being studied), inputs, a selection of controllable and observable assets, as
well as a clearly defined control objective (we refer the reader to Figure 1
and Table 1 for further details). While users can—and are encouraged to—create
their own scenarios, at the time of writing, pystorms provides an initial
collection of seven scenarios, all drawn from real-world smart stormwater
systems across North America and Europe. The collection of scenarios spans a
variety of stormwater systems that address a diverse set of urban watershed
needs with various control objectives. The subcatchment areas range from
$0.12-67\,\mathrm{km^{2}}$ in size, and include both combined and separated
stormwater arrangements. A summary of the collection of scenarios are
presented in Table 2, with their more detailed descriptions provided
throughout this paper’s appendices.

To demonstrate a simple example of a pystorms scenario, we can focus on
Scenario theta, an idealized stormwater network synthesized for unit testing
and rapid algorithm exploration. Scenario theta’s _network topology_ includes
two $1000\,\mathrm{m^{3}}$ storage basins connected in parallel and draining
into a shared downstream water body. The _event driver_ is a synthetic rain
event lasting $9\,\mathrm{hr}$ with a peak intensity of
$3.2\,\mathrm{in}\,\mathrm{hr^{-1}}$. The _observable states_ are the water
levels at the two basins, reported at $15\,\mathrm{min}$ time-steps. The
_controllable assets_ are valves at the outlet of both storage basins,
adjustable at each time-step between $0-100\,\mathrm{\%}$ open. The _control
objective_ is to maintain the outflow into the downstream water body below a
specified threshold of $0.5\,\mathrm{m^{3}s^{-1}}$, while simultaneously
preventing flooding at the basins. The ability of a control strategy to meet
theta’s control objective is quantified using a pre-defined performance metric
that computes a penalty for violating the control objective at each time-step,
and sums these penalties across the whole simulation. We provide the specific
details on this performance metric (eq. 1a) in Section 4 where we evaluate the
performance of two different example control strategies applied to theta.

While our aim is for this collection of scenarios to represent a myriad of
smart stormwater control applications, we recognize that it is certainly not
exhaustive. Ultimately, we aspire to grow the pystorms repository of
stormwater scenarios through community-driven contributions of new scenarios.
Accordingly, we provide extensive documentation 333open-
storm.org/pystorms/docs/build-scenarios for users to contribute their own
scenarios, or modify the existing ones.

Figure 2. This code snippet is an example implementation of pystorms for
Scenario theta, which is described in greater detail throughout Section 4. In
this example, the controller is implemented as a Python function block.

###  3.2. User Experience

pystorms provides a suite of pre-defined smart stormwater systems, and is
designed to be both intuitive and accessible for users at any level of
expertise or experience of control systems and stormwater management. Via
three intuitive function calls, the user is able to iterate through the
simulation of a smart stormwater scenario, and interact with the scenario at
any of the simulation timesteps by querying its states or changing the
settings of its control assets. pystorms provides the computational
environment of a smart stormwater system; the user then provides the control
algorithm dictating how it should operate. For developing control strategies,
pystorms allows users the flexibility to utilize any additional computational
tools at their disposal. This can be done either by leveraging any additional
computational software stacks of Python, or interfacing pystorms with other
computing platforms or languages (e.g., MATLAB, Julia).

The user first initializes a pystorms scenario by creating an instance of it
using the statement: pystorms.scenarios.<scenario name>(). As seen in the code
example (Fig. 2), theta can be initialized with pystorms.scenarios.theta().
The initialization configures the stormwater simulator with the computational
representations necessary to simulate the respective scenario, and returns it
as a Python object. This Python object (env in Fig. 2) can be used to progress
and/or pause the stormwater simulator, read and/or write parameters to the
network, and utilize any additional pystorms functionality.

The pystorms programming interface is inspired by the principles of control
theory, where the control of a system is abstracted as a control loop in which
a controller monitors the underlying state(s) of the system and makes
calculated adjustments to the system for it to achieve a desired behavior. The
basic control loop is implemented using the following steps:

  1. (1)

Query the set of observable states in the stormwater network at the current
time-step;

  2. (2)

Compute control actions to manipulate the system to achieve a desired
behavior; and

  3. (3)

Implement the control actions by adjusting the settings of the controllable
assets.

The state of the underlying stormwater network in the scenario can be queried
using the <scenario object>.state() call (see env.state() in Fig. 2).
<scenario object>.step(<actions>) implements the control actions in the
stormwater network, progresses the simulation forward a time-step, and returns
the current status of the simulation (True when the simulation has terminated
and False otherwise). In Fig. 2, done = env.step(actions) implements actions
in the stormwater network and progresses the simulation being handled by the
env Python object, which in this case is the Scenario theta (assigned via
pystorms.scenario.theta()). done is assigned True when the simulation has
terminated, and False otherwise.

During the each time-step of the simulation, the metrics that underly the
scenario’s control objective are evaluated. This computed value is then stored
for each time-step, and can be accessed at any time during the simulation
using <scenario object>.performance() (env.performance() in Fig. 2).
Additional parameters are logged throughout the simulation. While an initial
set of these logged parameters is predefined, the user is able to customize
this set for any additional parameters of interest. Users carry out Steps 1
and 3 with expressions <scenario object>.state(), and
<scenario object>.step(<actions>), respectively. The user defines the
controller (Step 2), which maps the observed states to control actions.

###  3.3. Architecture

The pystorms architecture follows an object oriented programming paradigm and
relies on classes as its core building blocks. While the pystorms programming
interface is designed with the intent to be intuitive for all potential users,
it particularly caters to those who may only have a rudimentary understanding
of stormwater dynamics and/or basic familiarity with programming in Python.
However, it can also be customized to meet the requirements of researchers who
want to incorporate advanced functionality, such as custom water quality or
rainfall-runoff modules. For details on how to utilize pystorms modularity and
customization, we direct the reader to its online documentation.

The pystorms architecture is organized to accomplish two tasks: (1) configure
the pystorms scenario, and (2) execute the pystorms scenario. These two tasks
are carried out using three core interacting modules: environment, scenario,
and config. These three modules interface with each other to build and execute
a given scenario. Fig. 3 provides a schematic of this architecture. The first
two modules handle the stormwater simulation, while the latter handles the
computational representation of the stormwater networks and the metadata
pertaining to the control problem (i.e. states, actions, and objectives).

Figure 3. pystorms is built with three interacting core modules: (i) config
represents the metadata and computational representations of the stormwater
network and event driver; (ii) environment acts as an interface for scenarios
to interact with the stormwater simulators; and (iii) scenario provides a
consistent structure for the scenarios in the package. A scenario object in
pystorms inherits (represented by arrows) from the base scenario class, and
interfaces (represented by the line) with the stormwater simulator though the
environment.

####  3.3.1. Configuration

The config module is used to manage the configuration in the pystorms
architecture. config contains a configuration file for each scenario, which
specifies the stormwater network, and then delineates its observable states,
controllable assets, and the set of parameters that are used to compute its
control objective’s corresponding performance metric. The configuration files
are written using YAML, a mark-up language commonly used for developing
configuration files in software applications. With YAML, the parameters of
interest defined in the configuration file are formatted as vertical lists
rather than data structures. As a result, the configuration file becomes more
human-readable, and creates a simple but scalable workflow for developing
scenarios.

####  3.3.2. Simulation

Scenarios in pystorms are implemented as Python classes. To ensure consistent
functionality across scenarios, each scenario is instantiated as its own
independent class with an inherited structure from a base scenario module. The
scenario classes interface their corresponding configuration files with the
stormwater simulator and implement any of the functions specific to that
scenario (e.g. functions used for computing performance metrics of
corresponding control objectives).

The environment module is the interface between the stormwater simulator (e.g.
EPA-SWMM) and the scenarios. This module is specifically included to ensure
pystorms is able to remain agnostic to whatever stormwater simulator is used.
For instance, if a user wants to utilize a customized hydrologic solver for
simulating stormwater, they can do so by modifying the environment module to
call their solver when the scenarios query it, thus ensuring compatibility to
a wide array of simulators with minimal overhead.

pystorms uses SWMM as its default stormwater simulator. SWMM, developed by the
U.S. EPA, is an open-source stormwater simulation model that is extensively
used for the design and analysis of stormwater systems across the world. SWMM
is written in C, a low-level programming language that results in significant
computational efficiency. However, the tradeoff for using C becomes SWMM’s
subsequent difficulty at being interfaced with the latest scientific
libraries, which are primarily developed in high-level programming languages,
such as Python. As a result, there have been several efforts over the years to
build wrappers for SWMM such that its functionality can be exploited via these
high-level languages.

PySWMM is a Python package that not only provides a wrapper to communicate
with SWMM, but also yields a high-level user interface for querying the
various stormwater parameters (McDonnell et al., 2020). pystorms — by means of
the environment module — interfaces with SWMM using PySWMM, and as a result,
all functionality included in PySWMM can also be accessed using pystorms.
Readers are directed to the documentation for additional details and examples
to customize pystorms to meet their requirements.

###  3.4. Software Availability

Developed in Python, pystorms is supported on all major operating systems
(OSX, Windows, and Linux) and can be installed using
pip444pypi.org/project/pystorms. pystorms is distributed under the GNU General
Public GPLv3 license555gnu.org/licenses/gpl-3.0.html, which ensures that this
package and its derivatives remain open-source and can be used free of cost.
Additionally, source code for the package is available on
Github666github.com/kLabUM/pystorms alongside comprehensive documentation and
tutorials to utilize and contribute to its broader utilization and
development777pystorms.org.

Figure 4. The Equal-filling Controller maintains the flows at the outlet below
a desired threshold by coordinating its actions such that it equally utilizes
the storage in the controllable assets of the network. Algorithm 1 and the
corresponding code snippet illustrate the algorithm and its implementation as
a function block in Python. An interactive example of the algorithm
implementation and its evaluation on Scenario theta can be accessed at
pystorms.org/#ipynb/GoogleColab/theta.

##  4\. Demo: Implementing and Evaluating Control Strategies

Here, we demonstrate how pystorms facilitates developing smart stormwater
control strategies by evaluating the performance of two control algorithms
applied to Scenario theta.

###  4.1. Scenario theta

As stated in Section 3.1, Scenario theta has been developed for rapid
prototyping and unit testing of new control strategies. Because Scenario theta
is an idealized case, its corresponding performance objective is defined such
that a “perfect” score of $0$ is achievable. We demonstrate how to utilize
Scenario theta in this manner by presenting two different control strategies
applied.

####  4.1.1. Implementating Control Strategies

While there exist many control strategies that can be adopted to achieve
theta’s control objective, we implement and compare two basic strategies here:
a simple rule-based control strategy, and the Equal-filling Degree control
strategy. Both control strategies adjust the valve openings of theta’s two
basin outlets to either retain or release storage depending on the observed
states; however, the rule-based control strategy illustrates a simple example
that may be explored by a first time user, while the Equal-filling Degree
control strategy is presented as an example of an established methodology
widely used by stormwater control practitioners.

#### Rule-based Controller.

The rule-based control strategy adjusts our basin outlets based on their
respective water levels. Specifically, each basin’s outlet setting is equal to
its relative water level (i.e., the current water level of the basin divided
by its maximum depth). Therefore, our control algorithm will set a full
basin’s outlet to 100% open, and a basin that is half full will have its
outlet set to 50% open, etc. While this strategy provides a means to mitigate
local flooding at each basin, it notably does not consider the other control
objective for the network’s outflow into the downstream water body to stay
below a given threshold.

#### Equal-filling Degree Controller.

The equal-filling degree is a control strategy often applied to stormwater
networks with distributed stormwater storage assets, and has been used in some
cases as a starting point when comparing more than one control strategies
(Borsányi et al., 2008; Campisano et al., 2000; Dirckx et al., 2011; Kroll et
al., 2016; Vezzaro and Grum, 2014). For this strategy, we begin by defining a
storage asset’s “filling degree”—which is typically the ratio a storage asset
is full based on its volume or depth—and compute it for each asset in the
collection system. The algorithm seeks to “balance” these filling degrees
across the system based on its average. The exact manner in which this
balancing is carried out is not necessarily consistent in literature. Our
method for this balancing is delineated in the algorithm in Fig. 4. If all
assets have a filling degree equal to the average (i.e., all assets are
equally filled), then each should release an equal fraction of the target
outflow. Otherwise, the released flows across the assets should be
differentiated such that, when an asset has a filling degree less than the
average, it does not release any flow; but if an asset is greater than the
average, it releases flows based on its deviation from the average.

The implementation of the equal-filling degree algorithm using pystorms can be
seen in Figure 4. We carry out the simulation for each of the two algorithms,
as well as for the _uncontrolled case_ , in which control actions are never
implemented and the basin outlets are always open. The resulting hydraulic
behavior at the two basins and the network’s outflow for each of these
simulation runs can be seen in Figure 5.

####  4.1.2. Evaluating Control Strategies

The aim of Scenario theta is to find a control strategy that can meet theta’s
_control objective_ to maintain the outflow into the downstream water body
below a specified threshold of $0.5\,\mathrm{m^{3}s^{-1}}$ and also minimize
flooding at the basins. As discussed in Section 3.1, we pre-define a
performance metric to quantify the control algorithm’s ability to meet the
corresponding control objective. For Scenario theta, this performance metric,
$P$, is defined as:

(1a)
$P={\sum_{t=0}^{T}}\left(\mathcal{H}_{t}+\displaystyle\sum_{i=1}^{2}\mathcal{G}_{i,t}\right)$
(1b) $\displaystyle\mathcal{H}_{t}$
$\displaystyle=\begin{cases}Q_{t}-0.5\,,&\text{if \,}Q_{t}>0.5\\\
0.0\,,&\text{otherwise}\\\ \end{cases}$ (1c) $\displaystyle\mathcal{G}_{i,t}$
$\displaystyle=\begin{cases}10^{3}\,,&\text{if any flooding at basin \,}i\\\
0.0\,,&\text{otherwise}\\\ \end{cases}$

where $\mathcal{H}_{t}$ is a flow exceedance penalty of the stormwater
network’s outflow, $Q_{t}$, over the $0.5\,\mathrm{m^{3}s^{-1}}$ threshold;
and $\mathcal{G}_{i,t}$ is an arbitrary flooding penalty of $10^{3}$ added
whenever there exists flooding at either of our two basins, both calculated
and summed across every time-step $t$ in the simulation. The performance
metric, which is calculated across the simulations for the uncontrolled case
and both of the implemented control algorithms, can be seen in Table 3.
Additionally, corresponding hydraulic behavior for all three cases at their
network outlet and both basins can be seen in Fig. 5. The equal-filling degree
strategy is able to achieve the control objective of the outflow threshold, as
well as avoidance of flooding. Alternatively, the rule-based control strategy
only is able to avoid flooding at the basins. The stormwater network behavior
for both strategies follow their corresponding implemented algorithm. For
example, as the rule-based control strategy does not directly consider the
outflow threshold when determining the implemented control actions, it follows
that the outflow in the network’s outlet exceeds this threshold (see the
outlet plot in Fig. 5).

The results for each implemented control strategy versus the uncontrolled case
are also captured using theta’s performance metric seen in Equation 1. As the
performance metric is ultimately a sum of penalties for violating the control
objective, a smaller calculated performance metric value indicates a better
performing control algorithm. The respective performance metric values for
each control strategy presented here can be seen in Table 3. With a calculated
performance metric of $0$, the equal-filling degree strategy meets theta’s
control objective; comparatively, the rule-based and uncontrolled cases have
higher performance metric values, and thus, we can conclude perform worse than
the equal-filling degree.

Figure 5. In Scenario theta, the equal-filling degree control strategy is successfully able to maintain the flows at the outlet of the watershed below the desired threshold of $0.5\,\mathrm{m^{3}s^{-1}}$ by uniformly using the storage in the networks. Static rule-based control and uncontrolled responses of the networks are also presented for comparison. The maximum depth in each of the two basins is $2\,\mathrm{m}$. Table 3. Calculated performance metric values from Equation 1 for simulations corresponding to the two implemented control algorithms and the uncontrolled simulation. As can be seen, the equal-filling degree control strategy performs better than the rule-based control strategy, which then outperforms the uncontrolled case. Control Strategy | Performance Metric
---|---
Uncontrolled | 1630
Rule-based | 1624
Equal-filling Degree | 0

###  4.2. Additional demos

For demonstration purposes, Scenario theta was chosen due to its simplicity.
For more complex examples, we direct the users to the Github repository where
a Jupyter Notebook has been developed for each of the scenarios presented in
Table 2, and a control strategy is implemented and evaluated against the
uncontrolled case using the corresponding performance metric. The details for
each scenario are provided in the Supplementary Information. In particular, we
direct the reader to Scenarios beta (SI-2.), gamma (SI-3.), epsilon (SI-5.),
and zeta (SI-7.), in which advanced controllers have been implemented, and the
development of these controllers documented in previous research from Sadler
et al. (2020), Mullapudi et al. (2020), Troutman et al. (2020), and Sun et al.
(2020), respectively.

##  5\. Discussion

The ability for stormwater systems to be rapidly modified is critical as
communities prepare for climate change and more frequent, uncertain, and
destructive weather events. Moreover, _often the most basic control
strategies_ can have large-scale impacts on the complex, dynamic systems they
operate, potentially leading to millions of dollars in savings for the
communities they serve. Even though sensor-actuator components may be
successfully deployed at individual sites throughout a stormwater network,
determining strategies for their coordination across the entire watershed may
only add further complexity. As a result, there is a great need — along with
numerous opportunities — to develop and implement novel control strategies to
transform stormwater systems. The sandboxing efforts of pystorms serves as an
initial effort to foster the development of these strategies. Moreover, we
intend for pystorms to serve as a catalyst for our research community to be
more expansive and comprehensive in its analysis of smart stormwater control.

A critical limitation to progressing smart stormwater control research forward
is the inability to systematically develop and then analyze smart stormwater
simulation workflows and control strategies. pystorms directly enables this
development and analysis to become more extensive. pystorms can be customized
and adapted for a multitude of applications beyond its initially-provided
collection of scenarios, and additional research questions can be studied by
assembling new scenarios from the assortment of components of this scenario
collection. For example, for each of the scenarios we provide at the outset,
pystorms specifies only a subset of the scenario’s total observable states
that are able to be queried throughout the simulation. But this initial subset
of observable states is never claimed as the optimal; and actually,
identifying an optimal set of observable states is in and of itself a rich
area of research (Chacon-Hurtado et al., 2017; Sambito et al., 2019; Bartos
and Kerkez, 2020; Van Nguyen et al., 2021). Thus, new scenarios can be made
with different subsets of observable states (e.g. flows, pollutant
concentrations), and new research questions can now be asked about which
states may be most critical for informing control actions to be taken.

In the current iteration of pystorms, we present one means for assessing a
control strategy via the scenario’s corresponding performance metric (e.g.
maintain flow below a threshold, avoid flooding). However, pystorms can serve
as a mechanism for developing and implementing additional metrics. For
example, by increasing the number of controllable assets available out of the
eleven pond outlets presented in Scenario gamma, one can assess the
_scalability_ of a control algorithm as the state-action space increases.
Additionally, control algorithm _generalizability_ across storm
characteristics can be assessed with the multiple rain events provided in
Scenario epsilon.

Beyond the coordination and integration of smart stormwater control methods,
we also view a more expansive — and potentially consequential — opportunity
for pystorms to drive our research community’s analysis of “control” to
include the social and ethical implications of its implementation. We
recognize that the control of stormwater systems operates within broader, and
far more complex, socio-techinical systems, and as a result, the decisions
made can have a profound impact on the lives of people within those systems.
Thus, in addition to water quantity and quality analyses, we impel our
community to also scrutinize their control strategies through the lens of
social equity and longterm community resilience and adaptiveness, such as the
decision framework developed by Ewing and Demir (2021). We believe pystorms
can help facilitate these analyses. For example, for any of the provided
scenarios, one could study the longterm social implications of an adverse
outcome that might occur regularly in the same community due to an implemented
control strategy by modifying a scenario’s control objective and corresponding
performance metric.

Finally, we encourage future users of pystorms to resist the natural
inclination towards fixating on a control algorithm’s _performance_ at the
expense of its _appropriateness_. Our development of pystorms was partly
motivated by similar sets of data and libraries developed by the electrical
and computer science communities over the past two decades in an effort to
evaluate their ever-growing assortment of machine-learning algorithms (Deng,
2012; Russakovsky et al., 2015; Brockman et al., 2016; Henderson et al.,
2017). While their efforts have been wildly successful at propelling the
development of novel algorithms, it has also been noted that much of that
effort has been expended in the “fine-tuning” and “hacking” of the various
algorithms to achieve some sort of arbitrarily-defined “state-of-the-art”
result (Torralba and Efros, 2011). While fine-tuning an algorithm’s behavior
for a specific dataset may indeed yield a better performance metric in
pystorms, this improved performance metric is not guaranteed to translate to
the physical system at hand. Validating these stormwater control strategies
requires much further analysis via their actual physical deployment.
Accordingly, we intend for pystorms to foster a collaborative and thoughtful
evaluation of stormwater control algorithms rather than a myopic focus on
performance metrics and competition.

##  6\. Conclusions and Next Steps

pystorms provides a curated collection of scenarios, coupled with an
accessible programming interface, to enable the development and quantitative
evaluation of stormwater control algorithms. We have developed pystorms with
the intent to make smart stormwater control research more methodical and
efficient. As shown with the demos in Section 4 and accompanying tutorials, we
have demonstrated how users can quickly download and test pystorms and its
scenarios on a basic computer with only a few lines of code. Additionally
shown in Section 4, pystorms facilitates rigorous evaluation by extricating
the control algorithm implementation of a stormwater control strategy to be
applied and quantitatively compared across the example scenarios.

It is our hope that this package will emerge as a community-driven resource
that is able to address key knowledge gaps and enable the advancement of smart
stormwater systems. To this extent, we see proximate opportunities for the
broader research community to collaborate on pystorms by contributing their
own stormwater scenarios and/or control algorithms to the package initiated
here. Likewise, we encourage the broader research community to further build
upon pystorms by imparting their own smart stormwater control instances using
the pystorms architecture and integrating their own stormwater control
simulation workflows into it.

###### Acknowledgements.

This research was supported U.S. National Science Foundation, Award Numbers:
1737432 and 1750744. Additionally, Argonne National Laboratory’s contribution
is based upon work supported by Laboratory Directed Research and Development
(LDRD) funding from Argonne National Laboratory, provided by the Director,
Office of Science, of the U.S. Department of Energy under Contract No. DE-
AC02-06CH11357.

## References

  * (1)
  * Advanced Research Projects Agency–Energy, U.S. Department of Energy (2016) Advanced Research Projects Agency–Energy, U.S. Department of Energy. 2016.  Generating Realistic Information for the Development of Distribution and Transmission Algorithms (GRID DATA) Program.  https://arpa-e.energy.gov/technologies/programs/grid-data.
  * Bartos and Kerkez (2020) Matt Bartos and Branko Kerkez. 2020.  Observability-based sensor placement improves contaminant tracing in river networks.  _Earth and Space Science Open Archive_ (2020).  https://doi.org/10.1002/essoar.10504108.1
  * Batty et al. (2012) Michael Batty, Kay W Axhausen, Fosca Giannotti, Alexei Pozdnoukhov, Armando Bazzani, Monica Wachowicz, Georgios Ouzounis, and Yuval Portugali. 2012.  Smart cities of the future.  _The European Physical Journal Special Topics_ 214, 1 (2012), 481–518.  https://doi.org/10.1140/epjst/e2012-01703-3
  * Bibri and Krogstie (2017) Simon Elias Bibri and John Krogstie. 2017.  Smart sustainable cities of the future: An extensive interdisciplinary literature review.  _Sustainable cities and society_ 31 (2017), 183–212.  https://doi.org/10.1016/j.scs.2017.02.016
  * Borsányi et al. (2008) Péter Borsányi, Lorenzo Benedetti, Geert Dirckx, Webbey De Keyser, Dirk Muschalla, Anne-Marie Solvi, Veronique Vandenberghe, Michael Weyand, and Peter A. Vanrolleghem. 2008.  Modelling real-time control options on virtual sewer systems.  _Journal of Environmental Engineering and Science_ 7, 4 (2008), 395–410.  https://doi.org/10.1139/S08-004
  * Bowes et al. (2021) Benjamin D Bowes, Arash Tavakoli, Cheng Wang, Arsalan Heydarian, Madhur Behl, Peter A Beling, and Jonathan L Goodall. 2021.  Flood mitigation in coastal urban catchments using real-time stormwater infrastructure control and reinforcement learning.  _Journal of Hydroinformatics_ 23, 3 (2021), 529–547.  https://doi.org/10.2166/hydro.2020.080
  * Brockman et al. (2016) Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016.  OpenAI Gym.  arXiv:arXiv:1606.01540
  * Campisano et al. (2000) Alberto Campisano, Wolfgang Schilling, and Carlo Modica. 2000\.  Regulators’ setup with application to the Roma–Cecchignola combined sewer system.  _Urban Water_ 2, 3 (2000), 235–242.  https://doi.org/10.1016/S1462-0758(00)00061-3
  * Chacon-Hurtado et al. (2017) J. C. Chacon-Hurtado, L. Alfonso, and D. P. Solomatine. 2017\.  Rainfall and streamflow sensor network design: a review of applications, classification, and a proposed framework.  _Hydrology and Earth System Sciences_ 21, 6 (2017), 3071–3091.  https://doi.org/10.5194/hess-21-3071-2017
  * Chourabi et al. (2012) Hafedh Chourabi, Taewoo Nam, Shawn Walker, J. Ramon Gil-Garcia, Sehl Mellouli, Karine Nahon, Theresa A. Pardo, and Hans Jochen Scholl. 2012.  Understanding smart cities: An integrative framework. In _Proceedings of the Annual Hawaii International Conference on System Sciences_. IEEE, 2289–2297.  https://doi.org/10.1109/HICSS.2012.615
  * Deng (2012) Li Deng. 2012.  The MNIST database of handwritten digit images for machine learning research [best of the web].  _IEEE Signal Processing Magazine_ 29, 6 (2012), 141–142.
  * Dirckx et al. (2011) G. Dirckx, M. Schütze, S. Kroll, Ch. Thoeye, G. De Gueldre, and B. Van De Steene. 2011.  Cost-efficiency of RTC for CSO impact mitigation.  _Urban Water Journal_ 8, 6 (2011), 367–377.  https://doi.org/10.1080/1573062X.2011.630092
  * Eggimann et al. (2017) Sven Eggimann, Lena Mutzner, Omar Wani, Mariane Yvonne Schneider, Dorothee Spuhler, Matthew Moy de Vitry, Philipp Beutler, and Max Maurer. 2017.  The potential of knowing more: A review of data-driven urban water management.  _Environmental science & technology_ 51, 5 (2017), 2538–2553.  https://doi.org/10.1021/acs.est.6b04267
  * Ewing and Demir (2021) Gregory Ewing and Ibrahim Demir. 2021.  An ethical decision-making framework with serious gaming: a smart water case study on flooding.  _Journal of Hydroinformatics_ (03 2021).  https://doi.org/10.2166/hydro.2021.097
  * Gaborit et al. (2016) E. Gaborit, François Anctil, Geneviève Pelletier, and Peter A. Vanrolleghem. 2016.  Exploring forecast-based management strategies for stormwater detention ponds.  _Urban Water Journal_ 13, 8 (2016).  https://doi.org/10.1080/1573062X.2015.1057172
  * Gaborit et al. (2013) E. Gaborit, Dirk Muschalla, Bertrand Vallet, Peter A. Vanrolleghem, and François Anctil. 2013\.  Improving the performance of stormwater detention basins by real-time control using rainfall forecasts.  _Urban Water Journal_ 10, 4 (2013), 230–246.  https://doi.org/10.1080/1573062X.2012.726229
  * García et al. (2015) L. García, J. Barreiro-Gomez, E. Escobar, D. Téllez, N. Quijano, and C. Ocampo-Martinez. 2015.  Modeling and real-time control of urban drainage systems: A review.  _Advances in Water Resources_ 85 (2015), 120–132.  https://doi.org/10.1016/j.advwatres.2015.08.007
  * Gironás et al. (2010) Jorge Gironás, Larry A Roesner, Lewis A Rossman, and Jennifer Davis. 2010.  A new applications manual for the Storm Water Management Model (SWMM).  _Environmental Modelling and Software_ 25, 6 (2010), 813–814.
  * Harrison and Donnelly (2011) Colin Harrison and Ian Abbott Donnelly. 2011.  A theory of smart cities. In _Proceedings of the 55th Annual Meeting of the International Society for the Systems Sciences_ , Vol. 55.
  * Henderson et al. (2017) Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup, and David Meger. 2017\.  Deep reinforcement learning that matters.  _arXiv preprint arXiv:1709.06560_ (2017).
  * Kerkez et al. (2016) Branko Kerkez, Cyndee Gruden, Matthew Lewis, Luis Montestruque, Marcus Quigley, Brandon Wong, Alex Bedig, Ruben Kertesz, Tim Braun, Owen Cadwalader, Aaron Poresky, and Carrie Pak. 2016\.  Smarter stormwater systems.  _Environmental Science and Technology_ 50, 14 (2016), 7267–7273.  https://doi.org/10.1021/acs.est.5b05870
  * Kitchin (2014) Rob Kitchin. 2014\.  The real-time city? Big data and smart urbanism.  _GeoJournal_ 79, 1 (2014), 1–14.  https://doi.org/10.1007/s10708-013-9516-8
  * Kroll et al. (2016) Stefan Kroll, Geert Dirckx, Brecht M.R. Donckels, Mieke Van Dorpe, Marjoleine Weemaes, and Patrick Willems. 2016.  Modelling real-time control of WWTP influent flow under data scarcity.  _Water Science and Technology_ 73, 7 (2016), 1637–1643.  https://doi.org/10.2166/wst.2015.641
  * Lund et al. (2018) Nadia Schou Vorndran Lund, Anne Katrine Vinther Falk, Morten Borup, Henrik Madsen, and Peter Steen Mikkelsen. 2018.  Model predictive control of urban drainage systems: A review and perspective towards smart real-time water management.  _Critical reviews in environmental science and technology_ 48, 3 (2018), 279–339.
  * Marchi et al. (2014) Angela Marchi, Elad Salomons, Avi Ostfeld, Zoran Kapelan, Angus R. Simpson, Aaron C. Zecchin, Holger R. Maier, Zheng Yi Wu, Samir M. Elsayed, Yuan Song, Tom Walski, Christopher Stokes, Wenyan Wu, Graeme C. Dandy, Stefano Alvisi, Enrico Creaco, Marco Franchini, Juan Saldarriaga, Diego Páez, David Hernández, Jessica Bohórquez, Russell Bent, Carleton Coffrin, David Judi, Tim McPherson, Pascal van Hentenryck, José Pedro Matos, António Jorge Monteiro, Natércia Matias, Do Guen Yoo, Ho Min Lee, Joong Hoon Kim, Pedro L. Iglesias-Rey, Francisco J. Martínez-Solano, Daniel Mora-Meliá, José V. Ribelles-Aguilar, Michele Guidolin, Guangtao Fu, Patrick Reed, Qi Wang, Haixing Liu, Kent McClymont, Matthew Johns, Edward Keedwell, Venu Kandiah, Micah Nathanael Jasper, Kristen Drake, Ehsan Shafiee, Mehdy Amirkhanzadeh Barandouzi, Andrew David Berglund, Downey Brill, Gnanamanikam Mahinthakumar, Ranji Ranjithan, Emily Michelle Zechman, Mark S. Morley, Carla Tricarico, Giovanni de Marinis, Bryan A. Tolson, Ayman Khedr, and Masoud Asadzadeh. 2014\.  Battle of the Water Networks II.  _Journal of Water Resources Planning and Management_ 140, 7 (2014), 04014009.  https://doi.org/10.1061/(ASCE)WR.1943-5452.0000378
  * McDonnell et al. (2020) Bryant E McDonnell, Katherine Ratliff, Michael E Tryby, Jennifer Jia Xin Wu, and Abhiram Mullapudi. 2020.  PySWMM: The Python Interface to Stormwater Management Model (SWMM).  _Journal of Open Source Software_ 5, 52 (2020), 2292\.
  * Montestruque (2018) Luis A. Montestruque. 2018\.  An agent-based storm water management system.  In _Smart Water Grids: A Cyber-Physical Systems Approach_ , Panagiotis Tsakalides, Athanasia Panousopoulou, Grigorios Tsagkatakis, and Luis Montestruque (Eds.). CRC Press, Chapter 6, 151–168.
  * Mullapudi et al. (2020) Abhiram Mullapudi, Matthew J Lewis, Cyndee L Gruden, and Branko Kerkez. 2020.  Deep reinforcement learning for the real time control of stormwater systems.  _Advances in Water Resources_ 140 (2020), 103600.
  * Mullapudi et al. (2017) Abhiram Mullapudi, Brandon P. Wong, and Branko Kerkez. 2017\.  Emerging investigators series: Building a theory for smart stormwater systems.  _Environmental Science: Water Resources and Technology_ 3, 1 (2017).  https://doi.org/10.1039/C6EW00211K
  * Ocampo-Martinez (2010) Carlos Ocampo-Martinez. 2010\.  _Model predictive control of wastewater systems_.  Springer. 1–236 pages.  https://doi.org/10.1007/978-1-84996-353-4
  * Ostfeld et al. (2008) Avi Ostfeld, James G. Uber, Elad Salomons, Jonathan W. Berry, William E. Hart, Cindy A. Phillips, Jean-Paul Watson, Gianluca Dorini, Philip Jonkergouw, Zoran Kapelan, Francesco di Pierro, Soon-Thiam Khu, Dragan Savic, Demetrios Eliades, Marios Polycarpou, Santosh R. Ghimire, Brian D. Barkdoll, Roberto Gueli, Jinhui J. Huang, Edward A. McBean, William James, Andreas Krause, Jure Leskovec, Shannon Isovitsch, Jianhua Xu, Carlos Guestrin, Jeanne VanBriesen, Mitchell Small, Paul Fischbeck, Ami Preis, Marco Propato, Olivier Piller, Gary B. Trachtman, Zheng Yi Wu, and Tom Walski. 2008.  The Battle of the Water Sensor Networks (BWSN): A Design Challenge for Engineers and Algorithms.  _Journal of Water Resources Planning and Management_ 134, 6 (2008), 556–568.  https://doi.org/10.1061/(ASCE)0733-9496(2008)134:6(556)
  * Persaud et al. (2019) P.P. Persaud, A.A. Akin, B. Kerkez, D.T. McCarthy, and J.M. Hathaway. 2019.  Real time control schemes for improving water quality from bioretention cells.  _Blue-Green Systems_ 1, 1 (2019), 55–71.  https://doi.org/10.2166/bgs.2019.924
  * Ramke (2018) Hans-Günter Ramke. 2018\.  Drainage Systems.  In _Solid Waste Landfilling: Concepts, Processes, Technology_ , Raffaello Cossu and Rainer Stegmann (Eds.). Elsevier, Chapter 8, 343–416.
  * Rossman (2015) L. Rossman. 2015\.  _Storm Water Management Model User’s Manual Version 5.1 - Manual_ (epa/600/r-14/413 (ntis epa/600/r-14/413b) ed.).  US EPA Office of Research and Development, Washington, DC.
  * Rossman (2017) L. Rossman. 2017\.  _Storm Water Management Model Reference Manual Volume II – Hydraulics_ (epa/600/r-17/111 ed.).  US EPA Office of Research and Development, Washington, DC.
  * Rossman and Huber (2015) L. Rossman and W. Huber. 2015.  _Storm Water Management Model Reference Manual Volume I – Hydrology_ (epa/600/r-15/162a ed.).  US EPA Office of Research and Development, Washington, DC.
  * Rossman and Huber (2016) L. Rossman and W. Huber. 2016.  _Storm Water Management Model Reference Manual Volume III – Water Quality_ (epa/600/r-16/093 ed.).  US EPA Office of Research and Development, Washington, DC.
  * Russakovsky et al. (2015) Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, and et al. 2015.  ImageNet Large Scale Visual Recognition Challenge.  _International Journal of Computer Vision_ 115, 3 (Dec. 2015), 211–252.  https://doi.org/10.1007/s11263-015-0816-y
  * Sadler et al. (2020) Jeffrey M. Sadler, Jonathan L. Goodall, Madhur Behl, Benjamin D. Bowes, and Mohamed M. Morsy. 2020.  Exploring Real-time Control of Stormwater Systems for Sea Level Rise.  _Journal of Hydrology_ (2020).  https://doi.org/10.1016/j.jhydrol.2020.124571
  * Sadler et al. (2019) Jeffrey M. Sadler, Jonathan L. Goodall, Madhur Behl, Mohamed M. Morsy, Teresa Culver, and Benjamin D. Bowes. 2019.  Leveraging open source software and parallel computing for model predictive control of urban drainage systems using EPA-SWMM5.  _Environmental Modelling and Software_ (2019).  https://doi.org/10.1016/j.envsoft.2019.07.009
  * Sambito et al. (2019) Mariacrocetta Sambito, Cristiana Di Cristo, Gabriele Freni, and Angelo Leopardi. 2019.  Optimal water quality sensor positioning in urban drainage systems for illicit intrusion identification.  _Journal of Hydroinformatics_ 22, 1 (09 2019), 46–60.  https://doi.org/10.2166/hydro.2019.036
  * Schilling (1989) Wolfgang Schilling. 1989\.  Real-time control of urban drainage systems: The state-of-the-art.  _IAWPRC Task Group on Real-Time Control of Urban Drainage Systems, London_ (1989).
  * Schütze et al. (2004) Manfred Schütze, Alberto Campisano, Hubert Colas, Wolfgang Schilling, and Peter A. Vanrolleghem. 2004.  Real time control of urban wastewater systems - Where do we stand today?  _Journal of Hydrology_ 299, 3-4 (2004), 335–348.  https://doi.org/10.1016/j.jhydrol.2004.08.010
  * Schütze et al. (2017) Manfred Schütze, Maja Lange, Michael Pabst, and Ulrich Haas. 2017.  Astlingen - a benchmark for real time control (RTC).  _Water Science and Technology_ 2017, 2 (2017), 552–560.
  * Shishegar et al. (2018) Shadab Shishegar, Sophie Duchesne, and Geneviève Pelletier. 2018\.  Optimization methods applied to stormwater management problems: A review.  _Urban Water Journal_ 15, 3 (2018), 276–286.
  * Sun et al. (2020) Congcong Sun, Jan Lorenz Svensen, Morten Borup, Vicenç Puig, Gabriela Cembrano, and Luca Vezzaro. 2020.  An MPC-enabled SWMM implementation of the Astlingen RTC benchmarking network.  _Water_ 12, 1034 (2020), 1–13.  https://doi.org/10.3390/w12041034
  * Torralba and Efros (2011) Antonio Torralba and Alexei A Efros. 2011.  Unbiased look at dataset bias. In _CVPR 2011_. IEEE, 1521–1528.
  * Trotta et al. (1977) Paul D. Trotta, J. W. Labadie, and N. S. Grigg. 1977\.  Automatic control strategies for urban stormwater.  _Journal of the Hydraulics Division_ (1977), 1443–1459.
  * Troutman et al. (2020) Sara C. Troutman, Nancy G. Love, and Branko Kerkez. 2020\.  Balancing water quality and flows in combined sewer systems using real-time control.  _Environmental Science: Water Research & Technology_ 6, 5 (2020), 1357–1369.
  * van Daal et al. (2017) Petra van Daal, Günter Gruber, Jeroen Langeveld, Dirk Muschalla, and François Clemens. 2017\.  Performance evaluation of real time control in urban wastewater systems in practice: Review and perspective.  _Environmental Modelling and Software_ 95 (2017), 90–101.
  * Van Nguyen et al. (2021) Lam Van Nguyen, Dieu Tien Bui, and Razak Seidu. 2021.  Identification of Sensitive Factors for Placement of Flood Monitoring Sensors in Wastewater/Stormwater Network Using GIS-Based Fuzzy Analytical Hierarchy Process: A Case of Study in Ålesund, Norway. In _Proceedings of the International Conference on Innovations for Sustainable and Responsible Mining_ , Dieu Tien Bui, Hai Thanh Tran, and Xuan-Nam Bui (Eds.). Springer International Publishing, 79–97.
  * Vanrolleghem et al. (2005) Peter A. Vanrolleghem, Lorenzo Benedetti, and J. Meirlaen. 2005\.  Modelling and real-time control of the integrated urban wastewater system.  _Environmental Modelling and Software_ 20, 4 (2005), 427–442.  https://doi.org/10.1016/j.envsoft.2004.02.004
  * Vezzaro and Grum (2014) Luca Vezzaro and Morten Grum. 2014.  A generalised Dynamic Overflow Risk Assessment (DORA) for real time control of urban drainage systems.  _Journal of Hydrology_ 515 (2014), 292–303.  https://doi.org/10.1016/j.jhydrol.2014.05.019
  * Walski et al. (1987) Thomas M. Walski, E. Downey Brill, Johannes Gessler, Ian C. Goulter, Roland M. Jeppson, Kevin Lansey, Han-Lin Lee, Jon C. Liebman, Larry Mays, David R. Morgan, and Lindell Ormsbee. 1987\.  Battle of the Network Models: Epilogue.  _Journal of Water Resources Planning and Management_ 113, 2 (1987), 191–203.  https://doi.org/10.1061/(ASCE)0733-9496(1987)113:2(191)
  * Yuan et al. (2019) Zhiguo Yuan, Gustaf Olsson, Rachel Cardell-Oliver, Kim van Schagen, Angela Marchi, Ana Deletic, Christian Urich, Wolfgang Rauch, Yanchen Liu, and Guangming Jiang. 2019\.  Sweating the assets - The role of instrumentation, control and automation in urban water systems.  _Water Research_ 155 (2019), 381–402.  https://doi.org/10.1016/j.watres.2019.02.034

## Supplementary Information.

###  SI-1.. Scenario alpha

Figure S1. Topology of Scenario alpha’s stormwater network. Regulators control flow of combined stormwater/dry-weather flows to WWTP, and to stream when capacity to treatment plant is exceeded. Plot shows flowrates at downstream locations of stream and interceptor for uncontrolled case. Table S1. Scenario alpha Configuration Network Topology |  Scenario alpha is a combined-sewer overflow system in a $0.12$ $\,\mathrm{km}^{2}$ residential area, and is based on the SWMM Applications Manual, Example 8 (Gironás et al., 2010). This specific network is chosen to demonstrate how combined sewer overflow (CSO) regulators, which are static in the SWMM Applications’ interpretation, can become active (e.g. valves whose openings can be remotely controlled), and their coordination can then potentially improve performance.
Scenario alpha has a series of interceptor pipes that are able to capture and
convey 100% of sanitary flows from sewers to a WWTP during dry weather events.
During wet weather events, the capacity of the interceptor pipes may be
exceeded. Thus, the sewers are also connected to a nearby water body in order
to discharge the excess wastewater during such events (i.e. CSO events).
To control the flow between the sewer system, the interceptor pipes, and the
water body, a set of five flow regulators are implemented in parallel to one
another. The five flow regulators are implemented as transverse weirs with
orifices, in which the sanitary and stormwater flows are conveyed to the
interceptor through a controllable orifice, and excess flow is then diverted
to the overflow outlet into the body of water via a weir upstream of the
orifice.
The network topology for Scenario alpha is in Figure S1.
---|---
Event Driver |  Scenario alpha is driven by a 10-year, 2-hour storm event. It is assumed the entire subcatchment experiences uniform rainfall.
Controllable Assets |  The controllable assets of Scenario alpha are a set of five orifice valves that control the wastewater flow to the interceptor pipes. The valve openings are controllable between 0% – 100%, where the percentage of each orifice valve corresponds to the percent opening for the wastewater flow into the interceptor pipes; all orifice valves are initially set to 100% open.
The orifice valves are one component of a flow regulator. For each flow
regulator, once the flow capacity through its orifice valve is reached, the
remaining wastewater is diverted upstream, potentially overtopping a
transverse weir, and then _overflowing_ into the nearby stream via the
corresponding CSO outfall.
In the computational implementation of Scenario alpha, the controllable valves
for each of the five regulators are identified with the pattern <Or1, Or2,
Or3, Or4, Or5>.
Observable States |  The observable states for Scenario alpha include the following:  • Depth at each runoff entry point into sewer network • Depth if the junctions nodes at the sewer inlet into the flow regulators • Depth at the CSO outfalls • Depth at each segment of the stream • Flowrate at final interceptor pipe connected to WWTP The values for each can be queried throughout the simulation to inform any control decisions.
Control Objectives |  The primary objective for Scenario alpha is to minimize the total CSO volume across the system, while also avoiding flooding. The ability to achieve the objective of Scenario alpha is quantified via its performance metric, $P$, in the equation below. $\begin{aligned} P=\displaystyle\sum_{t=0}^{T}\left[\displaystyle\sum_{i=1}^{5}V_{\mathrm{CSO}_{i}}(t)+\left(10^{6}\cdot\displaystyle\sum_{j=1}^{26}V_{\mathrm{flood}_{j}}(t)\right)\right]\end{aligned}$ where $V_{\mathrm{CSO}_{i}}(t)$ is the volume of CSO overflow at regulator $i$ over timestep $t$, and $V_{\mathrm{flood}_{j}}(t)$ is the volume of flooding over timestep $t$ at all of the 26 junction nodes in the network. (Note: the flooding volume is purposely penalized by a factor of $10^{6}$.)
Additional Notes |  As stated previously, Scenario alpha is based on the eighth example model presented in the SWMM 5.0 Applications Manual (Gironás et al., 2010), which was developed to demonstrate how to represent a combined sewer system in SWMM, and included flow regulators as well as a downstream pump station as control assets. In the SWMM Applications Manual, the outflow of the interceptor leads to a storage well that is then pumped through a forcemain pipe to the WWTP, and a rule-based controller dictates when a pump is operated based on depth of the well. This forcemain pipe and corresponding pump station are not included in Scenario beta. However, these components could be added as an additional component to this network for further dynamic control testing and exploration. Interactive example: pystorms.org/#ipynb/GoogleColab/alpha

###  SI-2.. Scenario beta

Figure S2. The network schematic for Scenario beta. Orange circles and blue lines in the figure represent, respectively, the controllable storage elements and channels draining into the outlet (black triangle). Scenario beta has three control assets: furthest upstream is a valve at the outlet of a stormwater pond for controlling the pond’s storage, connected in parallel to a pump used to increase the hydraulic head of the underlying stormwater pipes, which both drain into shared downstream conduit with inline storage controlled by an inflatable dam. To demonstrate the tidal fluctuation in this scenario, the flowrate at the node with the lowest elevation (i.e. junction J33) is plotted against the depth at the network outfall OUT0, which has an assumed increase in sea level of $3\,\mathrm{ft}$. Table S2. Scenario beta Configuration Network Topology |  Scenario beta is a separated storm system in a highly-urbanized area that drains to a tidally influenced river. The drainage area is $\approx 1.3\ km^{2}$. The elevation of the system is low and the outfall of the system becomes inundated with certain high tide conditions, making the drainage of the water through the outfall impossible. The system has an upstream stormwater pond, a pump that can help reduce flooding within the system, and variable downstream inline storage via an inflatable dam. Given the constraints on the system, totally eliminating flooding is not always possible. The stormwater network topology for Scenario beta is shown in Figure S2.
---|---
Event Driver |  Scenario beta is driven by a 2-year, 12-hour storm event with a SCS Type II temporal distribution. Scenario beta also has predefined tidal conditions, which are based on a typical tide cycle from a nearby tide gauge. The tidal conditions were adjusted such that high tides equaled between the $99\%$ and $50\%$ annual tide level for the station. Finally, to test potential future sea level rise scenarios, Scenario beta also assumes a high sea level rise of $3.0\,\mathrm{ft}$, which is incorporated into the scenario via the pre-defined tidal conditions.
Controllable Assets |  Scenario beta has three controllable assets: (i) a valve at the outlet of a stormwater pond for controlling the pond’s storage, (ii) a pump to increase the hydraulic head of the underlying stormwater pipes, and (iii) an inflatable dam for utilizing inline storage of stormwater pipes. The valve is computationally implemented as an orifice, identified as O2 and adjustable to a setting within a range of $0.0$ \- $1.0$ (i.e. 0% to 100% open). The pump, identified as P0, is computationally implemented with a predefined pump flowrate curve, and can be set to either $0.0$ (i.e., OFF) or $1.0$ (i.e., ON). The inline storage, identified as W0, is computationally implemented as a weir and adjustable to a setting within a range of $0.0$ \- $1.0$ (i.e. 0% to 100% open). The settings of the three control assets are initially set to $[1.0,0.0,1.0]$, which corresponds to a completely open orifice, a pump set to OFF, and a completely open weir.
Observable States |  The states in Scenario beta are observable at seven different locations, including <J33, J64, J98, J102, OUT0, ST0, ST2>.
Control Objectives |  The objective for Scenario beta is to minimize flooding at the most critical nodes within the system. This is quantified in the below performance equation. $\begin{aligned} P=\displaystyle\sum_{i=1}^{12}\displaystyle\sum_{t=0}^{T}V_{\mathrm{flood}_{i}}(t)\end{aligned}$ where $V_{\mathrm{flood}_{i}}$ is the flooding volume (in gallons) over timestep $t$ across a set of $i=12$ critical nodes. The set of critical nodes is: <J4, J8, J13, J33, J53, J54, J64, J65, J98, J102, J145, J146>.
Additional Notes |  Scenario beta was developed based on a stormwater model developed and tested by Sadler et al. (2020), with the model, data, and the implementation of its proposed control strategy shared via the HydroShare collaborative website and available at https://doi.org/10.4211/hs.5148675c6a5841e686a3b6aec67a38ee.
A notebook demonstrating the implementation of Scenario beta is available
here: pystorms.org/#ipynb/GoogleColab/beta.

###  SI-3.. Scenario gamma

Figure S3. Topology for the separated stormwater network of Scenario gamma. Orange circles and blue lines in the figure represent respectively the controllable storage elements and channels draining into the outlet (black triangle). During a storm event, each node in the network receives runoff generated by connected subcatchments in addition to discharge from upstream nodes. The system’s response to the given storm event for the uncontrolled case (i.e. all 11 of its valves set to 100% open) is depicted via plotting the outlet’s flows against the target exceedance threshold for each timestep of the simulation. For this scenario, the controller is penalized for any flowrates above given exceedance thresholds for each of the network’s 11 links situated immediately downstream from a corresponding control asset. Table S3. Scenario gamma Configuration Network Topology |  Scenario gamma is a separated system in a semi-urbanized watershed of $\approx 4\ km^{2}$. Scenario gamma’s stormwater network is comprised of 11 storage basins, connected in both series and parallel, that drain into a shared downstream water-body. The stormwater network topology for Scenario gamma can be seen in Figure S3.
---|---
Event Driver |  Scenario gamma is driven by a 25-year, 6-hour storm event. It is assumed the entire subcatchment experiences uniform rainfall.
Controllable Assets |  Each of Scenario gamma’s storage basins are control assets equipped with valves adjustable to a setting within a range of (0% to 100%) during the simulation. The percentage of each valve corresponds to the percent opening of the corresponding storage basin’s outlet, with all valves initially set to 100% open. In the computational implementation of Scenario gamma, each of the controllable valves are identified with the pattern O<storage asset ID> (e.g. O1, O2, …O11).
Observable States |  The observable states for Scenario gamma are the water levels in the the storage basins, which can be queried throughout the simulation to inform any control decisions.
Control Objectives |  The primary objective for Scenario gamma is to maintain flow in each connecting channel below a given threshold ($0.11\ m^{3}/s$) while also avoiding flooding. The ability to achieve Scenario gamma’s objective is quantified via its performance, $P$, in the equation below. As seen from the equation, flooding constitutes a much greater penalty than threshold exceedance. Thus, it is presumed a control algorithm will permit threshold exceedance when necessary to avoid flooding. $\begin{aligned} P_{i}(t)&=\begin{cases}0.0,&Q_{i}(t)\leq 0.11\quad and\quad F_{i}(t)=0\\\ Q_{i}(t)-0.11,&Q_{i}(t)>0.11\quad and\quad F_{i}(t)=0\\\ 10^{6},&F_{i}(t)>0\end{cases}\\\ P&=\sum^{11}_{i=1}\sum^{T}_{t=0}P_{i}(t)\end{aligned}$ where $Q_{i}(t)$ and $F_{i}(t)$ represent the outflow and flooding, respectively, at the $i^{th}$ node of the network for time $t$.
Additional Notes |  Interactive example: pystorms.org/#ipynb/GoogleColab/gamma

###  SI-4.. Scenario delta

Figure S4. Topology for the residential network of Scenario delta. Orange circles and blue lines in the figure represent, respectively, the controllable storage elements and channels draining the outflow (black triangle) of the subnetwork. During a storm event, each storage element in the network receives runoff generated by connected subcatchments, in addition to discharge from upstream storage elements. The system’s response to the given storm event for the uncontrolled case (i.e. all 5 of its valves set to 100% open) is depicted for each timestep of the simulation by plotting the depths of the basins against their operational bounds and exceedence thresholds, as well as the subnetwork’s outflow against its exceedance threshold. For this scenario, the controller is penalized for any depths and flows outside of their thresholds, as well as for any network flooding. Table S4. Scenario delta Configuration Network Topology |  Scenario delta is a subnetwork of a broader combined urban stormwater network. It is a in a residential neighborhood covering approximately $\approx 2.5\ km^{2}$. Scenario delta’s stormwater network is comprised of 6 storage basins: 5 basins connected in series, 4 of which have controllable outlet valves (North Basin 1, North Basin 2, North Basin 3, and Central Basin), and the furthest downstream storage basin (Central Basin) draining into the broader stormwater network through the outfall conduit. Additionally, the Central Basin connects to an infiltration basin (South Basin); the South Basic is equipped with a valve controlling its inflow from the Central Basin.
While all of these basins are part of the broader network’s stormwater
control, five of them also serve as waterfront to homes in the community
(North Basin 1, North Basin 2, North Basin 3, North Basin 4, and Central
Basin). Thus, it is desired to maintain the water level at these basins within
a pre-specified range. (Note: As North Basin 4 drains directly into North
Basin 3 without an active control mechanism, we are only concerned with
monitoring the water level in North Basin 3.)
The stormwater network topology for Scenario delta can be seen in Figure S4.
---|---
Event Driver |  Scenario delta is driven by an observed storm event (13-hr, 0.93-in) that has an approximate return period of 2-months. It is assumed the entire subcatchment experiences uniform rainfall. Additionally, the conduit connecting the subnetwork to the broader stormwater system experiences constant wastewater flow.
Controllable Assets |  Five of Scenario delta’s storage basins are control assets equipped with valves adjustable to a setting within a range of (0% to 100%) during the simulation. The South Basin is an infiltration basin, thus, the valve controls flow into the basin. Otherwise, the valves of the other four basins (i.e. North Basin 1, North Basin 2, North Basin 3, and Central Basin) control the flow out of each basin. Each basin’s valve is initially set to 100% open. The basin furthest upstream (i.e. North Basin 4) is not controllable and instead flows directly into North Basin 3. In the computational implementation of Scenario delta, each of the controllable valves are identified as weir_N3, weir_N2, weir_N1, weir_C, and orifice_S.
Observable States |  The observable states for Scenario delta are the water levels at the storage basins with controllable valves (i.e., basin_N1, basin_N2, basin_N3, basin_C, and basin_S), and can be queried throughout the simulation to inform any control decisions.
Control Objectives |  The primary objective for Scenario delta is to maintain the depths at the five storage ponds within corresponding inner thresholds. Additionally, any control strategy should avoid flooding and maintain a network outflow below a flowrate threshold. The performance is quantified during each time step of the simulation with penalties increasing exponentially the greater the depth exceeds its predefined operational bounds, and a maximum penalty for depth’s at basin’s passing their given exceedence thresholds. The ability to achieve Scenario delta’s objective is quantified via its performance, $P$, in the following equation: $\begin{aligned} P=\sum^{T}_{t=0}\left(\sum^{5}_{i=1}P_{i}(t)+\sum^{9}_{j=1}P_{j}(t)+P_{\mathrm{out}}(t)\right)\end{aligned}$ with penalties for the depth at each of the basins computed with $\begin{aligned} P_{i}(t)&=\begin{dcases}0.0,&O_{l}\leq d_{i}\leq O_{u}\\\ \left[\left(10^{6}+1\right)^{\left(\frac{O_{l}-d_{i}}{O_{l}-E_{l}}\right)}\right]-1,&E_{l}\leq d_{i}\leq O_{l}\\\ \left[\left(10^{6}+1\right)^{\left(\frac{d_{i}-O_{u}}{E_{u}-O_{u}}\right)}\right]-1,&O_{u}\leq d_{i}\leq E_{u}\\\ 10^{6},&d_{i}<E_{l}\quad or\quad d_{i}>E_{u}\end{dcases}\end{aligned}$ for each basin $i$ in <basin_N3, basin_N2, basin_N1, basin_C, basin_S>, and where $O$ is the corresponding operational bound and $E$ is the exceedence threshold defined for each basin; penalties from flooding computed as $\begin{aligned} P_{j}(t)&=\begin{dcases}0.0,&F_{j}=0\\\ 10^{6},&F_{j}>0\end{dcases}\end{aligned}$ for $j$ in the following set of network nodes: <junc_N4sc, basin_N4, junc_N3sc, basin_N3, junc_N2sc, basin_N2, junc_N1sc, basin_N1, junc_Csc, basin_C, junc_Ssc, basin_S, junc_EinflowA >; and penalties for exceeding the outflow threshold computed as $\begin{aligned} P_{\mathrm{out}}(t)=\begin{dcases}0.0,&Q_{\mathrm{out}}\leq 12.0\\\ \left(1-\dfrac{12.0}{Q_{\mathrm{out}}}\right)^{6},&Q_{\mathrm{out}}>12.0\end{dcases}\end{aligned}$ at the network’s outflow conduit_Eout. The upper and lower operational bounds and exceedence thresholds for each basin are delineated in the following table (Note: the South Basin does not have a lower bound/threshold): Operational bound (O) Exceedence threshold (E) Lower limit, $O_{l}$ Upper limit, $O_{u}$ Lower limit, $E_{l}$ Upper limit, $E_{u}$ North Basin 3 $5.32$ $5.92$ $5.28$ $11.99$ North Basin 2 $4.44$ $5.04$ $4.04$ $6.59$ North Basin 1 $5.20$ $5.80$ $2.11$ $5.92$ Central Basin $3.28$ $3.80$ $2.21$ $5.70$ South Basin — $6.55$ — $9.55$
Additional Notes |  Interactive example: pystorms.org/#ipynb/GoogleColab/delta

###  SI-5.. Scenario epsilon

Figure S5. Topology for the separated combined sewer network of Scenario epsilon. Orange circles and blue lines in the figure represent respectively the controllable storage elements and channels draining into the outlet (black triangle). During a storm event, each node in the network receives runoff generated by connected subcatchments in addition to discharge from upstream nodes. The system’s response to the given storm event for the uncontrolled case (i.e. all 11 of its in-line storage dams are set to 100% open) is depicted via plotting the outlet’s TSS load against the target exceedance threshold for each timestep of the simulation. For this scenario, the controller is penalized for any TSS load above given exceedance thresholds for the downstream outlet. Table S5. Scenario epsilon Configuration Network Topology |  Scenario epsilon is a combined sewer system in an urbanized watershed of $\approx 67\ km^{2}$. Scenario epsilon’s network is comprised of 11 in-line storage dams, connected in both series and parallel, that discharge to the downstream water resource recovery facility (WRRF). The stormwater network topology for Scenario epsilon can be seen in Figure S5.
---|---
Event Driver |  Scenario epsilon is driven by a 4- day period of measured precipitation data collected from 7 rain gauges located across the network. Diurnal wastewater inflows are also included at the receiving nodes for each subcatchment. The magnitude of these inflows is proportional to the area of each subcatchment.
Controllable Assets |  Each of Scenario epsilon’s in-line storage dams are control assets modeled as weirs with adjustable settings within a range of (0% to 100%) during the simulation. The percentage of each weir corresponds to the percent opening of the corresponding storage dam, with all valves initially set to 100% open. In the computational implementation of Scenario epsilon, each of the controllable valves are identified with the pattern ISD<storage asset ID> (e.g. ISD001, ISD002, …ISD011).
Observable States |  The observable states for Scenario epsilon are the outflow, water levels, and TSS concentrations for all in-line storage dams in the network, which can be queried throughout the simulation to inform any control decisions. The outflow and TSS concentration at the outlet of the network are observable as well.
Control Objectives |  The primary objective for Scenario epsilon is to maintain the TSS load at the outlet of the sewer network below a given threshold ($1.05\ kg/s$), which is the maximum dry-weather TSS load observed, while also avoiding flooding. The ability to achieve Scenario epsilon’s objective is quantified via its performance, $P$, in the equation below. As seen from the equation, flooding constitutes a much greater penalty than threshold exceedance. Thus, it is presumed a control algorithm will permit threshold exceedance when necessary to avoid flooding. $\begin{aligned} P(t)&=\begin{cases}0.0,&TSS(t)\leq 1.05\quad and\quad F(t)=0\\\ TSS(t)-1.05,&TSS(t)>1.05\quad and\quad F(t)=0\\\ 10^{6},&F(t)>0\end{cases}\\\ P&=\sum^{T}_{t=0}P(t)\end{aligned}$ where $TSS(t)$ and $F(t)$ represent the TSS load at the network outlet and overall network flooding, respectively, for time $t$.
Additional Notes |  Interactive example: pystorms.org/#ipynb/GoogleColab/epsilon

###  SI-6.. Scenario theta

Figure S6. Topology for the separated stormwater network of Scenario theta. Orange circles and blue lines in the figure represent respectively the controllable storage elements and channels draining into the outlet (black triangle). During a storm event, each node in the network receives runoff generated by connected sub-catchments. The system’s response to the given storm event for the uncontrolled case (i.e. all 2 of its valves set to 100% open) is depicted via plotting the outlet’s flows against the target exceedance threshold for each timestep of the simulation. For this scenario, the controller is penalized for any flow rates above given exceedance threshold at the network outlet and causing flooding in the network.  Table S6. Scenario theta Configuration Network Topology |  Scenario theta is a simple network is designed for unit testing of control algorithms. It consists of two storage basins with a controllable outlet that discharges to an outfall. The stormwater network topology for Scenario theta can be seen in Figure. S6.
---|---
Event Driver |  This scenario is driven by synthetic storm event of 3 inches per hour peak intensity and distributed uniformly throughout the entire subcatchment. The uncontrolled response of the system at the outlet is illustrated in the Figure. S6.
Controllable Assets |  The two tanks in this scenario are equipped with 1 $m^{2}$ square orifice. These orifice’s opening percentage (e.g., 10%, 73%) can be adjusted throughout the duration of the simulation. These orifices are initially set to 100% open. In the computational implementation of Scenario theta, these control assets are identified as 1 and 2.
Observable States |  The observable states for Scenario theta are the outflow $[m^{3}/s]$, water level $[m]$, and valve setting $[\%]$ for the tank and orifice elements in network, which can be queried throughout the simulation to inform any control decisions.
Control Objectives |  Control objective Scenario theta is to maintain the outflows at the outlet below the desired threshold ($0.5m^{3}/s$) while also avoiding flooding. The ability to achieve Scenario theta’s objective is qualified via its performance metric, $P$, summarized below. As seen in the equation, flooding constitutes a much greater penalty than threshold exceedance. $\begin{aligned} P_{i}(t)&=\begin{cases}0.0,&Q_{i}(t)\leq 0.50\quad and\quad F_{i}(t)=0\\\ Q_{i}(t)-0.50,&Q_{i}(t)>0.50\quad and\quad F_{i}(t)=0\\\ 10^{6},&F_{i}(t)>0\end{cases}\\\ P&=\sum^{2}_{i=1}\sum^{T}_{t=0}P_{i}(t)\end{aligned}$ Where $Q_{i}(t)$ and $F_{i}(t)$ represents the outflow and flooding, respectively, at the $i^{th}$ node of the network for time $t$.
Additional Notes |  Interactive example: pystorms.org/#ipynb/GoogleColab/theta

###  SI-7.. Scenario zeta

Figure S7. Topology for the combined network of Scenario zeta. Orange circles and blue lines in the figure represent, respectively, the controllable storage elements and channels draining the outflow (black triangle) of the subnetwork. During a storm event, each storage element in the network receives runoff generated by connected subcatchments, in addition to discharge from upstream storage elements. The system’s response to the given storm event for the uncontrolled case (i.e. all 4 of its valves set to 100% open) is depicted for each timestep of the simulation by plotting the combined sewer overflow into the receiving creek and river, and the network outflow to the downstream wastewater treatment plant. For this scenario, the controller is penalized for any combined sewer overflow events, and rewarded for the outflow volume to the wastewater treatment plant. Table S7. Scenario zeta Configuration Network Topology |  Scenario zeta is a combined system with ten subcatchments covering an area of $\approx 1.8\,\mathrm{km^{2}}$, with six storage tanks (a total storage volume of $5900\,\mathrm{m^{3}}$) connected both in series and parallel, and discharging to the downstream wastewater treatment plant. Within the network, there are ten combined sewer overflow (CSO) structures (one at each storage basin, plus an additional four locations), which drain into two receiving bodies: a river and a creek. The combined sewer overflows of Tank 1, Tank 2, Tank 3, Tank 4, Tank 5, CSO8, and CSO10 overflow into the larger river, while Tank 6, CSO7, and CSO9 overflow into the smaller, more sensitive creek which flows through a local park. Of the six storage basins, four are equipped with controllable outlet valves to utilize system storage and minimize the volume and number of combined sewer overflow events (Tank 2, Tank 3, Tank 4, and Tank 6).
Scenario zeta is based on the SWMM implementation of the Astlingen Real-Time
Control Benchmarking Network (Sun et al., 2020). The topology of Scenario
zeta, specifically the controllable tanks, can be seen in Figure S7.
---|---
Event Driver |  Rainfall in Scenario zeta is spatially heterogeneous, with four rainfall gauges connected to the ten subcatchments. Precipitation for the original Astlingen network from Sun et al. (2020) was for over a 10-year period. However, Scenario zeta uses one precipitation event from this 10-year data as its event driver. Specifically, Scenario zeta is driven by an 3-day precipitation event, with the majority of precipitation occurring over a 12-hr period, averaging $53.06\,\mathrm{mm}$ of rainfall across four rain gages, characterizing this event as a 17-year storm. (Note, this event was characterized using annual recurrence interval and rainfall duration data for a similar geographic location presented in Ramke (2018)).
Controllable Assets |  Four of Scenario zeta’s storage tanks are equipped with valves at their outlets and adjustable to a setting within a range of (0% to 100%) during the simulation. The controllable valves are implemented as orifices and are identified as <V2, V3, V4, V6>, corresponding to tanks <T2, T3, T4, T6>, respectively.
Observable States |  The observable states for Scenario zeta are the water levels at the six storage tanks: <T1, T2, T3, T4, T5, T6>, and can be queried throughout the simulation to inform any control decisions.
Control Objectives |  The primary objective for Scenario zeta is to minimize combined sewer overflow into the receiving river and creek, and maximize the downstream flow to the wastewater treatment plant. As the creek is more sensitive to overflows than the river, it is desirable to minimize overflow to the creek over that of the river. Additionally, there is a desire to minimize the roughness of control actions. The ability to achieve Scenario zeta’s objective is quantified via its performance, $P$, in the following equation: $\begin{aligned} P=\sum^{T}_{t=0}\left[\left(\sum^{7}_{i=1}V_{\mathrm{CSO}_{i}}(t)\right)+2\left(\sum^{3}_{j=1}V_{\mathrm{CSO}_{j}}(t)\right)+0.01\left(\sum^{6}_{k=1}V_{\mathrm{throttle}_{k}}(t)\right)-0.1V_{\mathrm{WWTP}}(t)\right]\end{aligned}$ where $V_{\mathrm{CSO}_{i}}(t)$ is the CSO volume at the $i$ locations of <T1, T2, T3, T4, T5, CSO8, CSO10> that overflow into the river; $V_{\mathrm{CSO}_{j}}(t)$ is the CSO volume at the $j$ locations of <T6, CSO7, CSO9> that overflow into the creek; $V_{\mathrm{throttle}_{k}}(t)$ is the change volume change in flow at each of the controlled and uncontrolled $k$ tank outlets <V1, V2, V3, V4, V5, V6>; and $V_{\mathrm{WWTP}}(t)$ is the volume of wastewater flowing to the downstream wastewater treatment plant, all over timestep $t$.
Additional Notes |  As stated above, Scenario zeta is based on a SWMM implementation of the Astlingen Real-Time Control Benchmarking Network, which was originally developed by the Integral Real-Time Control working group of the German Water Association for the purpose of comparing control strategies applied to the network (Sun et al., 2020). Three different control strategies were presented and tested on the original SWMM implementation of the Astlingen network. We implement two of these control strategies—a rule-based and equal-filling degree approach—to Scenario zeta in the following interactive example: pystorms.org/#ipynb/GoogleColab/zeta.