# Machine learning-based patient selection in an emergency department

Furian, Nikolaus
Institute of Engineering and Business Informatics
Graz University of Technology
Graz, Austria
<EMAIL_ADDRESS>
&O’Sullivan, Michael
Department of Engineering Science
University of Auckland
Auckland, New Zealand
<EMAIL_ADDRESS>
&Walker, Cameron
Department of Engineering Science
University of Auckland
Auckland, New Zealand
<EMAIL_ADDRESS>
&Reuter-Oppermann, Melanie
Information Systems
Technical University of Darmstadt
Darmstadt, Germany
<EMAIL_ADDRESS>

###### Abstract

The performance of Emergency Departments is of great importance for any health
care system, as they serve as the entry point for many patients. However,
among other factors, the variability of patient acuity levels and
corresponding treatment requirements of patients visiting EDs imposes
significant challenges on decision makers. Balancing waiting times of patients
to be first seen by a physician with the overall length of stay over all
acuity levels is crucial to maintain an acceptable level of operational
performance for all patients. To address those requirements when assigning
idle resources to patients, several methods have been proposed in the past,
including the Accumulated Priority Queuing (APQ) method. The APQ method
linearly assigns priority scores to patients with respect to their time in the
system and acuity level. Hence, selection decisions are based on a simple
system representation that is used as an input for a selection function. This
paper investigates the potential of an Machine Learning (ML) based patient
selection method. It assumes that for a large set of training data, including
a multitude of different system states, (near) optimal assignments can be
computed by a (heuristic) optimizer, with respect to a chosen performance
metric, and aims to imitate such optimal behavior when applied to new
situations. Thereby, it incorporates a comprehensive state representation of
the system and a complex non-linear selection function. The motivation for the
proposed approach is that high quality selection decisions may depend on a
variety of factors describing the current state of the ED, not limited to
waiting times, which can be captured and utilized by the ML model. Results
show that the proposed method significantly outperforms the APQ method for a
majority of evaluated settings.

_K_ eywords OR in health services $\cdot$ Emergency Department $\cdot$ Machine
Learning $\cdot$ Accumulated Priority Queuing

##  1 Introduction

Emergency Departments (EDs) play a crucial role in health care systems
worldwide, as they serve as the entry point for many patients, witnessing
highly stochastic patient arrival patterns while operating 24 hours each day
of the year. However, many health care providers face major issues regarding
the operational performance of EDs, such as overcrowding, diversion, or long
and imbalanced waiting times for patients with varying priorities. The reasons
for those inefficiencies are manifold and include among others: budget
restrictions; the aging of populations; insufficient resources; or non urgent
patient visits ( [32, 50]).

As a consequence, improving ED operations by the design of new care delivery
processes or adequate allocation of resources has gained significant interest
for both health care stakeholders and researchers, over the last decades [22].
Proposed approaches include: virtual streams [59]; fast track units [53]; or
advanced patient and resource selection methods such as Accumulated Priority
Queuing (APQ) [60, 19, 14]. Modeling and simulation has been extensively used
by decision makers to evaluate the performance of these methods within a
virtual abstraction of the system, before actually implementing them in a real
world ED.

In particular, some recent studies, e.g. [14] or [19], argue that APQ is a
technique capable of improving the performance of EDs significantly. By the
classification of Furian et al. [22], APQ is a patient selection policy, i.e.
a method to select the next patient from a pool of waiting patients to be
assigned to an idle resource, e.g. a doctor, for treatment or consultation.
The main aim of the APQ method is to balance performance measures for patients
belonging to different priority categories and thereby to improve an overall
objective function, or measure, representing the operational performance of
the entire ED. While traditional queuing policies for EDs select patients
purely by their priorities (i.e. high priority patients are always seen before
low priority patients), and type of consultation (i.e. first or subsequent
visit of doctors), APQ further considers the time patients are already present
in the ED. Therefore, patients are separated in virtual queues of priority
grades and type of consultation and each queue is assigned a (fixed) weight
parameter. When selecting a patient, a score value is computed for each
patient being at the head of one of the queues. The score is obtained by the
product of the weight parameter of the corresponding queue and the time the
patient already spent in the system. The patient with the highest score is
then seen next by an idle resource.

In its essence, APQ can be abstracted in the following way. Whenever a
resource can be dispatched, a simple representation of the current state of
the ED is computed and passed to a mathematical function for selecting a
waiting patient. The time spent in the system by a patient that is currently
at the head of a virtual queue defines the system’s state, and the function
applies simple linear weight functions to each of those patients. Therefore,
the weight parameters for queues are chosen, e.g. via simulation optimization,
such that a pre-defined objective function, accounting for the performance of
the studied ED, is optimized over a training data set.

While the simplicity of the approach seems to be beneficial for practical
reasons, the question whether more comprehensive state representations and
complex selection functions may further improve the performance of an ED has
not been considered yet. In other words, can considering additional
information on the state of the ED (such as queue lengths, time of the day,
number of patients that are close to target times, etc.) and possibly non-
linear functions mapping those states to selection decisions result in
improved performance measures?

In theory, for every possible state of a specific ED there is at least one
optimal decision to select a patient to be seen. Obviously, these optimal
decision are not only dependent on the state of the ED at the time of decision
making, but also on future events (e.g. arrival of new patients, finishing of
ongoing consultations and treatments, future requirements of patients, etc.)
that may occur in the system. As systems like EDs are of a highly stochastic
nature, it is impossible to know, or even accurately predict, those future
events at the time of decision making. However, having a virtual abstraction
of the system, i.e. a model including all stochastic aspects of the ED, allows
for the generation (or sampling) of (in theory) unlimited realizations of the
ED for a chosen period of time. For example, it is possible to sample the
arrival process of patients, e.g. the realization of a stochastic process, and
treatment requirements and times, e.g. the realization of random variables, of
all patients arriving at the ED during a single day. For such a realization,
from now on referred to as an instance, patient related information could be
treated as deterministic data.

Further, assuming to have perfect and full information on such an instance
allows to treat the patient selection problem as a scheduling problem. In
other words, it is possible to compute an optimal plan with respect to a
chosen objective function, or schedule, determining at what time which patient
is seen by which resource, considering all patient requirements and events
throughout the entire time span covered by the instance. Obviously, such a
single optimal schedule is of no practical use, as perfect and complete
information on future events will never be available in a real world setting.
However, computing optimal solutions for a large set of instances for a
specific ED enables the definition, fitting and use of complex functions to
select patients at a given point in time - an approach that is analogous to
the use of simulation optimization to find optimal weights for the APQ policy.

The Machine Learning (ML) based patient selection method presented in this
paper uses such optimal plans for a set of training instances to fit a (non-
linear) ML model to learn optimal selection decisions with respect to a
comprehensive state representation of the ED. In other words, given a state of
the ED it predicts the selection of a patient that potentially leads to an
optimal, or at least high quality, decision. Thereby, the transformation of
optimal solutions for training instances to an ML model may be seen as
building a computational “experience” (or knowledge base) that enables the
selection of patients with respect to the state of the ED at the time of
decision making, and implicitly possible future events in the system.

A discrete event simulation model of a typical ED is proposed in order to
generate training instances, evaluate the approach on new unseen instances and
compare it with results obtained by standard methods from literature, e.g. APQ
patient selection. The model covers most relevant aspects of ED operations and
includes stochastic elements (e.g. stochastic processes for patient streams,
stochastic treatment times, etc.) to account for the dynamic nature of EDs.
Results show that the ML based patient selection method significantly
outperforms standard approaches for a majority of proposed scenarios and
definitions of objective functions.

The paper is structured as follows: in the next section relevant literature
and the contribution of the paper are outlined. In Section 3, the simulation
model used in this study is introduced, a generic and parameterizable
objective function is defined, and standard patient selection methods are
described. Section 4 introduces the ML-based patient selection method. Input
parameters and scenarios are defined in Section 5. Results are presented in
Section 6. The paper concludes with final remarks and suggestions for further
research in Section 7.

##  2 Related Work and Contribution

The analysis and improvement of ED operations has been addressed by many
researchers in the past decades. Thereby, modeling and simulation has been
extensively used as a decision support tool by researchers and stakeholders.
As a consequence, there exists a vast number of publications on modeling of ED
systems. For comprehensive reviews on ED modeling and simulation, the reader
is referred to [58] and [28]. A survey on simulation models dealing with
overcrowding effects in EDs is given by [54].

In a previous paper, the authors propose a guideline for ED modeling, that is
based on generic building blocks of ED operations, see [22]. In addition, a
variety of patient selection methods integrated in, and evaluated by, ED
simulation models are identified.

A majority of proposed models select patients by a purely priority-based
policy. In particular, patients are selected for consultation and/or treatment
based on their previously assigned triage grades. Patients with equivalent
triage grades are then selected based on the ‘first-in-first-out” (FIFO)
principle. Examples for such selection policies are given, among others, by
[3, 4, 6, 52, 18, 46].

A common extension to purely priority based selection policies is to further
classify patients with respect to the number of times they have been seen by a
doctor, i.e. new vs. old patients, and define rules considering those
classifications. Such approaches were proposed by [13, 18, 26, 59, 61, 64,
69].

In addition to the number of visits, [59] classify patients into virtual
streams with respect to an estimation of a patient being discharged or
admitted to the hospital at the end of the stay in the ED. The reasoning
behind this policy is that conflicting performance measures for patients in
either class may be of interest. Whereas for patients being discharged at the
end of their stay in the ED, the Length of Stay (LOS) is mainly of interest,
the Time to Doctor (TTD) may be more important for patients with a high
probability of being admitted to the hospital.

The work of [17] includes also non-ED patients being seen in diagnostic
facilities, and proposes a triage based selection policy for both patients
categories.

Less work has been published on policies that include resource related
information in patient selection policies, referred to as “combined patient
and resource selection” by [22]. For example, [15] introduce a policy that
aims to balance the share of currently seen high and low priority patients for
each doctor individually. A policy including skill and/or training levels of
doctors when selecting a patient is proposed by [30].

A two step patient selection policy based on a queuing network formulation
(rather than a simulation model) of an ED is introduced by [35]. They classify
patients between triage patient (TP) (i.e. first contact with a medical staff
member) and patients in process (IP). Whenever a physician becomes idle, it is
first determined whether an IP or TP patient is seen. This first step always
selects TPs, if any exist and are close to reach a predefined time limit for
getting triaged. In the second step, given that such TPs exist, the patient
whose time in the system is closest to the limit is selected. If a TP is to be
selected in the second step, the policy uses a modified $c\mu$-rule to decide
which patient to see next.

A queuing principle for ED systems that recently gained significant attention
is APQ. For a general discussion of the APQ method, i.e. without a focus on ED
operations, the reader is referred to [38] or [60]. However, those general
models do not include common features of EDs, such as patients being seen
multiple times by doctors or time-dependent arrival rates of patients. [30]
were among the first to propose a policy for ED patient selection that can be
considered to follow an APQ principle. They assign patients an “operating
priority” that is composed by a waiting time component (that increases over
time) and a “clinical priority” (similar to triage grades). [49] state that
patients are prioritized with respect to a function of triage categories and
waiting times that increases the priority the waiting time of a patient is
close to target. However, they do not exactly specify the function that is
used. Dynamically increasing priorities of patients are also proposed by [61].
In particular, they define three priority functions. The first assigns
priorities based on an exponential function of the estimated consultation
times of patients. The second uses an exponential function on the remaining
time of a patient until a LOS target is met, and a linear weight function when
for patients that already missed the target. The third policy combines those
by a linear weighted sum.

In a more recent study, [19] discuss the performance of the APQ method by a
simulation model based on a real world ED. The authors show that APQ
outperforms other strategies, such as fast track units or virtual streaming,
to balance Key Performance Indicator (KPI) over patients with different triage
grades. However, their results are based on the LOS as a performance measure,
and hence do not include an investigation on how APQ performs with respect to
target related KPIs.

The work of [14] addresses this shortcoming, and defines a performance measure
based on a weighted sum of the total waiting time of patients, and the share
of patients meeting a TTD target time. Besides a strictly linear APQ model,
the authors also propose a so called APQ-h model that prevents priorities to
increase over time as soon as the TTD target time is met by a patient.
However, they conclude that the difference in performance of both policies is
not significant for tested scenarios. They evaluate the APQ under a variety of
settings, including different patient mixes, arrival rates and resource
utilization rates.

The use or application of ML methods and tools to analyse and/or improve
health care systems has gained increased attention by practitioners and the
scientific community. Reuter-Oppermann and Kühl review publications that use
machine learning approaches either to directly address planning problems in
healthcare logistics or to derive input for these [57]. They state that a
significant share of publications has targeted emergency departments. To the
best of our knowledge, only few papers have used machine learning within ED
optimisation or simulation. Laskowski, for example, combines an agent based
model with a machine learning approach to build a hybrid system for decision
support in order to improve processes in an emergency department [42].
Ceglowski et al. integrate a data-mining approach into a discrete-event
simulation in order to analyse ED processes and identify bottlenecks in the
interface between the ED and the hospital wards [12].

Significantly more publications have used machine learning approaches to
predict arrival rates [1, 2, 48, 68], patient admissions [27, 40, 44], lengths
of stay [8, 10] or patient re-admissions [62, 71, 43, 25, 56, 16, 67].

Menke et al. and Xu et al. for example use artificial neural networks (ANN) to
predict the daily patient volumes arriving at an ED [48, 68], while Xu et al.
compare the ANN with the nonlinear least square regression (NLLSR) and the
multiple linear regression (MLR) [68]. Afilal et al. apply time-series based
forecasting models to predict longterm as well as short term arrival rates
[2].

Graham et al. compare logistic regression, decision trees and gradient boosted
machines (GBM) to predict which patients are admitted into the hospital from
the ED [27]. Krämer et al. use supervised machine learning techniques to
classify hospital admissions into urgent and elective care [40], while Lee et
al. [44] distinguish four admission classes, i.e. intensive care unit,
telemetry unit, general practice unit, and observation unit, for the ML-based
classification approaches.

Patient readmissions are addressed by Turgeman and May with a support vector
machine (SVM) [62] and by Zheng et al. with classification models that use
neural networks, random forest and support vector machines, with a specific
focus on heart failure patients [71]. Lee et al. have developed a clinical
tool based on machine learning for predicting patients who will return within
72 hours to the pediatric emergency department [43]. Futoma et al. compare
predictive and deep learning models for 30 day re-admissions [25].

The integration of classical OR methods, such as optimization and simulation,
with ML models, has gained significant interest within the scientific
community in recent years. For a comprehensive guide on the combined use of
those methodologies, the reader is referred to [9]. Due to the novelty of the
field, most published studies focus on classical combinatorial optimization
problems, such as the Traveling Salesmen problem, e.g. [7] and [36], variants
of the vehicle routing problem, e.g. [51] and [23], or the facility location
problem, e.g. [47]. However, some studies propose the integrated use of
classical optimization with ML models in more practical settings, e.g. [34]
introduce a learning assisted tree search for a container pre-marshalling
problem and [20] predict optimal objective values of an offshore wind-park
production optimization problem.

In general, [9] consider three ways to integrate ML into optimization tasks:
(1) end-to-end learning, i.e. heuristically computing solutions for the
optimization problem via an ML model; (2) using ML models to to gather
information on the problem at hand in a pre-processing step and pass that
information on to a classical optimization algorithm; and (3) utilize ML
models to to make online heuristic decisions within classical optimization
algorithms.

The proposed ML-based patient selection policy does not strictly fall in any
of those categories. Although the method uses an ML model to imitate optimal
behavior, i.e. optimized schedules of patients, and hence may be categorized
as end-to-end learning, it is used as a policy rather than an algorithm. In
particular, one does not solve an optimization problem in the classical sense
when selecting a patient for consultation.

To the best knowledge of the authors, the presented approach is the first
using ML models to select patients for consultation/treatment in a healthcare
setting, and in particular for EDs. While previously published methods rely on
relatively simple state representations of an EDs, our method includes a
comprehensive abstraction of an ED’s state for decision making. Further, the
use of deterministic optimization results as a knowledge base for deriving
selection policies has not been considered in previous research.

##  3 The ED Setting

The ED model proposed in this paper is not the abstraction of a specific real
world ED, but is rather intended to be a representation of “typical” ED
operations. In a previous study [22], the authors analyzed and classified a
wide range of published ED models and defined archetype building blocks of the
latter. The presented model is a composition of the most commonly used
building blocks, balancing implied simplifications and assumptions with the
requirement to represent generic and “typical” ED processes and resource
definitions. For example, the register and triage process of the proposed
model has been used by 18 out of 48 models analyzed in [22]. The patient
treatment processes in the model are a realizations of the generic
$P_{P,0}[A_{A,0},D_{D,0}]T_{T,0}$ process classified by [22] that covers about
60% of analyzed models. For detailed information on the nomenclature the
interested reader is referred to [22]. In the following, the components of the
model are outlined in detail.

Definition | Description
---|---
General Definitions
$J=\\{2,3,4,5\\}$ | Set representing triage grades
$P=\bigcup\limits_{j\in J}P_{j}$ | Set of patients, partitioned over triage grades
$p_{j}$ | Probability of a patient being of class $P_{j}$
$pd_{j}$ | Probability of a patient of class $j$ is requiring diagnostic tests
$pa_{j}$ | Probability of a patient of class $j$ is arriving via ambulance
$pw_{j}$ | Importance weight associated with patient class $j$
$D$ | Set of doctors
$\lambda_{t}$ | Arrival rate of patients at time $t$ of a given day
General Time Related Definitions
$t_{a}^{i},i\in P$ | Arrival time of patient $i$
$t_{l}^{i},i\in P$ | Discharging time of patient $i$
$t_{fc}^{i},i\in P$ | Start time of first consultation of patient $i$
$t_{sc}^{i},i\in P$ | Start time of second consultation of patient $i$
$ttd^{i}=t_{fc}^{i}-t_{a}^{i},i\in P$ | Time to doctor of patient $i$
$los^{i}=t_{l}^{i}-t_{a}^{i},i\in P$ | Length of stay of patient $i$
$t_{ttd}^{j},j\in J$ | Maximum time targets (in minutes) for $ttd$ over triage grades
$ts_{ttd}^{j},j\in J$ | Target share of patients that should meet TTD target times
$t_{sw_{3}}^{i},i\in P$ | Start of waiting time $w_{3}$ of patient $i$
$t_{sw_{4}}^{i},i\in P$ | Start of waiting time $w_{4}$ of patient $i$
Waiting Time Related Definitions
$w_{1}^{i},i\in P$ | Waiting time for register of patient $i$
$w_{2}^{i},i\in P$ | Waiting time for triage of patient $i$
$w_{3}^{i},i\in P$ | Waiting time for first consultation of patient $i$
$w_{4}^{i},i\in P$ | Waiting time for second consultation of patient $i$
$W_{1}^{i},i\in P$ | Total waiting time until first consultation of patient $i$
$W^{i},i\in P$ | Total waiting time of patient $i$
Queuing Related Definitions
$Q_{j,1},j\in J$ | patients of category $j$ waiting for the first consultation
$Q_{j,2}^{d},j\in J,d\in D$ | patients of category $j$ waiting for the second consultation with doctor $d$
Table 1: Summary of Definitions

###  3.1 Patient Categories and Processes

A standard method to categorize patients are triage grades which reflect the
severity of a patient’s condition upon arrival at the ED. Most commonly used
triage grading systems include the Emergency Severity Index (ESI) and the
Manchester Triage Scale (MTS). Both consist of 5 triage levels, ranging from
"immediately life threatening" to "not urgent" conditions of patients. Note,
that the ESI not only accounts for a patient’s condition, but also for the
expected amount of resources required. The presented model is also based on a
5 level triage grading system, where triage grade 1 is assigned to
resuscitation patients and triage grade 5 to the least urgent patients.

Since, resuscitation patients are usually admitted to trauma bays or
resuscitation facilities, and hence do not compete for resources with other
patients, the included triage grades range from 2 (high priority) to 5 (low
priority), similar to [14].

Besides triage grades, the mode of arrival is another criteria to classify
patients. Patients either arrive by ambulance or are classified as “walk-in”
patients, i.e. arriving by their own means. However, we assume that the model
of arrival affects the care process only for patients with triage grade 2.
Those are assumed to be triaged on the ambulance and therefore skip the first
process steps, as will be described below.

Upon arrival, patients wait to be registered at the ED. The registration
itself is followed by a wait for a triage activity. As mentioned above,
category 2 patients arriving by ambulance skip those two activities. Further,
category 2 patients arriving by a private mode of transportation, i.e. walk-in
patients, are assumed to skip the registration step and start their process
with a short triage activity. After triage (or arrival of category 2 patients
by ambulance) patients queue up to be first seen by a doctor, referred to as
“first consultation”.

This first consultation is possibly followed by some diagnostic tests. The
probability of requiring such tests is assumed to be triage category
dependent. However, no explicit resources for performing those tests are
included in the model, i.e. they are modeled as a (stochastic) time delay. We
assume that (if required) diagnostic tests start immediately after the first
consultation, and consequently any possible waiting time is included in the
time delay for testing. Patients not requiring diagnostic tests are assumed to
leave the ED after the first consultation.

After the diagnostic tests, patients queue up for the “second consultation” by
a doctor, which may be seen as the final treatment. The resulting waiting time
(denoted by $w_{4}$) starts immediately after the time delay for testing. It
is important to note that patients are required to be seen by the same doctor
for the second consultation who performed the first consultation. This policy
is often referred to as “same-doctor-policy” and is commonly found in
literature, see for example [59, 26, 39].

Table 1 summarizes relevant formal definitions to describe the system, while
figure 1 illustrates the patient process and important measures.

Figure 1: ED Patient Process

###  3.2 Resources

Patients require a set of resources along their care process. In the presented
model, only resources which patients compete for, and which are not directly
coupled to other resources, are included. For example, patients register with
a register clerk, who is modeled as an explicit resource. A register facility,
such as a registration desk, is not explicitly included, as we assume that
whenever the clerk is available for registration, the associated physical
facility is also available.

Further, we include a set of triage nurses, whose sole responsibility in the
model is to perform triage activities of arriving patients. In particular,
they do not participate in any consultation or test activities. For the first
and second consultation we include a set of doctors (or physicians) $D$ that
perform those activities. As stated in section 3.1, we do not include
resources for diagnostic tests, but rather model them as stochastic time
delays. Such a simplification is commonly used in ED models, see for example
[29, 53, 45, 64].

###  3.3 Performance Metrics

A wide range of performance indicators to evaluate ED operations are used and
reported in literature. Those range from simple time-based, such as LOS or
waiting times, to complex combined measures, such as the National Emergency
Department Crowding Scale (NEDCOS). In a previous paper [22], the authors
classified commonly used KPIs as: time-based, limit-based, state-based,
financial measures, and combined measures. Time-based measures denote for KPIs
that capture time spans along patient processes, such as LOS, TTD, or waiting
times. Limit-based KPIs usually set time-based measures in context with
predefined targets, for example measure the share of patients meeting a TTD
target time. State-based KPIs aggregate system states to a performance
indicator, e.g. resource utilization rates or diversion state rates. Combined
measures, such as the NEDCOS, [65], aggregate a variety of single measures to
an overall KPI.

In this paper, we use different performance measures that can all be
represented by a weighted sum of two individual indicators, i.e. are different
parametrization of a combined KPI. In the following, we will describe the
individual components and define the analytical as well as the empirical
performance metric.

Therefore, let $X_{\tilde{W}}^{j}$ be a random variable representing the total
waiting time for patients of category $j$ that need two consultations and
respectively $X_{\tilde{W}_{1}}^{j}$ be a random variable representing the
total waiting time for patients of category $j$ that leave the ED after the
first consultation. Further, let $x^{j}_{i,ttd}$ be a variable that is $1$ if
patient $i$ of category $j$ meets the TTD target time, i.e. if $ttd^{i}\leq
ttd^{j}$, and $0$ otherwise, and $X_{ttd}^{j}$ its random variable
representation.

The analytical combined measure is then defined by

$f_{ED}(P|\lambda_{W},\lambda_{ttd})=\sum_{j\in
J}p_{j}pw_{j}\left(\lambda_{W}\left((1-pd_{j})E(X_{\tilde{W}_{1}}^{j})+pd_{j}E(X_{\tilde{W}}^{j})\right)+\lambda_{ttd}\max\left(ts^{j}_{ttd}-E(X_{ttd}^{j}),0\right)\right),$
(1)

where $\lambda_{W}$ and $\lambda_{ttd}$ are the corresponding weights of
individual measures. In particular, $f_{ED}$ is a weighted sum of the expected
total waiting time of patients (per category and diagnostic requirement) and
the share of patients that are not meeting the TTD target (per patient
category).

Obviously, the real distributions of $X_{\tilde{W}}^{j}$,
$X_{\tilde{W}_{1}}^{j}$, and $X_{ttd}^{j}$ are not known, but for the
evaluation of a simulation result (or a data record) of $P$ patients, the
following estimator can be used:

$\hat{f}_{ED}(P|\lambda_{W},\lambda_{ttd})=\sum_{j\in
J}p_{j}pw_{j}\left(\lambda_{W}\frac{1}{|P_{j}|}\sum_{i\in
P_{j}}W^{i}+\lambda_{ttd}\left(ts^{j}_{ttd}-\frac{1}{|P_{j}|}\sum_{i\in
P_{j}}x^{j}_{i,ttd}\right)^{+}\right).$ (2)

The weight tuple $(\lambda_{W},\lambda_{ttd})$ is used to generate a set of
measures to evaluate the proposed method. Thereby, the goal is not to define a
universally applicable measure, but to demonstrate the performance of the
approach with respect to a variety of settings and goal functions.

###  3.4 Patient Selection Policies and Constraints

Besides entities and process definitions, the control logic or control
policies determine the content of a model [24]. For the model proposed in this
paper, the control policies are concerned with matching available resources
with patients queuing up for services. Those can be further categorized in
resource, patient and combined selection policies, see [22]. As this study is
mainly concerned with the selection of patients, those polices are described
in detail in the remainder of the section.

In general, patients queue for registration, triage, first and second
consultation. However, as a patient’s triage grade is not known before queuing
up for the first consultation, i.e. before the completion of the triage
activity or the arrival of a patient in case of ambulance arrivals of category
$2$ patients, the queuing policies for registration and triage are modeled by
the “first-in-first-out” (FIFO) principle.

In its most general form, the selection of patients to be seen by a doctor
(for first and second consultation) at a given point in time can be seen as
function that maps the set of available doctors to a sub-set of waiting
patients. Such a function does not necessarily require that the set of waiting
patients is structured by a system of queues. However, due to practical
implications, such as constraints on the patients to be selected and the
priority of patients imposed by the triage grades, most commonly used patient
selection policies are based on a set of virtual queues that structure the set
of waiting patients.

Therefore, in this paper, patients waiting to be seen by a doctor are further
split up in queues per triage category, i.e. $2$-$5$, by the type of
consultation, i.e. first or second, and in case of a second consultation by
the doctor they were seen during the first consultation (due to the “same-
doctor-policy”), similar to [14]. The resulting queuing system of $4+4|D|$
queues is illustrated by figure 2.

Figure 2: ED Queue Structures Corresponding to Triage Grades and Doctors

In the setting illustrated by Figure 2, and if we assume that $d\in D$ is an
idle doctor and $S_{ED}^{t}$ fully describes the state of the ED at time $t$,
a patient selection policy for $d$ may be described by a function

$f_{ps}^{d}:S_{ED}^{t}\rightarrow\\{Q_{2,1},Q_{3,1},Q_{4,1},Q_{5,1},Q_{2,2}^{d},Q_{3,2}^{d},Q_{4,2}^{d},Q_{5,2}^{d}\\}.$
(3)

Note that the above definition of function $f_{ps}^{d}$ implies some
limitations and constraints. First, defining $f_{ps}^{d}$ per doctor implies
that idle doctors are dispatched one at a time, e.g. by a possible ordering of
idle doctors. In particular, we assume that doctors are dispatched in the
order by their idle times, i.e. the doctor being idle the longest is
considered first. This assumption prevents the use of policies based on more
complex resource selection mechanisms, e.g. team selection [31] or dynamic
work load calculation [37]. However, such complex resource selection policies
are beyond the scope of this study and may be included in future research.

Second, the definition of $f_{ps}^{d}$ implies that patients within a specific
queue are chosen with respect to the FIFO principle, i.e. within one queue
always the patient waiting longest is seen first. This means that a selection
policy may let patients with a lower priority “overtake” a patient with a
higher priority, as they are not in the same queue, but within a single queue
“overtaking” is not allowed. Note that due to stochastic registration and
triage durations, the order of patients in a queue $Q_{j,k}$ for $j\in
J,k\in\\{1,2\\}$ may not be the same as the order of arrival of those
patients, i.e. implied by $a_{i}$ for $i\in Q_{j,k}$. Analogously, the order
of patients of triage category $j$ seen by doctor $d$ may be different to the
order in which they were seen for the first consultation due to stochastic
diagnostic times. In particular, the order of patients in $Q_{j,1}$ are
implied by $t_{sw_{3}}^{i}$ and by $t_{sw_{4}}^{i}$ for patients in
$Q_{j,2}^{d}$ for $d\in D$.

In the following, two established versions of function $f_{ps}^{d}$ are
introduced, which serve as a benchmark for the proposed ML based patient
selection method.

####  3.4.1 Basic Queuing Principles

Basic queuing principles, or “pure priority based orderings” [14], define
$f_{ps}^{d}$ in a way that queues are always considered in a fixed (priority
based) ordering, and the first non-empty queue is selected when dispatching an
idle doctor. Hence, at time $t$, a patient in $Q_{j_{1},1}$ will always be
seen before a patient in $Q_{j_{2},1}$ if $j_{1}<j_{2}$, $|Q_{j_{1},1}|>0$ and
$|Q_{j_{2},1}|>0$ (analogously for second consultation queues). Different
versions of basic queuing principle are a result of ordering first and second
consultation queues, often referred to as “prioritizing old vs. new patients”
or vice-versa, see for example [13, 18, 26, 59, 61, 64, 69].

[14] identify 4 different pure priority based strategies based on three
different triage categories, where one is designed to reflect the actual
queuing policy at the ED their study is built on. Based on their definitions
we propose 4 orderings of queues $Q_{j,k}$ for $j\in J,k\in\\{1,2\\}$ that are
summarized in Table 2.

Policy | Ordering
---|---
| 1st | 2nd | 3rd | 4th | 5th | 6th | 7th | 8th
$QP1$ | $Q_{2,1}$ | $Q_{3,1}$ | $Q_{4,1}$ | $Q_{5,1}$ | $Q_{2,2}$ | $Q_{3,2}$ | $Q_{4,2}$ | $Q_{5,2}$
$QP2$ | $Q_{2,1}$ | $Q_{2,2}$ | $Q_{3,1}$ | $Q_{3,2}$ | $Q_{4,1}$ | $Q_{4,2}$ | $Q_{5,1}$ | $Q_{5,2}$
$QP3$ | $Q_{2,2}$ | $Q_{3,2}$ | $Q_{4,2}$ | $Q_{5,2}$ | $Q_{2,1}$ | $Q_{3,1}$ | $Q_{4,1}$ | $Q_{5,1}$
$QP4$ | $Q_{2,2}$ | $Q_{2,1}$ | $Q_{3,2}$ | $Q_{3,1}$ | $Q_{4,2}$ | $Q_{4,1}$ | $Q_{5,2}$ | $Q_{5,1}$
Table 2: Basic Queuing Policies Defined by Their Queue Ordering

Policies $QP1$ and $QP3$ order queues $Q_{j,k}$ first with respect to $k$,
i.e. type of consultation, and within those categories with respect to triage
grades $j$. Hence, $QP1$ will always select “new” patients over old patients,
and $QP3$ vice-versa. On the other hand, $QP2$ and $QP4$ first order queues by
triage grade $j$ followed by the type of consultation $k$. Within a triage
category $QP2$ prioritizes “new” over “old” patients, and $QP4$ “old” over
“new”. Note that other orderings than $QP1-QP4$ would be feasible, but
arguably lack practicability.

####  3.4.2 Accumulated Priority Queuing

The APQ policy generalizes pure priority queuing by relaxing the constraint
that the selection of patients of $Q_{j,1}$ with $j\in J$ is strictly done
with respect to triage grades (and analogously for selecting patients of
$Q_{j,2}^{d}$ with $j\in J,d\in D$), see for example [14]. In particular,
patients accumulate priority points over time at a predefined rate that is
constant per patient class. When a doctor becomes idle, the patient with the
most priority points is selected for consultation. Thereby, patients may
“overtake” patients with higher priorities, depending on waiting times and
accumulation rates. Analogously to [14], we define patient classes over triage
grades and type of consultation. Hence, there is exactly one patient class for
each type of queue $Q_{j,1}$ with $j\in J$ and one patient class for each set
of queues $\\{Q_{j,2}^{d}|d\in D\\}$ with $j\in J$. Let $\beta_{j,k}$ be the
corresponding accumulation rates. The accumulated priority points for a
patient $i$ with triage grade $j$ waiting for consultation type $k$ at time
$t$, is then given by the linear function $\beta_{j,k}(t-t^{i}_{a})$. Further,
let $Q^{d}=\\{Q_{j,1}|j\in J\\}\cup\\{Q^{d}_{j,2}|j\in J\\}$ be the set of all
queues doctor $d$ may select patients from, $\hat{Q}^{d}$ the non-empty queues
for doctor $d$ (with at least one patient waiting), $\beta_{q}$ the
corresponding weight and $p_{q}^{1}$ the first patient of a queue
$q\in\hat{Q}^{d}$. The selection function $f_{ps}^{d}$ at time $t$ for doctor
$d$ in case of APQ patient selection can be written as

$f_{APQ}^{d}=\begin{cases}\operatorname*{arg\,max}_{q\in\hat{Q}^{d}}\left(\beta_{q}(t-t_{a}^{p_{q}^{1}})\right),\text{
if }|\hat{Q}^{d}|>0\\\ None,\text{ otherwise.}\end{cases}$ (4)

Note that the above definition, still satisfies constraints that patients of a
particular queue $q\in Q^{d}$ are seen in the order they join the queue.

The used procedure to optimize weights $\beta$ and corresponding results are
outlined in the Appendix of the paper.

##  4 ML Based Patient Selection

In its essence, APQ patient selection aims to balance the impact of patients
of different triage grades on a chosen objective function, thereby improving
the overall performance of the ED with respect to that measure. Therefore,
additional information on the system’s state, i.e. patients’ time in the
system, are used within a linear function as a basis for decision making.
Naturally, the question arises, if such a method could be improved by
considering an extended information base and using more complex functions than
linear equations.

In an ideal, but not realistic, setting, one would not only have perfect
information on the current state of the ED, but also information on future
events in the ED. If such information was available, patient selection would
essentially become a question of scheduling tasks to resources, e.g. doctors.
In that case, a deterministic optimization algorithm, exact or heuristic,
could be used to optimize $\hat{f}_{ED}$, and make optimal (or at least close
to optimal) patient selection decisions at any time. However, as mentioned
before, this would require that all future arriving patients, all activity
durations and requirements of diagnostic tests are known at the time of
planning, i.e. when scheduling patient requests.

Obviously, such information will never be available at the time of decision
making in a real-world setting with stochastic behavior. However, having a
model representing the real system allows for the generation of a
(theoretically infinite) set of instances of the studied ED. For those
instances, one is able to use a deterministic optimization algorithm to
compute optimal (or close to optimal) patient selection decisions with respect
to $\hat{f}_{ED}$ and thereby generate an information basis for enhanced
decision making.

The proposed ML based patient selection policy learns from optimal decisions
made by a deterministic optimizer during a training phase and transforms the
gained “knowledge” into a ML model. This model can then be used for patient
selection for future unseen instances. Thereby, it extends the considered
system state of APQ by any feature set describing the ED at a given point in
time and replaces the linear function of APQ by a more complex and non-linear
function, i.e. the ML model. The main principle of the approach is outlined by
figure 3

Figure 3: General Principle and Architecture of the Design and Evaluation of
the ML Based Patient Selection Policy

During a training phase a heuristic optimization algorithm is used to solve a
set of randomly generated training instances. The generation of instances uses
the same stochastic processes and probability distributions that are embedded
in the simulation model and will be described in detail in section 5.1. Hence,
for a chosen period of time, a set of patients with associated parameters,
i.e. arrival times, triage grades activity durations, and diagnostic test
requirements, are sampled per instance from those distributions and processes.
Since we are assuming time-of-the-day dependent arrival rates of patients (see
section 5.1), the smallest reasonable period of time covered by one instance
is 24 hours. Instances covering more than 24 hours increase the complexity of
the scheduling problems to solve, without adding much value to the gained
information. Hence, generated instances include patients arriving during a 24
hour period starting at midnight. It is important to note that resulting
schedules may stretch over a longer period of time, as all activities of
patients arriving within that 24 hour period are scheduled. This results in a
slight inaccuracy for the last activities to schedule, as newly arriving
patients outside the time window of 24 hours are not considered. However, as
ED are typically less frequently visited during night hours (as will be
apparent by the chosen patient arrival pattern), this small bias should be
negligible.

The optimizer treats the generated instances as a deterministic input to a
scheduling problem. The scheduling of registration and triage tasks is done
with respect to the FIFO principle, but the assignment of consultation tasks
to doctors is subjected to optimization. The same set of constraints as for
other queuing policies, i.e. no overtaking within queues and the same-doctor-
policy is considered by the optimizer. The result of this process are
schedules per doctor, i.e. at what point in time which patient is seen, for
each generated training instance.

The feature generator transforms the optimized schedules to a set of features.
Each point in time where a patient starts a consultation activity with a
doctor corresponds to a decision made in the abstracted system. Therefore, the
feature generator iterates over each of those time steps, stores information
of the state of the system at the specific point in time as a sample and the
“optimal” decision as a response. Thereby, the decision is represented by the
queue identifier of the queue the patient is selected from. Note that, for
defining responses, we do not distinguish between queues per doctors for
second consultations. Hence, there are eight possible responses
$QR=\\{Q_{2,1},Q_{3,1},Q_{4,1},Q_{5,1},Q_{2,2},Q_{3,2},Q_{4,2},Q_{5,2}\\}$.

The resulting sample and response training set is then used to fit a ML
classifier model with eight possible classes, i.e. $QR$.

In the evaluation phase, the ML classifier is used within the simulation model
of the ED to select patients from queues whenever a doctor becomes idle and
there are patients either waiting for the first consultation or the second
consultation with that particular doctor. The simulation model is then
evaluated on a set of evaluation instances that are generated with respect to
the same distributions and processes as the training instances. Note that
those steps are not only dependent on the input data, e.g. stochastic
components used, but also on the used weights of the objective function
$\hat{f}_{ED}$. This requires the optimization of training instance and
fitting of the ML for each parametrization of $\hat{f}_{ED}$ with different
weights.

In the following, the most important components of the proposed ML based
patient selection method are described in detail individually.

###  4.1 Optimizing ED instances

The training base for the proposed ML patient selection method is built on
optimal, or near optimal, schedules for one day instances of the ED. The
underlying optimization problem is basically a job-shop scheduling problem of
assigning consultation tasks (jobs) to doctors (machines or resources). Exact
methods to solve such problems often rely on column generation within a
branch-and-price or branch-and-price-and-cut framework. However, to use column
generation, (or decomposition in general) effectively, it is usually required
that schedules can be independently computed and priced. Unfortunately, this
is not the case for the underlying optimization problem of scheduling patient
consultations to doctors.

Re-ordering of jobs within a schedule of a single doctor may make schedules of
other doctors infeasible, as patients within one queue need to be selected
exactly in that order. The same observation holds for removing patients from a
doctor’s schedule and inserting it in another one’s schedule. In particular,
the earliest possible time a patient may be seen by a doctor may depend on the
schedule of other doctors. This inter-dependency limits the usage of exact
methods.

However, as results will show, even heuristically computed near-optimal
solutions are effective as a training base for the ML approach. Thus, we use a
Simulated Annealing-based adaptive neighborhood search algorithm to
heuristically optimize one-day instances of the ED. Further, to stabilize the
search, the Simulated Annealing method is embedded in a (simplified) race
algorithm.

In the following, the solution decoding and encoding, the neighborhood
definitions, adaptive mechanisms and the final search heuristic are described
in detail.

####  4.1.1 Solution Coding and Decoding

The majority of meta-heuristics used for optimization (as Simulated Annealing)
require a solution representation that can easily be decoded to an actual
solution and vice-versa. A key requirement of such a solution representation
is to allow the definition of effective operators, such as neighborhood
operators. Initially, we experimented with solution representations that were
based on actual schedules per doctors. However, the intra-queue constraints of
the underlying problem made the definition of operators cumbersome, and led to
an inferior performance of the search algorithm.

Hence, the presented heuristic is based on a simpler representation of
solutions. As the set of patients and their treatment requirements are treated
as an input and are known, the number of patients that will eventually be in a
queue $q\in
QR=\\{Q_{2,1},Q_{3,1},Q_{4,1},Q_{5,1},Q_{2,2},Q_{3,2},Q_{4,2},Q_{5,2}\\}$ is
also known upon execution of the algorithm. Therefore, we encode a solution as
the sequence of queues, where the number of occurrences of a queue identifier
$q\in QR$ matches exactly the number of consultation requirements of patients
of the corresponding triage grade and consultation type. For example, if
during the day in total 20 triage grade 4 patients arrive at the ED, out of
which 15 require diagnostic tests and a second consultation, any sequence
representing a solution to that instance contains exactly 20 $Q_{4,1}$ and 15
$Q_{4,2}$ entries.

It is important to note that such a sequence is only defining the order of
queues by which patients get selected and is not explicitly defining the order
by which patients are selected. Further, queues for the second consultation
are not considered by doctors in the sequence, i.e. there are no distinct
identifiers for each doctor.

To decode such a sequence to an actual solution, i.e. a schedule, the
simulation model described in Section 3 is used. But instead of using a pure
priority or APQ policy for dispatching idle doctors, a sequence based policy
is used. Therefore, the sequence of queues is passed to the simulation model
upon initialization and the model is started. Whenever a doctor $d$ becomes
idle, the patient to be seen is determined in the following way. The sequence
is searched for the first entry where the corresponding queue is not empty at
the current time. For entries representing queues for the second consultation,
only the queue for doctor $d$ is considered. For example, if the first entry
in the sequence is $Q_{3,2}$ but queue $Q_{3,2}^{d}$ is empty, the search
moves to the next entry of the sequence. When the first non-empty queue is
found, the first patient in that particular queue, i.e. the patient waiting
longest, is seen by doctor $d$. Finally, the corresponding queue identifier is
removed from the sequence and the simulation model continues.

Since sequences are subjected to neighborhood operators, it regularly occurs
that the actual queue a patient is selected from is not the first queue in the
sequence. In other words, the sequence of queues by which patients are
actually selected might differ from the sequence passed to the simulation
model. To improve the search, whenever a solution representation is decoded,
the actual queue sequence is stored along with the solution itself, i.e. the
schedule, and returned by the simulation model upon termination. The overall
principle is illustrated by Figure 4.

Figure 4: Simulation Based Solution Decoding and Encoding

####  4.1.2 Neighborhood Definitions

Based on the solution representation and decoding principle described in the
previous section, we will define in total four different neighborhood
operators. A common principle for sequence or permutation based
representations, is to define neighborhood operators based on swapping of
elements, or removal and insertion of elements, in the sequence or
permutation.

Basically, the proposed operators follow that principle. However, as swapping
for example two $Q_{4,1}$ identifiers will not change the sequence, we define
operators based on positions in the sequence where two consecutive identifiers
differ. In the following, such positions are referred to as “change points”.

The first operator, referred to as the block neighborhood, randomly picks a
change point in the sequence and swaps a chosen number of queue identifiers
before the change point, i.e. a block, with the same sized block after the
change point. The size of blocks is randomly picked from an interval
$\left[1,b\right]$, where $b$ is a parameter that is adapted along the search.

The other operators are based on a removal and insertion principle. First, a
change point is randomly selected. Second, a block size is randomly selected
from $\left[1,b\right]$ and queue identifiers in the block before and after
the change point are removed. Finally, a new sub-sequence is formed by the
removed identifiers and this sequence is inserted in the original sequence at
the position of removal.

In general, the sub-sequence is constructed randomly, but three different
versions are considered:

  1. 1.

The “random insertion operator” inserts the removed identifiers in a purely
random fashion.

  2. 2.

The “objective insertion operator” assigns weights to the identifiers that are
based on the objective value of the current solution. In particular, each
identifier is associated with an actual patient $i$ in the decoded solution.
The contribution of that particular patient to the overall objective
$\hat{f}_{ED}$, i.e. the term corresponding to patient $i$ in equation 2, is
used as a weight. The sum of weights of patients in the removed block is
normalized to 1 and probabilities are assigned correspondingly. Identifiers
are then iteratively picked and added to the new sub-sequence with respect to
those probabilities.

  3. 3.

The “queue insertion operator” works similar as the objective based operator,
but instead of using the contribution to the objective value as weights for
queue identifiers, queue lengths are used as weights. Therefore, the state of
queues in the ED is checked after dispatching the last patient before the
removed block in the decoded solution of the current sequence. The number of
patients in corresponding queues at that point in time in the solution are
then used as weights for removed queue identifiers. Note that queues for
different doctors are aggregated to a single weight, i.e. summed up.

The general principle of change points and the four neighborhood operators is
outlined by Figure 5.

Figure 5: Definition of Neighborhood Operators

####  4.1.3 Competitive Simulated Annealing Heuristic

The heuristic to optimize ED instances proposed in this paper is based on the
Simulated Annealing meta-heuristic. In its simplest form, Simulated Annealing
iteratively updates a current solution by randomly picking a neighborhood
solution (e.g. via randomly applying a neighborhood operator) and comparing
its objective with the objective of the current solution. If the objective is
improving, the move to the new solutions is accepted, i.e. the current
solution is updated. If the objective is non-improving, the move is only
accepted with a certain probability that is dependent on the distance between
objectives and a so-called temperature. This temperature is assumed to
decrease over iterations, i.e. making it less likely for worse solutions to be
accepted.

Several adaptions and extensions to that base principle have been proposed to
solve real-world problems. Two commonly integrated adaptions, that are also
used in the presented version, are adaptive mechanisms and diversification
moves.

Adaptive mechanisms are usually used to pick operators from a set of operators
at a specific iteration along the search with respect to their performance at
previous iterations, see for example [5]. Assuming that $h$ such operators are
available, each with a weight $w_{k},k=1,\ldots,h$, the probability of
choosing operator $k$ is given by

$\frac{w_{k}}{\sum_{i=1}^{h}w_{i}}.$ (5)

Weights $w_{k}$ are initialized uniformly and updated after a consecutive
number of iterations, referred to as a segment. After each segment, weights
are updated by

$w_{k}^{sg}=\gamma w_{k}^{sg-1}+(1-\gamma)\pi_{k}^{sg-1},$ (6)

where $w_{k}^{sg-1}$ are the weights of the previous segment, $\gamma$ is a
parameter, and $\pi_{k}^{sg-1}$ is a performance score of operator $k$ during
the previous segment, see for example [5]. At the start of each segment
$\pi_{k}^{sg-1}$ are set to $0$ and updated each iteration by the following
rule

$\pi_{k}^{t+1}=\pi_{k}^{t}+\begin{cases}\sigma_{1}\text{ if a new best
solution has been produced,}\\\ \sigma_{2}\text{ if the new solution is better
than the current solution,}\\\ \sigma_{3}\text{ if the new solution is worse
than the current solution, but has been accepted,}\end{cases}$ (7)

where $\sigma_{1}$, $\sigma_{2}$, and $\sigma_{3}$ are parameters to chose.

We use this adaptive mechanism to select a neighborhood operator from the
following operators: block neighborhood, random insertion neighborhood and
objective insertion neighborhood. Based on experimentation, the segment size
is chosen as 5000 iterations, $\gamma$ is chosen as $0.95$ and sigma weights
are chosen as $0.5$, $2$, and $0.25$.

However, we use the same principle not only for selecting operators, but also
for randomly selecting a change point in the current sequence. Therefore, the
positions in a sequence reflected by change points are kept in buckets of size
10, e.g. change points at position 22 and 27 would both be in the same bucket.
The number of buckets is limited by the length of the sequence divided by 10
plus 1. Weights are kept and updated for each bucket in the same way as
weights for operators are maintained. When selecting a change point, first a
bucket is chosen with respect to current weights, and second a change point is
randomly chosen within the bucket with respect to uniform probabilities.
Parameter $\gamma$ is set to $0.99$ and parameter $\sigma_{2}$ is set to $1$,
where all other parameters are the same as for the selection of neighborhood
operators.

The queue insertion neighborhood operator is used within diversification
steps. In particular, whenever the search fails to improve the overall best
solution for consecutive $d_{max}$ iterations, a diversification move is
performed. Such moves include the application of the queue based insertion
neighborhood operator with block size $b_{max}$ and resetting weights for
operators and change point buckets to uniformly distributed weights.

As a starting solution, the best solution of the set of solutions resulting
from applying the pure queuing principles $QP1$-$QP4$ is chosen. The resulting
procedure is described by algorithm 1. Therefore, let $Decode(seq)$ be a
simulation based method that decodes a sequence of queues based on the
principle described in section 4.1.1 and returns the corresponding objective
value and actual queue sequence.

Algorithm 1 Adaptive Simulated Annealing

1: Input: Instance $I$, $init$, $iter_{max}$

2: Output: Optimized Schedule

3: if $init$ then

4: Set $iter=0$, $b=b_{max}$, $lastImprov=0$, $T=2$

5: Initialize Weights for NB Operators and Change Point Buckets

6: Compute start solution $seq_{start}$ with objective $f_{start}$

7: Set
$best_{seq}=seq_{start},f_{best}=f_{start},cur_{seq}=seq_{start},f_{cur}=f_{start}$

8: else

9: Restore $iter,best_{seq},f_{best},cur_{seq},f_{cur},T,lastImprov$, NB and
Change Point Weights and Scores from Last Run

10: end if

11: while $iter<iter_{max}$ do

12: Set $iter=iter+1$ and $T=\max(T\cdot 0.0.999975,0.3)$

13: if $iter\mod 5000=0$ then

14: Update Weights for NB and Change Points by (6)

15: end if

16: Select NB Operator with weights by (5)

17: Select Change Point Bucket with weights by by (5) and Change Point in
Bucket

18: Choose NB Size as $randInt(1,b)$

19: Compute New Sequence $new_{seq}$ with Selected NB and Size

20: Set $new_{seq},f_{new}=Decode(new_{seq})$

21: if $f_{new}<f_{cur}$ then

22: Set $f_{cur}=f_{new},cur_{seq}=new_{seq},b=\max(1,b-1)$

23: if $f_{new}<f_{best}$ then

24: Set $best_{seq}=new_{seq},f_{best}=f_{new},lastImprov=iter$

25: end if

26: else if $rand(0,1)<\exp^{-(f_{new}-f_{cur})/T}$ then

27: Set $f_{cur}=f_{new},cur_{seq}=new_{seq}$

28: else

29: Set $b=\min(b+1,b_{max})$

30: end if

31: Update Scores $\pi$ for NB and Change Points with (7)

32: if $iter-lastImprov>d_{max}$ then

33: Set $lastImprov=iter$, $b=b_{max}$, $T=\min(T\cdot 2,2)$

34: Reset Weights for NB and Change Points to Uniform Weights

35: Set Scores for NB and Change Points to $0$

36: Compute New Sequence $new_{seq}$ with Queue Based Insertion NB

37: Set $new_{seq},f_{new}=Decode(new_{seq})$

38: Set $f_{cur}=f_{new},cur_{seq}=new_{seq}$

39: end if

40: end while

41: Return $best_{seq},f_{best}$

By its definitions, Simulated Annealing in general as well as the proposed
adaptive version outlined by Algorithm 1 are stochastic algorithms. Hence,
multiple runs on the same instance (with varying random seeds) may lead to
different results. In order to stabilize the search, Algorithm 1 is embedded
in a race-like algorithm. In particular, the search of Algorithm 1 is divided
in three phases of iterations. Let $iter_{max}^{1}$, $iter_{max}^{2}$ and
$iter_{max}^{3}=iter_{max}$ denote the number of iterations of each phase.
Initially, $n_{s}^{1}$ instances of Algorithm 1 are initialized (with varying
random seeds) and for each instance $iter_{max}^{1}$ iterations are performed.
After completion of those $iter_{max}^{1}$ iterations, the current best
objectives, $f_{best}$ of all instances are compared and the $n_{s}^{2}$ best
instances are passed to the second phase. For those, the search is continued
until $iter_{max}^{2}$ iterations are reached. Only the instances of the
search with $f_{best}$ among the $n_{s}^{3}$ best objectives are kept and
passed to the last phase. During that last phase the search is continued until
the overall maximum number of iterations, $iter_{max}$, is reached. The final
best solution found is chosen from those $n_{s}^{3}$ instances in the last
phase. The parameters used in this paper are
$(iter_{max}^{1},iter_{max}^{2},iter_{max})=(25000,50000,20000)$ and
$(n_{s}^{1},n_{s}^{2},n_{s}^{3})=(10,3,1)$.

###  4.2 ML Model Integration

Given that doctor $d\in D$ is idle at time $t$, the task of the ML is to pick
the patient to be seen next by doctor $d$. However, as overtaking within
queues is not allowed, this task can be represented as picking a (non-empty)
queue from
$\\{Q_{2,1},Q_{3,1},Q_{4,1},Q_{5,1},Q^{d}_{2,2},Q^{d}_{3,2},Q^{d}_{4,2},Q^{d}_{5,2}\\}$.
The first patient in the chosen queue is then selected for consultation (first
or second) by the idle doctor.

Picking a queue from the reference set can also be seen as a classification
task. In other words, for any possible situation in the ED, i.e. any possible
state, there exists at least one optimal selection, and the ML aims to predict
this optimal decision for a given state.

There exist several classifier models in ML literature, such as, for example,
decision trees, random forests, neural networks, logistic regression and
support-vector-machine.

Experiments have shown that, apart from single decision trees, all of the
above mentioned approaches lead to very similar results in terms of accuracy.
In this paper a random forest approach is use. The exact definition of the
feature set used for the predictor model is outlined in the Appendix of the
paper.

Note that it is theoretically possible that a random forest model predicts, or
selects, an empty queue for a given unseen state. Obviously, an empty queue is
never part of a solution generated from the optimizer, and hence part of the
training set of samples. However, due to techniques like feature sampling and
bagging, during the generation of trees within the forest, it is very
unlikely, but possible to occur. To avoid the inefficiency of picking an empty
queue, the classes
$\\{Q_{2,1},Q_{3,1},Q_{4,1},Q_{5,1},Q^{d}_{2,2},Q^{d}_{3,2},Q^{d}_{4,2},Q^{d}_{5,2}\\}$
are sorted by their prediction probabilities provided by the random forest
model, and the first non-empty queue is picked.

##  5 Data Generation and Scenario Definition

In this section, we describe the parametrization of the model and the design
of scenarios that are used for experimentation and evaluation of the proposed
approach. In section 5.1 the “base” parametrization or scenario is described,
whereas some of these inputs are altered in section 5.2 to define different
scenarios. Presented inputs for the base scenario are not based on a real ED,
but are representative typical values that can be found in literature.
Wherever possible, references to data sources are provided.

###  5.1 ED Data

To paremeterize the model, three different sets of input measures have to be
defined: patient related measures (arrival processes, triage grade
distributions, etc.) activity durations, and resource availabilities.

Arrival streams of patients are most often modeled via a Poisson process with
exponential inter-arrival times, see for example [69, 18, 15, 64, 59]. Typical
variations to identically distributed inter-arrival times are so called hour-
of-day dependent arrival rates and/or day-of-week dependent arrival rates. In
other words, it is typical that the number of patients arriving to an ED is
dependent on the time of the day (see for example [29, 11, 4, 13, 33, 37, 70])
and sometimes also the weekday ([26, 45, 31, 63]). However, it is interesting
to see that the distribution of arrival rates over the time of the day shows
similar characteristics over various published ED models, see [22], with a
peak around noon and less patients arriving during night hours. Hence, we used
the slightly adapted shape of the arrival rate curve proposed by [21], as
shown by Figure 6.

Figure 6: Arrival Stream of Patients

The distribution over triage grades of patients is chosen in the same way as
proposed by [19], with the slight adaption that the probability mass of
category 1 patients is shifted to category 2 patients. The probabilities of
patients requiring a diagnostic test, and consequently a second consultation
with a doctor, as well as the arrival modes are obtained from [29]. Note that
the arrival mode has no influence on the processes of patients with a triage
category other than 2. Target times and shares are chosen in the same way as
by [14]. The importance of patient categories $j\in J$ is based on the values
used by [14], where for $j=2$ we defined the importance to be 3. All
parameters and chosen importance of patient categories $pw_{j}$ are summarized
by table 3.

Triage Grade | 2 | 3 | 4 | 5
---|---|---|---|---
Probability | 6.5% | 20% | 51.5% | 22%
Probability Diagnostic Test | 74% | 62% | 53% | 31%
Probability Ambulance Arrival | 60% | - | - | -
Patient Importance $pw_{j}$ | 3 | 2 | 1.5 | 1
$t^{j}_{ttd}$ | 15 | 30 | 60 | 120
$ts^{j}_{ttd}$ | 95% | 90% | 83% | 80%
Table 3: Probabilities, Importance, Target Time and Shares of Patient Classes

The probability distributions for the duration of activities are summarized by
table 4. For register and triage activities we assumed that the duration
follows a triangular distribution with associated parameters. The duration of
registering is independent of the triage grade (which is also not known at the
time of register) and parameters are chosen in the same way as in [21]. The
duration of triage activities is chosen to be slightly shorter than that
proposed by [19], as this led to more realistic shares of patients meeting the
TTD targets.

Similar to [14], the duration of first and second consultation activities are
assumed to be Log-Normally distributed. Thereby, the $\sigma$-parameter is
chosen exactly as by [14] for all consultation activities. The $\mu$ parameter
was used to achieve the desired expected value of the specific duration. The
first consultation task is assumed to have an expected duration of $17$
minutes for category 2 patients, as also proposed by [19], $20$ minutes for
category 3 patients, as [14], $18$ minutes for category 4 patients (as a mix
from $16$ minutes by [14] and $20$ minutes of [19]) and $15$ minutes for
category 5 patients (as a mix from $13$ minutes by [14], $22$ minutes of [19]
and $10$ minutes of [21]).

For the second consultation, the same values as reported in [14] were used for
category 3, 4 and 5 patients. A $12$ minute expected duration is assumed for
category 2 patients (those are not included in the study of [14]).

The delay of diagnostic tests (including possible waiting times) is modelled
with a triangular distribution of mean $30$ minutes and $\pm 15$ minutes as
lower and upper limits.

Triage Grade | 2 | 3 | 4 | 5
---|---|---|---|---
Register | - | $T(3,4,5)$ | $T(3,4,5)$ | $T(3,4,5)$
Triage | $T(2,3,4)$ | $T(4,6,8)$ | $T(3,5,7)$ | $T(3,5,7)$
First Consultation | $LogN(2.73,0.45)$ | $LogN(2.89,0.45)$ | $LogN(2.79,0.45)$ | $LogN(2.60,0.45)$
First Consultation $E$ | 17 | 20 | 18 | 15
Second Consultation | $LogN(2.38,0.45)$ | $LogN(2.29,0.45)$ | $LogN(2.10,0.45)$ | $LogN(1.84,0.45)$
Second Consultation $E$ | 12 | 11 | 9 | 7
Diagnostic Tests | $T(15,30,45)$ | $T(15,30,45)$ | $T(15,30,45)$ | $T(15,30,45)$
Table 4: Stochastic Distributions of Activity Durations: $T(l,m,u)$ Triangular
Distributions (lower, mode, upper), $LogN(\mu,\sigma)$ Log-Normal Distribution
with Expected Value $E$ in minutes

We did not include shift-models in the ED models, and assumed that 4 doctors,
2 triage nurses and 1 register clerk are present throughout the simulation
period. As will be shown in section 6, the described input parameters of the
base scenario led to typical and reasonable results in terms of waiting times,
proportion of patient that meet TTD targets and doctor utilization rates.

###  5.2 Scenarios

The data and parameters presented in section 5.1 defines the baseline scenario
for experimentation. Hence, in the remainder it is referred to as “Base”.
However, to evaluate the performance of the proposed method under a variety of
settings, additional scenarios based on alterations of some parameters are
defined.

Parameters used for scenario definition include the arrival rate of patients,
the variance of consultation times, and the distribution of patients over
triage grades.

To test the ML based selection method under the setting of a “more severely”
crowded ED, we included a scenario that is based on an 10% increase of the
hourly arrival rates shown by figure 6. Note that the hourly pattern of
arriving patients was left unchanged.

The variance of consultation times may influence the “predictability” of how
long patients will occupy doctors during consultation. Consequently, the ML
based selection policy may suffer from high variances and vice-versa benefit
from lower variances of consultation times. Hence, we define two additional
scenarios with increased/decreased $\sigma$ parameters of the Log-Normal
distributions for consultation times. Note that the corresponding $\mu$
parameters were adapted such that the expected duration of activities shown by
table 4 remain unchanged.

Last, the distribution of patients over triage grades by [19] assigns a
relatively low probability mass to category 2 patients and almost a triangular
shape to category 3 to 5 patients centered at category 4 patients. We define
three alterations to this distribution. One, were category 3-5 patients have
approximately uniformly distributed probabilities, and two where the peak of
category 4 patients is shifted to either category 3 or 5 patients.

The resulting parameter sets and the scenario identifiers are summarized by
table 5.

For each scenario we randomly generated a set of 10,000 training instances and
10,000 evaluation instances. Training instances are used to twofold: (1) to
compute near optimal solutions that are used to train ML models; (2) and to
optimize weights $\beta$ for the APQ algorithm. Each instance covers 24h hours
starting at midnight. However, only patients arriving between 8am and 8pm
(regardless of the time they leave the ED) are used to compute performance
measures and evaluate results. Hence, the time between midnight and 8am may be
seen as a warm-up period of the models.

Besides different scenarios, we evaluate the proposed method with respect to a
set of performance measures based on parameters $\lambda_{W}$,
$\lambda_{ttd}$. The used objective functions include the “pure” ones, i.e.
where only the waiting time or the TTD target is considered, and two combined
measures (indicated by the prefix C). For the combined measures one percent of
target increase accounts for $\lambda_{W}$ minutes of waiting time reduction.
All four combinations are summarized in Table 5.

Scenario Name | Arrival Rate Factor | $\sigma$ | Patient Mix
---|---|---|---
Base | 1 | 0.45 | (0.065, 0.2, 0.515, 0.22)
HR | 1.1 | 0.45 | (0.065, 0.2, 0.515, 0.22)
LV | 1 | 0.35 | (0.065, 0.2, 0.515, 0.22)
HV | 1 | 0.55 | (0.065, 0.2, 0.515, 0.22)
U | 1 | 0.45 | (0.065, 0.312, 0.312, 0.311)
T3 | 1 | 0.45 | (0.065, 0.515, 0.2, 0.22)
T5 | 1 | 0.45 | (0.065, 0.2, 0.22, 0.515)
Objective Name | $\lambda_{ttd}$ | $\lambda_{W}$ |
TTDL | 1 | 0 |
C-30 | 30 | 1 |
C-15 | 15 | 1 |
TWT | 0 | 1 |
Table 5: Summary of Scenario and Objective Function Definitions

##  6 Results

In this section we present the experimental results for all proposed scenario
and objective function combinations. Algorithms and simulation models were
coded in C# and experiments were conducted on a desktop machine with an AMD
Threadripper Pro 3995WX Processor containing 64 cores, each up to 4.2 GHz.
Machine learning models were built using the Python scikit-learn 0.24.2
library [55] and then converted to C# models. Each forest consists of 200
trees. To avoid over-fitting, each tree is based on a sample set consisting of
0.5 times the number of samples of the entire training set that are selected
using bootstrapping. To account for imbalanced feature sets, e.g. in the base
scenario the majority of patients are of category 4, we use over-sampling to
maintain a balance of observed responses. However, we over-sample the first
and second consultation responses individually. In particular, we choose the
class for each type of consultation with the most responses, e.g. 41 for the
first consultation in the base scenario, and for each other class we randomly
bootstrap samples from the entire set belonging to this class until the same
number of samples as for the maximum (in this example 41) is reached.

To parametrize and tune the Genetic Algorithm and the Simulated Annealing
algorithm, we used the base scenario and the C-15 objective. Hence, the
performance of those algorithms for other scenario/objective combinations,
would possibly benefit from specific parametrization. In order to keep the
computational effort at a reasonable level, we used the same versions of those
algorithms for all settings. However, experiments have shown that the
Simulated Annealing algorithms perform worse for the pure TTDL objective. This
is mainly due to the approximately step-wise behavior of the objective
function for a single day instance. In particular, the objective function only
changes when a patient meets the target compared to a previous solution where
the patient missed the target (and vice-versa). This results in many
neighborhood moves not changing the objective value, which consequently limits
the performance of the algorithm. Therefore, for computing optimal solutions
we used a C-120 objective, i.e. $\lambda_{ttd}=120$ and $\lambda_{W}=1$, while
presented results are evaluated by the TTDL objective.

The section is structured as follows. In Section 6.1 the prediction quality of
the proposed ML models is evaluated. Section 6.2 outlines experimental results
regarding the base scenario, while results for additional scenarios are
summarized in Section 6.3.

###  6.1 Prediction Results

The patient selection method is based on a Random Forest classifier. To assess
the quality of predictions compared to the optimal solutions, we further split
the training instances (for which optimal solutions are known) into ML
training (90%) and ML test instances (10% of all training instances). The
fitted models are then evaluated on the ML test instances. Overall results and
results per triage category are summarized in Table 6. For each included
combination of scenarios and objective functions, we report the accuracy of
the predictor and the Matthews Correlation Coefficient (MCC). We define
accuracy as the number of correctly predicted observations of a class divided
by the total number of predictions of that particular class. The MCC is often
used to assess classifiers when the underlying feature-response set is
strongly unbalanced. It is bounded by $-1$ and $1$, where $1$ denotes a
perfect prediction, $0$ a random prediction and $-1$ a prediction doing
exactly the opposite to real world observations. Values above $0.4$ are
considered to indicate a strong positive correlation between predictions and
real world observations, and values above $0.7$ a very strong correlation.

|  |  |  |  | Accuracy Per Triage Grade
---|---|---|---|---|---
|  |  |  | First Consultation | Second Consultation
Scenario | Objective | MCC | Ov. Acc. | 2 | 3 | 4 | 5 | 2 | 3 | 4 | 5
Base | TTDL | 0.71 | 77.3% | 67.0% | 67.6% | 79.3% | 77.6% | 65.1% | 82.0% | 95.1% | 86.6%
C-30 | 0.63 | 70.0% | 65.9% | 65.1% | 78.6% | 68.5% | 45.7% | 62.2% | 74.0% | 59.9%
C-15 | 0.63 | 70.0% | 66.8% | 65.6% | 78.4% | 67.3% | 47.0% | 61.4% | 74.1% | 60.4%
TWT | 0.90 | 92.4% | 89.6% | 92.2% | 96.6% | 94.0% | 85.6% | 87.4% | 90.8% | 79.8%
HR | C-15 | 0.56 | 64.6% | 64.0% | 64.3% | 77.1% | 56.2% | 38.9% | 52.7% | 65.1% | 48.1%
LV | 0.62 | 69.6% | 68.0% | 65.7% | 78.3% | 67.2% | 46.4% | 60.1% | 73.1% | 57.7%
HV | 0.62 | 69.9% | 67.7% | 65.4% | 78.5% | 67.8% | 45.5% | 61.4% | 73.1% | 59.0%
U | 0.62 | 68.6% | 65.0% | 71.2% | 72.6% | 75.7% | 42.1% | 64.9% | 63.3% | 62.1%
T3 | 0.61 | 68.7% | 63.6% | 78.6% | 66.1% | 63.7% | 44.9% | 70.3% | 52.1% | 52.6%
T5 | 0.70 | 75.8% | 71.1% | 71.5% | 67.7% | 86.5% | 56.6% | 71.4% | 69.3% | 83.4%
Table 6: Prediction Accuracy and MCC for the ML Classifier

Results show that for most of the settings the MCC is above 0.6 with
corresponding accuracy values larger than 68% indicating a relatively strong
correlation between predictions and optimal solutions. In particular, for the
base scenario, results for the TDDL and TWT objectives (i.e. the single
measures) are significantly higher, where the overall accuracy even exceeds
90% for the TWT objective. Hence, it may be concluded that predicting optimal
decisions is “easier” with respect to a single performance measure compared to
a combined measure of at least partly conflicting objectives.

However, it has to be noted that a wrong prediction with respect to the
optimal solution is not necessarily a bad decision for the given state of the
ED.

###  6.2 Base Scenario Results

In this section we analyze the results for the base scenario and different
objectives. Table 7 summarizes the estimated performance measure
$\hat{f}_{ED}$ for all included combinations of $\lambda_{W}$ and
$\lambda_{ttd}$ and patient selection policies. The selection policies
include: the pure priority based policies QP1-4, the APQ method, and the ML
based method denoted by ML-OPT. Further, we report results based on the (near)
optimal solutions (denoted by OPT) of the training instances as a benchmark.
Thereby, it has to be noted that results for the OPT policy are based on
different instances than the other policies. Further, it has to be noted that
results for the APQ (and the ML-OPT) policy are based on different models for
each objective function, but reported in a single line in table 7. In other
words, the objective value for the C-15 measure and the ML-OPT policy results
from the model that was fit on optimal solutions with respect to the C-15
objective. The same model would lead to a different result in terms of the
C-30 objective than reported by Table 7, which itself is based on optimal
solutions computed with respect to the C-30 objective.

| Objective Function
---|---
Selection Policy | TTDL | C-30 | C-15 | TWT
QP1 | 7.65 | 445.3 | 330.6 | 215.9
QP2 | 17.22 | 649.6 | 391.3 | 132.9
QP3 | 31.43 | 1045.6 | 574.2 | 102.8
QP4 | 28.79 | 970.0 | 538.1 | 106.2
Min QPX | 7.65 | 445.3 | 330.6 | 102.8
APQ | 5.97 | 395.2 | 298.3 | 100.9
ML-OPT | 6.00 | 362.0 | 272.8 | 102.5
OPT | 4.89 | 316.0 | 243.4 | 102.4
APQ-Improvement | 21.9% | 11.3% | 9.8% | 1.8%
ML-OPT-Improvement | 21.6% | 18.7% | 17.5% | 0.3%
OPT-Improvement | 36.1% | 29.0% | 26.4% | 0.4%
Table 7: Summary of Objective Values $\hat{f}_{ED}$ and Improvement with
Respect to Pure Priority-Based Queuing per Selection Policy for the Base
Scenario

Experiments show that for objective functions that are based on a single
measure, i.e. either TTD targets or TWT, APQ and the ML-OPT approach lead to
similar results. While the APQ policy performs slightly better, both fail to
improve the QP3 policy significantly for the TWT objective. For the TTDL
objective, both methods yield an improvement above 21%. In terms of pure
priority based policies, the QP1 policy yields the best results for all
objectives but for the TWT objective where QP3 performs best.

However, for objectives including both total waiting times and TTD targets,
the ML-OPT policy clearly outperforms the APQ method. Hence, it may be
concluded that the ML-OPT is more capable of balancing those (at least partly)
conflicting objectives. To further analyze the behavior of the proposed
policies, Tables 8 and 9 report expected TTD target shares $E(X_{ttd}^{j})$
and expected waiting time measures $E(X_{\tilde{W}_{1}})$ and
$E(X_{\tilde{W}})$ per patient category $j$ and per policy. Further, the
tables include 95% confidence intervals of above mentioned measures to assess
the accuracy of results.

| Triage Category
---|---
Selection Policy | Overall | 2 | 3 | 4 | 5
QP1 | 83.7 $\pm$ 0.06% | 99.0 $\pm$ 0.06% | 75.9 $\pm$ 0.16% | 90.3 $\pm$ 0.07% | 70.9 $\pm$ 0.16%
QP2 | 71.1 $\pm$ 0.08% | 99.1 $\pm$ 0.06% | 75.9 $\pm$ 0.16% | 80.1 $\pm$ 0.09% | 37.6 $\pm$ 0.17%
QP3 | 61.4 $\pm$ 0.08% | 96.1 $\pm$ 0.13% | 71.9 $\pm$ 0.17% | 65.0 $\pm$ 0.11% | 33.2 $\pm$ 0.17%
QP4 | 63.2 $\pm$ 0.08% | 99.0 $\pm$ 0.06% | 75.0 $\pm$ 0.16% | 66.7 $\pm$ 0.11% | 33.7 $\pm$ 0.17%
APQ-TTDL | 81.4 $\pm$ 0.07% | 95.2 $\pm$ 0.14% | 76.5 $\pm$ 0.16% | 83.3 $\pm$ 0.09% | 77.5 $\pm$ 0.15%
ML-OPT-TTDL | 83.8 $\pm$ 0.10% | 98.4 $\pm$ 0.16% | 76.1 $\pm$ 0.14% | 87.4 $\pm$ 0.04% | 78.0 $\pm$ 0.42%
OPT-TTDL | 83.5 $\pm$ 0.08% | 98.9 $\pm$ 0.27% | 78.3 $\pm$ 0.14% | 85.5 $\pm$ 0.03% | 79.1 $\pm$ 0.33%
APQ-C-30 | 81.2 $\pm$ 0.07% | 94.5 $\pm$ 0.15% | 76.2 $\pm$ 0.16% | 83.4 $\pm$ 0.09% | 76.8 $\pm$ 0.15%
ML-OPT-C-30 | 83.6 $\pm$ 0.06% | 98.2 $\pm$ 0.09% | 76.1 $\pm$ 0.16% | 87.3 $\pm$ 0.08% | 77.5 $\pm$ 0.15%
OPT-C-30 | 83.3 $\pm$ 0.06% | 98.7 $\pm$ 0.07% | 78.3 $\pm$ 0.15% | 85.5 $\pm$ 0.08% | 78.3 $\pm$ 0.15%
APQ-C-15 | 81.2 $\pm$ 0.07% | 95.1 $\pm$ 0.14% | 76.3 $\pm$ 0.16% | 83.4 $\pm$ 0.09% | 76.5 $\pm$ 0.15%
ML-OPT-C-15 | 83.5 $\pm$ 0.06% | 98.2 $\pm$ 0.09% | 76.2 $\pm$ 0.16% | 87.4 $\pm$ 0.08% | 76.5 $\pm$ 0.15%
OPT-C-15 | 82.8 $\pm$ 0.06% | 98.6 $\pm$ 0.08% | 78.2 $\pm$ 0.16% | 85.4 $\pm$ 0.08% | 76.4 $\pm$ 0.15%
APQ-TWT | 51.6 $\pm$ 0.08% | 95.8 $\pm$ 0.13% | 51.7 $\pm$ 0.19% | 52.6 $\pm$ 0.12% | 36.2 $\pm$ 0.17%
ML-OPT-TWT | 61.4 $\pm$ 0.08% | 96.0 $\pm$ 0.13% | 71.8 $\pm$ 0.17% | 65.0 $\pm$ 0.11% | 33.3 $\pm$ 0.17%
OPT-TWT | 61.7 $\pm$ 0.08% | 91.3 $\pm$ 0.19% | 67.9 $\pm$ 0.18% | 66.0 $\pm$ 0.11% | 37.3 $\pm$ 0.17%
Table 8: Estimated Share of Patients Meeting TTD target and 95% Confidence
Intervals per Selection Policy and Triage Grade for the Base Scenario

Table 8 shows that the performance of the APQ method and the ML-OPT policy are
relatively similar in terms of TTD target shares for all objectives, except
for the TWT, which is based only on waiting times. Interestingly, both the OPT
and ML-OPT policy yield a target rate above 98% for patients of category 2
which is about 3% higher than the target rate of 95%. In other words, those 3%
do not contribute to the improvement of those methods compared to the APQ
policy. Similarly for category 4 patients, the ML-OPT policy exceeds the
target share of 83%. For the TWT objective, the performance of the ML-OPT
policy is similar to that of the best pure priority based policy, QP3, in
terms of TTD target shares, while applying the APQ policy results in
significantly lower shares of patients meeting target times for all but
category 5 patients.

| Triage Category
---|---
Selection Policy | Overall | 2 | 3 | 4 | 5
| $E(X_{\tilde{W}_{1}})$
QP1 | 33.7 $\pm$ 0.1 | 3.1 $\pm$ 0.0 | 13.7 $\pm$ 0.1 | 19.9 $\pm$ 0.1 | 80.6 $\pm$ 0.4
QP2 | 73.6 $\pm$ 0.3 | 3.0 $\pm$ 0.0 | 13.7 $\pm$ 0.1 | 28.9 $\pm$ 0.1 | 217.4 $\pm$ 0.8
QP3 | 89.2 $\pm$ 0.3 | 4.1 $\pm$ 0.0 | 15.4 $\pm$ 0.1 | 45.5 $\pm$ 0.1 | 241.9 $\pm$ 0.8
QP4 | 87.4 $\pm$ 0.3 | 3.0 $\pm$ 0.0 | 14.1 $\pm$ 0.1 | 43.7 $\pm$ 0.1 | 239.8 $\pm$ 0.8
APQ-TTDL | 32.7 $\pm$ 0.1 | 5.0 $\pm$ 0.0 | 13.5 $\pm$ 0.1 | 25.9 $\pm$ 0.1 | 63.5 $\pm$ 0.3
ML-OPT-TTDL | 38.7 $\pm$ 0.1 | 4.5 $\pm$ 0.1 | 16.9 $\pm$ 0.2 | 27.8 $\pm$ 0.1 | 80.7 $\pm$ 0.2
OPT-TTDL | 40.9 $\pm$ 0.0 | 5.6 $\pm$ 0.0 | 21.9 $\pm$ 0.2 | 30.5 $\pm$ 0.1 | 80.8 $\pm$ -0.2
APQ-C-30 | 34.8 $\pm$ 0.1 | 5.4 $\pm$ 0.1 | 13.7 $\pm$ 0.1 | 26.9 $\pm$ 0.1 | 69.7 $\pm$ 0.3
ML-OPT-C-30 | 39.0 $\pm$ 0.1 | 4.8 $\pm$ 0.2 | 16.8 $\pm$ 0.1 | 28.0 $\pm$ 0.1 | 81.6 $\pm$ 0.3
OPT-C-30 | 41.5 $\pm$ 0.1 | 5.8 $\pm$ 0.1 | 21.9 $\pm$ 0.2 | 30.6 $\pm$ 0.1 | 83.0 $\pm$ 0.4
APQ-C-15 | 35.1 $\pm$ 0.1 | 5.2 $\pm$ 0.0 | 13.7 $\pm$ 0.1 | 27.0 $\pm$ 0.1 | 70.5 $\pm$ 0.3
ML-OPT-C-15 | 39.2 $\pm$ 0.1 | 4.7 $\pm$ 0.2 | 16.4 $\pm$ 0.1 | 28.0 $\pm$ 0.1 | 82.9 $\pm$ 0.4
OPT-C-15 | 43.0 $\pm$ 0.1 | 5.7 $\pm$ 0.1 | 21.9 $\pm$ 0.2 | 30.9 $\pm$ 0.1 | 88.1 $\pm$ 0.4
APQ-TWT | 81.8 $\pm$ 0.2 | 4.7 $\pm$ 0.0 | 22.0 $\pm$ 0.1 | 54.0 $\pm$ 0.1 | 190.7 $\pm$ 0.7
ML-OPT-TWT | 88.6 $\pm$ 0.3 | 4.6 $\pm$ 0.1 | 15.5 $\pm$ 0.1 | 45.4 $\pm$ 0.1 | 239.6 $\pm$ 0.8
OPT-TWT | 83.2 $\pm$ 0.3 | 7.6 $\pm$ 0.3 | 21.4 $\pm$ 0.2 | 43.9 $\pm$ 0.1 | 218.1 $\pm$ 0.8
| $E(X_{\tilde{W}})$
QP1 | 295.1 $\pm$ 0.6 | 198.4 $\pm$ 1.5 | 241.6 $\pm$ 1.0 | 316.6 $\pm$ 0.8 | 376.0 $\pm$ 1.6
QP2 | 154.9 $\pm$ 0.5 | 14.1 $\pm$ 0.1 | 31.1 $\pm$ 0.1 | 167.5 $\pm$ 0.6 | 395.6 $\pm$ 1.6
QP3 | 77.5 $\pm$ 0.3 | 14.3 $\pm$ 0.1 | 26.1 $\pm$ 0.1 | 57.0 $\pm$ 0.2 | 252.1 $\pm$ 1.3
QP4 | 88.8 $\pm$ 0.3 | 13.1 $\pm$ 0.1 | 25.7 $\pm$ 0.1 | 62.6 $\pm$ 0.2 | 304.5 $\pm$ 1.2
APQ-TTDL | 290.5 $\pm$ 0.5 | 190.5 $\pm$ 1.3 | 243.4 $\pm$ 0.9 | 311.2 $\pm$ 0.8 | 365.6 $\pm$ 1.5
ML-OPT-TTDL | 227.8 $\pm$ 0.2 | 168.9 $\pm$ -3.9 | 204.4 $\pm$ 3.2 | 235.2 $\pm$ 0.4 | 278.7 $\pm$ 1.0
OPT-TTDL | 200.1 $\pm$ 1.2 | 185.9 $\pm$ -2.0 | 189.7 $\pm$ 2.2 | 192.0 $\pm$ 2.1 | 247.5 $\pm$ 1.4
APQ-C-30 | 268.8 $\pm$ 0.5 | 241.1 $\pm$ 1.3 | 254.2 $\pm$ 0.9 | 256.5 $\pm$ 0.6 | 342.1 $\pm$ 1.4
ML-OPT-C-30 | 225.9 $\pm$ 0.4 | 180.1 $\pm$ 1.4 | 205.1 $\pm$ 0.9 | 229.2 $\pm$ 0.6 | 276.2 $\pm$ 1.2
OPT-C-30 | 197.8 $\pm$ 0.4 | 186.8 $\pm$ 1.5 | 185.9 $\pm$ 0.9 | 188.9 $\pm$ 0.6 | 247.5 $\pm$ 1.2
APQ-C-15 | 269.9 $\pm$ 0.5 | 218.4 $\pm$ 1.3 | 246.2 $\pm$ 0.8 | 260.6 $\pm$ 0.7 | 363.4 $\pm$ 1.5
ML-OPT-C-15 | 225.9 $\pm$ 0.4 | 175.4 $\pm$ 1.4 | 202.4 $\pm$ 0.8 | 229.8 $\pm$ 0.6 | 281.5 $\pm$ 1.2
OPT-C-15 | 192.0 $\pm$ 0.4 | 178.2 $\pm$ 1.5 | 177.7 $\pm$ 0.8 | 182.5 $\pm$ 0.6 | 248.9 $\pm$ 1.2
APQ-TWT | 74.7 $\pm$ 0.2 | 14.9 $\pm$ 0.1 | 32.6 $\pm$ 0.1 | 65.7 $\pm$ 0.2 | 201.1 $\pm$ 1.0
ML-OPT-TWT | 77.4 $\pm$ 0.3 | 14.7 $\pm$ 0.1 | 26.3 $\pm$ 0.1 | 57.0 $\pm$ 0.2 | 250.7 $\pm$ 1.2
OPT-TWT | 82.2 $\pm$ 0.3 | 25.9 $\pm$ 0.5 | 38.2 $\pm$ 0.3 | 59.9 $\pm$ 0.2 | 246.2 $\pm$ 1.2
Table 9: Estimated Total Waiting Times $E(X_{\tilde{W}_{1}}$ and
$E(X_{\tilde{W}}$ and 95% Confidence Intervals per Selection Policy and Triage
Grade for the Base Scenario

Comparing expected total waiting times for patients leaving the ED after the
first consultation ($E(X_{\tilde{W}_{1}})$) and patients requiring a second
consultation ($E(X_{\tilde{W}})$) reveals the major difference in behavior of
the APQ and the ML-OPT policies. For all objectives that include TTD target
shares (i.e. TTDL, C-15 and C-30), ML-OPT (and also OPT) results in longer
expected waiting times than APQ for patients of categories 3-5 that leave the
ED after the first consultation. Especially for patients of category 5 the
difference is significant. However, ML-OPT (and also) OPT significantly reduce
the expected total waiting time $E(X_{\tilde{W}})$ for patients requiring a
second consultation regardless of the triage category.

In particular, the results of the APQ policy with respect to objectives C-15
and C-30 reveal only a slight increase in terms of $E(X_{\tilde{W}})$ over
patient categories 2-4, but significantly longer waiting times for patients of
category 5. This is especially interesting, as the APQ method is designed to
balance waiting times between high and low priority patients, but fails to do
so for patients of very low priority, i.e. patients of category 5. On the
other hand, for C-15 and C-30 objectives, the ML-OPT method results in a
steeper curve of $E(X_{\tilde{W}})$ values over patient categories but on a
generally lower level than the APQ policy . Hence, one may conclude that ML-
OPT is better suited to balance conflicting TTD target shares and expected
total waiting times for all patient categories. This may result from the
observation that APQ weight vectors $\beta$ for C-15 and C-30 only assign
about 1.1% of their mass to second consultation patients (see table 13), i.e.
APQ is only rarely mixing “new” and “old”. Clearly, the ML-OPT method results
in a more balanced mix of patients waiting for the first and second
consultation.

For the pure target-based objective TTDL, the results for APQ in terms of
$E(X_{\tilde{W}_{1}})$ are similar to the results for the combined measures.
This can be explained by the observations that $E(X_{\tilde{W}})$ values more
or less correlate with TTD target shares for those patients. However, results
regarding $E(X_{\tilde{W}})$ show significantly longer waiting times for
patients with low priority that require a second consultation. This could have
been expected as those waiting times do not contribute to the TTDL objective,
and consequently the sum of weights in vector $\beta$ for second consultation
patients is about 0.3% (see table 13). The performance of the ML-OPT method is
relatively similar for the TTDL and C-30 objectives. This may be because the
computation of the optimal solutions was done using a C-120 objective rather
than the pure TTDL objective, when training the ML models.

Results for the pure TWT objective show that both policies, i.e. APQ and ML-
OPT, result in waiting times for category 5 patients significantly longer than
for patients of higher priority. However, it has to be noted that APQ does
slightly better than QP3, while ML-OPT performs on a similar level compared to
the best pure priority based policy.

###  6.3 Additional Scenarios Results

In this section, we present results for scenarios with altered parameters. All
results were obtained by applying the C-15 objective, i.e. a mix of TTD target
shares and expected total waiting times.

Table 10 shows the objective values and improvement with respect to pure
priority based policies for all additional scenarios.

| Scenario
---|---
Selection Policy | HR | LV | HV | U | T3 | T5
QP1 | 669.3 | 324.6 | 340.7 | 327.5 | 706.6 | 232.9
QP2 | 795.2 | 375.5 | 416.3 | 375.8 | 812.1 | 256.4
QP3 | 977.8 | 564.6 | 600.0 | 519.0 | 1056.2 | 333.6
QP4 | 946.5 | 526.3 | 558.0 | 474.8 | 999.7 | 317.8
Min QPX | 669.3 | 324.6 | 340.7 | 327.5 | 706.6 | 232.9
APQ | 635.5 | 291.1 | 319.4 | 369.0 | 704.7 | 180.4
ML-OPT | 584.4 | 263.0 | 286.5 | 342.8 | 659.3 | 193.7
OPT | 516.3 | 244.3 | 250.2 | 284.6 | 546.6 | 164.7
APQ-Improvement | 5.1% | 10.3% | 6.2% | 5.5% | 0.3% | 22.5%
ML-OPT-Improvement | 12.7% | 19.0% | 15.9% | 12.2% | 6.7% | 16.8%
OPT-Improvement | 22.9% | 24.7% | 26.5% | 27.1% | 22.6% | 29.3%
Table 10: Summary of Objective Values $\hat{f}_{ED}$ and Improvement with
Respect to Pure Priority Based Queuing per Selection Policy for the Additional
Scenarios

Results for the HR scenario, i.e. an increased number of arriving patients,
show that the improvement of both the APQ and the ML-OPT policy is less than
for the base scenario. In other words, for a highly crowded ED there is less
potential for improvement than in the more typical setting of the base
scenario. However, it has to be noted that the crowded setting impacts the
improvement of the APQ method to a larger extent (i.e. from 9.8% to 5.1%) than
it impacts the ML-OPT method (i.e. from 17.5% to 12.7%). Comparing the values
of the (not realistic) OPT policy shows that there generally a lower potential
improvement that theoretically could be achieved, as the improvement drops
from 26.4% to 22.9%.

Under scenarios of higher and lower variance in the consultation time
distributions, i.e. LV and HV, the methods perform as expected. A lower
variance in consultation times leads to an increased improvement for both the
APQ and the ML-OPT method, and the improvement decreases for a setting with
higher consultation time variances. However, it is interesting to observe that
the APQ method seems to be impacted more than the ML-OPT method by increasing
stochasticity in the system.

The performance of the methods is more differentiated for scenarios that are
based on altered patient mixes. ML-OPT clearly outperforms APQ for scenario U,
where the probability of a patient belonging to triage categories 3-5 is
almost constant. However, the gap between ML-OPT and the theoretical optimum
OPT is significantly larger than for the base scenario. A possible explanation
could be, that for uniformly distributed patient arrivals of category 3-5, at
the time of decision making the variability of future states of the ED is
larger than for patient mixes with a dominant patient category.

The T3 scenario, i.e. a majority of patients arriving belong to category 3,
leads to significantly less improvement of all policies compared to QP1. In
particular, APQ does not yield any significant improvement, while ML-OPT only
improves the overall objective by 6.7%. Besides a low number of category 2
patients, the majority of patients have very high priority in this scenario.
This mix is even more extreme for second consultation queues, as category 3
patients more often require diagnostic tests and consequently a second
consultation. Hence, queues of category 3 patients will be most “crowded”,
which leaves less potential for improvement.

The APQ policy outperforms ML-OPT for scenario T5, i.e., the majority of
patients belonging to category 5. In some sense this scenario is the extreme
case of what motivates the APQ approach, with most patients having the lowest
priority, and so it is not surprising this method performs well.

To further analyse the behavior of the proposed policies for the additional
scenarios, Tables 11 and 12 report expected TTD target shares $E(X_{ttd}^{j})$
and expected waiting time measures $E(X_{\tilde{W}_{1}})$ and
$E(X_{\tilde{W}})$ per patient category $j$.

|  | Triage Category
---|---|---
Scenario | Selection Policy | Overall | 2 | 3 | 4 | 5
HR | Min QPX | 69.5% | 98.9% | 59.6% | 77.9% | 50.0%
APQ | 64.1% | 95.3% | 60.3% | 78.5% | 24.3%
ML-OPT | 70.3% | 97.5% | 59.6% | 77.6% | 54.9%
OPT | 72.4% | 97.3% | 63.1% | 78.9% | 58.2%
LV | Min QPX | 84.5% | 97.3% | 76.4% | 91.6% | 71.5%
APQ | 81.8% | 93.0% | 77.0% | 83.7% | 78.2%
ML-OPT | 84.3% | 96.6% | 76.7% | 88.4% | 77.9%
OPT | 82.9% | 98.8% | 78.4% | 85.3% | 76.5%
HV | Min QPX | 82.3% | 96.7% | 75.2% | 88.6% | 69.9%
APQ | 80.3% | 93.9% | 75.5% | 83.2% | 73.7%
ML-OPT | 81.7% | 95.2% | 75.3% | 85.7% | 74.4%
OPT | 82.4% | 98.0% | 77.7% | 85.3% | 75.0%
U | Min QPX | 81.9% | 99.0% | 74.5% | 91.4% | 76.2%
APQ | 79.7% | 93.2% | 74.7% | 82.5% | 79.2%
ML-OPT | 80.2% | 98.7% | 74.7% | 85.7% | 76.5%
OPT | 82.0% | 98.8% | 78.3% | 85.9% | 78.1%
T3 | Min QPX | 71.9% | 98.6% | 70.7% | 77.9% | 61.5%
APQ | 71.3% | 93.3% | 70.9% | 77.5% | 59.9%
ML-OPT | 71.4% | 97.3% | 70.1% | 76.5% | 62.3%
OPT | 74.1% | 95.7% | 75.6% | 75.6% | 62.8%
T5 | Min QPX | 90.8% | 99.3% | 76.2% | 96.1% | 93.2%
APQ | 81.2% | 94.9% | 77.0% | 83.5% | 80.2%
ML-OPT | 86.6% | 99.1% | 76.5% | 94.5% | 85.5%
OPT | 88.5% | 99.6% | 79.9% | 90.9% | 89.5%
Table 11: Estimated Share of Patients Meeting TTD target per Selection Policy
and Triage Grade for Additional Scenarios

As could have been expected, an increased arrival rate of patients decreases
the number of patients meeting TTD target times. While results for category 2
patients remain at a similar level compared to the base scenario, patients of
lower priority meet their target times less often. However, a large difference
between APQ and ML-OPT can only be observed for patients of category 5, where
ML-OPT significantly outperforms APQ.

In a similar fashion, altered consultation time variances seem to impact
category 5 patients the most, when comparing APQ and ML-OPT results.

For scenarios based on adapted patient mixes, only marginal differences below
corresponding thresholds can be observed for patients with priority 2 or 3
(see Table 11). For scenario U, ML-OPT prioritises category 4 patients,
leading to worse performance for category 5 patients compared to APQ. For
scenario T3 the ML-OPT policy leads to better results for category 5 patients,
while only performing slightly worse for category 4 patients. Interestingly,
ML-OPT prioritises category 4 patients in terms of TTD target shares for the
T5 scenario, but also increases the share of category 5 patients meeting
target times. However, it achieves this increase by “trading off against”
total waiting times, as shown in Table 12.

|  | $E(X_{\tilde{W}_{1}})$ | $E(X_{\tilde{W}})$
---|---|---|---
|  | Triage Category | Triage Category
|  | Overall | 2 | 3 | 4 | 5 | Overall | 2 | 3 | 4 | 5
HR | Min QPX | 57.4 | 3.4 | 21.6 | 30.9 | 145.0 | 437.4 | 323.2 | 377.1 | 462.9 | 527.2
APQ | 110.4 | 5.1 | 21.2 | 31.3 | 351.4 | 232.4 | 172.8 | 194.0 | 211.1 | 386.9
ML-OPT | 61.9 | 6.8 | 30.0 | 37.3 | 143.2 | 312.5 | 275.1 | 295.7 | 311.6 | 363.2
OPT | 67.8 | 7.3 | 39.0 | 41.6 | 151.2 | 275.8 | 252.3 | 255.6 | 263.3 | 356.2
LV | Min QPX | 32.6 | 3.1 | 13.5 | 18.9 | 78.4 | 295.0 | 198.3 | 241.0 | 316.8 | 376.2
APQ | 34.6 | 5.2 | 13.4 | 27.3 | 68.1 | 265.5 | 201.4 | 252.4 | 254.7 | 354.2
ML-OPT | 38.8 | 5.4 | 16.1 | 27.7 | 81.7 | 223.0 | 174.0 | 201.1 | 226.0 | 278.0
OPT | 43.2 | 5.3 | 22.2 | 31.1 | 88.1 | 195.2 | 182.2 | 180.9 | 185.2 | 253.0
HV | Min QPX | 35.4 | 3.2 | 13.9 | 21.2 | 84.3 | 296.9 | 202.0 | 245.4 | 317.3 | 376.8
APQ | 36.5 | 4.6 | 13.8 | 26.8 | 76.3 | 273.6 | 221.3 | 245.0 | 277.5 | 338.7
ML-OPT | 41.0 | 7.5 | 17.0 | 28.7 | 87.4 | 225.2 | 175.1 | 206.1 | 228.2 | 276.4
OPT | 43.9 | 5.7 | 21.9 | 30.9 | 91.4 | 190.4 | 172.5 | 174.3 | 182.9 | 247.3
U | Min QPX | 34.7 | 3.1 | 14.2 | 18.9 | 66.9 | 301.0 | 197.8 | 258.1 | 333.3 | 378.7
APQ | 37.2 | 5.6 | 14.3 | 27.8 | 65.0 | 271.0 | 162.9 | 255.7 | 303.4 | 303.9
ML-OPT | 43.2 | 3.9 | 15.4 | 30.4 | 78.7 | 222.6 | 161.1 | 197.1 | 237.0 | 275.1
OPT | 45.3 | 5.4 | 18.6 | 30.2 | 82.1 | 194.0 | 174.1 | 168.4 | 197.4 | 242.5
T3 | Min QPX | 45.0 | 3.4 | 15.9 | 33.5 | 107.6 | 364.7 | 243.0 | 336.5 | 439.8 | 468.8
APQ | 47.0 | 5.6 | 15.8 | 34.6 | 113.7 | 340.3 | 239.1 | 344.8 | 351.5 | 370.6
ML-OPT | 50.7 | 5.9 | 17.6 | 44.9 | 115.4 | 277.7 | 253.7 | 256.3 | 315.2 | 336.5
OPT | 59.2 | 7.6 | 18.8 | 59.7 | 131.6 | 237.2 | 214.5 | 208.7 | 280.0 | 319.2
T5 | Min QPX | 25.8 | 2.8 | 13.4 | 14.5 | 35.1 | 232.4 | 153.9 | 191.1 | 234.1 | 278.9
APQ | 44.8 | 5.2 | 13.2 | 27.5 | 62.6 | 146.2 | 17.4 | 66.9 | 150.4 | 229.4
ML-OPT | 39.3 | 3.4 | 13.9 | 19.5 | 56.2 | 162.5 | 74.5 | 109.9 | 130.7 | 235.8
OPT | 39.0 | 4.2 | 16.9 | 23.3 | 53.4 | 133.6 | 101.6 | 109.1 | 128.1 | 160.8
Table 12: Estimated Total Waiting Times $E(X_{\tilde{W}_{1}}$ and
$E(X_{\tilde{W}}$ per Selection Policy and Triage Grade for the Additional
Scenarios

Table 12 summarizes expected total waiting times for all additional scenarios
and the C-15 objective. Investigating results for the HR scenario reveals some
interesting behavior. While for the base scenario, the improvement of the ML-
OPT method compared to APQ mainly resulted from a reduction in
$E(X_{\tilde{W}})$, results for HR differ. $E(X_{\tilde{W}})$ for this
scenario is higher for patient categories 2-4 with the ML-OPT method than with
the APQ method, but ML-OPT performs significantly better for category 5
patients. Consequently, it may be concluded that the improvement of ML-OPT
mainly results from “not neglecting” those low priority patients.

Observed waiting time behavior for scenarios LV and HV is fairly similar
compared to the base scenario and objective C-15. For scenarios U and T3 the
improvement of ML-OPT seems to be due to the decreased $E(X_{\tilde{W}})$
compared to the APQ policy. This may lead to the same conclusion as for the
base scenario, i.e. that ML-OPT is able to achieve a improved mix of “new” and
“old” patients.

As outlined previously, the APQ method outperforms the ML-OPT method for the
T5 scenario. Analysing $E(X_{\tilde{W}_{1}})$ and $E(X_{\tilde{W}})$ measures
for OPT reveals that the theoretical optimal policy balances
$E(X_{\tilde{W}})$ over all triage grades, but at a significantly higher level
than APQ. While this leads to a theoretical improvement with respect to the
C-15 objective, ML-OPT fails to imitate this behavior, especially for category
5 patients. Hence, it seems that the general ML model that was parametrized
using the base scenario, is not capable of sufficiently capturing the dynamics
of this particular scenario.

To sum up, the ML-OPT approach outperforms the APQ policy for a majority of
settings where the performance metric is based on TTD targets and total
expected waiting times. In most cases, the performance gain is yielded by an
improved mix of first and second consultation patients.

##  7 Conclusion and Further Research

We proposed a learning-based patient selection method for assigning idle
resources, i.e. doctors, to waiting patients in an ED setting. Once a doctor
becomes idle, the policy picks a patient at the head of eight possible queues,
one for each triage category and type of consultation. Thereby, it aims to
imitate the behavior of (near) optimal solutions computed by a Simulated
Annealing algorithm. In particular, the Simulated Annealing algorithm is used
to heuristically optimize a large set of training instances, each covering a
single typical day of the ED setting. Thereby, the instances are treated as a
deterministic input, with a priori known patient arrivals, categories,
diagnostic requirements and consultation times. From this “data-base” of
optimal solutions, a feature set is created, which should comprehend
information on optimal decisions with respect to specific states of the ED.
This feature set is used to fit a standard machine learning classifier that
builds the basis for the proposed selection policy. The method is compared to
established patient selection policies, such as the APQ method. Mathematically
speaking, the main difference between the proposed method and APQ is that our
approach is based on a more comprehensive state representation as a basis for
decision making, and a more dynamic and non-linear selection function, i.e.
the ML model. The proposed method is evaluated using a Discrete Event
Simulation Model which imitates typical ED settings that can be found in
literature. Experiments show that the ML based policy outperforms the APQ
method in a majority of settings where the performance of the ED is assessed
by a combination of total waiting times and the share of patients meeting TTD
target times.

Results indicate that the proposed ML based selection method is able to
generate an improved mix of selected patients waiting to be seen either the
first or the second time by a doctor. This results in lower overall waiting
times, while keeping the share of patients meeting TTD targets at a comparable
level.

The presented method is evaluated using a typical ED setting that is not based
on a particular real world ED. Hence, future research should include the
transfer of the theoretical approach to a real world setting. Obviously, any
such setting must include a computerized ED management system, that could
incorporate such a ML based policy. Further, a simulation model of the real
system is required for training of the ML model. Although, this imposes some
barriers for the practical use of the method, the same holds for the APQ
method, as one may doubt that priority scores will ever be computed manually
to select patients. In addition, unless choosing weights arbitrarily, a
simulation model based weight optimization is also required for the APQ
method.

Further, the presented study aims to demonstrate the theoretical potential of
rather complex patient selection policies based on ML models. When
transferring the approach to a real world setting, it would be interesting to
investigate, if such complex policies could be used to derive simpler rule
sets that are more easily interpretable, but extend the strictly linear logic
of APQ. In other words, future research should include the design and
evaluation of models that cover the “middle ground” between simple APQ
policies and “black box” methods such as ML models. However, the results
presented in this paper, based on a “black box” method, clearly demonstrate
that there is a significant improvement potential for the management of ED
resources that can be achieved using more sophisticated state representations
and selection functions.

## References

  * [1] S. Acid, L. M. de Campos, J. M. Fernández-Luna, S. Rodrıguez, J. M. Rodrıguez, and J. L. Salcedo.  A comparison of learning algorithms for bayesian networks: a case study based on data from an emergency medical service.  Artificial intelligence in medicine, 30(3):215–232, 2004.
  * [2] M. Afilal, F. Yalaoui, F. Dugardin, L. Amodeo, D. Laplanche, and P. Blua.  Emergency department flow: A new practical patients classification and forecasting daily attendance.  IFAC-PapersOnLine, 49(12):721–726, 2016.
  * [3] M. A. Ahmed and T. M. Alkhamis.  Simulation optimization for an emergency department healthcare unit in kuwait.  European Journal of Operational Research, 198(3):936–942, 2009\.
  * [4] O. M. Ashour and G. E. Okudan Kremer.  A simulation analysis of the impact of fahp-maut triage algorithm on the emergency department performance measures.  Expert Systems with Applications, 40(1):177–187, 2013.
  * [5] N. Azi, M. Gendreau, and J.-Y. Potvin.  An adaptive large neighborhood search for a vehicle routing problem with multiple routes.  Computers & Operations Research, 41:167–173, 2014.
  * [6] A. E. Bair, W. T. Song, Y.-C. Chen, and B. A. Morris.  The impact of inpatient boarding on ed efficiency: a discrete-event simulation study.  Journal of medical systems, 34(5):919–929, 2010.
  * [7] I. Bello, H. Pham, Q. V. Le, M. Norouzi, and S. Bengio.  Neural combinatorial optimization with reinforcement learning.  arXiv preprint arXiv:1611.09940, 2016.
  * [8] S. Benbelkacem, F. Kadri, B. Atmani, and S. Chaabane.  Machine learning for emergency department management.  International Journal of Information Systems in the Service Sector (IJISSS), 11(3):19–36, 2019.
  * [9] Y. Bengio, A. Lodi, and A. Prouvost.  Machine learning for combinatorial optimization: A methodological tour d’horizon.  European Journal of Operational Research, 290(2):405–421, 2021\.
  * [10] X. Cai, O. Perez-Concha, E. Coiera, F. Martin-Sanchez, R. Day, D. Roffe, and B. Gallego.  Real-time prediction of mortality, readmission, and length of stay using electronic health record data.  Journal of the American Medical Informatics Association, 23(3):553–561, 2015.
  * [11] R. Carmen, M. Defraeye, and I. Van Nieuwenhuyse.  A decision support system for capacity planning in emergency departments.  International Journal of Simulation Modelling, 14(2):299–312, 2015\.
  * [12] R. Ceglowski, L. Churilov, and J. Wasserthiel.  Combining data mining and discrete event simulation for a value-added view of a hospital emergency department.  Journal of the Operational Research Society, 58(2):246–254, 2007\.
  * [13] S. Chonde, C. Parra, and C.-J. Chang.  Minimizing flow-time and time-to-first-treatment in an emergency department through simulation.  In 2013 Winter Simulations Conference (WSC), pages 2374–2385, 2013\.
  * [14] M. Cildoz, A. Ibarra, and F. Mallor.  Accumulating priority queues versus pure priority queues for managing patients in emergency departments.  Operations Research for Health Care, 23:100224, 2019.
  * [15] L. G. Connelly and A. E. Bair.  Discrete event simulation of emergency department activity: A platform for system-level operations research.  Academic Emergency Medicine, 11(11):1177–1185, 2004.
  * [16] I. Eigner, F. Bodendorf, and N. Wickramasinghe.  A theoretical framework for research on readmission risk prediction.  In 32ND Bled eConference Humanizing Technology for a Sustainable Society, pages 387 – 410, 2019.
  * [17] H. Eskandari, M. Riyahifard, S. Khosravi, and C. D. Geiger.  Improving the emergency department performance using simulation and mcdm methods.  In Proceedings of the 2011 Winter Simulation Conference (WSC), pages 1211–1222, 2011.
  * [18] P. Facchin, E. Rizzato, and G. Romanin-Jacur.  Emergency department generalized flexible simulation model.  In 2010 IEEE Workshop on Health Care Management (WHCM), pages 1–6, 2010.
  * [19] Y. B. Ferrand, M. J. Magazine, U. S. Rao, and T. F. Glass.  Managing responsiveness in the emergency department: Comparing dynamic priority queue with fast track.  Journal of Operations Management, 58-59:15–26, 2018.
  * [20] M. Fischetti and M. Fraccaro.  Machine learning meets mathematical optimization to predict the optimal production of offshore wind parks.  Computers & Operations Research, 106:289–297, 2019.
  * [21] N. Furian, C. Gutschi, D. Neubacher, C. Walker, and M. O’Sullivan.  The activity-entity-impact method: Understanding bottleneck behavior of simulation models demonstrated by an emergency department model.  In 2019 Winter Simulation Conference (WSC), pages 1148–1159, 2019\.
  * [22] N. Furian, D. Neubacher, M. O’Sullivan, C. Walker, and C. Pizzera.  Gedmod - towards a generic toolkit for emergency department modeling.  Simulation Modelling Practice and Theory, 87:239–273, 2018.
  * [23] N. Furian, M. O’Sullivan, C. Walker, and E. Çela.  A machine learning-based branch and price algorithm for a sampled vehicle routing problem.  OR Spectrum, pages 1–40, 2021.
  * [24] N. Furian, M. O’Sullivan, C. Walker, S. Voessner, and D. Neubacher.  A conceptual modeling framework for discrete event simulation using hierarchical control structures.  Simulation Modelling Practice and Theory, 56:82–96, 2015.
  * [25] J. Futoma, J. Morris, and J. Lucas.  A comparison of models for predicting early hospital readmissions.  Journal of Biomedical Informatics, 56:229–238, 2015.
  * [26] K. Ghanes, O. Jouini, Z. Jemai, M. Wargon, R. Hellmann, V. Thomas, and G. Koole.  A comprehensive simulation modeling of an emergency department: A case study for simulation optimization of staffing levels.  In Proceedings of the Winter Simulation Conference 2014, pages 1421–1432, 2014.
  * [27] B. Graham, R. Bond, M. Quinn, and M. Mulvenna.  Using data mining to predict hospital admissions from the emergency department.  IEEE Access, 6:10458–10469, 2018.
  * [28] M. Gul and A. F. Guneri.  A comprehensive review of emergency department simulation applications for normal and disaster conditions.  Computers & Industrial Engineering, 83:327–344, 2015.
  * [29] M. M. Gunal and M. Pidd.  Understanding accident and emergency department performance using simulation.  In Proceedings of the 2006 Winter Simulation Conference, pages 446–452, 2006.
  * [30] A. M. Hay, E. C. Valentin, and R. A. Bijlsma.  Modeling emergency care in hospitals: A paradox - the patient should not drive the process.  In Proceedings of the 2006 Winter Simulation Conference, pages 439–445, 2006.
  * [31] L. B. Holm and F. A. Dahl.  Simulating the effect of physician triage in the emergency department of akershus university hospital.  In Proceedings of the 2009 Winter Simulation Conference (WSC), pages 1896–1905, 2009.
  * [32] N. R. Hoot and D. Aronsky.  Systematic review of emergency department crowding: Causes, effects, and solutions.  Annals of Emergency Medicine, 52(2):126–136.e1, 2008.
  * [33] N. R. Hoot, L. J. LeBlanc, I. Jones, S. R. Levin, C. Zhou, C. S. Gadd, and D. Aronsky.  Forecasting emergency department crowding: A discrete event simulation.  Annals of Emergency Medicine, 52(2):116–125, 2008.
  * [34] A. Hottung, S. Tanaka, and K. Tierney.  Deep learning assisted heuristic tree search for the container pre-marshalling problem.  Computers & Operations Research, 113:104781, 2020.
  * [35] J. Huang, B. Carmeli, and A. Mandelbaum.  Control of patient flow in emergency departments, or multiclass queues with deadlines and feedback.  Operations Research, 63(4):892–908, 2015.
  * [36] C. K. Joshi, T. Laurent, and X. Bresson.  An efficient graph convolutional network technique for the travelling salesman problem.  arXiv preprint arXiv:1906.01227, 2019.
  * [37] H. Kang and J. M. Lobo.  Identifying the optimal configuration of an express care area in an emergency department: A des and metamodeling approach.  In 2016 Winter Simulation Conference (WSC), pages 1961–1969, 2016\.
  * [38] L. Kleinrock.  A delay dependent queue discipline.  Naval Research Logistics ly, 11(3-4):329–341, 1964.
  * [39] R. Konrad, K. DeSotto, A. Grocela, P. McAuley, J. Wang, J. Lyons, and M. Bruin.  Modeling the impact of changing patient flow processes in an emergency department: Insights from a computer simulation study.  Operations Research for Health Care, 2(4):66–74, 2013.
  * [40] J. Krämer, J. Schreyögg, and R. Busse.  Classification of hospital admissions into emergency and elective care: a machine learning approach.  Health care management science, 22(1):85–105, 2019.
  * [41] M. Laguna.  Optimization of complex systems with optquest.  A White Paper from OptTek Systems, Inc, 1997.
  * [42] M. Laskowski.  A prototype agent based model and machine learning hybrid system for healthcare decision support.  International Journal of E-Health and Medical Communications (IJEHMC), 2(4):67–90, 2011.
  * [43] E. K. Lee, F. Yuan, D. A. Hirsh, M. D. Mallory, and H. K. Simon.  A clinical decision tool for predicting patient care characteristics: patients returning within 72 hours in the emergency department.  In AMIA Annual Symposium Proceedings, volume 2012, page 495. American Medical Informatics Association, 2012.
  * [44] S.-Y. Lee, R. B. Chinnam, E. Dalkiran, S. Krupp, and M. Nauss.  Prediction of emergency department patient disposition decision for proactive resource allocation for admission.  Health care management science, pages 1–21, 2019.
  * [45] M. E. Lim, A. Worster, R. Goeree, and J.-É. Tarride.  Simulating an emergency department: the importance of modeling the interactions between physicians and delegates in a discrete event simulation.  BMC Medical Informatics and Decision Making, 13(1):59, May 2013\.
  * [46] C.-H. Lin, C.-Y. Kao, and C.-Y. Huang.  Managing emergency department overcrowding via ambulance diversion: A discrete event simulation model.  Journal of the Formosan Medical Association, 114(1):64–71, 2015\.
  * [47] A. Lodi, L. Mossina, and E. Rachelson.  Learning to handle parameter perturbations in combinatorial optimization: An application to facility location.  EURO Journal on Transportation and Logistics, 9(4):100023, 2020\.
  * [48] N. B. Menke, N. Caputo, R. Fraser, J. Haber, C. Shields, and M. N. Menke.  A retrospective analysis of the utility of an artificial neural network to predict ed volume.  The American journal of emergency medicine, 32(6):614–617, 2014\.
  * [49] M. Mes and M. Bruens.  A generalized simulation model of an integrated emergency post.  In Proceedings of the 2012 Winter Simulation Conference (WSC), pages 1–11, 2012.
  * [50] J. C. Moskop, D. P. Sklar, J. M. Geiderman, R. M. Schears, and K. J. Bookman.  Emergency department crowding, part 1 - concept, causes, and moral consequences.  Annals of Emergency Medicine, 53(5):605–611, 2009.
  * [51] M. Nazari, A. Oroojlooy, L. V. Snyder, and M. Takáč.  Reinforcement learning for solving the vehicle routing problem.  arXiv preprint arXiv:1802.04240, 2018.
  * [52] C. Oh, A. M. Novotny, P. L. Carter, R. K. Ready, D. D. Campbell, and M. C. Leckie.  Use of a simulation-based decision support tool to improve emergency department throughput.  Operations Research for Health Care, 9:29–39, 2016.
  * [53] J. A. Paul and L. Lin.  Models for improving patient throughput and waiting at hospital emergency departments.  The Journal of Emergency Medicine, 43(6):1119–1126, 2012.
  * [54] S. A. Paul, M. C. Reddy, and C. J. DeFlitch.  A systematic review of simulation studies investigating emergency department overcrowding.  SIMULATION, 86(8-9):559–571, 2010.
  * [55] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay.  Scikit-learn: Machine learning in Python.  Journal of Machine Learning Research, 12:2825–2830, 2011.
  * [56] S. Rana, S. Gupta, D. Phung, and S. Venkatesh.  A predictive framework for modeling healthcare data with evolving clinical interventions.  Statistical Analysis and Data Mining: The ASA Data Science Journal, 8(3):162–182, 2015.
  * [57] M. Reuter-Oppermann and N. Kühl.  Artificial Intelligence for Healthcare Logistics: An Overview and Research Agenda, pages 1–22.  Springer International Publishing, Cham, 2021.
  * [58] S. Saghafian, G. Austin, and S. J. Traub.  Operations research/management contributions to emergency department patient flow optimization: Review and research prospects.  IIE Transactions on Healthcare Systems Engineering, 5(2):101–123, 2015.
  * [59] S. Saghafian, W. J. Hopp, M. P. Van Oyen, J. S. Desmond, and S. L. Kronick.  Patient streaming as a mechanism for improving responsiveness in emergency departments.  Operations Research, 60(5):1080–1097, 2012.
  * [60] D. A. Stanford, P. Taylor, and I. Ziedins.  Waiting time distributions in the accumulating priority queue.  Queueing Systems, 77(3):297–330, 2014.
  * [61] K. W. Tan, C. Wang, and H. C. Lau.  Improving patient flow in emergency department through dynamic priority queue.  In 2012 IEEE International Conference on Automation Science and Engineering (CASE), pages 125–130, 2012.
  * [62] L. Turgeman and J. H. May.  A mixed-ensemble model for hospital readmission.  Artificial intelligence in medicine, 72:72–82, 2016.
  * [63] A. G. Uriarte, E. R. Zúñiga, M. U. Moris, and A. H. Ng.  How can decision makers be supported in the improvement of an emergency department? a simulation, optimization and data mining approach.  Operations Research for Health Care, 15:102–122, 2017.
  * [64] M. van de Vrugt and R. J. Boucherie.  Assigning treatment rooms at the emergency department.  Operations Research for Health Care, 8:62–70, 2016.
  * [65] S. J. Weiss, R. Derlet, J. Arndahl, A. A. Ernst, J. Richards, M. Fernández-Frankelton, R. Schwab, T. O. Stair, P. Vicellio, D. Levy, M. Brautigan, A. Johnson, and T. G. Nick.  Estimating the degree of emergency department overcrowding in academic medical centers: Results of the national ed overcrowding study (nedocs).  Academic Emergency Medicine, 11(1):38–50, 2004.
  * [66] D. Whitley.  A genetic algorithm tutorial.  Statistics and computing, 4(2):65–85, 1994.
  * [67] N. Wickramasinghe, D. M. Delgano, and S. Mcconchie.  Real-time prediction of the risk of hospital readmissions.  In 32ND Bled eConference Humanizing Technology for a Sustainable Society, pages 85 – 102, 2019.
  * [68] M. Xu, T. C. Wong, and K.-S. Chin.  Modeling daily patient arrivals at emergency department and quantifying the relative importance of contributing variables using artificial neural network.  Decision Support Systems, 54(3):1488–1498, 2013.
  * [69] K. K. Yang, S. S. W. Lam, J. M. Low, and M. E. H. Ong.  Managing emergency department crowding through improved triaging and resource allocation.  Operations Research for Health Care, 10:13–22, 2016.
  * [70] F. Zeinali, M. Mahootchi, and M. M. Sepehri.  Resource planning in the emergency departments: A simulation-based metamodeling approach.  Simulation Modelling Practice and Theory, 53:123–138, 2015.
  * [71] B. Zheng, J. Zhang, S. W. Yoon, S. S. Lam, M. Khasawneh, and S. Poranki.  Predictive modeling of hospital readmissions using metaheuristics and data mining.  Expert Systems with Applications, 42(20):7110–7120, 2015.

##  Appendix A Appendix

###  A.1 APQ Weight Optimization

The implementation of APQ patient selection requires the definition of weights
$\beta=(\beta_{j,k})_{j\in J,1\leq k\leq 2}$. [14] use the OptQuest
simulation-optimization package within the Arena simulation tool [41] to
optimize weights $\beta$. In this paper, we use a genetic algorithm to
optimize weights $\beta$ for a training set of simulation instances. Note that
the training set may differ over scenarios, which have been defined in Section
5.2, but the main optimization procedures remain the same and will be outlined
in the following. For a general description of genetic algorithms the reader
is referred to [66].

The main principle of the algorithm is outlined in Figure 7.

Figure 7: General Principle of the Genetic Algorithm Used to Optimize APQ
Weights $\beta$

The algorithm is initialized with a set of randomly generated start solutions,
i.e. randomly generated weight vectors $\beta$. To evaluate the fitness of an
individual, a set of patients and associated data (i.e. arrival times, triage
grades, activity durations and diagnostic requirements) is fed to the
simulated model described in section 3. The APQ method is used as a patient
selection mechanism and the objective value $\hat{f}_{ED}$ is computed based
on the data stored by the simulation.

The population at each generation during the execution of the genetic
algorithm is divided into an elite-set of individuals (those with lowest
objective value) and a non-elite-set of individuals (the remaining
individuals). The newly formed population consists of the current elite-set, a
set of off-spring generated by a crossover operator and randomly generated
mutant individuals (computed in the same way as the starting population).
Parent individuals for crossover are randomly selected from the elite and non-
elite set, i.e. one from each pool. The crossover operator randomly computes
weights for each patient category individually, by using a Gaussian
distribution with mean at the center of the two parent weights for that
particular patient category and a standard deviation chosen as the square of
the distance of the two weights divided by 3, see Figure 7. Further, each
offspring is subjected to a mutation operator with respect to a chosen
mutation probability.

The fitness of newly generated individuals is evaluated using the simulation
model, as described above. Finally, all individuals of the new population are
sorted with respect to their fitness value and the elite and non-elite sets
are updated.

To avoid getting stuck in a local minimum, i.e. low heterogeneity in the
population, a diversification step is included in the algorithm. The
diversification step is applied if two conditions are satisfied: (1) the
fitness values of the dominant half of the population (i.e. with lower fitness
values) varies less than 1% and (2) during the last 50 generations no
improvement of the overall best solution was achieved.

As mentioned above, the generated training instances build the basis for the
computation of objective values within the genetic algorithm. In particular,
to compute the fitness of a weight vector $\beta$, all 10,000 training
instances are simulated and the resulting patient data sets are aggregated to
a single data set that is then used to compute the corresponding objective
value.

The genetic algorithm is parametrized as follows. The population size is
chosen to be 80 individuals, out of which 20 form the elite set of each
generation. Of the remaining 60 individuals, 50 are generated by the crossover
operator while 10 are randomly generated. The mutation probability $p_{mut}$
for each offspring is set to 0.75, i.e. with a 75% chance weight is shifted
from $\beta_{i}$ to $\beta_{j}$ where $i$ and $j$ are randomly selected. The
share of weight $\beta_{i}$ that is shifted, is uniformly chosen between 0 and
50%.

After each generation of a new weight vector $\beta$ (either by mutation or
crossover), $\beta$ is corrected to satisfy two pre-defined conditions. First,
the sum of all weights must be 10 (to ensure comparability). Second, weights
per consultation type must be strictly decreasing with respect to patient
priorities, i.e. $b_{i,k}>b_{j,k}$ for $i<j$ and $k=1,2$. Note that randomly
generated weight vectors are also required to satisfy those constraints.

Finally, the number of generations is chosen as 200. The resulting weights per
scenario and objective functions are summarized in Table 13.

Scenario | Objective | $\beta_{2,1}$ | $\beta_{3,1}$ | $\beta_{4,1}$ | $\beta_{5,1}$ | $\sum\beta_{j,1}$ | $\beta_{2,2}$ | $\beta_{3,2}$ | $\beta_{4,2}$ | $\beta_{5,2}$ | $\sum\beta_{j,2}$
---|---|---|---|---|---|---|---|---|---|---|---
Base | TTDL | 6.727 | 2.587 | 0.480 | 0.174 | 9.967 | 0.021 | 0.008 | 0.003 | 0.001 | 0.033
C-30 | 6.606 | 2.522 | 0.558 | 0.199 | 9.885 | 0.035 | 0.032 | 0.032 | 0.016 | 0.115
C-15 | 6.737 | 2.438 | 0.524 | 0.187 | 9.886 | 0.041 | 0.033 | 0.030 | 0.011 | 0.114
TWT | 3.335 | 0.157 | 0.060 | 0.016 | 3.568 | 3.421 | 1.853 | 0.579 | 0.579 | 6.432
HR | C-15 | 7.400 | 2.086 | 0.392 | 0.006 | 9.884 | 0.035 | 0.029 | 0.026 | 0.026 | 0.116
LV | 6.375 | 2.720 | 0.548 | 0.211 | 9.855 | 0.055 | 0.037 | 0.037 | 0.016 | 0.145
HV | 7.206 | 2.184 | 0.409 | 0.124 | 9.923 | 0.026 | 0.022 | 0.018 | 0.011 | 0.077
U | 6.8124 | 2.2876 | 0.5341 | 0.2124 | 9.847 | 0.0713 | 0.0332 | 0.0246 | 0.0245 | 0.154
T3 | 8.016 | 1.732 | 0.213 | 0.023 | 9.984 | 0.007 | 0.003 | 0.003 | 0.003 | 0.016
T5 | 5.4179 | 2.8171 | 0.4584 | 0.2023 | 8.896 | 0.814 | 0.1625 | 0.082 | 0.0457 | 1.104
Table 13: APQ Weights for Different Scenario and Objective Function
Combinations Computed by the Genetic Algorithm

For the base scenario, one may observe that weights for the first consultation
are relatively similar if the objective function includes the TTD target. In
particular, weights for category 2 patients are approximately 2.5 to 3 times