 EDs is particularly problematic since delayed treatment has been linked with numerous negative clinical outcomes \cite{sudarshan2021performance}. However, inadequate staff availability also carries risks which may lead to an unsafe practice resulting from acute conditions being overlooked \cite{Batal2001}.

Therefore it is vital to ensure that strategies are implemented which can efficiently allocate human resources and manage patient demand in these contexts. Effective forecasting of patient demand is one of the strategies which can enable more optimized resourcing of medical services beyond primary care providers. Forecasting of this kind can be in the form of predicting daily patient arrivals as well as more granular predictions at an hourly basis. Forecasting models that also alert in real-time of impending surges in patient arrivals has also been recognized as having high value.

This study develops patient forecasting models for two UCCs providing services to patients experiencing sudden illness or accident-related injury in the Auckland region of New Zealand.  The clinics provide patient walk-in as well as overflow services to local hospitals when their EDs are congested, resulting in ambulances being diverted to them. 

Predicting patient numbers for the UCCs, and similarly for EDs, is a challenging task as patterns from influencing factors have been shown to possess significant variability over time.  The COVID-19 pandemic has added an additional level of complexity from 2020 onward, complicating the existing in-house strategies used by the clinics' administrators for developing effective roster schedules.

Existing literature has primarily focused on predicting patient demand at EDs with limited studies covering UCCs. Generally, most studies have employed traditional statistical methods in preference to machine learning approaches. Some have used Artificial Neural Networks (ANNs) \cite{Khatri2017,yucesan2018multi,khaldi2019forecasting}. Majority of models in literature have incorporated calendar variables and public holiday indicators, with some extending these holidays to the immediately subsequent days and including school holiday flags.  A minority of studies explored features capturing relevant Google search terms \cite{ho2019forecasting}, weather \cite{sudarshan2021performance} and pollution information \cite{sahu2014hierarchical}. However, no literature exists highlighting the use of machine learning to predict patient demand in UCCs within the New Zealand context, having the capability to incorporate mechanisms to handle fluctuating demand caused by the COVID-19 pandemic. 

The aim of this study is to leverage machine learning in order to improve on existing resource planning strategies within the clinics and improve patient-demand forecasting in order to help provide high quality patient care at all times. The study develops models which predict patient demand at a daily basis with the predictions extending three months ahead. The paper presents comprehensive comparisons of forecasting models generated by numerous algorithms and provides analyses into the driving factors influencing patient presentation volumes. The study is specifically targetted for UCCs; however, the methods and findings are also transferable for EDs.  \par

The research questions (RQ) addressed in this paper are:
\begin{itemize}
\item (RQ1) Are machine learning models able to improve on existing in-house strategies for forecasting patient demand at UCCs?

\item (RQ2) What are the most effective machine learning algorithms for predicting daily demand at UCCs across a three-month forecasting horizon?

\item (RQ3) Which features are the key drivers in predicting patient demand at UCCs?
\end{itemize}

\section{Literature Review}

Given the paucity of prior research in forecasting demand for UCCs as well as some similarity between UCCs and the EDs, we include in this literature review studies that have focused on EDs. While UCCs are specifically designed to manage lower-acuity conditions than EDs, it has been estimated that up to around a quarter of all ED visits could effectively be serviced by UCCs \cite{carlson2020impact}. Findings\footnote{Royal New Zealand College of Urgent Care}  indicate that cities with UCCs have significantly lower ED presentations, thus confirming the correlation between the patient demand for EDs and UCCs, and ultimately the existence of similar drivers of demand across both types of medical facilities. Therefore, approaches that have been effective on the ED domain include transferable insights for the UCC context, and vice versa. Indeed some prior studies \cite{Wargon395} have used literature from EDs and UCCs interchangeably.     

Some of the earliest forecasting research into patient demand has made use of the simple modelling approaches like linear regression for time-series prediction by making adjustments to the input data to incorporate calendar information. Seasonal and cyclic effects can be added using dummy variables representing the day of the week, month of the year, public and school holiday flags and more. \citet{Batal2001} successfully used this modelling approach at the UCC located at the Denver Health Medical Center in Colorado, USA.  With the inclusion of weather and calendar variables to the model, they showed weather contributed little value to the accuracy of predictions.  However with incorporating a selection of calendar variables like day of the week, month, season, and holiday flags, they believed their regression models produced more accurate forecasts of patient numbers. The average daily patient volume at the clinic varied between 115 on Mondays decreasing steadily down to 69 on Sundays and their study highlighted the expected increase in patients during the winter months. Interestingly they discovered an 11\% increase in patient presentations following a holiday, with patients choosing to delay medical help to avoid holiday disruption or participating in holiday activities, thus seeking medical help the next day.

\citet{Jones2008} used a similar approach with multiple linear regression to create a benchmark model for predicting patient presentations in three hospital EDs in Utah and Idaho, USA.  They included calendar variables representing day of the week, month of the year, and holiday flags. Unlike the clinics studied by \citet{Batal2001}, patient presentations were higher in the weekend and dipped mid-week. They enhanced their model to include interaction terms between calendar variables and climatic variables.  By accounting for serial autocorrelation using ARIMA models, they saw little improvement in the accuracy of their predictions.  The ARIMA models were enhanced to SARIMA models to account for seasonality, however they noted that these univariate models could not incorporate other factors like calendar and weather variables.  After considering an exponential smoothing model and a neural network model, they concluded their regression models with calendar variables resulted in the best predictions, with potential improvement in accuracy by including location specific information such as special days.  Their research used a forecast horizon of 30 days with an iterative testing strategy. The model was estimated on a training set and measured on an unseen 30 day test set. 

\citet{Aboagye_Sarfo_2015} used both multivariate vector VARMA and univariate ARMA time series models to forecast patient demand in public hospital emergency departments throughout the state of Western Australia.  Their work identified the VARMA multivariate models were more accurate than univariate ARMA models to predict demand for the purpose of resource allocation and strategic planning.  The study did not incorporate drivers for patient presentations, however it identified seasonal trends by age group, place of treatment, triage category and disposition. The data was supplied on a monthly granularity, limiting its usefulness. However the models were built on 5 years of data and tested using a 2 year period, with MAE, RMSE and MAPE  evaluation metrics calculated, providing a more stable evaluation than \citet{Jones2008}. 

Part of the study carried out by \citet{Boyle2011} analysed ED presentations in two Queensland hospitals in Australia over 5 years from 2002 to 2007.  They used multiple regression, ARIMA and exponential smoothing models to predict daily demand, evaluating performance with the MAPE metric only.  They noted that days with matching characteristics, like day of week and public holidays tended to have the most similar patient demand. Highlighting the importance of testing model performance over both the summer and winter months, they allocated the final year in the dataset  to the testing period. Rather than using a gradually expanding training set and a moving test set, they kept the test set static and used 4 training sets, from 1 to 4 years in duration. To investigate the applicability of their models to other emergency departments, they tested them on data from 2005 to 2009 for 27 other Queensland hospitals.

\citet{Champion_2007} built forecasting models to predict the monthly demand at a hospital in the Australian state of Victoria.  Using data from 2000 to 2005, they used exponential smoothing and ARIMA methods to predict demand for the first 5 months of 2006.  Using RMSE metrics to compare models, they concluded their best exponential smoothing model with a simple seasonal component had smaller error than their best ARIMA(0,1,1) non-seasonal moving average model. However resulting forecasts from both models were similar.  Their exponential model predicted seasonal fluctuations with ease however no performance improvement was seen when including seasonal terms into the ARIMA model.  But as previously discussed, measuring model performance with less than a 1-year cycle of data may produce misleading conclusions about forecast accuracy, and forecasting at such large granularity of monthly demand provides limited usefulness in resource allocation.


\citet{Jilani2019} present a unique study using Fuzzy Time Series (FTS), a concept taken from Fuzzy Logic \cite{Aladag2017}.  Using data from 2011 - 2015 over four emergency departments in the UK, they predicted daily demand for a 4 week horizon. They created a time-series model for each day of the week as a method of modelling week day variations, then built a FTS model for each day.  Performance was measured using MAPE and RM