model the performance of applications?

-   RQ2: Can we measure how good the effect of RQ1 would have on an arbitrary cluster?

In this paper, we develop a parametric model to estimate execution time of a given MapReduce application for a given cluster of nodes running YARN  [@vavilapalli2013apache] (MapReduce 2.0 or greater). To achieve this, we first study and identify the performance drivers of each generic phase of the MapReduce application execution life cycle. This is achieved by executing generic MapReduce workloads to collect relevant performance metrics from logs. The combination of varying modelling approaches and cross-validation methods are used to obtain the models with their respective parameters. We have also developed a proof of concept application to test the models with different datasets. This links for the application and source code are is available at:

<https://sc306.host.cs.st-andrews.ac.uk/cbm/> and\
<https://github.com/sneceesay77/mr-performance-modelling>

The main contributions of our work are:

-   A parametric models for predicting the execution time of a big data application running on a YARN cluster.

-   A generic performance modelling approach to model the execution time of MapReduce (YARN) [@vavilapalli2013apache] applications tested on different clusters and data sizes.

-   An insight into the performance characteristics of the generic phases of 