shown in Figure \[phaseprofile\], the second stage in the pipeline is the YARN log parser. For each job and tasks, YARN logs are collated, aggregated and parsed to extract the relevant counters and their respective values for further processing. These new data are grouped into the various phases of the MapReduce framework. The final stage of the phase profiling pipeline uses linear regression to generate the actual model parameters using the $R$ statistical programming language

Generating the Actual Models and Parameters Using Regression and Cross Validation {#actualmodels}
=================================================================================

Figure \[crossvalidation\] gives the results of the models on the processed YARN log data, using a 10 fold cross-validation on each dataset.

![**Results of 10 Fold Cross-Validation and Prediction on the 8 Nodes cluster**[]{data-label="crossvalidation"}](images/crossvalidated){width="6.5cm"}

The observations for each are discussed in the following subsections. The plot shows the results of the predicted and actual values. To select the right parameters and evaluate the accuracy of the model of each phase, we consider the best practices for model selection. For each model, we study the effect of $p$-value, *Root Mean Squared Error* $RMSE$ [@barnston1992correspondence], $Adjusted - R-squared$ [@miles2005r] and $Multiple-R-squared$ [@miles2005r]. First, to select the most important parameters of the model, we performed 