title: Plot learning curve for multiclass classification with cross validation using caret package in R
body: <p>I have a multiclass classification problem (with 10 classes)that I am trying to solve using the neural network option 'mxnet' in the caret package in R. I'm using a 10-fold cross validation during training and would like to plot a learning curve for this to figure out whether/how the model is overfitting. I have modified the solution given in this post (<a href="https://stackoverflow.com/questions/20370827/plot-learning-curves-with-caret-package-and-r#">Plot learning curves with caret package and R</a>) to fit my data. However, since the learning curve is being recorded over each one of the resamples, not all factors/classes (1-10) are present in each fold, which leads to the following error: </p>
  
  <blockquote>
    <p>Error: One or more factor levels in the outcome has no data </p>
  </blockquote>
  
  <p>I have also tried to use the builtin function of caret with learning_curve_dat, but I encounter the same error message. </p>
  
  <p>Is there a way to bypass this problem of not all factors being present in each one of the folds? </p>