title: Unable to get the proper multi-class ROC Curve using the pROC library in R
body: <p>My prediction column has spam, not spam and undefinable. I used the ensemble method-stacking method to predict it.I was able to get an accuracy of around 77% I am able to plot the ROC Curve but I just don't think it is right. </p>
  
  <p>Ensemle Technique code:</p>
  
  <pre><code># Generate level-one dataset for training the ensemble metalearner
  predDF &lt;- data.frame(dataTest.pred, NB_Predictions, RF_Predictions,SVM_Predictions, spam = validation$spam, stringsAsFactors = F)
  
  # Train the ensemble
  
  # define training control
  set.seed(1958)
  # Train the ensemble
  modelStack &lt;- caret::train(spam ~ ., data = predDF, method = "rf")
  </code></pre>
  
  <p>Here is my code for the ROC:</p>
  
  <pre><code>#ROC Curve
  pre&lt;-predict(modelStack, testPredLevelOne,type='prob')
  # AUC measure
  modelroc = mean(
    c(as.numeric(multiclass.roc(testPredLevelOne$spam, pre[,1])$auc),
      as.numeric(multiclass.roc(testPredLevelOne$spam, pre[,2])$auc),
      as.numeric(multiclass.roc(testPredLevelOne$spam, pre[,3])$auc)
    )
  )
  </code></pre>
  
  <p>I don't think the AUC measure is also right.</p>
  
  <pre><code>&gt; modelroc
  [1] 0.500903
  </code></pre>
  
  <p>Here is another method that I tried to plot the ROC curve. I don't think that is working either.</p>
  
  <pre><code>#Method 2 ROC
  predictions&lt;-as.numeric(predict(modelStack, testPredLevelOne))
  roc.multi&lt;-multiclass.roc(testPredLevelOne$spam,predictions)
  auc(roc.multi)
  rs &lt;- roc.multi[['rocs']]
  plot.roc(rs[[1]])
  
  sapply(2:length(rs),function(i) lines.roc(rs[[i]],col=i))
  </code></pre>
  
  <p>Here is the Plot:</p>
  
  <p><a href="https://i.stack.imgur.com/nmBmC.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/nmBmC.png" alt="enter image description here"></a></p>
  
  <p>Could someone please help me out with this? Thanks a lot.</p>