title: R and Random Forest: How caret and pROC deal with positive and negative class?
body: <p>In the past days, I've been analyzing the performance of R's implementation of Random Forest and the different tools available in order to obtain:</p>
  
  <ul>
  <li>AUC</li>
  <li>Sensitivity</li>
  <li>Specificity</li>
  </ul>
  
  <p>Thus, I've used two different methods:</p>
  
  <ul>
  <li>mroc and coords from <strong>pROC</strong> library in order to obtain the performance of the model at different cutoff points.</li>
  <li>confusionMatrix from <strong>caret</strong> library in order to obtain the optimal performance of the model (AUC, Accuracy, Sensitivity, Specificity, ...)</li>
  </ul>
  
  <p>The point is that I've realized that there is some differences between both approaches.</p>
  
  <p>I've developed the following code:</p>
  
  <pre><code>suppressMessages(library(randomForest))
  suppressMessages(library(pROC))
  suppressMessages(library(caret))
  
  set.seed(100)
  
  t_x &lt;- as.data.frame(matrix(runif(100),ncol=10))
  t_y &lt;- factor(sample(c("A","B"), 10, replace = T), levels=c("A","B"))
  
  v_x  &lt;- as.data.frame(matrix(runif(50),ncol=10))
  v_y &lt;- factor(sample(c("A","B"), 5, replace = T), levels=c("A","B"))
  
  model &lt;- randomForest(t_x, t_y, ntree=1000, importance=T);
  prob.out &lt;- predict(model, v_x, type="prob")[,1];
  prediction.out &lt;- predict(model, v_x, type="response");
  
  mroc &lt;- roc(v_y,prob.out,plot=F)
  
  results &lt;- coords(mroc,seq(0, 1, by = 0.01),input=c("threshold"),ret=c("sensitivity","specificity","ppv","npv"))
  
  accuracyData &lt;- confusionMatrix(prediction.out,v_y)
  </code></pre>
  
  <p>If you compare the <strong>results</strong> and <strong>accuracyData</strong> variables, you can see that the relationship between sensitivity and specificity is the opposite.</p>
  
  <p>That is, the confusionMatrix results are:</p>
  
  <pre><code>Confusion Matrix and Statistics
  
            Reference
  Prediction A B
           A 1 1
           B 2 1
  
                 Accuracy : 0.4             
                   95% CI : (0.0527, 0.8534)
      No Information Rate : 0.6             
      P-Value [Acc &gt; NIR] : 0.913           
  
                    Kappa : -0.1538         
   Mcnemar's Test P-Value : 1.000           
  
              Sensitivity : 0.3333          
              Specificity : 0.5000          
           Pos Pred Value : 0.5000          
           Neg Pred Value : 0.3333          
               Prevalence : 0.6000          
           Detection Rate : 0.2000          
     Detection Prevalence : 0.4000          
        Balanced Accuracy : 0.4167          
  
         'Positive' Class : A 
  </code></pre>
  
  <p>But if I look for such Sensitivity and Specificity in the coords calculation, I find them swapped:</p>
  
  <pre><code>     sensitivity specificity       ppv       npv
  0.32         0.5   0.3333333 0.3333333 0.5000000
  </code></pre>
  
  <p>Apparently, Sensitivity and Specificity is are opposite in coords and confusionMatrix.</p>
  
  <p>Taking into account that confusionMatrix identifies correctly the positive class, I assume that this good interpretation of Sensitivity and Specificity.</p>
  
  <p>My question is: <em>Is there any way of forcing coords to interpret the positive and negative classes in the way I want to?</em></p>