title: Accuracy of model is 0.86 while AUC is 0.50?
body: <p>I ran a few models in sklearn. Here is the code for the same. </p>
  
  <pre><code># Function for Stochastic Gradient Descent Logistic Regression with Elastic Net
  def SGDlogistic(k_fold,train_X,train_Y):
          """Method to implement Multi-class SVM using 
          Stochastic Gradient Descent
          """
  
          from sklearn.linear_model import SGDClassifier
          scores_sgd_lr = []
  
          for train_indices, test_indices in k_fold:
              train_X_cv = train_X[train_indices]
              train_Y_cv= train_Y[train_indices]
  
              test_X_cv = train_X[test_indices]
              test_Y_cv= train_Y[test_indices]
  
              sgd_lr=SGDClassifier(loss='log',penalty='elasticnet')
              scores_sgd_lr.append(sgd_lr.fit(train_X_cv,train_Y_cv).score(test_X_cv,test_Y_cv))
  
          print("The mean accuracy of Stochastic Gradient Descent Logistic on CV data is:", np.mean(scores_sgd_lr)) 
  
          return sgd_lr
  
  
  
  def test_performance(test_X,test_Y,classifier,name):
          """This method checks the performance of each algorithm on test data."""
  
          from sklearn import metrics
  
          # For SGD
          print ("The accuracy of "+ name + " on test data is:",classifier.score(test_X,test_Y))
          print 'Classification Metrics for'
          print metrics.classification_report(test_Y, classifier.predict(test_X))
          print "Confusion matrix"
          print metrics.confusion_matrix(test_Y, classifier.predict(test_X))
  
  
  
  
  def plot_ROC(test_X,test_Y,classifier):
      """ This functions plots the ROC curve of the classifier"""
  
      from sklearn.metrics import roc_curve, auc
      false_positive_rate, true_positive_rate, thresholds =roc_curve(test_Y, classifier.predict(test_X))
      roc_auc= auc(false_positive_rate, true_positive_rate)
      plt.title('Receiver Operating Characteristic')
      plt.plot(false_positive_rate, true_positive_rate, 'b',label='AUC = %0.2f'% roc_auc)
      plt.legend(loc='lower right')
      plt.ylabel('True Positive Rate')
      plt.xlabel('False Positive Rate')
  </code></pre>
  
  <p>The first function does logistic regression with Elastic net penalty. 
  The second function is testing the perfomance of the algo on the test data. This gives confusion matrix and the accuracy. </p>
  
  <p>While plot_ROC plots the ROC curve on test data. </p>
  
  <p>Here is what I see. </p>
  
  <pre><code>('The accuracy of Logistic with Elastic Net on test data is:', 0.90566607467092586)
  Classification Metrics for
               precision    recall  f1-score   support
  
            0       0.91      1.00      0.95    227948
            1       0.50      0.00      0.00     23743
  
  avg / total       0.87      0.91      0.86    251691
  
  Confusion matrix
  [[227944      4]
   [ 23739      4]]
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/wH2Ys.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/wH2Ys.png" alt="enter image description here"></a></p>
  
  <pre><code>(array([ 0.        ,  0.00001755,  1.        ]),
   array([ 0.        ,  0.00016847,  1.        ]),
   array([2, 1, 0]))
  </code></pre>
  
  <p>If you see , the accuracy on test data 90% and even the confusion matrix shows good precision and recall. So it's not just accuracy which might be misleading. But the ROC and AUC it gives is like 0.50?. That's so weird. It is behaving as a random guess as per ROC while the accuracy and Confusion matrix show a different picture. </p>
  
  <p>Help pls</p>
  
  <p>Edit 2: </p>
  
  <p>Ok. SO I added the code for using probabilities instead of actual classification in AUC. </p>
  
  <p>This is what I get now. </p>
  
  <p><a href="https://i.stack.imgur.com/z1Ki7.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/z1Ki7.png" alt="enter image description here"></a></p>
  
  <p>As you see AUC of <strong>0.71</strong>.  I haven't done anything for class imbalance. One question. How do I convert the prediction scores to probabilities for SVM etc. Currently it has predict_proba for only log loss or Huber Loss functions. That means I can't go beyond Logistic regression to get AUC?</p>