title: How to compute precision, recall, accuracy and f1-score for the multiclass case with scikit learn?
body: <p>I'm working in a sentiment analysis problem the data looks like this:</p>
  
  <pre><code>label instances
      5    1190
      4     838
      3     239
      1     204
      2     127
  </code></pre>
  
  <p>So my data is unbalanced since 1190 <code>instances</code> are labeled with <code>5</code>. For the classification Im using scikit's <a href="http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html" rel="noreferrer">SVC</a>. The problem is I do not know how to balance my data in the right way in order to compute accurately the precision, recall, accuracy and f1-score for the multiclass case. So I tried the following approaches:</p>
  
  <p>First:</p>
  
  <pre><code>    wclf = SVC(kernel='linear', C= 1, class_weight={1: 10})
      wclf.fit(X, y)
      weighted_prediction = wclf.predict(X_test)
  
  print 'Accuracy:', accuracy_score(y_test, weighted_prediction)
  print 'F1 score:', f1_score(y_test, weighted_prediction,average='weighted')
  print 'Recall:', recall_score(y_test, weighted_prediction,
                                average='weighted')
  print 'Precision:', precision_score(y_test, weighted_prediction,
                                      average='weighted')
  print '\n clasification report:\n', classification_report(y_test, weighted_prediction)
  print '\n confussion matrix:\n',confusion_matrix(y_test, weighted_prediction)
  </code></pre>
  
  <p>Second:</p>
  
  <pre><code>auto_wclf = SVC(kernel='linear', C= 1, class_weight='auto')
  auto_wclf.fit(X, y)
  auto_weighted_prediction = auto_wclf.predict(X_test)
  
  print 'Accuracy:', accuracy_score(y_test, auto_weighted_prediction)
  
  print 'F1 score:', f1_score(y_test, auto_weighted_prediction,
                              average='weighted')
  
  print 'Recall:', recall_score(y_test, auto_weighted_prediction,
                                average='weighted')
  
  print 'Precision:', precision_score(y_test, auto_weighted_prediction,
                                      average='weighted')
  
  print '\n clasification report:\n', classification_report(y_test,auto_weighted_prediction)
  
  print '\n confussion matrix:\n',confusion_matrix(y_test, auto_weighted_prediction)
  </code></pre>
  
  <p>Third:</p>
  
  <pre><code>clf = SVC(kernel='linear', C= 1)
  clf.fit(X, y)
  prediction = clf.predict(X_test)
  
  
  from sklearn.metrics import precision_score, \
      recall_score, confusion_matrix, classification_report, \
      accuracy_score, f1_score
  
  print 'Accuracy:', accuracy_score(y_test, prediction)
  print 'F1 score:', f1_score(y_test, prediction)
  print 'Recall:', recall_score(y_test, prediction)
  print 'Precision:', precision_score(y_test, prediction)
  print '\n clasification report:\n', classification_report(y_test,prediction)
  print '\n confussion matrix:\n',confusion_matrix(y_test, prediction)
  
  
  F1 score:/usr/local/lib/python2.7/site-packages/sklearn/metrics/classification.py:676: DeprecationWarning: The default `weighted` averaging is deprecated, and from version 0.18, use of precision, recall or F-score with multiclass or multilabel data or pos_label=None will result in an exception. Please set an explicit value for `average`, one of (None, 'micro', 'macro', 'weighted', 'samples'). In cross validation use, for instance, scoring="f1_weighted" instead of scoring="f1".
    sample_weight=sample_weight)
  /usr/local/lib/python2.7/site-packages/sklearn/metrics/classification.py:1172: DeprecationWarning: The default `weighted` averaging is deprecated, and from version 0.18, use of precision, recall or F-score with multiclass or multilabel data or pos_label=None will result in an exception. Please set an explicit value for `average`, one of (None, 'micro', 'macro', 'weighted', 'samples'). In cross validation use, for instance, scoring="f1_weighted" instead of scoring="f1".
    sample_weight=sample_weight)
  /usr/local/lib/python2.7/site-packages/sklearn/metrics/classification.py:1082: DeprecationWarning: The default `weighted` averaging is deprecated, and from version 0.18, use of precision, recall or F-score with multiclass or multilabel data or pos_label=None will result in an exception. Please set an explicit value for `average`, one of (None, 'micro', 'macro', 'weighted', 'samples'). In cross validation use, for instance, scoring="f1_weighted" instead of scoring="f1".
    sample_weight=sample_weight)
   0.930416613529
  </code></pre>
  
  <p>However, Im getting warnings like this:</p>
  
  <pre><code>/usr/local/lib/python2.7/site-packages/sklearn/metrics/classification.py:1172:
  DeprecationWarning: The default `weighted` averaging is deprecated,
  and from version 0.18, use of precision, recall or F-score with 
  multiclass or multilabel data or pos_label=None will result in an 
  exception. Please set an explicit value for `average`, one of (None, 
  'micro', 'macro', 'weighted', 'samples'). In cross validation use, for 
  instance, scoring="f1_weighted" instead of scoring="f1"
  </code></pre>
  
  <p>How can I deal correctly with my unbalanced data in order to compute in the right way classifier's metrics?</p>