title: sklearn - Cross validation with multiple scores
body: <p>I would like to compute the <strong>recall</strong>, <strong>precision</strong> and <strong>f-measure</strong> of a cross validation test for different classifiers.
  <em>scikit-learn</em> comes with <a href="http://scikit-learn.org/stable/modules/generated/sklearn.cross_validation.cross_val_score.html">cross_val_score</a> but unfortunately such method does not return multiple values.</p>
  
  <p>I could compute such measures by calling <strong>three times</strong> <em>cross_val_score</em> but that is not efficient. Is there any better solution?</p>
  
  <p>By now I wrote this function:</p>
  
  <pre><code>from sklearn import metrics
  
  def mean_scores(X, y, clf, skf):
  
      cm = np.zeros(len(np.unique(y)) ** 2)
      for i, (train, test) in enumerate(skf):
          clf.fit(X[train], y[train])
          y_pred = clf.predict(X[test])
          cm += metrics.confusion_matrix(y[test], y_pred).flatten()
  
      return compute_measures(*cm / skf.n_folds)
  
  def compute_measures(tp, fp, fn, tn):
       """Computes effectiveness measures given a confusion matrix."""
       specificity = tn / (tn + fp)
       sensitivity = tp / (tp + fn)
       fmeasure = 2 * (specificity * sensitivity) / (specificity + sensitivity)
       return sensitivity, specificity, fmeasure
  </code></pre>
  
  <p>It basically sums up the confusion matrix values and once you have <em>false positive</em>, <em>false negative</em> etc you can easily compute the recall, precision etc... But still I don't like this solution :)</p>