title: Calculate sklearn.roc_auc_score for multi-class
body: <p>I would like to calculate AUC, precision, accuracy for my classifier.
  I am doing supervised learning:</p>
  
  <p>Here is my working code.
  This code is working fine for binary class, but not for multi class. 
  Please assume that you have a dataframe with binary classes:</p>
  
  <pre><code>sample_features_dataframe = self._get_sample_features_dataframe()
  labeled_sample_features_dataframe = retrieve_labeled_sample_dataframe(sample_features_dataframe)
  labeled_sample_features_dataframe, binary_class_series, multi_class_series = self._prepare_dataframe_for_learning(labeled_sample_features_dataframe)
  
  k = 10
  k_folds = StratifiedKFold(binary_class_series, k)
  for train_indexes, test_indexes in k_folds:
      train_set_dataframe = labeled_sample_features_dataframe.loc[train_indexes.tolist()]
      test_set_dataframe = labeled_sample_features_dataframe.loc[test_indexes.tolist()]
  
      train_class = binary_class_series[train_indexes]
      test_class = binary_class_series[test_indexes]
      selected_classifier = RandomForestClassifier(n_estimators=100)
      selected_classifier.fit(train_set_dataframe, train_class)
      predictions = selected_classifier.predict(test_set_dataframe)
      predictions_proba = selected_classifier.predict_proba(test_set_dataframe)
  
      roc += roc_auc_score(test_class, predictions_proba[:,1])
      accuracy += accuracy_score(test_class, predictions)
      recall += recall_score(test_class, predictions)
      precision += precision_score(test_class, predictions)
  </code></pre>
  
  <p>In the end I divided the results in K of course for getting average AUC, precision, etc. 
  This code is working fine. 
  However, I cannot calculate the same for multi class: </p>
  
  <pre><code>    train_class = multi_class_series[train_indexes]
      test_class = multi_class_series[test_indexes]
  
      selected_classifier = RandomForestClassifier(n_estimators=100)
      selected_classifier.fit(train_set_dataframe, train_class)
  
      predictions = selected_classifier.predict(test_set_dataframe)
      predictions_proba = selected_classifier.predict_proba(test_set_dataframe)
  </code></pre>
  
  <p>I found that for multi class I have to add the parameter "weighted" for average.</p>
  
  <pre><code>    roc += roc_auc_score(test_class, predictions_proba[:,1], average="weighted")
  </code></pre>
  
  <p>I got an error: raise ValueError("{0} format is not supported".format(y_type))</p>
  
  <p>ValueError: multiclass format is not supported</p>