title: Sci kit learn confusion matrix always look almost the same
body: <p>So I'm pretty new to both Machine Learning and Python but have managed to be able to classify my data and print the confusion matrix with various classifiers with the code below:</p>
  
  <pre><code>def classify_data(df, feature_cols, file):
  nbr_folds = 5
  attributes = df.loc[:, feature_cols]  # Also known as x
  class_label = df['task']  # Class label, also known as y.
  file.write("\nFeatures used: ")
  for feature in feature_cols:
      file.write(feature + ",")
  print("Features used", feature_cols)
  
  print("MLP")
  file.write("MLP")
  mlp = MLPClassifier(solver='lbfgs', alpha=1e-5, hidden_layer_sizes=(5, 2), random_state=1)
  class_label_predicted = cross_val_predict(mlp, attributes, class_label, cv=nbr_folds)
  conf_mat = confusion_matrix(class_label, class_label_predicted)
  print(conf_mat)
  accuracy = accuracy_score(class_label, class_label_predicted)
  print("\nRows classified: " + str(len(class_label_predicted)))
  print("\nAccuracy: {0:.3f}%\n".format(accuracy * 100))
  file.write("\nClassifier settings:" + str(mlp) + "\n")
  file.write("\nRows classified: " + str(len(class_label_predicted)))
  file.write("\nAccuracy: {0:.3f}%\n".format(accuracy * 100))
  file.writelines('\t'.join(str(j) for j in i) + '\n' for i in conf_mat)
  
  print("RandomForest")
  file.write("\nRandomForest")
  #sv = svm.SVC(kernel="linear")
  clf = RandomForestClassifier(max_depth=2, random_state=0)
  class_label_predicted = cross_val_predict(clf, attributes, class_label, cv=nbr_folds)
  conf_mat = confusion_matrix(class_label, class_label_predicted)
  print(conf_mat)
  accuracy = accuracy_score(class_label, class_label_predicted)
  print("Rows classified: " + str(len(class_label_predicted)))
  print("Accuracy: {0:.3f}%\n".format(accuracy * 100))
  file.write("\nClassifier settings:" + str(clf) + "\n")
  file.write("\nRows classified: " + str(len(class_label_predicted)))
  file.write("\nAccuracy: {0:.3f}%\n".format(accuracy * 100))
  file.writelines('\t'.join(str(j) for j in i) + '\n' for i in conf_mat)
  </code></pre>
  
  <p>However, I am starting to wonder if I am doing something wrong here as the confusion matrix is almost always the same, putting everything in on my fifth feature. When I run the exact same data set with the same attributes in the Weka application, I get different results. Below is an example:</p>
  
  <pre><code>sci kit learn:
  MLP
  Rows classified: 6881
  Accuracy: 25.970%
  0   0   0   0   412 12  0   0   25  1   0   0   0
  0   0   0   0   540 50  0   0   8   0   0   0   0
  0   0   0   0   111 3   0   0   6   2   0   0   0
  0   0   0   0   139 19  0   0   4   2   0   0   0
  0   0   0   0   1630    54  0   0   106 18  0   0   0
  0   0   0   0   554 63  0   0   22  0   0   0   0
  0   0   0   0   246 8   0   0   33  10  0   0   0
  0   0   0   0   324 39  0   0   8   0   0   0   0
  0   0   0   0   605 60  0   0   90  5   0   0   0
  0   0   0   0   519 31  0   0   72  4   0   0   0
  0   0   0   0   455 19  0   0   10  1   0   0   0
  0   0   0   0   260 11  0   0   21  1   0   0   0
  0   0   0   0   236 8   0   0   21  3   0   0   0
  
  RandomForest:
  Rows classified: 6881
  Accuracy: 26.174%
  0   0   0   0   440 0   0   0   10  0   0   0   0
  0   0   0   0   597 0   0   0   0   1   0   0   0
  0   0   0   0   119 0   0   0   3   0   0   0   0
  0   0   0   0   164 0   0   0   0   0   0   0   0
  0   0   0   0   1774    0   0   0   34  0   0   0   0
  0   0   0   0   629 0   0   0   10  0   0   0   0
  0   0   0   0   268 0   0   0   29  0   0   0   0
  0   0   0   0   371 0   0   0   0   0   0   0   0
  0   0   0   0   733 0   0   0   27  0   0   0   0
  0   0   0   0   605 0   0   0   21  0   0   0   0
  0   0   0   0   484 0   0   0   1   0   0   0   0
  0   0   0   0   286 0   0   0   7   0   0   0   0
  0   0   0   0   263 0   0   0   5   0   0   0   0
  
  Weka
  MLP
      a    b    c    d    e    f    g    h    i    j    k    l    m   &lt;-- classified as
      5  504   50    1    0    0   10   28    0    0    0    0    0 |    a = t1
      2 1511   56    1    4    1   83  135    0    2   12    0    1 |    b = t12
      4  467   88    0    1    3   30   45    0    0    0    1    0 |    c = t2
      1  227   15    2    2    0   36   13    0    1    0    0    0 |    d = t3
      4  369   18    2    1    0   25   31    0    0    0    0    0 |    e = t0
      3  306   43    0    1    2   10    6    0    0    0    0    0 |    f = t4
      5  463   36    2    4    0  178   69    0    0    2    0    1 |    g = t5
      3  371   23    1    0    0   49  176    0    0    2    1    0 |    h = t6
      4  398   14    1    1    0   28   33    0    0    5    1    0 |    i = t7
      1  252   13    0    0    0   16    8    0    1    2    0    0 |    j = t8
      1  213    9    0    0    0   20   24    0    1    0    0    0 |    k = t9
      1   96    3    0    0    0    4   16    0    0    2    0    0 |    l = t10
      1  133    7    0    0    0    7   15    0    0    1    0    0 |    m = t11
  </code></pre>
  
  <p><strong>I also wonder if it possible to print the confusion matrix with the class labels as Weka is doing? Here it looks like column b is somewhat equal to the fifth column in sci kit learn but it is difficult to tell what column it represents.</strong></p>