title: How to rank the instances based on prediction probability in sklearn
body: <p>I am using sklearn's support vector machine (<code>SVC</code>) as follows to get the prediction probability of my instances in my dataset as follows using <code>10-fold cross validation</code>.</p>
  
  <pre class="lang-py prettyprint-override"><code>from sklearn import datasets
  iris = datasets.load_iris()
  
  X = iris.data
  y = iris.target
  
  clf=SVC(class_weight="balanced")
  proba = cross_val_predict(clf, X, y, cv=10, method='predict_proba')
  
  print(clf.classes_)
  print(proba[:,1])
  print(np.argsort(proba[:,1]))
  </code></pre>
  
  <p>My expected output is as follows for <code>print(proba[:,1])</code> and <code>print(np.argsort(proba[:,1]))</code> where the first one indicates the <em>prediction probability of all instances for class <code>1</code></em> and the second one indicates the <em>corresponding index of the data instance for each probability</em>.</p>
  
  <pre><code>[0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.1 0.  0.  0.
   0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.  0.
   0.2 0.  0.  0.  0.  0.1 0.  0.  0.  0.  0.  0.  0.  0.  0.9 1.  0.7 1.
   1.  1.  1.  0.7 1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  0.9 0.9 0.1 1.
   0.6 1.  1.  1.  0.9 0.  1.  1.  1.  1.  1.  0.4 0.9 0.9 1.  1.  1.  0.9
   1.  1.  1.  1.  1.  1.  1.  1.  1.  1.  0.  0.  0.  0.  0.  0.  0.9 0.
   0.1 0.  0.  0.  0.  0.  0.  0.  0.1 0.  0.  0.8 0.  0.1 0.  0.1 0.  0.1
   0.3 0.2 0.  0.6 0.  0.  0.  0.6 0.4 0.  0.  0.  0.8 0.  0.  0.  0.  0.
   0.  0.  0.  0.  0.  0. ]
  
  [  0 113 112 111 110 109 107 105 104 114 103 101 100  77 148  49  48  47
    46 102 115 117 118 147 146 145 144 143 142 141 140 139 137 136 135 132
   131 130 128 124 122 120  45  44 149  42  15  26  16  17  18  19  20  21
    22  43  23  24  35  34  33  32  31  30  29  28  27  37  13  25   9  10
     7   6   5   4   3   8  11   2   1  38  39  40  12 108 116  41 121  70
    14 123 125  36 127 126 134  83  72 133 129  52  57 119 138  89  76  50
    84 106  85  69  68  97  98  66  65  64  63  62  61  67  60  58  56  55
    54  53  51  59  71  73  75  96  95  94  93  92  91  90  88  87  86  82
    81  80  79  78  99  74]
  </code></pre>
  
  <p>My first question is; it seems like <code>SVC</code> does not support <code>predict_proba</code>. Therefore, is it correct if I use <code>proba = cross_val_predict(clf, X, y, cv=10, method='decision_function')</code> instead?</p>
  
  <p>My second question is how to print the classes of prediction probability? I tried <code>clf_classes_</code>. But, I get an error saying <code>AttributeError: 'SVC' object has no attribute 'classes_'</code>. Is there a way to resolve this issue?</p>
  
  <p>Note: I want to get the prediction probability for all the instances using cross validation.</p>
  
  <p><strong>EDIT:</strong></p>
  
  <p>The answer of @KRKirov is great. However, I do not need <code>GridSearchCV</code> and only want to use normal <code>cross validation</code>. Therefore, I changed his code use <code>cross_val_score</code>. Now, I am getting the error <code>NotFittedError: Call fit before prediction</code>.</p>
  
  <p>Is there a way to resolve this issue?</p>
  
  <p>I am happy to provide more details if needed.</p>