title: GridSearchCV performs worse than vanilla SVM using the SAME parameters
body: <p>community.</p>
  <p>I was coding some ML to classify some data into groups.</p>
  <p>I tried different methods, but when I got to SVM I encountered this problem.</p>
  <p>I have a simple set of data (3 classes, 6 features) and when I use SVM with fixed parameters (<strong>C=10</strong>, <strong>gamma=1</strong>) and I predict on the same data I get 100% accuracy (these could be overfitted vectors, but that's another issue).</p>
  <p>What I find difficult to understand is that then I try GridSearchCV (sklearn.model_selection.GridSearchCV) and I sweep over all powers of 10 from 10^-5 to 10^5 for C and gamma (this includes of course C=10 and gamma=1) and it finds as best_params: <strong>C=10^-5</strong> and <strong>gamma=10^-5</strong>. With those parameters, the accuracy is 41% and all of the predictions fall into <strong>one category</strong>.</p>
  <p>At least I should be able to predict the same parameters as the FIXED SVM.
  What is also puzzling is that the same code worked before for other datasets...</p>
  <p>My issue NOW is NOT (so please leave these discussions aside if you answer):</p>
  <ol>
  <li><p>overfitting, or the use of the same data for training and testing;</p>
  </li>
  <li><p>an unbalanced set of data;</p>
  </li>
  <li><p>dataset issues.</p>
  </li>
  </ol>
  <p>My issue is only WHY do GridSearchCV behaves differently than normal SVM. I am sure it must be something I am coding wrong, or else they really don't work as expected.</p>
  <p>Here is the code:</p>
  <pre><code>import pandas as pd
  import numpy as np
  from sklearn import svm
  from sklearn.model_selection import GridSearchCV
  from sklearn import preprocessing
  
  xl= pd.ExcelFile('3classes_6_features.xlsx')
  cont= xl.parse()
  
   # Encode labels:
  labeling = preprocessing.LabelEncoder()
  labeling.fit(cont[cont.keys()[0]])
  
  ### cont.keys()[0] == &quot;GS&quot;
  y_all= labeling.transform(np.array(cont[&quot;GS&quot;]))
  X_all= np.array(cont.drop([&quot;GS&quot;],1))
  
  # NORMAL SVM:
  SVMclassifier= svm.SVC(kernel='rbf', gamma=1, C=10, random_state=0)
  SVMclassifier.fit(X_all,y_all)
  
  # SVM with HYPERPARAMETRIC TUNING:
  log_sweep= [10**(i)/j for i in range(-5,5) for j in [2,1]]
  SVMparam_grid = {'C': log_sweep, 'gamma': log_sweep}
  
  SVMgrid_classifier= GridSearchCV(svm.SVC(kernel='rbf', random_state=0), SVMparam_grid)
  SVMgrid_classifier= SVMgrid_classifier.fit(X_all,y_all)
  
  
  print(&quot;INITIAL CLASSES: &quot;, y_all)
  print(&quot;NORMAL SVM prediction: &quot;, SVMclassifier.predict(X_all))
  print(&quot;TUNED SVM prediction: &quot;, SVMgrid_classifier.predict(X_all))
  </code></pre>
  <p>The result is:</p>
  <blockquote>
  <p>INITIAL CLASSES:  [0 1 2 2 0 0 2 0 1 0 0 1 2 0 0 1 1 1 2 1 2]</p>
  <p>NORMAL SVM prediction:  [0 1 2 2 0 0 2 0 1 0 0 1 2 0 0 1 1 1 2 1 2]</p>
  <p>TUNED SVM prediction:  [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]</p>
  </blockquote>
  <p>I attach the data in case you want to try it yourselves:</p>
  <p><a href="https://drive.google.com/open?id=1LPRiMFNeSXu790lG_-NP3HAkvq8vyLcV" rel="nofollow noreferrer">https://drive.google.com/open?id=1LPRiMFNeSXu790lG_-NP3HAkvq8vyLcV</a></p>
  <p>Thanks in advance!</p>
  <p>Regards,</p>
  <p>Luke</p>