title: GridSearchCV.best_score_ meaning when scoring set to 'accuracy' and CV
body: <p>I'm trying to find the best model Neural Network model applied for the classification of breast cancer samples on the well-known Wisconsin Cancer dataset (569 samples, 31 features + target). I'm using sklearn 0.18.1. I'm not using Normalization so far. I'll add it when I solve this question.</p>
  
  <pre><code># some init code omitted
  X_train, X_test, y_train, y_test = train_test_split(X, y)
  </code></pre>
  
  <p>Define params NN params for the GridSearchCV</p>
  
  <pre><code>tuned_params = [{'solver': ['sgd'], 'learning_rate': ['constant'], "learning_rate_init" : [0.001, 0.01, 0.05, 0.1]},
                  {"learning_rate_init" : [0.001, 0.01, 0.05, 0.1]}]
  </code></pre>
  
  <p>CV method and model</p>
  
  <pre><code>cv_method = KFold(n_splits=4, shuffle=True)
  model = MLPClassifier()
  </code></pre>
  
  <p>Apply grid</p>
  
  <pre><code>grid = GridSearchCV(estimator=model, param_grid=tuned_params, cv=cv_method, scoring='accuracy')
  grid.fit(X_train, y_train)
  y_pred = grid.predict(X_test)
  </code></pre>
  
  <p>And if I run:</p>
  
  <pre><code>print(grid.best_score_)
  print(accuracy_score(y_test, y_pred))
  </code></pre>
  
  <p>The result is <em>0.746478873239</em> and <em>0.902097902098</em> </p>
  
  <p>According to the doc "best_score_ : float, Score of best_estimator on the left out data". I assume it is the best accuracy among the ones obtained running the 8 different configuration as especified in <em>tuned_params</em> the number of times especified by KFold, on the left out data as especified by KFold. Am I right?</p>
  
  <p>One more question. Is there a method to find the optimal size of test data to use in <em>train_test_split</em> which defaults to 0.25?</p>
  
  <p>Thanks a lot</p>
  
  <p>REFERENCES</p>
  
  <ul>
  <li><a href="http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html" rel="noreferrer">http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html</a></li>
  <li><a href="http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html#sklearn.model_selection.GridSearchCV" rel="noreferrer">http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html#sklearn.model_selection.GridSearchCV</a></li>
  <li><a href="http://scikit-learn.org/stable/modules/grid_search.html" rel="noreferrer">http://scikit-learn.org/stable/modules/grid_search.html</a></li>
  <li><a href="http://scikit-learn.org/stable/modules/cross_validation.html" rel="noreferrer">http://scikit-learn.org/stable/modules/cross_validation.html</a></li>
  <li><a href="http://scikit-learn.org/stable/auto_examples/model_selection/plot_nested_cross_validation_iris.html#sphx-glr-auto-examples-model-selection-plot-nested-cross-validation-iris-py" rel="noreferrer">http://scikit-learn.org/stable/auto_examples/model_selection/plot_nested_cross_validation_iris.html#sphx-glr-auto-examples-model-selection-plot-nested-cross-validation-iris-py</a></li>
  </ul>