title: How to get comparable and reproducible results from LogisticRegressionCV and GridSearchCV
body: <p>I want to score different classifiers with different parameters.</p>
  <p>For speedup on <code>LogisticRegression</code> I use <code>LogisticRegressionCV</code> (which at least 2x faster) and plan use <code>GridSearchCV</code> for others.</p>
  <p>But problem while it give me equal <code>C</code> parameters, but not the <code>AUC ROC</code> scoring.</p>
  <p>I'll try fix many parameters like <code>scorer</code>, <code>random_state</code>, <code>solver</code>, <code>max_iter</code>, <code>tol</code>...
  Please look at example (real data have no mater):</p>
  <p>Test data and common part:</p>
  <pre><code>from sklearn import datasets
  boston = datasets.load_boston()
  X = boston.data
  y = boston.target
  y[y &lt;= y.mean()] = 0; y[y &gt; 0] = 1
  
  import numpy as np
  from sklearn.cross_validation import KFold
  from sklearn.linear_model import LogisticRegression
  from sklearn.grid_search import GridSearchCV
  from sklearn.linear_model import LogisticRegressionCV
  
  fold = KFold(len(y), n_folds=5, shuffle=True, random_state=777)
  </code></pre>
  <h2>GridSearchCV</h2>
  <pre><code>grid = {
      'C': np.power(10.0, np.arange(-10, 10))
       , 'solver': ['newton-cg']
  }
  clf = LogisticRegression(penalty='l2', random_state=777, max_iter=10000, tol=10)
  gs = GridSearchCV(clf, grid, scoring='roc_auc', cv=fold)
  gs.fit(X, y)
  
  print ('gs.best_score_:', gs.best_score_)
  </code></pre>
  <blockquote>
  <p>gs.best_score_: 0.939162082194</p>
  </blockquote>
  <h2>LogisticRegressionCV</h2>
  <pre><code>searchCV = LogisticRegressionCV(
      Cs=list(np.power(10.0, np.arange(-10, 10)))
      ,penalty='l2'
      ,scoring='roc_auc'
      ,cv=fold
      ,random_state=777
      ,max_iter=10000
      ,fit_intercept=True
      ,solver='newton-cg'
      ,tol=10
  )
  searchCV.fit(X, y)
  
  print ('Max auc_roc:', searchCV.scores_[1].max())
  </code></pre>
  <blockquote>
  <p>Max auc_roc: 0.970588235294</p>
  </blockquote>
  <p>Solver <code>newton-cg</code> used just to provide fixed value, other tried too.
  What I forgot?</p>
  <p>P.S. In both cases I also got warning &quot;<em>/usr/lib64/python3.4/site-packages/sklearn/utils/optimize.py:193: UserWarning: Line Search failed
  warnings.warn('Line Search failed')</em>&quot; which I can't understand too. I'll be happy if someone also describe what it mean, but I hope it is not relevant to my main question.</p>
  <h3>EDIT UPDATES</h3>
  <p>By @joeln comment add <code>max_iter=10000</code> and <code>tol=10</code> parameters too. It does not change result in any digit, but the warning disappeared.</p>