title: Recursive feature elimination and grid search using scikit-learn
body: <p>I would like to perform recursive feature elimination with nested grid search and cross-validation for each feature subset using scikit-learn. From the <strong>RFECV</strong> documentation it sounds like this type of operation is supported using the <code>estimator_params</code> parameter:</p>
  
  <pre><code>estimator_params : dict
  
      Parameters for the external estimator. Useful for doing grid searches.
  </code></pre>
  
  <p>However, when I try to pass a grid of hyperparameters to the RFECV object </p>
  
  <pre><code>from sklearn.datasets import make_friedman1
  from sklearn.feature_selection import RFECV
  from sklearn.svm import SVR
  X, y = make_friedman1(n_samples=50, n_features=10, random_state=0)
  estimator = SVR(kernel="linear")
  selector = RFECV(estimator, step=1, cv=5, estimator_params={'C': [0.1, 10, 100, 1000]})
  selector = selector.fit(X, y)
  </code></pre>
  
  <p>I get an error like</p>
  
  <pre><code>  File "U:/My Documents/Code/ModelFeatures/bin/model_rcc_gene_features.py", line 130, in &lt;module&gt;
      selector = selector.fit(X, y)
    File "C:\Python27\lib\site-packages\sklearn\feature_selection\rfe.py", line 336, in fit
      ranking_ = rfe.fit(X_train, y_train).ranking_
    File "C:\Python27\lib\site-packages\sklearn\feature_selection\rfe.py", line 146, in fit
      estimator.fit(X[:, features], y)
    File "C:\Python27\lib\site-packages\sklearn\svm\base.py", line 178, in fit
      fit(X, y, sample_weight, solver_type, kernel, random_seed=seed)
    File "C:\Python27\lib\site-packages\sklearn\svm\base.py", line 233, in _dense_fit
      max_iter=self.max_iter, random_seed=random_seed)
    File "libsvm.pyx", line 59, in sklearn.svm.libsvm.fit (sklearn\svm\libsvm.c:1628)
  TypeError: a float is required
  </code></pre>
  
  <p>If anyone could show me what I'm doing wrong it would be greatly appreciated, thanks!</p>
  
  <p><strong>EDIT:</strong></p>
  
  <p>After Andreas' response things became clearer, below is a working example of RFECV combined with grid search.</p>
  
  <pre><code>from sklearn.datasets import make_friedman1
  from sklearn.feature_selection import RFECV
  from sklearn.grid_search import GridSearchCV
  from sklearn.svm import SVR
  X, y = make_friedman1(n_samples=50, n_features=10, random_state=0)
  param_grid = [{'C': 0.01}, {'C': 0.1}, {'C': 1.0}, {'C': 10.0}, {'C': 100.0}, {'C': 1000.0}, {'C': 10000.0}]
  estimator = SVR(kernel="linear")
  selector = RFECV(estimator, step=1, cv=4)
  clf = GridSearchCV(selector, {'estimator_params': param_grid}, cv=7)
  clf.fit(X, y)
  clf.best_estimator_.estimator_
  clf.best_estimator_.grid_scores_
  clf.best_estimator_.ranking_
  </code></pre>