title: scikit-learn pipeline: grid search over parameters of transformer to generate data
body: <p>I would like to use the first step of a scikit-learn pipeline to generate a toy data set in order to evaluate the performance of my analysis. An as-simple-as-it-gets-example solution I came up with looks like the following:</p>
  
  <pre><code>import numpy as np
  from sklearn.pipeline import Pipeline
  from sklearn.grid_search import GridSearchCV
  from sklearn.base import TransformerMixin
  from sklearn import cluster
  
  class FeatureGenerator(TransformerMixin):
  
      def __init__(self, num_features=None):
          self.num_features = num_features
  
      def fit(self, X, y=None, **fit_params):
          return self
  
      def transform(self, X, **transform_params):
          return np.array(
              range(self.num_features*self.num_features)
          ).reshape(self.num_features,
                    self.num_features)
  
      def get_params(self, deep=True):
          return {"num_features": self.num_features}
  
      def set_params(self, **parameters):
          self.num_features = parameters["num_features"]
          return self
  </code></pre>
  
  <p>This transformer in action would e. g. be called like this:</p>
  
  <pre><code>pipeline = Pipeline([
      ('pick_features', FeatureGenerator(100)),
      ('kmeans', cluster.KMeans())
  ])
  
  pipeline = pipeline.fit(None)
  classes = pipeline.predict(None)
  print classes
  </code></pre>
  
  <p>It gets tricky for me as soon as I try to grid search over this pipeline:</p>
  
  <pre><code>parameter_sets = {
      'pick_features__num_features' : [10,20,30],
      'kmeans__n_clusters' : [2,3,4]
  }
  
  pipeline = Pipeline([
      ('pick_features', FeatureGenerator()),
      ('kmeans', cluster.KMeans())
  ])
  
  g_search_estimator = GridSearchCV(pipeline, parameter_sets)
  
  g_search_estimator.fit(None,None)
  </code></pre>
  
  <p>The grid search expects the samples and the labels as input and is not as robust as the pipeline, which does not complain about <code>None</code> as input parameter:</p>
  
  <pre><code>TypeError: Expected sequence or array-like, got &lt;type 'NoneType'&gt;
  </code></pre>
  
  <p>This makes sense, because the grid search needs to divide the data set in to different cv-partitions.  </p>
  
  <hr>
  
  <p>Other than in the above example, I have a lot of parameters, that can be adjusted in the data set generation step. I thus need a solution to include this step into my parameter selection cross-validation.</p>
  
  <p><strong>Question:</strong> Is there a way to set the <code>X</code>s and <code>y</code>s of the GridSearch from inside the first transformer? Or how could a solution look like, that calls the GridSearch with multiple different data sets (preferably parallel)? Or has anyone tried to customize <code>GridSearchCV</code> or can point to some reading materials on this?</p>