title: How to combine SGD like SVM Kernel Approximation with Feature Selection on a Multiclass Dataset
body: <p>I want to train a relatively large recordset. (200000 rows and 400 columns) in a pipeline. Only a weak notebook is available for the task.
  This dataset has 15 independent classes and mixed categorical and numerical features. An SVM-like algorithm should be chosen.</p>
  
  <p>I already tried to put some code together.</p>
  
  <pre><code>from sklearn.datasets import make_classification
  from sklearn.preprocessing import LabelBinarizer,StandardScaler, OneHotEncoder
  from sklearn.compose import ColumnTransformer
  from sklearn.kernel_approximation import RBFSampler
  from sklearn.linear_model import SGDClassifier
  from sklearn.model_selection import StratifiedShuffleSplit
  from sklearn.pipeline import Pipeline
  from sklearn.feature_selection import RFECV
  from sklearn.multiclass import OneVsRestClassifier
  
  X, y= make_classification(n_samples=200000, n_features=130, n_informative=105,
                            n_redundant=25, n_classes=15, n_clusters_per_class=15)
  
  #add some categorical columns
  X [:,:2]= np.abs(X[:,:2]).astype(int)
  X = pd.DataFrame(X, columns=[f'F{i}' for i in range(X.shape[1])])
  cols = X.columns.tolist()
  
  y = LabelBinarizer().fit_transform(y)
  
  #%%Transformation  
  full_pipeline = ColumnTransformer([
          ('numerical', StandardScaler(), cols[2:]),
          ('categorical', OneHotEncoder(categories='auto'), cols[:2])
          ])
  
  #Sparse matrix
  X = full_pipeline.fit_transform(X)
  
  #set start
  rbf = RBFSampler(gamma=0.1, random_state=42)
  semi_svm = SGDClassifier(loss="hinge", penalty="l2", max_iter=50)
  
  clf_pipe = Pipeline([
          ('rbf', rbf),
          ('svm', semi_svm)
          ])
  
  cv = StratifiedShuffleSplit(n_splits=5)
  
  grid_search = RFECV(estimator=OneVsRestClassifier(clf_pipe), step=3, cv=cv, 
                scoring='accuracy', n_jobs=-1, verbose=10)
  
  grid_search.fit(X, y)
  </code></pre>
  
  <blockquote>
    <p>ValueError: bad input shape (200000, 15)</p>
  </blockquote>
  
  <p>How to handle the multiclass error in this case?</p>