title: KerasClassifier issue when using custom scoring with GridSearchCV for NN with multiclass output
body: <p>Using custom scoring with Multiclass outputs from Keras model returns the same error for cross_val_score or GridSearchCV as below (it's on Iris, so you can run it directly to test):</p>
  
  
  
  <pre class="lang-python prettyprint-override"><code>import numpy as np
  from sklearn import datasets
  from sklearn.model_selection import train_test_split, cross_val_score, GridSearchCV
  from keras.models import Sequential
  from keras.layers import Dense
  from keras.utils import to_categorical
  from keras.wrappers.scikit_learn import KerasClassifier
  
  iris = datasets.load_iris()
  X= iris.data
  Y = to_categorical(iris.target)
  
  X_train, X_test, Y_train, Y_test = train_test_split(X, Y, train_size=0.8, random_state=1000)
  
  def create_model(optimizer='rmsprop'):
      model = Sequential()
      model.add(Dense(8,activation='relu',input_shape = (4,)))
      model.add(Dense(3,activation='softmax'))
      model.compile(optimizer = optimizer,
                    loss='categorical_crossentropy',
                    metrics=['accuracy'])
      return model
  
  
  model = KerasClassifier(build_fn=create_model,
                          epochs=10, 
                          batch_size=5,
                          verbose=0)
  
  #results = cross_val_score(model, X_train, Y_train, scoring='precision_macro')
  
  param_grid = {'optimizer':('rmsprop','adam')}
  grid = GridSearchCV(model,
                      param_grid=param_grid,
                      return_train_score=True,
                      scoring=['accuracy','precision_macro','recall_macro'],
                      refit='precision_macro')
  
  grid_results = grid.fit(X_train,Y_train)
  </code></pre>
  
  <p><strong>So I get this error</strong></p>
  
  <p><em>I bypassed the full stack as you can reproduce it by copying the code above.</em></p>
  
  <pre class="lang-python prettyprint-override"><code>ValueError: Classification metrics can't handle a mix of multilabel-indicator and binary targets
  </code></pre>
  
  <p>When I, remove the scoring parameters, it works.</p>
  
  <p>Is there any way to avoid that and enable having a f1, precision or any custom score ? without having to rewrite my own grid search code, of course. </p>
  
  <p>Thanks for your help</p>
  
  <p><strong>UPDATE : I've just found how to resolve it</strong></p>
  
  <p>First this doc (<a href="http://scikit-learn.org/stable/modules/multiclass.html#multilabel-classification-format" rel="nofollow noreferrer">http://scikit-learn.org/stable/modules/multiclass.html#multilabel-classification-format</a>) show that the one-hot representation used in Keras is interpreted as <em>multilabel</em> in scikit-learn.</p>
  
  <p>Then looking at <code>scikit_learn.py</code> implementing KerasClassifier class : <a href="https://github.com/keras-team/keras/blob/master/keras/wrappers/scikit_learn.py" rel="nofollow noreferrer">https://github.com/keras-team/keras/blob/master/keras/wrappers/scikit_learn.py</a></p>
  
  <p>The fit function in the BaseWrapper class includes this line of code :</p>
  
  <pre class="lang-python prettyprint-override"><code>if loss_name == 'categorical_crossentropy' and len(y.shape) != 2:
              y = to_categorical(y)
  </code></pre>
  
  <p>The Wrapper does the categorical transformation by itself.</p>
  
  <p>It seems that Keras, to avoid this issue, due to the difference in the multiclass representation with scikit-learn, can takes a scikit-learn style multiclass <code>[0,1,2,1,0,2]</code> and transform it into categorical representation just for the NN model fit. </p>
  
  <p>So, I simply tried removing the categorical transformation when passing the model to the sklearn functions. </p>
  
  <p><strong>And it work now</strong></p>
  
  <pre class="lang-python prettyprint-override"><code>import numpy as np
  from sklearn import datasets
  from sklearn.model_selection import train_test_split, cross_val_score, GridSearchCV
  from keras.models import Sequential
  from keras.layers import Dense, Activation
  from keras.utils import to_categorical
  from keras.wrappers.scikit_learn import KerasClassifier
  
  iris = datasets.load_iris()
  X= iris.data
  #Y = to_categorical(iris.target,3)
  Y = iris.target
  
  X_train, X_test, Y_train, Y_test = train_test_split(X, Y, train_size=0.8, random_state=1000)
  
  def create_model(optimizer='rmsprop'):
      model = Sequential()
      model.add(Dense(8,activation='relu',input_shape = (4,)))
      model.add(Dense(3,activation='softmax'))
      model.compile(optimizer = optimizer,
                    loss='categorical_crossentropy',
                    metrics=['accuracy'])
      return model
  
  
  model = KerasClassifier(build_fn=create_model,
                          epochs=10, 
                          batch_size=5,
                          verbose=0)
  
  #results = cross_val_score(model, X_train, Y_train, scoring='precision_macro')
  
  param_grid = {'optimizer':('rmsprop','adam')}
  grid = GridSearchCV(model,
                      param_grid=param_grid,
                      return_train_score=True,
                     scoring=['precision_macro','recall_macro','f1_macro'],
                      refit='precision_macro')
  grid_results = grid.fit(X_train,Y_train)
  </code></pre>