title: From SKLearn to Keras - What is the difference?
body: <p>I'm trying to go from <strong>SKLearn</strong> to <strong>Keras</strong> in order to make specific improvements to my models.
  However, I can't get the same performance I had with my <strong>SKLearn</strong> model :</p>
  
  <pre><code>mlp = MLPClassifier(
      solver='adam', activation='relu',
      beta_1=0.9, beta_2=0.999, learning_rate='constant',
      alpha=0, hidden_layer_sizes=(238,),
      max_iter=300
  )
  dev_score(mlp)
  </code></pre>
  
  <p>Gives ~0.65 score everytime</p>
  
  <p>Here is my corresponding <strong>Keras</strong> code :</p>
  
  <pre><code>def build_model(alpha):
      level_moreargs = {'kernel_regularizer':l2(alpha), 'kernel_initializer': 'glorot_uniform'}
      model = Sequential()
      model.add(Dense(units=238, input_dim=X.shape[1], **level_moreargs))
      model.add(Activation('relu'))
      model.add(Dense(units=class_names.shape[0], **level_moreargs)) # output
      model.add(Activation('softmax'))
  
      model.compile(loss=keras.losses.categorical_crossentropy, # like sklearn
                    optimizer=keras.optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0),
                    metrics=['accuracy'])
      return model
  
  k_dnn = KerasClassifier(build_fn=build_model, epochs=300, batch_size=200, validation_data=None, shuffle=True, alpha=0.5, verbose=0)
  
  dev_score(k_dnn)
  </code></pre>
  
  <p>From looking at the documentation (and digging into SKLearn code), <em>this should correspond exactly to the same thing</em>.</p>
  
  <p>However, I get ~0.5 accuracy when I run this model, <em>which is very bad</em>.</p>
  
  <p>And if I set alpha to 0, SKLearn's score barely changes (0.63), while Keras's goes random from 0.2 to 0.4.</p>
  
  <p>What is the difference between these models ? Why is Keras, although being supposed to be better than SKLearn, outperformed by so far here ? <em>What's my mistake</em> ?</p>
  
  <p>Thanks,</p>