title: Keras, Score v.s Predict
body: <p>I'm currently training a Keras model for classification (20 classes), text classification problem.</p>
  
  <p>When I train the model with a training and validation set, I get an output like this:</p>
  
  <blockquote>
    <p>Epoch 19/20 59500/59500 [==============================] - 22s
    371us/step - loss: 0.0038 - accuracy: 0.9989 - val_loss: 0.6645 -
    val_accuracy: 0.9550</p>
  </blockquote>
  
  <p>Suggesting a very good validation accuracy. Afterwards, when I call my model to evaluate the score (on the <em>same</em> validation set) I get:</p>
  
  <pre><code>from sklearn.metrics import accuracy_score
  predictions = classifier.predict(X_test)
  print(f"Keras Score: {classifier.score(X_test, valid_labels)}")
  print(f"Accuracy on test set: {100 * accuracy_score(np.argmax(valid_labels, axis=1),    predictions):.2f}%")
  </code></pre>
  
  <p>Yielding:</p>
  
  <blockquote>
    <p>10500/10500 [==============================] - 1s 66us/step
    10500/10500 [==============================] - 1s 80us/step 
    Keras Score: 0.9546093940734863
    Accuracy on test set: 49.58%</p>
  </blockquote>
  
  <p>I expected the scores to be exactly the same, since it is the same data, the same measure and everything. The labels are encoded as one-hot vectors, thus the argmax in the accuracy calculation. </p>
  
  <p>I would of course like the achieve the performance the model seems to be having while training, I am simply confused as to why the performance is not even the same. While I understand the model might not be correct, I don't understand why the model predicts something different than what it uses to calculate the score (metrics is accuracy).</p>
  
  <p>The model is trained with:</p>
  
  <pre><code>history = classifier.fit(X_train, train_labels,
                      epochs=20,
                      verbose=True,
                      validation_data=(X_test, valid_labels),
                      batch_size=20)
  </code></pre>
  
  <p>Thanks!</p>
  
  <p><strong>EDIT</strong></p>
  
  <p>Predictions returned by the model are not a one-hot vector. They are an integer that I believe is the class predicted.</p>