title: The accuracy that the fit generator outputs in Keras differs from the manually calculated accuracy
body: <p>It worked fine when I used fit, but when I used fit_generator, I got a problem.</p>
  
  <p>I used the call back method to find the confusion matrix at the end of each train epoch.</p>
  
  <p>However, the accuracy obtained from the confusion matrix and the validation accuracy output from keras differ.</p>
  
  <p><a href="https://i.stack.imgur.com/hPeer.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/hPeer.png" alt="enter image description here"></a></p>
  
  <p>My code is below.</p>
  
  <pre><code>    metrics = Valid_checker(model_name, args.patience, (x_valid, y_valid), x_length_valid)       
      model.compile(optimizer=optimizers.RMSprop(lr=args.lr),
        loss=[first_loss],
        loss_weights=[1.],
        metrics={'capsnet': 'accuracy'})
      callback_list = [lr_decay, metrics]
  
      model.fit_generator(
                  no_decoder_generator(x_train, y_train),
                  steps_per_epoch=len(x_train),
                  epochs=args.epochs,
                  validation_data=no_decoder_generator(x_valid, y_valid),
                  validation_steps=len(x_valid),
                  callbacks=callback_list,
                  #class_weight=class_weights,
                  verbose=1)
  </code></pre>
  
  <p><code>Valid check</code> is my callback method. <code>no_decoder_generator</code> is my decoder generator. and my batch size of train and validation is 1. </p>
  
  <p>This is my <code>Valid_check</code> class. (below)</p>
  
  <pre><code>class Valid_checker(keras.callbacks.Callback):
          def __init__(self, model_name, patience, val_data, x_length):
              super().__init__()
              self.best_score = 0
              self.patience = patience
              self.current_patience = 0 
              self.model_name = model_name
              self.validation_data = val_data
              self.x_length = x_length
  
  
          def on_epoch_end(self, epoch, logs={}):
              X_val, y_val = self.validation_data
              if args.decoder==1:
                  y_predict, x_predict = model.predict_generator(no_decoder_generator(X_val, y_val), steps=len(X_val))
                  y_predict = np.asarray(y_predict)
                  x_predict = np.asarray(x_predict)                       
  
              else:
                  y_predict = np.asarray(model.predict_generator(predict_generator(X_val), steps=len(X_val)))
  
              y_val, y_predict = get_utterence_label_pred(y_val, y_predict, self.x_length )
              cnf_matrix = get_accuracy_and_cnf_matrix(y_val, y_predict)[1]
              val_acc_custom =  get_accuracy_and_cnf_matrix(y_val, y_predict)[0]
              war = val_acc_custom[0]
              uar = val_acc_custom[1]
              score = round(0.2*war+0.8*uar,2)
  
              loss_message=''
              # custom ModelCheckpoint &amp; early stopping by using UAR            
              loss_message='loss: %s - acc: %s - val_loss: %s - val_acc: %s'%(round(logs.get('loss'),4), round(logs.get('acc'),4), round(logs.get('val_loss'),4), round(logs.get('val_acc'),4))
              log('[Epoch %03d/%03d]'%(epoch+1, args.epochs))
              log(loss_message)
              log('Confusion matrix:')
              log('%s'%cnf_matrix)
              log('Valid [WAR] [UAR] [Custom] : %s [%s]'%(val_acc_custom,score))
  
              if score &gt; self.best_score :
                  model.save_weights(model_name)
                  log('Epoch %05d: val_uar_acc improved from %s to %s saving model to %s'%(epoch+1, self.best_score, score, self.model_name))
                  self.best_score = score
                  self.current_patience = 0
  
              else :
                  self.current_patience+=1
  
              # early stopping
              if self.current_patience == (self.patience-1):
                  self.model.stop_training = True
                  log('Epoch %05d: early stopping' % (epoch + 1)) 
              return        
  </code></pre>
  
  <p>It should be equal to <code>val_acc</code> output by keras and <code>war</code>. However, the value is different. Why does this happen?  I have confirmed that there are no problems with <code>get_utterence_label_pred</code> and <code>get_accuracy_and_cnf_matrix</code>. 
  It works well when I use the fit function.</p>
  
  <p>My generator is below. </p>
  
  <pre><code>def predict_generator(x):
  while True:
      for index in range(len(x)):
          feature = x[index]
          feature = np.expand_dims(x[index],-1)
          feature = np.expand_dims(feature,0) # make (1,input_height,input_width,1) 
          yield (feature)
  
  def no_decoder_generator(x, y):
  while True:
      indexes = np.arange(len(x))
      np.random.shuffle(indexes)
      for index in indexes:
          feature = x[index]
          feature = np.expand_dims(x[index],-1)
          feature = np.expand_dims(feature,0) # make (1,input_height,input_width,1) 
          label = y[index]
          label = np.expand_dims(label,0)
          yield (feature, label)
  </code></pre>
  
  <blockquote>
    <p>Epoch 1/70<br>
    1858/1858 [==============================] - 558s 300ms/step - loss:
    1.0708 - acc: 0.5684 - val_loss: 0.9087 - val_acc: 0.6244      [Epoch 001/070]<br>
    loss: 1.0708 - acc: 0.5684 - val_loss: 0.9087 - val_acc: 0.6244<br>
    Confusion matrix:<br>
    [[ 0.   28.   68.    4.  ]<br>
    [ 0.   13.33 80.    6.67]<br>
    [ 0.96  2.88 64.42 31.73]<br>
    [ 0.    0.    3.28 96.72]]<br>
    Valid [WAR] [UAR] [Custom] : [62.44 43.62] [47.38] </p>
    
    <p>Epoch 2/70
    1858/1858 [==============================] - 262s 141ms/step - loss:
    0.9526 - acc: 0.6254 - val_loss: 1.1998 - val_acc: 0.4537      [Epoch 002/070]<br>
    loss: 0.9526 - acc: 0.6254 - val_loss: 1.1998 - val_acc: <strong>0.4537</strong><br>
    Confusion matrix:<br>
    [[ 36.    12.    24.    28.  ]<br>
    [ 20.     0.    46.67  33.33]<br>
    [  4.81   0.96  24.04  70.19]<br>
    [  0.     0.     0.   100.  ]]<br>
    Valid [WAR] [UAR] [Custom] : [<strong>46.34</strong> 40.01] [41.28]</p>
  </blockquote>