title: Keras: fit_generator get much lower accuracy then the evaluate_generator just one line above?
body: <p>I am following the <a href="https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html" rel="nofollow noreferrer">blog</a> for transfer learning: </p>
  
  <pre><code>##First I compute the saved the bottleneck features and build a new model and train it with the bottleneck features: 
  input_layer = Input(shape=base_model.output_shape[1:])
  x = GlobalAveragePooling2D()(input_layer)
  x = Dense(512, activation='relu',name='fc_new_1')(x)
  x = Dropout(0.2)(x)
  x = Dense(512, activation='relu',name='fc_new_2')(x)
  x = Dense(num_classes, activation='softmax',name='logit_new')(x)
  Add_layers = Model(inputs=input_layer, outputs=x,name='Add_layers')
  
  ##Then I put this new model at the end of pretrained models: 
  base_model = ResNet50(include_top=False, weights='imagenet', input_shape=
  (img_shape[0],img_shape[1],3))
  x = base_model.output
  predictions = Add_layers(x)
  model = Model(inputs=base_model.input, outputs=predictions)
  model.compile(optimizer=sgd, loss='categorical_crossentropy', metrics=
  ['accuracy'])
  
  ##Then, I evaluate the model : 
  score = model.evaluate_generator(train_generator, nb_train_samples // 
  batch_size_finetuning)
  print('The evaluation of the entire model before fine tuning : ')
  print(score)
  score = model.evaluate_generator(validation_generator, nb_validation_samples 
  // batch_size_evaluation)
  print('The evaluation of the entire model before fine tuning : ')
  print(score)
  </code></pre>
  
  <p>And get training loss and accuracy : <code>[0.015362062912073827, 1.0]</code>
  validation loss and accuracy : <code>[0.89740632474422455, 0.75]</code></p>
  
  <pre><code>##Just one line below it, I trained the new model: 
  model.fit_generator(train_generator,
                  steps_per_epoch= nb_train_samples // batch_size_finetuning,
                  epochs=finetuning_epoch,
                  validation_data=validation_generator,
                  validation_steps=nb_validation_samples //batch_size_evaluation,
                  callbacks=[checkpointer_finetuning, 
   history_finetuning,TB_finetuning,
  lrate_finetuning,Eartly_Stopping_finetuning]);
  </code></pre>
  
  <p>Then the output is : </p>
  
  <pre><code>31/31 [==============================] - 35s - loss: 3.4004 - acc: 0.3297 - val_loss: 0.9591 - val_acc: 0.7083
  </code></pre>
  
  <p>Weird thing is: This problem only happens if I use <code>Resnet50</code> and <code>InceptionV3</code> but not with <code>vgg16</code>. I am pretty sure that changing the pretrained model is the only difference. I understand that the dropout might make it different but it should not be this large and <code>vgg16</code> has no obvious problem at all.  </p>
  
  <p>Another weird thing is: If I change every layer to be <code>.trainable = False</code> and compile, the validation accuracy will still decrease dramatically. I even checked the weights of every layer and if <code>.trainable = False</code> weights will not change and <code>.trainable = True</code> weights will change. </p>
  
  <p>Any help is appreciated!!! THANKS!!!</p>