title: Siamese neural network with two pre-trained ResNet 50 - strange behavior while testing model
body: <p>I built siamese neural network, using Keras lib for it.
  My model has two inputs with shape (64,64,3), two pre-trained ResNet-50.
  Loss function is binary cross entropy.</p>
  
  <p>The model is based on this paper <a href="https://www.researchgate.net/publication/328458023_Siamese-ResNet_Implementing_Loop_Closure_Detection_based_on_Siamese_Network" rel="nofollow noreferrer">a link</a></p>
  
  <p>During train I have very good trait/val accuracy, about 0.99/0.98, and low loss 0.01/0.05.</p>
  
  <p>But when I test my saved model, I get bad results. The model can't recognize even two the same pictures. </p>
  
  <p>Also I noticed strange behavior:
  the greater the number of epochs the result is worse. 
  For example, comparing two identical images, trained model with <strong>10 epoch</strong>  gives prediction:
  <em>"8.jpg": 0.5180479884147644</em>
  but the same model trained with <strong>100</strong> <strong>epoch</strong> gives
  <em>"8.jpg": 5.579867080537926E-13</em>
  However for <strong>100 epoch</strong> I have better train results.</p>
  
  <p>I've tried different model for CNN: ResNet18, different input shapes, like <em>(224,224,3)</em> or <em>(128,128,3)</em>. </p>
  
  <p>Also I've triad use not pre-train model, only <strong>ResNet50/ResNet18</strong> without pre-trained weights. But I have the same bad results while testing real model.</p>
  
  <p>My code is</p>
  
  <pre><code>def create_base_model(image_shape, dropout_rate, suffix=''):
      I1 = Input(shape=image_shape)
      model = ResNet50(include_top=False, weights='imagenet', input_tensor=I1, pooling=None)
      model.layers.pop()
      model.outputs = [model.layers[-1].output]
      model.layers[-1].outbound_nodes = []
  
      for layer in model.layers:
          layer.name = layer.name + str(suffix)
          layer.trainable = False
  
      flatten_name = 'flatten' + str(suffix)
  
      x = model.output
      x = Flatten(name=flatten_name)(x)
      x = Dense(1024, activation='relu')(x)
      x = Dropout(dropout_rate)(x)
      x = Dense(512, activation='relu')(x)
      x = Dropout(dropout_rate)(x)
  
      return x, model.input
  
  
  def create_siamese_model(image_shape, dropout_rate):
  
      output_left, input_left = create_base_model(image_shape, dropout_rate)
      output_right, input_right = create_base_model(image_shape, dropout_rate, suffix="_2")
  
      L1_layer = Lambda(lambda tensors: tf.abs(tensors[0] - tensors[1]))
      L1_distance = L1_layer([output_left, output_right])
      L1_prediction = Dense(1, use_bias=True,
                            activation='sigmoid',
                            kernel_initializer=RandomNormal(mean=0.0, stddev=0.001),
                            name='weighted-average')(L1_distance)
  
      prediction = Dropout(0.2)(L1_prediction)
  
      siamese_model = Model(inputs=[input_left, input_right], outputs=prediction)
  
      return siamese_model
  
  siamese_model = create_siamese_model(image_shape=(64, 64, 3),
                                           dropout_rate=0.2)
  
  siamese_model.compile(loss='binary_crossentropy',
                        optimizer=Adam(lr=0.0001),
                        metrics=['binary_crossentropy', 'acc'])
  siamese_model.fit_generator(train_gen,
                              steps_per_epoch=1000,
                              epochs=10,
                              verbose=1,
                              callbacks=[checkpoint, tensor_board_callback, lr_reducer, early_stopper, csv_logger],
                              validation_data=validation_data,
                              max_q_size=3)
  
  siamese_model.save('siamese_model.h5')
  
  
  
  # and the my prediction
  siamese_net = load_model('siamese_model.h5', custom_objects={"tf": tf})
  
  X_1 = [image, ] * len(markers)
  batch = [markers, X_1]
  result = siamese_net.predict_on_batch(batch)
  
  # I've tried also to check identical images 
  markers = [image]
  X_1 = [image, ] * len(markers)
  batch = [markers, X_1]
  result = siamese_net.predict_on_batch(batch)
  </code></pre>
  
  <p>I have some doubts about my prediction method.
  Could someone please help me to find what is wrong with predictions?</p>