<p>I want to mention few things here which may be useful to others:</p>
<p><strong>1) Data stratification / random sampling</strong></p>
<p>When you use <code>validation_split</code> Keras uses the last x percent of data as validation data. This means that if the data is ordered by class, e.g. because &quot;pairs&quot; or &quot;tripletts&quot; are made in a sequence, validation data will only come from classes (or the class) contained in the last x percent of data. In this case, the validation set will be of no use. Thus <strong>it is essential to suffle input</strong> data to make sure that the validation set contains random samples from each class.</p>
<p>The docs for <code>validation_split</code> say:</p>
<blockquote>
<p>Float between 0 and 1. Fraction of the training data to be used as
validation data. The model will set apart this fraction of the
training data, will not train on it, and will evaluate the loss and
any model metrics on this data at the end of each epoch. The
validation data is selected from the <strong>last samples</strong> in the x and y data
provided, before shuffling</p>
</blockquote>
<p><strong>2) Choice of optimizer</strong></p>
<p>In <code>model.compile()</code> choosing <code>optimizer='sgd'</code> may not be the best approach since <code>sgd</code> can get stuck in local minima etc. <code>Adam</code> (<a href="https://keras.io/api/optimizers/adam/" rel="nofollow noreferrer">see docs</a>) seems to be a good choice to start with since it...</p>
<blockquote>
<p>[...] combines the advantages of [...] AdaGrad to deal with sparse
gradients, and the ability of RMSProp to deal with non-stationary
objectives.</p>
</blockquote>
<p>according to <a href="https://arxiv.org/abs/1412.6980" rel="nofollow noreferrer">Kingma and Ba</a> (2014, page 10).</p>
<pre><code>from keras.optimizers import Adam
...
model.compile(loss=contrastive_loss, optimizer=keras.optimizers.Adam(lr=0.0001))
</code></pre>
<p><strong>3) Early stopping / learning rate</strong></p>
<p>Using <a href="https://keras.io/api/callbacks/early_stopping/" rel="nofollow noreferrer">early stopping</a> and <a href="https://keras.io/api/callbacks/reduce_lr_on_plateau/" rel="nofollow noreferrer">adjusting the learning rate</a> during training may also be highly useful to achieve good results. So the model can train until there is no more success (stop automatically in this case).</p>
<pre><code>from keras.callbacks import EarlyStopping
from keras.callbacks import ReduceLROnPlateau
...
early_stopping = EarlyStopping(monitor='val_loss', patience=50, mode='auto', restore_best_weights=True)
reduce_on_plateau = ReduceLROnPlateau(monitor=&quot;val_loss&quot;, factor=0.8, patience=15, cooldown=5, verbose=0)
...
hist = model.fit([img_1, img_2], y, 
            validation_split=.2, 
            batch_size=128, 
            verbose=1, 
            epochs=9999,
            callbacks=[early_stopping])
</code></pre>
<p><strong>4) Kernel initialization</strong></p>
<p>Kernel initialization (with a small SD) may be helpful as well.</p>
<pre><code># Layer 1
    seq.add(Conv2D(8, (5,5), input_shape=input_shape, 
        kernel_initializer=keras.initializers.TruncatedNormal(mean=0.0, stddev=0.01, seed=None), 
        data_format=&quot;channels_first&quot;))
    seq.add(Activation('relu'))
    seq.add(MaxPooling2D(pool_size=(2, 2))) 
    seq.add(Dropout(0.1))
</code></pre>
<p><strong>5) Overfitting</strong></p>
<p>I noticed that instead of using dropout to fight overfitting, adding some noise can be rather helpful. In this case simply add some <a href="https://keras.io/api/layers/regularization_layers/gaussian_noise/" rel="nofollow noreferrer">GaussianNoise</a> at the top of the network.</p>
Keras Model for Siamese Network not Learning and always predicting the same ouput
<p>I am trying to train a Siamese neural network using Keras, with the goal of identifying if 2 images belong to same class or not. My data is shuffled and has equal number of positive examples and negative examples. My model is not learning anything and it is predicting the same output always. I am getting the same loss, validation accuracy, and validation loss every time. </p>

<p><a href="https://i.stack.imgur.com/dRfLF.png" rel="nofollow noreferrer">Training Output</a></p>

<pre class="lang-py prettyprint-override"><code>def convert(row):
    return imread(row)

def contrastive_loss(y_true, y_pred):
    margin = 1
    square_pred = K.square(y_pred)
    margin_square = K.square(K.maximum(margin - y_pred, 0))
    return K.mean(y_true * square_pred + (1 - y_true) * margin_square)

def SiameseNetwork(input_shape):
    top_input = Input(input_shape)

    bottom_input = Input(input_shape)

    # Network
    model = Sequential()
    model.add(Conv2D(96,(7,7),activation='relu',input_shape=input_shape))
    model.add(MaxPooling2D())
    model.add(Conv2D(256,(5,5),activation='relu',input_shape=input_shape))
    model.add(MaxPooling2D())
    model.add(Conv2D(256,(5,5),activation='relu',input_shape=input_shape))
    model.add(MaxPooling2D())
    model.add(Flatten())
    model.add(Dense(4096,activation='relu'))
    model.add(Dropout(0.5))
    model.add(Dense(1024,activation='relu'))
    model.add(Dropout(0.5))
    model.add(Dense(512,activation='relu'))
    model.add(Dropout(0.5))

    encoded_top = model(top_input)
    encoded_bottom = model(bottom_input)

    L1_layer = Lambda(lambda tensors:K.abs(tensors[0] - tensors[1]))    
    L1_distance = L1_layer([encoded_top, encoded_bottom])

    prediction = Dense(1,activation='sigmoid')(L1_distance)
    siamesenet = Model(inputs=[top_input,bottom_input],outputs=prediction)
    return siamesenet

data = pd.read_csv('shuffleddata.csv')
print('Converting X1....')

X1 = [convert(x) for x in data['X1']]

print('Converting X2....')

X2 = [convert(x) for x in data['X2']]

print('Converting Y.....')
Y = [0 if data['Y'][i] == 'Negative' else 1 for i in range(len(data['Y']))]

input_shape = (53,121,3,)
model = SiameseNetwork(input_shape)
model.compile(loss=contrastive_loss,optimizer='sgd',metrics=['accuracy'])
print(model.summary())
model.fit(X1,Y,batch_size=32,epochs=20,shuffle=True,validation_split = 0.2)
model.save('Siamese.h5')

</code></pre>