<p>In addition to already said in the answers:</p>

<ul>
<li>You can have several <code>Dropout</code> layers with different probabilities, e.g. after the pooling layers. Early layers often have higher keep probability, since they use fewer filters.</li>
<li><a href="https://machinelearningmastery.com/image-augmentation-deep-learning-keras/" rel="nofollow noreferrer">Image data augmentation</a> is another way towards generalization and in my experience it always improves the result, at least slightly (of course, provided that input transformation is not severe).</li>
<li><a href="https://keras.io/layers/normalization/" rel="nofollow noreferrer">Batch normalization</a> (and its successors, <a href="https://arxiv.org/abs/1602.07868" rel="nofollow noreferrer">weight normalization</a> and <a href="https://arxiv.org/abs/1607.06450" rel="nofollow noreferrer">layer normalization</a>) is a modern regularization method that reduces the required dropout intensity, sometimes completely, i.e. you can get rid of dropout layers. In addition, batchnorm improves activations statistics, which often makes the network learn faster. I used it in addition to dropout and it worked pretty well.</li>
<li>A technique called Scaled Exponential Linear Units (SELU) has been <a href="https://arxiv.org/pdf/1706.02515.pdf" rel="nofollow noreferrer">published very recently</a>, which is said to have implicit self-normalizing properties. It's even <a href="https://keras.io/activations/" rel="nofollow noreferrer">already implemented</a> in keras.</li>
<li>The good old L2 or L1 regularizer is still in use. If nothing else helps, consider adding it too. But I'm pretty sure that batchnorm, selu and few dropout layers will be enough.</li>
</ul>
Overfitting problems for key points model (detecting wrists, elbows, shoulders, etc.)
<p>I'm building a model to detect keypoints of body parts. To do that I'm using the COCO dataset (<a href="http://cocodataset.org/#download" rel="nofollow noreferrer">http://cocodataset.org/#download</a>). I'm trying to understand why I'm running into overfitting issues (training loss converges, but I reach a ceiling really early for testing loss). In the model, I've tried adding layers of dropout (gradually adding more layers with higher probabilities, but I quickly get to a point when training loss stops decreasing which is just as bad. <strong>My theory is that the model I use isn't complex enough but I'd like to know if that's the likely reason or if it could be something else. The models I've found online are all extremely deep (30+ layers).</strong> </p>

<p><strong>Data</strong></p>

<p>I'm using 10,000 RGB images each of which has a single person in it. They each have different sizes but a max of 640 length and width. As a preprocessing step, I make every image the size 640x640 by filling any extra area (bottom and right of image) with (0,0,0) or black. </p>

<p><strong>Targets</strong></p>

<p>The full dataset has many keypoints but I'm only interested in the right shoulder, right elbow, and right wrist. Each body part has 2 keypoints (X coordinate and Y coordinate) so my target is a list of length 6.</p>

<p><strong>Model</strong></p>

<pre><code>activation_function = 'relu'
batch_size = 16  # ##
epoch_count = 40  # ##
loss_function = 'mean_squared_error'
opt = 'adam'

model = Sequential()
model.add(Conv2D(filters=16, kernel_size=(3, 3), input_shape=inp_shape))
# model.add(Conv2D(filters=16, kernel_size=(3, 3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(filters=32, kernel_size=(3, 3)))
# model.add(Conv2D(filters=32, kernel_size=(3, 3)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(300, activation=activation_function))
model.add(Dropout(rate=0.1))
model.add(Dense(300, activation=activation_function))
model.add(Dense(num_targets))
model.summary()
model.compile(loss=loss_function, optimizer=opt)
hist = model.fit(x_train, y_train, batch_size=batch_size, epochs=epoch_count,
                 verbose=verbose_level,
                 validation_data=(x_valid, y_valid))
</code></pre>