title: Training design / sequential loading of images for Mask-RCNN
body: <p>I am training a deep learning model using Mask RCNN from the following git repository: matterport/Mask_RCNN. I rely on a heavy augmentation of my dataset (original dataset: 59 images of 1988x1355x3 with each &gt; 80 annotations), which I store locally (necessary to evaluate type/degree of augmentation vs validation metrics). The augmented dataset counts 6000 images. This dataset varies in x and y dimensions of the image, because of reducing resolution and affine transformations - I assume the different x,y-dimensions will not affect the final tests.</p>
  <p>However, my Python kernel crashes whenever I load more than 'X' images to train the model.</p>
  <p>Hence, I came up with the idea of splitting the dataset in sub-datasets and iterate through the sub-dataset, using the 'last' trained weights as starting point for the new round. But I am not sure if the results will be the same (read: same, taken the stochastic nature of 'stochastic gradient descent' into account)?</p>
  <p>I wonder, if the results would be the same, if I don't iterate through the sub-datasets per epoch, but train Y epochs (eg. 20 for 'heads' only, 10 for 'all layers')?</p>
  <p>Yet, I am sure this is not the most efficient way of solving this issues. Ideas for improvement are welcome.</p>
  <p><strong>Note</strong>, I am not using <code>keras.preprocessing.image.ImageDataGenerator()</code>, as I have understood it, it randomly generates data and feeds it to the model by replacing the input for the epoch, whereas I would like to feed the whole dataset to the model.</p>