<p>One CNN architecture that achieves you goal is the U-Net, originally introduced by <a href="https://arxiv.org/abs/1505.04597" rel="nofollow noreferrer">this</a> paper.</p>

<p>It uses a sequence of convolutional and pooling layers to create a pyramid. Note that it's not an image pyramid of the input image, but the idea is to learn what is useful at different scales, not directly feed the pyramid.</p>

<p>Now, think about how the <code>AveragePooling2D</code> works. You select a patch of the original image, replace it with the average and then move to the next patch. This is exactly what you describe in generating an image pyramid: smoothing is achieved by the averaging, and replacing the patch with one pixel is a downsampling.</p>
Implementing an image-pyramid with Keras
<p>To get a scale invariance (or to detect objects at any scale) on my CNN model, I want to implement <a href="https://en.wikipedia.org/wiki/Pyramid_(image_processing)" rel="nofollow noreferrer">Image Pyramids</a>. As the article explains, while creating image-pyramids, image is subjected to repeated smoothing and subsampling. </p>

<p>I am implementing a CNN in Keras. Is there a way with Keras to implement image pyramids? I read <a href="https://stackoverflow.com/questions/48420092/how-to-down-scale-image-in-keras-for-an-image-pyramid">one of the SO post</a> that says to use <a href="https://keras.io/layers/pooling/#averagepooling2d" rel="nofollow noreferrer">AveragePooling2D</a> to achieve the pyramid effect.</p>

<p>Is that even correct? How could <code>AveragePooling2D</code> layer give the pyramid effect? </p>