<p>Yes, you are right with the concept of <strong>CONVLSTM2D</strong>.<br />
<strong>CONVLSTM2D</strong> architecture combines gating of LSTM with 2D convolutions.</p>
<p>As you have mentioned, CONVLSTM layers will do a similar task to LSTM but instead of matrix multiplications, it does convolution operations and retains the input dimensions.</p>
<p>Another different approach would be that the images pass through the convolution layer and the result will be a flattened 1D array and this will be the input to the LSTM layers with a set of features over time.</p>
<p><strong>Input of Kera's CONVLSTM layer:</strong> is a 5D tensor with shape</p>
<p><code>(samples, time, channels, rows, cols)</code> if it is channels first.<br />
<code>(samples, time, rows, cols, channels)</code> if it is channels last.</p>
<p><strong>Output of a CONVLSTM layer:</strong></p>
<p>If <code>return_sequences = True</code> then it is a 5D tensor with shape</p>
<pre><code>(samples, time, filters, rows, cols)  
</code></pre>
<p>If return_sequences = False then it is a 4D tensor with shape.</p>
<pre><code>(samples, filters, rows, cols)  
</code></pre>
<p>You can refer to <a href="https://arxiv.org/pdf/1506.04214v1.pdf" rel="nofollow noreferrer">this</a> paper from where the implementation of CONVLSTM is done.</p>
What does the keras ConvLSTM2D layer do?
<p>I would like to understand the <a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/ConvLSTM2D" rel="nofollow noreferrer">ConvLSTM2D</a> Keras layer a bit better.</p>
<p>Does it execute an 2D convolution on a 2D input (image) and then average/ flatten its ouptut and feed that into a LSTM module?
But I guess it is basically an LSTM cell, where the matrix multiplications are replaced with convolution operations. Is that correct?</p>
<p>Thank you</p>