$$\label{eq-dotproductatt}
W_\text{dot-prod} = \phi(G_1(f'_\text{in})  G_2(f'_\text{in})^{T}).$$ Here, $G_1$ and $G_2$ are both a 1D convolutional layer with kernel size as 1 and they both output a feature map of shape $(T, C')$. Let $Q = G_1(f'_\text{in})$ and $K = G_2(f'_\text{in})$. $QK^{T}$ computes an similarity matrix between the features of all the temporal frames. We then use $\phi$, an activation function of our choice, e.g., the softmax function, to convert the similarity matrix into attention weights. Note that different from $W_\text{map}$, $W_\text{dot-prod}$ is a full matrix instead of a diagonal matrix.

When being applied to the spatiotemporal dimension, dot-product attention generates a $THW \times THW$ attention weight matrix. When applying dot-product attention to the spatial dimension, each frame has its own attention weights (a $HW \times HW$ matrix), where $G_1$ and $G_2$ are shared among different frames.

#### Apply the attention weights.

We apply the attention weight matrix to the input feature map through matrix multiplication to obtain the attended feature map: $$\label{eq-applyattweights}
\begin{split}
f_\text{out}  = \texttt{ReshapeTo2D}^{-1} (  W \texttt{ReshapeTo2D}(G_3(f_\text{in})) ).
\end{split}$$ $W$ is the weight matrix generated by map-based attention $(W_\text{map})$ or dot-product attention $(W_\text{dot-prod})$. $G_3$ is a $1\times 1 \times 1$ convolutional layer to reduce the number of channels of 