transition, we propose a hybrid model that 1) predicts a saliency map from each video frame, 2) predicts an attention map by exploiting temporal context of gaze fixations, and 3) fuses the saliency map and the attention map to output a final gaze map.

The model architecture is shown in Figure \[fig:wholemodel\]. The feature encoding module is composed by a spatial Convolutional Neural Network (S-CNN) and a temporal Convolutional Neural Network (T-CNN), which extract latent representations from a single RGB image and stacked optical flow images respectively. The saliency prediction module generates a saliency map based on the extracted latent representation. The attention transition module generates an attention map based on previous gaze fixations and head motion. The late fusion module combines the results of saliency prediction and attention transition to generate a final gaze map. The details of each module will be given in the following part.

Feature Encoding
----------------

At time $t$, the current video frame $I_t$ and stacked optical flow $O_{t-\tau, t}$ are fed into S-CNN and T-CNN to extract latent representations $F^{S}_t = h^S(I_t)$ from the current RGB frame, and $F^{T}_t = h^T(O_{t-\tau, t})$from the stacked optical flow images for later use. Here $\tau$ is fixed 