49.3      29.5      17.9      10.9      37.2            17.0         54.1           84.2%       2.73$\pm$0.96   76.2%
     SentiCap       93.2       49.1      29.1      17.5      10.8      36.5            16.8         54.4           88.4%       2.86$\pm$0.97   84.6%
     CNN+RNN        0.8        47.6      27.5      16.3      9.8       36.1            15.0         54.6           –           2.81$\pm$0.94   –
     ANP-Replace    85.5       48.1      28.8      17.7      10.9      36.3            16.0         56.5           61.4%       2.51$\pm$0.93   73.7%
     ANP-Scoring    85.5       47.9      28.7      17.7      11.1      36.2            16.0         57.1           64.5%       2.52$\pm$0.94   76.0%
     RNN-Transfer   73.4       47.8      29.0      18.7      12.1      36.7            16.2         55.9           68.1%       2.52$\pm$0.96   70.3%
     SentiCap       97.4       50.0      31.2      20.3      13.1      37.9            16.8         61.8           72.5%       2.40$\pm$0.89   65.0%

[[**[Implementation details.]{}**]{}]{} We implement RNNs with LSTM units using the Theano package [@BastienTheano2012]. Our implementation of CNN+RNN reproduces caption generation performance in recent work [@Karpathy2015CVPR]. The visual input to the switching RNN is 4096-dimensional feature vector from the second last layer of the Oxford VGG CNN [@Simonyan2015]. These features are [linearly embedded into a $D=512$ dimensional space]{}. Our word embeddings ${\mathbf{Ey}}$ are 512 dimensions and the hidden state ${\mathbf{h}}$ and memory cell ${\mathbf{c}}$ of the LSTM module also have 512 dimensions. The size of our vocabulary for generating sentences is 8,787, and becomes 8,811 after including additional sentiment words.

[We train the model using Stochastic Gradient Descent (SGD) with mini-batching and the momentum update rule. Mini-batches of size 