deviation), while those for the negative captions ([Neg]{}) have a significant decrease (by 0.73), likely because the notion of negativity is diverse.

[We also ask whether the sentiment of the sentence matches the image. Each rating task is completed by 3 different AMT workers.]{} In the [*correct sentiment*]{} column of Figure \[fig:mturk\_eval\](b), we record the number of votes each caption received for bearing a sentiment that matches the image. We can see that the vast majority of the captions are unanimously considered emotionally appropriate ($94\%$, or 315/335 for [Pos]{}; $82\%$, or 250/305 for [Neg]{}). Among the captions with less than unanimous votes received, most of them (20 for [Pos]{} and 49 for [Neg]{}) still have majority agreement for having the correct sentiment, which is on par with the level of noise (16 for [Coco]{} captions).

Experiments {#sec:exp}
===========

                    [sen]{}%   [B-1]{}   [B-2]{}   [B-3]{}   [B-4]{}   [Rouge]{}$_L$   [Meteor]{}   [Cide]{}$_r$   [Senti]{}   [Desc]{}        [DescCmp]{}
  -- -------------- ---------- --------- --------- --------- --------- --------------- ------------ -------------- ----------- --------------- -------------
     CNN+RNN        1.0        48.7      28.1      17.0      10.7      36.6            15.3         55.6           –           2.90$\pm$0.90   –
     ANP-Replace    90.3       48.2      27.8      16.4      10.1      36.6            16.5         55.2           84.8%       2.89$\pm$0.92   95.0%
     ANP-Scoring    90.3       48.3      27.9      16.6      10.1      36.5            16.6         55.4           84.8%       2.86$\pm$0.96   95.3%
     RNN-Transfer   86.5       