We proposed SentiCap, a switching RNN model for generating image captions with sentiments. One novel feature of this model is a specialized word-level supervision scheme to effectively make use of a small amount of training data with sentiments. We also designed a crowd-sourced caption re-writing task to generate sentimental yet descriptive captions. We demonstrate the effectiveness of the proposed model using both automatic and crowd-sourced evaluations, with the SentiCap model able to generate an emotional caption for over 90% of the images, and the vast majority of the generated captions are rated as having the appropriate sentiment by crowd workers. Future work can include unified model for positive and negative sentiment; models for linguistic styles (including sentiments) beyond the word level, and designing generative models for a richer set of emotions such as pride, shame, anger.

[ [**Acknowledgments**]{} NICTA is funded by the Australian Government as represented by the Dept. of Communications and the ARC through the ICT Centre of Excellence program. This work is also supported in part by the Australian Research Council via the Discovery Project program. The Tesla K40 used for this research was donated by the NVIDIA Corporation. ]{}

Ames, M., and Naaman, M. 2007. Why 