* Mnih and Hinton (2007b) Andriy Mnih and Geoffrey Hinton. 2007b.  Three new graphical models for statistical language modelling. In _Proceedings of the 24th international conference on Machine learning_. ACM, 641–648.
  * Och (2003) Franz Josef Och. 2003\.  Minimum error rate training in statistical machine translation. In _Proceedings of the 41st Annual Meeting on Association for Computational Linguistics-Volume 1_. Association for Computational Linguistics, 160–167.
  * Ojala et al. (2000) Timo Ojala, Matti Pietikäinen, and Topi Mäenpää. 2000\.  Gray scale and rotation invariant texture classification with local binary patterns. In _European Conference on Computer Vision_. Springer, 404–420.
  * Ordonez et al. (2011) Vicente Ordonez, Girish Kulkarni, and Tamara L Berg. 2011\.  Im2text: Describing images using 1 million captioned photographs. In _Advances in Neural Information Processing Systems_. 1143–1151.
  * Papadopoulos et al. (2014) Dim P Papadopoulos, Alasdair DF Clarke, Frank Keller, and Vittorio Ferrari. 2014.  Training object class detectors from eye tracking data. In _European Conference on Computer Vision_. Springer, 361–376.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002\.  BLEU: a method for automatic evaluation of machine translation. In _Proceedings of the 40th annual meeting on association for computational linguistics_. Association for Computational Linguistics, 311–318.
  * Park et al. (2017) Cesc Chunseong Park, Byeongchang Kim, and Gunhee Kim. 2017\.  Attend to You: Personalized Image Captioning with Context Sequence Memory Networks. In _Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)_. 6432–6440.
  * Pedersoli et al. (2017) Marco Pedersoli, Thomas Lucas, Cordelia Schmid, and Jakob Verbeek. 2017.  Areas of Attention for Image Captioning. In _Proceedings of the IEEE international conference on computer vision_. 1251–1259.
  * Plummer et al. (2015) Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. 2015.  Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In _Proceedings of the IEEE international conference on computer vision_. 2641–2649.
  * Ranzato et al. (2016) Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2016.  Sequence level training with recurrent neural networks. In _International Conference on learning Representations (ICLR)_.
  * Reed et al. (2016) Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. 2016\.  Generative adversarial text to image synthesis. In _Proceedings of Machine Learning Research_ , Vol. 48. 1060–1069.
  * Ren et al. (2015a) Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015a.  Faster R-CNN: Towards real-time object detection with region proposal networks. In _Advances in neural information processing systems_. 91–99.
  * Ren et al. (2015b) Zhou Ren, Hailin Jin, Zhe Lin, Chen Fang, and Alan Yuille. 2015b.  Multi-instance visual-semantic embedding.  _arXiv preprint arXiv:1512.06963_ (2015).
  * Ren et al. (2016) Zhou Ren, Hailin Jin, Zhe Lin, Chen Fang, and Alan Yuille. 2016.  Joint image-text representation by gaussian visual-semantic embedding. In _Proceedings of the 2016 ACM on Multimedia Conference_. ACM, 207–211.
  * Ren et al. (2017) Zhou Ren, Xiaoyu Wang, Ning Zhang, Xutao Lv, and Li-Jia Li. 2017.  Deep Reinforcement Learning-based Image Captioning with Embedding Reward. In _Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)_. 1151–1159.
  * Rennie et al. (2017) Steven J Rennie, Etienne Marcheret, Youssef Mroueh, Jarret Ross, and Vaibhava Goel. 2017.  Self-critical sequence training for image captioning. In _Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)_. 1179–1195.
  * Robertson (2004) Stephen Robertson. 2004\.  Understanding inverse document frequency: on theoretical arguments for IDF.  _Journal of documentation_ 60, 5 (2004), 503–520.
  * Sattar et al. (2015) Hosnieh Sattar, Sabine Muller, Mario Fritz, and Andreas Bulling. 2015.  Prediction of search targets from fixations in open-world settings. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 981–990.
  * Schuster et al. (2015) Sebastian Schuster, Ranjay Krishna, Angel Chang, Li Fei-Fei, and Christopher D Manning. 2015.  Generating semantically precise scene graphs from textual descriptions for improved image retrieval. In _Proceedings of the fourth workshop on vision and language_ , Vol. 2.
  * Shanmuga Vadivel et al. (2015) Karthikeyan Shanmuga Vadivel, Thuyen Ngo, Miguel Eckstein, and BS Manjunath. 2015.  Eye tracking assisted extraction of attentionally important objects from videos. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 3241–3250.
  * Sharif et al. (2018) Naeha Sharif, Lyndon White, Mohammed Bennamoun, and Syed Afaq Ali Shah. 2018.  Learning-based Composite Metrics for Improved Caption Evaluation. In _Proceedings of ACL 2018, Student Research Workshop_. 14–20.
  * Shetty et al. (2017) Rakshith Shetty, Marcus Rohrbach, Lisa Anne Hendricks, Mario Fritz, and Bernt Schiele. 2017\.  Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training. In _IEEE International Conference on Computer Vision (ICCV)_. 4155–4164.
  * Simonyan and Zisserman (2015) Karen Simonyan and Andrew Zisserman. 2015.  Very deep convolutional networks for large-scale image recognition. In _International Conference on Learning Representations (ICLR)_.
  * Socher et al. (2014) Richard Socher, Andrej Karpathy, Quoc V Le, Christopher D Manning, and Andrew Y Ng. 2014\.  Grounded compositional semantics for finding and describing images with sentences.  _Transactions of the Association for Computational Linguistics_ 2 (2014), 207–218.
  * Sugano and Bulling (2016) Yusuke Sugano and Andreas Bulling. 2016.  Seeing with humans: Gaze-assisted neural image captioning.  _arXiv preprint arXiv:1608.05203_ (2016).
  * Sun et al. (2015) Chen Sun, Chuang Gan, and Ram Nevatia. 2015.  Automatic concept discovery from parallel text and visual corpora. In _Proceedings of the IEEE International Conference on Computer Vision_. 2596–2604.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014\.  Sequence to sequence learning with neural networks. In _Advances in neural information processing systems_. 3104–3112.
  * Sutton et al. (2000) Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. 2000.  Policy gradient methods for reinforcement learning with function approximation. In _Advances in neural information processing systems_. 1057–1063.
  * Szegedy et al. (2015) Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015.  Going deeper with convolutions. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 1–9.
  * Tavakoli et al. (2017) Hamed R Tavakoli, Rakshith Shetty, Ali Borji, and Jorma Laaksonen. 2017.  Paying Attention to Descriptions Generated by Image Captioning Models. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 2487–2496.
  * Tran et al. (2016) Kenneth Tran, Xiaodong He, Lei Zhang, Jian Sun, Cornelia Carapcea, Chris Thrasher, Chris Buehler, and Chris Sienkiewicz. 2016\.  Rich image captioning in the wild. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops_. 49–56.
  * van Deemter et al. (2006) Kees van Deemter, Ielka van der Sluis, and Albert Gatt. 2006\.  Building a semantically transparent corpus for the generation of referring expressions. In _Proceedings of the Fourth International Natural Language Generation Conference_. Association for Computational Linguistics, 130–132.
  * van den Oord et al. (2016) Aaron van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. 2016\.  Conditional image generation with pixelcnn decoders. In _Advances in Neural Information Processing Systems_. 4790–4798.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need. In _Advances in Neural Information Processing Systems_. 5998–6008.
  * Vedantam et al. (2015) Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015\.  Cider: Consensus-based image description evaluation. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 4566–4575.
  * Venugopalan et al. (2017) Subhashini Venugopalan, Lisa Anne Hendricks, Marcus Rohrbach, Raymond Mooney, Trevor Darrell, and Kate Saenko. 2017.  Captioning images with diverse objects. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 1170–1178.
  * Viethen and Dale (2008) Jette Viethen and Robert Dale. 2008.  The use of spatial relations in referring expression generation. In _Proceedings of the Fifth International Natural Language Generation Conference_. Association for Computational Linguistics, 59–67.
  * Vinyals et al. (2015) Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015.  Show and tell: A neural image caption generator. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 3156–3164.
  * Vinyals et al. (2017) Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2017.  Show and tell: Lessons learned from the 2015 mscoco image captioning challenge.  _IEEE transactions on pattern analysis and machine intelligence_ 39, 4 (2017), 652–663.
  * Wang et al. (2016b) Cheng Wang, Haojin Yang, Christian Bartz, and Christoph Meinel. 2016b.  Image captioning with deep bidirectional LSTMs. In _Proceedings of the 2016 ACM on Multimedia Conference_. ACM, 988–997.
  * Wang et al. (2018) Heng Wang, Zengchang Qin, and Tao Wan. 2018.  Text Generation Based on Generative Adversarial Nets with Latent Variables. In _Pacific-Asia Conference on Knowledge Discovery and Data Mining_. Springer, 92–103.
  * Wang et al. (2016a) Minsi Wang, Li Song, Xiaokang Yang, and Chuanfei Luo. 2016a.  A parallel-fusion RNN-LSTM architecture for image caption generation. In _Image Processing (ICIP), 2016 IEEE International Conference on_. IEEE, 4448–4452.
  * Wang and Chan (2018) Qingzhong Wang and Antoni B Chan. 2018.  CNN+ CNN: Convolutional Decoders for Image Captioning.  _arXiv preprint arXiv:1805.09019_.
  * Wang et al. (2017) Yufei Wang, Zhe Lin, Xiaohui Shen, Scott Cohen, and Garrison W Cottrell. 2017.  Skeleton Key: Image Captioning by Skeleton-Attribute Decomposition. In _Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)_. 7378–7387.
  * Wu et al. (2015) Qi Wu, Chunhua Shen, Anton van den Hengel, Lingqiao Liu, and Anthony Dick. 2015.  Image captioning with an intermediate attributes layer.  _arXiv preprint arXiv:1506.01144_ (2015).
  * Wu et al. (2018) Qi Wu, Chunhua Shen, Peng Wang, Anthony Dick, and Anton van den Hengel. 2018.  Image captioning and visual question answering based on attributes and external knowledge.  _IEEE transactions on pattern analysis and machine intelligence_ 40, 6, 1367–1381.
  * Wu and Cohen (2016) Zhilin Yang Ye Yuan Yuexin Wu and Ruslan Salakhutdinov William W Cohen. 2016\.  Encode, Review, and Decode: Reviewer Module for Caption Generation. In _30th Conference on Neural Image Processing System(NIPS)_.
  * Xu et al. (2015) Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. 2015\.  Show, attend and tell: Neural image caption generation with visual attention. In _International Conference on Machine Learning_. 2048–2057.
  * Yang et al. (2016) Linjie Yang, Kevin Tang, Jianchao Yang, and Li-Jia Li. 2016\.  Dense Captioning with Joint Inference and Visual Context. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. 1978–1987.
  * Yao et al. (2017a) Ting Yao, Yingwei Pan, Yehao Li, and Tao Mei. 2017a.  Incorporating copying mechanism in image captioning for learning novel objects. In _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, 5263–5271.
  * Yao et al. (2017b) Ting Yao, Yingwei Pan, Yehao Li, Zhaofan Qiu, and Tao Mei. 2017b.  Boosting image captioning with attributes. In _IEEE International Conference on Computer Vision (ICCV)_. 4904–4912.
  * You et al. (2016) Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo. 2016.  Image captioning with semantic attention. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 4651–4659.
  * Yu et al. (2017) Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu Seqgan. 2017\.  Sequence generative adversarial nets with policy gradient.. In _Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence_.
  * Yun et al. (2013) Kiwon Yun, Yifan Peng, Dimitris Samaras, Gregory J Zelinsky, and Tamara L Berg. 2013.  Exploring the role of gaze behavior and object detection in scene understanding.  _Frontiers in psychology_ 4 (2013).
  * Zeiler and Fergus (2014) Matthew D Zeiler and Rob Fergus. 2014.  Visualizing and understanding convolutional networks. In _European conference on computer vision_. Springer, 818–833.
  * Zelinsky (2013) Gregory J Zelinsky. 2013\.  Understanding scene understanding.  _Frontiers in psychology_ 4 (2013).
  * Zhang et al. (2017) Li Zhang, Flood Sung, Feng Liu, Tao Xiang, Shaogang Gong, Yongxin Yang, and Timothy M Hospedales. 2017.  Actor-critic sequence training for image captioning.  _arXiv preprint arXiv:1706.09601_.