* Antunes, Saponaro, Morse, Jamone, Santos-Victor, & Cangelosi Antunes, A., Saponaro, G., Morse, A., Jamone, L., Santos-Victor, J., & Cangelosi, A. (2017).  Learn, plan, remember: A developmental robot architecture for task solving.  In 2017 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), pp. 283–289. IEEE.
  * Astington & Baird Astington, J. W., & Baird, J. A. (2005).  Why language matters for theory of mind.  Oxford University Press.
  * Atzmon, Berant, Kezami, Globerson, & Chechik Atzmon, Y., Berant, J., Kezami, V., Globerson, A., & Chechik, G. (2016).  Learning to generalize to new compositions in image understanding..
  * Austin Austin, J. L. (1975).  How to do things with words.  Oxford university press.
  * Bahdanau, Hill, Leike, Hughes, Hosseini, Kohli, & Grefenstette Bahdanau, D., Hill, F., Leike, J., Hughes, E., Hosseini, A., Kohli, P., & Grefenstette, E. (2018a).  Learning to understand goal specifications by modelling reward..
  * Bahdanau, Murty, Noukhovitch, Nguyen, de Vries, & Courville Bahdanau, D., Murty, S., Noukhovitch, M., Nguyen, T. H., de Vries, H., & Courville, A. (2018b).  Systematic generalization: what is required and can it be learned?..
  * Baltrušaitis, Ahuja, & Morency Baltrušaitis, T., Ahuja, C., & Morency, L.-P. (2018).  Multimodal machine learning: A survey and taxonomy..  Vol. 41, pp. 423–443. IEEE.
  * Barsalou Barsalou, L. W. (1999).  Perceptual symbol systems..  Vol. 22, pp. 577–660. Cambridge University Press.
  * Barsalou Barsalou, L. W. (2008).  Grounded cognition..  Vol. 59, pp. 617–645. Annual Reviews.
  * Barwise & Perry Barwise, J., & Perry, J. (1981).  Situations and attitudes..  Vol. 78, pp. 668–691. JSTOR.
  * Batali Batali, J. (1998).  Computational simulations of the emergence of grammar..  pp. 405–426. Cambridge University Press.
  * Beattie, Leibo, Teplyashin, Ward, Wainwright, Küttler, Lefrancq, Green, Valdés, Sadik, et al. Beattie, C., Leibo, J. Z., Teplyashin, D., Ward, T., Wainwright, M., Küttler, H., Lefrancq, A., Green, S., Valdés, V., Sadik, A., et al. (2016).  Deepmind lab..
  * Becerra-Bonache, Christiansen, & Jiménez-López Becerra-Bonache, L., Christiansen, H., & Jiménez-López, M. D. (2018).  A Gold Standard to Measure Relative Linguistic Complexity with a Grounded Language Learning Model.  In Proceedings of the Workshop on Linguistic Complexity and Natural Language Processing, pp. 1–9.
  * Bender & Koller Bender, E. M., & Koller, A. (2020).  Climbing towards NLU: On meaning, form, and understanding in the age of data.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5185–5198.
  * Bengio, Courville, & Vincent Bengio, Y., Courville, A., & Vincent, P. (2013).  Representation learning: A review and new perspectives..  Vol. 35, pp. 1798–1828. IEEE.
  * Benotti & Blackburn Benotti, L., & Blackburn, P. (2021).  Grounding as a Collaborative Process.  In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 515–531 Online. Association for Computational Linguistics.
  * Bisk, Holtzman, Thomason, Andreas, Bengio, Chai, Lapata, Lazaridou, May, Nisnevich, et al. Bisk, Y., Holtzman, A., Thomason, J., Andreas, J., Bengio, Y., Chai, J., Lapata, M., Lazaridou, A., May, J., Nisnevich, A., et al. (2020a).  Experience Grounds Language.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 8718–8735.
  * Bisk, Holtzman, Thomason, Andreas, Bengio, Chai, Lapata, Lazaridou, May, Nisnevich, et al. Bisk, Y., Holtzman, A., Thomason, J., Andreas, J., Bengio, Y., Chai, J., Lapata, M., Lazaridou, A., May, J., Nisnevich, A., et al. (2020b).  Experience Grounds Language.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 8718–8735.
  * Bisk, Yuret, & Marcu Bisk, Y., Yuret, D., & Marcu, D. (2016).  Natural language communication with robots.  In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 751–761.
  * Bogin, Gupta, Gardner, & Berant Bogin, B., Gupta, S., Gardner, M., & Berant, J. (2021a).  COVR: A Test-Bed for Visually Grounded Compositional Generalization with Real Images.  In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 9824–9846.
  * Bogin, Subramanian, Gardner, & Berant Bogin, B., Subramanian, S., Gardner, M., & Berant, J. (2021b).  Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering..  Vol. 9, pp. 195–210. MIT Press.
  * Bojanowski, Grave, Joulin, & Mikolov Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017).  Enriching Word Vectors with Subword Information..  Vol. 5, pp. 135–146.
  * Bouchacourt & Baroni Bouchacourt, D., & Baroni, M. (2018).  How agents see things: On visual representations in an emergent language game..
  * Bowman, Angeli, Potts, & Manning Bowman, S., Angeli, G., Potts, C., & Manning, C. D. (2015).  A large annotated corpus for learning natural language inference.  In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 632–642.
  * Brennan & Clark Brennan, S. E., & Clark, H. H. (1996).  Conceptual pacts and lexical choice in conversation..  Vol. 22, pp. 482–1493.
  * Brodeur, Perez, Anand, Golemo, Celotti, Strub, Rouat, Larochelle, & Courville Brodeur, S., Perez, E., Anand, A., Golemo, F., Celotti, L., Strub, F., Rouat, J., Larochelle, H., & Courville, A. (2017).  HoME: A household multimodal environment..
  * Brown, Mann, Ryder, Subbiah, Kaplan, Dhariwal, Neelakantan, Shyam, Sastry, Askell, et al. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020).  Language models are few-shot learners..  Vol. 33, pp. 1877–1901.
  * Bugliarello, Cotterell, Okazaki, & Elliott Bugliarello, E., Cotterell, R., Okazaki, N., & Elliott, D. (2021).  Multimodal pretraining unmasked: A meta-analysis and a unified framework of vision-and-language BERTs..  Vol. 9, pp. 978–994. MIT Press.
  * Bugliarello & Elliott Bugliarello, E., & Elliott, D. (2021).  The Role of Syntactic Planning in Compositional Image Captioning.  In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 593–607.
  * Bugliarello, Liu, Pfeiffer, Reddy, Elliott, Ponti, & Vulić Bugliarello, E., Liu, F., Pfeiffer, J., Reddy, S., Elliott, D., Ponti, E. M., & Vulić, I. (2022).  IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages..
  * Chai, Fang, Liu, & She Chai, J. Y., Fang, R., Liu, C., & She, L. (2016).  Collaborative language grounding toward situated human-robot dialogue..  Vol. 37, pp. 32–45.
  * Chandu, Bisk, & Black Chandu, K. R., Bisk, Y., & Black, A. W. (2021).  Grounding ‘Grounding’ in NLP.  In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 4283–4305 Online. Association for Computational Linguistics.
  * Chang, Dai, Funkhouser, Halber, Niebner, Savva, Song, Zeng, & Zhang Chang, A., Dai, A., Funkhouser, T., Halber, M., Niebner, M., Savva, M., Song, S., Zeng, A., & Zhang, Y. (2017).  Matterport3D: Learning from RGB-D Data in Indoor Environments.  In 2017 International Conference on 3D Vision (3DV), pp. 667–676. IEEE.
  * Chen, Suhr, Misra, Snavely, & Artzi Chen, H., Suhr, A., Misra, D., Snavely, N., & Artzi, Y. (2019).  Touchdown: Natural language navigation and spatial reasoning in visual street environments.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12538–12547.
  * Chen, Li, Yu, Kholy, Ahmed, Gan, Cheng, & Liu Chen, Y.-C., Li, L., Yu, L., Kholy, A. E., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2020).  Uniter: Universal image-text representation learning.  In European Conference on Computer Vision (ECCV).
  * Chevalier-Boisvert, Bahdanau, Lahlou, Willems, Saharia, Nguyen, & Bengio Chevalier-Boisvert, M., Bahdanau, D., Lahlou, S., Willems, L., Saharia, C., Nguyen, T. H., & Bengio, Y. (2018).  BabyAI: First Steps Towards Grounded Language Learning With a Human In the Loop..
  * Choi, Lazaridou, & de Freitas Choi, E., Lazaridou, A., & de Freitas, N. (2018).  Compositional obverter communication learning from raw visual input..
  * Chou, Chen, Zeng, Hu, Fu, & Sun Chou, S.-H., Chen, Y.-C., Zeng, K.-H., Hu, H.-N., Fu, J., & Sun, M. (2018).  Self-view grounding given a narrated 360 video.  In Thirty-Second AAAI Conference on Artificial Intelligence.
  * Chrupała Chrupała, G. (2019).  Symbolic Inductive Bias for Visually Grounded Learning of Spoken Language.  In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 6452–6462.
  * Clark, Salvador, Schwenk, Bonafilia, Yatskar, Kolve, Herrasti, Choi, Mehta, Skjonsberg, et al. Clark, C., Salvador, J., Schwenk, D., Bonafilia, D., Yatskar, M., Kolve, E., Herrasti, A., Choi, J., Mehta, S., Skjonsberg, S., et al. (2021).  Iconary: A Pictionary-Based Game for Testing Multimodal Communication with Drawings and Text.  In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 1864–1886.
  * Clark Clark, H. H. (1996).  Using language.  Cambridge university press.
  * Clark & Brennan Clark, H. H., & Brennan, S. E. (1991).  Grounding in communication..  American Psychological Association.
  * Co-Reyes, Gupta, Sanjeev, Altieri, DeNero, Abbeel, & Levine Co-Reyes, J. D., Gupta, A., Sanjeev, S., Altieri, N., DeNero, J., Abbeel, P., & Levine, S. (2018).  Guiding Policies with Language via Meta-Learning..
  * Csurka, Dance, Fan, Willamowski, & Bray Csurka, G., Dance, C., Fan, L., Willamowski, J., & Bray, C. (2004).  Visual categorization with bags of keypoints.  In Workshop on statistical learning in computer vision, ECCV, Vol. 1, pp. 1–2. Prague.
  * Da, Forbes, Zellers, Zheng, Hwang, Bosselut, & Choi Da, J., Forbes, M., Zellers, R., Zheng, A., Hwang, J. D., Bosselut, A., & Choi, Y. (2021).  Edited Media Understanding Frames: Reasoning About the Intent and Implications of Visual Misinformation.  In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 2026–2039.
  * Das, Datta, Gkioxari, Lee, Parikh, & Batra Das, A., Datta, S., Gkioxari, G., Lee, S., Parikh, D., & Batra, D. (2018).  Embodied question answering.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2054–2063.
  * Das, Kottur, Gupta, Singh, Yadav, Moura, Parikh, & Batra Das, A., Kottur, S., Gupta, K., Singh, A., Yadav, D., Moura, J. M., Parikh, D., & Batra, D. (2017a).  Visual dialog.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 326–335.
  * Das, Kottur, Gupta, Singh, Yadav, Moura, Parikh, & Batra Das, A., Kottur, S., Gupta, K., Singh, A., Yadav, D., Moura, J. M., Parikh, D., & Batra, D. (2017b).  Visual Dialog.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
  * De Vries, Bahdanau, & Manning De Vries, H., Bahdanau, D., & Manning, C. (2020).  Towards ecologically valid research on language user interfaces..
  * De Vries, Shuster, Batra, Parikh, Weston, & Kiela De Vries, H., Shuster, K., Batra, D., Parikh, D., Weston, J., & Kiela, D. (2018).  Talk the walk: Navigating new york city through grounded dialogue..
  * De Vries, Strub, Chandar, Pietquin, Larochelle, & Courville De Vries, H., Strub, F., Chandar, S., Pietquin, O., Larochelle, H., & Courville, A. (2017).  Guesswhat?! visual object discovery through multi-modal dialogue.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5503–5512.
  * Deitke, Han, Herrasti, Kembhavi, Kolve, Mottaghi, Salvador, Schwenk, VanderBilt, Wallingford, et al. Deitke, M., Han, W., Herrasti, A., Kembhavi, A., Kolve, E., Mottaghi, R., Salvador, J., Schwenk, D., VanderBilt, E., Wallingford, M., et al. (2020).  Robothor: An open simulation-to-real embodied ai platform.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3164–3174.
  * Deng, Wu, Wu, Hu, Lyu, & Tan Deng, C., Wu, Q., Wu, Q., Hu, F., Lyu, F., & Tan, M. (2018).  Visual grounding via accumulated attention.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7746–7755.
  * Deng, Dong, Socher, Li, Li, & Fei-Fei Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009).  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255. Ieee.
  * Devlin, Chang, Lee, & Toutanova Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018).  Bert: Pre-training of deep bidirectional transformers for language understanding..
  * Devlin, Chang, Lee, & Toutanova Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019a).  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.  In North American Chapter of the Association for Computational Linguistics (NAACL).
  * Devlin, Chang, Lee, & Toutanova Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019b).  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186.
  * Dobnik, Kelleher, & Howes Dobnik, S., Kelleher, J. D., & Howes, C. (2020).  Local alignment of frame of reference assignment in English and Swedish dialogue.  In German Conference on Spatial Cognition, pp. 251–267. Springer.
  * Dosovitskiy, Beyer, Kolesnikov, Weissenborn, Zhai, Unterthiner, Dehghani, Minderer, Heigold, Gelly, et al. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020).  An image is worth 16x16 words: Transformers for image recognition at scale..
  * Elliott, Frank, Sima’an, & Specia Elliott, D., Frank, S., Sima’an, K., & Specia, L. (2016).  Multi30K: Multilingual English-German Image Descriptions.  In Proceedings of the 5th Workshop on Vision and Language, pp. 70–74.
  * Elman Elman, J. L. (1990).  Finding structure in time..  Vol. 14, pp. 179–211. Wiley Online Library.
  * Ferraro, Mostafazadeh, Huang, Vanderwende, Devlin, Galley, & Mitchell Ferraro, F., Mostafazadeh, N., Huang, T.-H., Vanderwende, L., Devlin, J., Galley, M., & Mitchell, M. (2015).  A Survey of Current Datasets for Vision and Language Research.  In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 207–213.
  * Fodor Fodor, J. A. (1983).  The modularity of mind.  MIT press.
  * Fodor & Lepore Fodor, J. A., & Lepore, E. (2002).  The compositionality papers.  Oxford University Press.
  * Fodor & Pylyshyn Fodor, J. A., & Pylyshyn, Z. W. (1988).  Connectionism and cognitive architecture: A critical analysis..  Vol. 28, pp. 3–71. Elsevier.
  * Frege Frege, G. (1892).  Über sinn und bedeutung..  Vol. 100, pp. 25–50.
  * Gao, Thattai, Gao, Shakiah, Pansare, Sharma, Sukhatme, Shi, Yang, Zheng, et al. Gao, Q., Thattai, G., Gao, X., Shakiah, S., Pansare, S., Sharma, V., Sukhatme, G., Shi, H., Yang, B., Zheng, D., et al. (2023).  Alexa arena: A user-centric interactive platform for embodied ai..
  * Gao, Gao, Gong, Lin, Thattai, & Sukhatme Gao, X., Gao, Q., Gong, R., Lin, K., Thattai, G., & Sukhatme, G. S. (2022).  Dialfred: Dialogue-enabled agents for embodied instruction following..  Vol. 7, pp. 10049–10056. IEEE.
  * Gardenfors Gardenfors, P. (2014).  The geometry of meaning: Semantics based on conceptual spaces.  MIT press.
  * Gibson Gibson, J. J. (1977).  The theory of affordances.  Hilldale, USA, 1(2), 67–82.
  * Glenberg & Kaschak Glenberg, A. M., & Kaschak, M. P. (2002).  Grounding language in action..  Vol. 9, pp. 558–565. Springer.
  * Gluck, Laird, & Lupp Gluck, K. A., Laird, J., & Lupp, J. (2018).  Interactive Task Learning: Humans, Robots, and Agents Acquiring New Tasks Through Natural Interactions.  MIT Press.
  * Gordon, Kembhavi, Rastegari, Redmon, Fox, & Farhadi Gordon, D., Kembhavi, A., Rastegari, M., Redmon, J., Fox, D., & Farhadi, A. (2018).  Iqa: Visual question answering in interactive environments.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4089–4098.
  * Goyal, Khot, Summers-Stay, Batra, & Parikh Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., & Parikh, D. (2017a).  Making the v in vqa matter: Elevating the role of image understanding in visual question answering.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6904–6913.
  * Goyal, Khot, Summers-Stay, Batra, & Parikh Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., & Parikh, D. (2017b).  Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6904–6913.
  * Grauman & Darrell Grauman, K., & Darrell, T. (2005).  The pyramid match kernel: Discriminative classification with sets of image features.  In Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1, Vol. 2, pp. 1458–1465. IEEE.
  * Graves, Wayne, & Danihelka Graves, A., Wayne, G., & Danihelka, I. (2014).  Neural turing machines..
  * Grubinger, Clough, Müller, & Deselaers Grubinger, M., Clough, P., Müller, H., & Deselaers, T. (2006).  The iapr tc-12 benchmark: A new evaluation resource for visual information systems.  In International workshop ontoImage, Vol. 2.
  * Gu, Stefani, Wu, Thomason, & Wang Gu, J., Stefani, E., Wu, Q., Thomason, J., & Wang, X. (2022).  Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions.  In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 7606–7623.
  * Gurari, Li, Stangl, Guo, Lin, Grauman, Luo, & Bigham Gurari, D., Li, Q., Stangl, A. J., Guo, A., Lin, C., Grauman, K., Luo, J., & Bigham, J. P. (2018).  Vizwiz grand challenge: Answering visual questions from blind people.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3608–3617.
  * Haber, Baumgärtner, Takmaz, Gelderloos, Bruni, & Fernández Haber, J., Baumgärtner, T., Takmaz, E., Gelderloos, L., Bruni, E., & Fernández, R. (2019).  The PhotoBook Dataset: Building Common Ground through Visually-Grounded Dialogue.  In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1895–1910.
  * Harnad Harnad, S. (1990).  The symbol grounding problem..  Vol. 42, pp. 335–346. Elsevier.
  * Harwath & Glass Harwath, D., & Glass, J. (2017).  Learning Word-Like Units from Joint Audio-Visual Analysis.  In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 506–517.
  * Havrylov & Titov Havrylov, S., & Titov, I. (2017).  Emergence of language with multi-agent games: Learning to communicate with sequences of symbols.  In Advances in neural information processing systems, pp. 2149–2159.
  * He, Zhang, Ren, & Sun He, K., Zhang, X., Ren, S., & Sun, J. (2015).  Deep residual learning for image recognition. eprint..
  * Hermann, Hill, Green, Wang, Faulkner, Soyer, Szepesvari, Czarnecki, Jaderberg, Teplyashin, et al. Hermann, K. M., Hill, F., Green, S., Wang, F., Faulkner, R., Soyer, H., Szepesvari, D., Czarnecki, W. M., Jaderberg, M., Teplyashin, D., et al. (2017).  Grounded language learning in a simulated 3d world..
  * Hong, Wu, Qi, Rodriguez-Opazo, & Gould Hong, Y., Wu, Q., Qi, Y., Rodriguez-Opazo, C., & Gould, S. (2020).  A Recurrent Vision-and-Language BERT for Navigation..
  * Huang, Ferraro, Mostafazadeh, Misra, Agrawal, Devlin, Girshick, He, Kohli, Batra, et al. Huang, T.-H., Ferraro, F., Mostafazadeh, N., Misra, I., Agrawal, A., Devlin, J., Girshick, R., He, X., Kohli, P., Batra, D., et al. (2016).  Visual storytelling.  In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1233–1239.
  * Huang, Xia, Xiao, Chan, Liang, Florence, Zeng, Tompson, Mordatch, Chebotar, et al. Huang, W., Xia, F., Xiao, T., Chan, H., Liang, J., Florence, P., Zeng, A., Tompson, J., Mordatch, I., Chebotar, Y., et al. (2022).  Inner monologue: Embodied reasoning through planning with language models..
  * Hudson & Manning Hudson, D. A., & Manning, C. D. (2018).  Compositional Attention Networks for Machine Reasoning.  In International Conference on Learning Representations.
  * Hudson & Manning Hudson, D. A., & Manning, C. D. (2019).  Gqa: A new dataset for real-world visual reasoning and compositional question answering.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6700–6709.
  * Hupkes, Dankers, Mul, & Bruni Hupkes, D., Dankers, V., Mul, M., & Bruni, E. (2020).  Compositionality decomposed: how do neural networks generalise?..  Vol. 67, pp. 757–795.
  * Ilinykh, Zarrieß, & Schlangen Ilinykh, N., Zarrieß, S., & Schlangen, D. (2019a).  Meetup! a corpus of joint activity dialogues in a visual environment..
  * Ilinykh, Zarrieß, & Schlangen Ilinykh, N., Zarrieß, S., & Schlangen, D. (2019b).  Tell me more: A dataset of visual scene description sequences.  In Proceedings of the 12th international conference on natural language generation, pp. 152–157.
  * Jain, Weihs, Kolve, Rastegari, Lazebnik, Farhadi, Schwing, & Kembhavi Jain, U., Weihs, L., Kolve, E., Rastegari, M., Lazebnik, S., Farhadi, A., Schwing, A. G., & Kembhavi, A. (2019).  Two body problem: Collaborative visual task completion.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6689–6699.
  * Jiang, Misra, Rohrbach, Learned-Miller, & Chen Jiang, H., Misra, I., Rohrbach, M., Learned-Miller, E., & Chen, X. (2020).  In defense of grid features for visual question answering.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10267–10276.
  * Jiang, Gu, Murphy, & Finn Jiang, Y., Gu, S. S., Murphy, K. P., & Finn, C. (2019).  Language as an Abstraction for Hierarchical Deep Reinforcement Learning..  Vol. 32, pp. 9419–9431.
  * Johnson, Hariharan, van der Maaten, Fei-Fei, Lawrence Zitnick, & Girshick Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Lawrence Zitnick, C., & Girshick, R. (2017).  Clevr: A diagnostic dataset for compositional language and elementary visual reasoning.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2901–2910.
  * Johnson, Karpathy, & Fei-Fei Johnson, J., Karpathy, A., & Fei-Fei, L. (2016).  Densecap: Fully convolutional localization networks for dense captioning.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4565–4574.
  * Kafle & Kanan Kafle, K., & Kanan, C. (2017).  An analysis of visual question answering algorithms.  In Proceedings of the IEEE International Conference on Computer Vision, pp. 1965–1973.
  * Kalyan, Lee, Kannan, & Batra Kalyan, A., Lee, S., Kannan, A., & Batra, D. (2018).  Learn from Your Neighbor: Learning Multi-modal Mappings from Sparse Annotations.  In International Conference on Machine Learning, pp. 2454–2463.
  * Kasami Kasami, T. (1966).  An efficient recognition and syntax-analysis algorithm for context-free languages..  Coordinated Science Laboratory, University of Illinois at Urbana-Champaign.
  * Kazemzadeh, Ordonez, Matten, & Berg Kazemzadeh, S., Ordonez, V., Matten, M., & Berg, T. (2014).  Referitgame: Referring to objects in photographs of natural scenes.  In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp. 787–798.
  * Kharitonov, Copet, Lakhotia, Nguyen, Tomasello, Lee, Elkahky, Hsu, Mohamed, Dupoux, et al. Kharitonov, E., Copet, J., Lakhotia, K., Nguyen, T. A., Tomasello, P., Lee, A., Elkahky, A., Hsu, W.-N., Mohamed, A., Dupoux, E., et al. (2022).  textless-lib: A library for textless spoken language processing..
  * Kiela Kiela, D. (2017).  Deep embodiment: grounding semantics in perceptual modalities.  Tech. rep. UCAM-CL-TR-899, University of Cambridge, Computer Laboratory.
  * Kiela, Conneau, Jabri, & Nickel Kiela, D., Conneau, A., Jabri, A., & Nickel, M. (2018).  Learning Visually Grounded Sentence Representations.  In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 408–418.
  * Kiela, Firooz, Mohan, Goswami, Singh, Ringshia, & Testuggine Kiela, D., Firooz, H., Mohan, A., Goswami, V., Singh, A., Ringshia, P., & Testuggine, D. (2020).  The hateful memes challenge: Detecting hate speech in multimodal memes..
  * Kim, Kitaev, Chen, Rohrbach, Zhang, Tian, Batra, & Parikh Kim, J.-H., Kitaev, N., Chen, X., Rohrbach, M., Zhang, B.-T., Tian, Y., Batra, D., & Parikh, D. (2019).  CoDraw: Collaborative Drawing as a Testbed for Grounded Goal-driven Communication.  In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 6495–6513.
  * Kim, Son, & Kim Kim, W., Son, B., & Kim, I. (2021).  Vilt: Vision-and-language transformer without convolution or region supervision..
  * Kiros, Chan, & Hinton Kiros, J., Chan, W., & Hinton, G. (2018).  Illustrative Language Understanding: Large-Scale Visual Grounding with Image Search.  In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 922–933.
  * Kiseleva, Li, Aliannejadi, Mohanty, ter Hoeve, Burtsev, Skrynnik, Zholus, Panov, Srinet, et al. Kiseleva, J., Li, Z., Aliannejadi, M., Mohanty, S., ter Hoeve, M., Burtsev, M., Skrynnik, A., Zholus, A., Panov, A., Srinet, K., et al. (2022).  Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021..
  * Kolve, Mottaghi, Han, VanderBilt, Weihs, Herrasti, Gordon, Zhu, Gupta, & Farhadi Kolve, E., Mottaghi, R., Han, W., VanderBilt, E., Weihs, L., Herrasti, A., Gordon, D., Zhu, Y., Gupta, A., & Farhadi, A. (2017).  AI2-THOR: An Interactive 3D Environment for Visual AI..
  * Kottur, Moon, Geramifard, & Damavandi Kottur, S., Moon, S., Geramifard, A., & Damavandi, B. (2021).  SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations..
  * Kottur, Moura, Lee, & Batra Kottur, S., Moura, J., Lee, S., & Batra, D. (2017).  Natural Language Does Not Emerge ‘Naturally’in Multi-Agent Dialog.  In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2962–2967.
  * Kottur, Moura, Parikh, Batra, & Rohrbach Kottur, S., Moura, J. M., Parikh, D., Batra, D., & Rohrbach, M. (2018).  Visual coreference resolution in visual dialog using neural module networks.  In Proceedings of the European Conference on Computer Vision (ECCV), pp. 153–169.
  * Kottur, Moura, Parikh, Batra, & Rohrbach Kottur, S., Moura, J. M., Parikh, D., Batra, D., & Rohrbach, M. (2019).  CLEVR-Dialog: A Diagnostic Dataset for Multi-Round Reasoning in Visual Dialog..
  * Krishna, Hata, Ren, Fei-Fei, & Carlos Niebles Krishna, R., Hata, K., Ren, F., Fei-Fei, L., & Carlos Niebles, J. (2017a).  Dense-captioning events in videos.  In Proceedings of the IEEE international conference on computer vision, pp. 706–715.
  * Krishna, Zhu, Groth, Johnson, Hata, Kravitz, Chen, Kalantidis, Li, Shamma, et al. Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017b).  Visual genome: Connecting language and vision using crowdsourced dense image annotations..  Vol. 123, pp. 32–73. Springer.
  * Krojer, Adlakha, Vineet, Goyal, Ponti, & Reddy Krojer, B., Adlakha, V., Vineet, V., Goyal, Y., Ponti, E., & Reddy, S. (2022).  Image Retrieval from Contextual Descriptions.  In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3426–3440.
  * Ku, Anderson, Patel, Ie, & Baldridge Ku, A., Anderson, P., Patel, R., Ie, E., & Baldridge, J. (2020).  Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4392–4412.
  * Kuo, Katz, & Barbu Kuo, Y.-L., Katz, B., & Barbu, A. (2020).  Compositional Networks Enable Systematic Generalization for Grounded Language Understanding..
  * Küttler, Nardelli, Miller, Raileanu, Selvatici, Grefenstette, & Rocktäschel Küttler, H., Nardelli, N., Miller, A. H., Raileanu, R., Selvatici, M., Grefenstette, E., & Rocktäschel, T. (2020).  The nethack learning environment..
  * Lake Lake, B. M. (2019).  Compositional generalization through meta sequence-to-sequence learning..
  * Lake & Murphy Lake, B. M., & Murphy, G. L. (2021).  Word meaning in minds and machines…  American Psychological Association.
  * Lake, Ullman, Tenenbaum, & Gershman Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017).  Building machines that learn and think like people..  Vol. 40. Cambridge University Press.
  * Landauer & Dumais Landauer, T. K., & Dumais, S. T. (1997).  A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge…  Vol. 104, p. 211. American Psychological Association.
  * Lazaridou, Chrupała, Fernández, & Baroni Lazaridou, A., Chrupała, G., Fernández, R., & Baroni, M. (2016).  Multimodal Semantic Learning from Child-Directed Input.  In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 387–392.
  * Lazaridou, Hermann, Tuyls, & Clark Lazaridou, A., Hermann, K. M., Tuyls, K., & Clark, S. (2018).  Emergence of linguistic communication from referential games with symbolic and pixel input..
  * Lazaridou, Peysakhovich, & Baroni Lazaridou, A., Peysakhovich, A., & Baroni, M. (2016).  Multi-agent cooperation and the emergence of (natural) language..
  * Lazaridou, Potapenko, & Tieleman Lazaridou, A., Potapenko, A., & Tieleman, O. (2020).  Multi-agent communication meets natural language: Synergies between functional and structural language learning..
  * LeCun, Bengio, & Hinton LeCun, Y., Bengio, Y., & Hinton, G. (2015).  Deep learning..  Vol. 521, pp. 436–444. Nature Publishing Group.
  * Lei, Yu, Berg, & Bansal Lei, J., Yu, L., Berg, T. L., & Bansal, M. (2020).  Tvr: A large-scale dataset for video-subtitle moment retrieval.  In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXI 16, pp. 447–463. Springer.
  * Lemon Lemon, O. (2022).  Conversational grounding in emergent communication – data and divergence.  In Emergent Communication (EmeComm) Workshop at ICLR.
  * Levesque, Davis, & Morgenstern Levesque, H., Davis, E., & Morgenstern, L. (2012).  The winograd schema challenge.  In Thirteenth international conference on the principles of knowledge representation and reasoning.
  * Lewis Lewis, D. K. (1969).  Convention: A Philosophical Study.  Wiley-Blackwell, Cambridge, MA, USA.
  * Li, Zhang, Zhang, Liu, Guo, Ni, Zhang, & Zhang Li, F., Zhang, H., Zhang, Y.-F., Liu, S., Guo, J., Ni, L. M., Zhang, P., & Zhang, L. (2022).  Vision-Language Intelligence: Tasks, Representation Learning, and Large Models..
  * Li, Ataman, & Sennrich Li, J., Ataman, D., & Sennrich, R. (2021).  Vision Matters When It Should: Sanity Checking Multimodal Machine Translation Models.  In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 8556–8562.
  * Li, Yatskar, Yin, Hsieh, & Chang Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., & Chang, K.-W. (2019).  VisualBERT: A simple and performant baseline for vision and language..
  * Li, Yin, Li, Hu, Zhang, Zhang, Wang, Hu, Dong, Wei, Choi, & Gao Li, X., Yin, X., Li, C., Hu, X., Zhang, P., Zhang, L., Wang, L., Hu, H., Dong, L., Wei, F., Choi, Y., & Gao, J. (2020).  Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks.  In European Conference on Computer Vision (ECCV).
  * Lin, Maire, Belongie, Hays, Perona, Ramanan, Dollár, & Zitnick Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014).  Microsoft coco: Common objects in context.  In European conference on computer vision, pp. 740–755. Springer.
  * Linzen Linzen, T. (2020).  How Can We Accelerate Progress Towards Human-like Linguistic Generalization?.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5210–5217.
  * Liu, Fang, She, & Chai Liu, C., Fang, R., She, L., & Chai, J. (2013).  Modeling collaborative referring for situated referential grounding.  In Proceedings of the SIGDIAL 2013 Conference, pp. 78–86.
  * Liu, Bugliarello, Ponti, Reddy, Collier, & Elliott Liu, F., Bugliarello, E., Ponti, E. M., Reddy, S., Collier, N., & Elliott, D. (2021).  Visually Grounded Reasoning across Languages and Cultures.  In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 10467–10485.
  * Lu, Batra, Parikh, & Lee Lu, J., Batra, D., Parikh, D., & Lee, S. (2019).  ViLBERT: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks.  In Neural Information Processing Systems (NeurIPS).
  * Lu, Yang, Batra, & Parikh Lu, J., Yang, J., Batra, D., & Parikh, D. (2018).  Neural baby talk.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7219–7228.
  * Ma, Ren, Zhao, Testuggine, & Peng Ma, M., Ren, J., Zhao, L., Testuggine, D., & Peng, X. (2022).  Are Multimodal Transformers Robust to Missing Modality?.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18177–18186.
  * Magassouba, Sugiura, & Kawai Magassouba, A., Sugiura, K., & Kawai, H. (2021).  CrossMap Transformer: A Crossmodal Masked Path Transformer Using Double Back-Translation for Vision-and-Language Navigation..
  * Majumdar, Shrivastava, Lee, Anderson, Parikh, & Batra Majumdar, A., Shrivastava, A., Lee, S., Anderson, P., Parikh, D., & Batra, D. (2020a).  Improving vision-and-language navigation with image-text pairs from the web.  In European Conference on Computer Vision, pp. 259–274. Springer.
  * Majumdar, Shrivastava, Lee, Anderson, Parikh, & Batra Majumdar, A., Shrivastava, A., Lee, S., Anderson, P., Parikh, D., & Batra, D. (2020b).  Improving Vision-and-Language Navigation with Image-Text Pairs from the Web.  In European Conference on Computer Vision (ECCV).
  * Marcus Marcus, G. (2018).  Deep learning: A critical appraisal..
  * Massiceti, Dokania, Siddharth, & Torr Massiceti, D., Dokania, P. K., Siddharth, N., & Torr, P. H. (2018).  Visual Dialogue without Vision or Dialogue..
  * McClelland, Hill, Rudolph, Baldridge, & Schütze McClelland, J. L., Hill, F., Rudolph, M., Baldridge, J., & Schütze, H. (2020).  Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models..  Vol. 117, pp. 25966–25974. National Acad Sciences.
  * Mikolov, Yih, & Zweig Mikolov, T., Yih, W.-t., & Zweig, G. (2013).  Linguistic regularities in continuous space word representations.  In Proceedings of the 2013 conference of the north american chapter of the association for computational linguistics: Human language technologies, pp. 746–751.
  * Miller Miller, G. A. (1995).  WordNet: a lexical database for English..  Vol. 38, pp. 39–41. ACM New York, NY, USA.
  * Min, Chaplot, Ravikumar, Bisk, & Salakhutdinov Min, S. Y., Chaplot, D. S., Ravikumar, P., Bisk, Y., & Salakhutdinov, R. (2021).  FILM: Following Instructions in Language with Modular Methods..
  * Mirowski, Banki-Horvath, Anderson, Teplyashin, Hermann, Malinowski, Grimes, Simonyan, Kavukcuoglu, Zisserman, et al. Mirowski, P., Banki-Horvath, A., Anderson, K., Teplyashin, D., Hermann, K. M., Malinowski, M., Grimes, M. K., Simonyan, K., Kavukcuoglu, K., Zisserman, A., et al. (2019).  The StreetLearn Environment and Dataset..
  * Misra, Bennett, Blukis, Niklasson, Shatkhin, & Artzi Misra, D., Bennett, A., Blukis, V., Niklasson, E., Shatkhin, M., & Artzi, Y. (2018a).  Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction.  In Empirical Methods in Natural Language Processing (EMNLP).
  * Misra, Girshick, Fergus, Hebert, Gupta, & van der Maaten Misra, I., Girshick, R., Fergus, R., Hebert, M., Gupta, A., & van der Maaten, L. (2018b).  Learning by asking questions.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11–20.
  * Mogadala, Kalimuthu, & Klakow Mogadala, A., Kalimuthu, M., & Klakow, D. (2021).  Trends in integration of vision and language research: A survey of tasks, datasets, and methods..  Vol. 71, pp. 1183–1317.
  * Montague Montague, R. (1970).  Universal grammar..  Vol. 36, pp. 373–398. Wiley Online Library.
  * Mordatch & Abbeel Mordatch, I., & Abbeel, P. (2018).  Emergence of grounded compositional language in multi-agent populations.  In Thirty-Second AAAI Conference on Artificial Intelligence.
  * Narayan-Chen, Jayannavar, & Hockenmaier Narayan-Chen, A., Jayannavar, P., & Hockenmaier, J. (2019).  Collaborative dialogue in Minecraft.  In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5405–5415.
  * Navigli & Ponzetto Navigli, R., & Ponzetto, S. P. (2010).  BabelNet: Building a very large multilingual semantic network.  In Proceedings of the 48th annual meeting of the association for computational linguistics, pp. 216–225.
  * Nenkova & McKeown Nenkova, A., & McKeown, K. (2011).  Automatic summarization.  Now Publishers Inc.
  * Newell, Simon, et al. Newell, A., Simon, H. A., et al. (1972).  Human problem solving, Vol. 104.  Prentice-hall Englewood Cliffs, NJ.
  * Nikolaus, Abdou, Lamm, Aralikatte, & Elliott Nikolaus, M., Abdou, M., Lamm, M., Aralikatte, R., & Elliott, D. (2019).  Compositional Generalization in Image Captioning.  In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pp. 87–98.
  * Ogden & Richards Ogden, C. K., & Richards, I. A. (1923).  The Meaning of Meaning: A Study of the Influence of Language upon Thought and of the Science of Symbolism, Vol. 29.  K. Paul, Trench, Trubner & Company, Limited.
  * OpenAI OpenAI (2022).  Introducing ChatGPT..
  * Padmakumar, Thomason, Shrivastava, Lange, Narayan-Chen, Gella, Piramuthu, Tur, & Hakkani-Tur Padmakumar, A., Thomason, J., Shrivastava, A., Lange, P., Narayan-Chen, A., Gella, S., Piramuthu, R., Tur, G., & Hakkani-Tur, D. (2021).  TEACh: Task-driven Embodied Agents that Chat..
  * Pantazopoulos, Suglia, & Eshghi Pantazopoulos, G., Suglia, A., & Eshghi, A. (2022).  Combine to Describe: Evaluating Compositional Generalization in Image Captioning.  In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pp. 115–131.
  * Papalampidi, Keller, Frermann, & Lapata Papalampidi, P., Keller, F., Frermann, L., & Lapata, M. (2020).  Screenplay Summarization Using Latent Narrative Structure.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1920–1933.
  * Park, Bhagavatula, Mottaghi, Farhadi, & Choi Park, J. S., Bhagavatula, C., Mottaghi, R., Farhadi, A., & Choi, Y. (2020).  Visualcomet: Reasoning about the dynamic context of a still image.  In European Conference on Computer Vision, pp. 508–524. Springer.
  * Pashevich, Schmid, & Sun Pashevich, A., Schmid, C., & Sun, C. (2021).  Episodic transformer for vision-and-language navigation.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15942–15952.
  * Patki & Howard Patki, S., & Howard, T. (2018).  Language-Guided Adaptive Perception for Efficient Grounded Communication with Robotic Manipulators in Cluttered Environments.  In Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue, pp. 151–160.
  * Peirce Peirce, C. S. (1902).  Logic as semiotic: The theory of signs..  p. 100.
  * Pennington, Socher, & Manning Pennington, J., Socher, R., & Manning, C. (2014).  Glove: Global vectors for word representation.  In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp. 1532–1543.
  * Perez, Strub, De Vries, Dumoulin, & Courville Perez, E., Strub, F., De Vries, H., Dumoulin, V., & Courville, A. (2018).  Film: Visual reasoning with a general conditioning layer.  In Thirty-Second AAAI Conference on Artificial Intelligence.
  * Pezzelle & Fernández Pezzelle, S., & Fernández, R. (2019).  Is the red square big? malevic: Modeling adjectives leveraging visual contexts..
  * Pezzelle, Greco, Gandolfi, Gualdoni, & Bernardi Pezzelle, S., Greco, C., Gandolfi, G., Gualdoni, E., & Bernardi, R. (2020).  Be different to be better! A benchmark to leverage the complementarity of language and vision.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pp. 2751–2767.
  * Pillai & Matuszek Pillai, N., & Matuszek, C. (2018).  Unsupervised selection of negative examples for grounded language learning.  In Thirty-Second AAAI Conference on Artificial Intelligence.
  * Puig, Shu, Li, Wang, Liao, Tenenbaum, Fidler, & Torralba Puig, X., Shu, T., Li, S., Wang, Z., Liao, Y.-H., Tenenbaum, J. B., Fidler, S., & Torralba, A. (2020).  Watch-and-help: A challenge for social perception and human-ai collaboration..
  * Pylyshyn Pylyshyn, Z. W. (1986).  Computation and cognition: Toward a foundation for cognitive science. The MIT Press.
  * Radford, Kim, Hallacy, Ramesh, Goh, Agarwal, Sastry, Askell, Mishkin, Clark, et al. Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021).  Learning transferable visual models from natural language supervision..
  * Radford, Wu, Amodei, Amodei, Clark, Brundage, & Sutskever Radford, A., Wu, J., Amodei, D., Amodei, D., Clark, J., Brundage, M., & Sutskever, I. (2019).  Better language models and their implications..  Vol. 1, p. 2.
  * Ramakrishnan, Gokaslan, Wijmans, Maksymets, Clegg, Turner, Undersander, Galuba, Westbury, Chang, et al. Ramakrishnan, S. K., Gokaslan, A., Wijmans, E., Maksymets, O., Clegg, A., Turner, J., Undersander, E., Galuba, W., Westbury, A., Chang, A. X., et al. (2021).  Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI..
  * Ramesh, Pavlov, Goh, Gray, Voss, Radford, Chen, & Sutskever Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021).  Zero-shot text-to-image generation..
  * Reiter & Dale Reiter, E., & Dale, R. (2000).  Building Natural Language Generation Systems.  Studies in Natural Language Processing. Cambridge University Press.
  * Rosenthal, Biswas, & Veloso Rosenthal, S., Biswas, J., & Veloso, M. M. (2010).  An effective personal mobile robot agent through symbiotic human-robot interaction..  In AAMAS, Vol. 10, pp. 915–922.
  * Ruis, Andreas, Baroni, Bouchacourt, & Lake Ruis, L., Andreas, J., Baroni, M., Bouchacourt, D., & Lake, B. M. (2020).  A Benchmark for Systematic Generalization in Grounded Language Understanding..  Vol. 33.
  * Rust, Lotz, Bugliarello, Salesky, de Lhoneux, & Elliott Rust, P., Lotz, J. F., Bugliarello, E., Salesky, E., de Lhoneux, M., & Elliott, D. (2022).  Language Modelling with Pixels..
  * Salesky, Etter, & Post Salesky, E., Etter, D., & Post, M. (2021).  Robust Open-Vocabulary Translation from Visual Text Representations.  In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 7235–7252 Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Savva, Kadian, Maksymets, Zhao, Wijmans, Jain, Straub, Liu, Koltun, Malik, et al. Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., Malik, J., et al. (2019).  Habitat: A platform for embodied ai research..
  * Schlangen Schlangen, D. (2019a).  Grounded Agreement Games: Emphasizing Conversational Grounding in Visual Dialogue Settings..
  * Schlangen Schlangen, D. (2019b).  Language tasks and language games: On methodology in current natural language processing research..
  * Schuster & Nakajima Schuster, M., & Nakajima, K. (2012).  Japanese and korean voice search.  In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5149–5152. IEEE.
  * Shah, Fiser, Faust, Kew, & Hakkani-Tur Shah, P., Fiser, M., Faust, A., Kew, J. C., & Hakkani-Tur, D. (2018).  Follownet: Robot navigation by following natural language directions with deep reinforcement learning..
  * Sharma, Ding, Goodman, & Soricut Sharma, P., Ding, N., Goodman, S., & Soricut, R. (2018).  Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning.  In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2556–2565.
  * Shekhar, Pezzelle, Klimovich, Herbelot, Nabi, Sangineto, & Bernardi Shekhar, R., Pezzelle, S., Klimovich, Y., Herbelot, A., Nabi, M., Sangineto, E., & Bernardi, R. (2017).  FOIL it! Find One mismatch between Image and Language caption.  In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 255–265.
  * Shekhar, Venkatesh, Baumgärtner, Bruni, Plank, Bernardi, & Fernández Shekhar, R., Venkatesh, A., Baumgärtner, T., Bruni, E., Plank, B., Bernardi, R., & Fernández, R. (2019).  Beyond task success: A closer look at jointly learning to see, ask, and GuessWhat.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 2578–2587.
  * Shi, Ball, Thattai, Zhang, Hu, Gao, Shakiah, Gao, Padmakumar, Yang, et al. Shi, H., Ball, L., Thattai, G., Zhang, D., Hu, L., Gao, Q. Q., Shakiah, S., Gao, X., Padmakumar, A., Yang, B., et al. (2023).  Alexa, play with robot: Introducing the first Alexa Prize SimBot Challenge on embodied AI..
  * Shridhar, Thomason, Gordon, Bisk, Han, Mottaghi, Zettlemoyer, & Fox Shridhar, M., Thomason, J., Gordon, D., Bisk, Y., Han, W., Mottaghi, R., Zettlemoyer, L., & Fox, D. (2020).  ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks.  In Computer Vision and Pattern Recognition (CVPR).
  * Shridhar, Yuan, Côté, Bisk, Trischler, & Hausknecht Shridhar, M., Yuan, X., Côté, M.-A., Bisk, Y., Trischler, A., & Hausknecht, M. (2021).  ALFWorld: Aligning Text and Embodied Environments for Interactive Learning.  In International Conference on Learning Representations (ICLR).
  * Singh, Hu, Goswami, Couairon, Galuba, Rohrbach, & Kiela Singh, A., Hu, R., Goswami, V., Couairon, G., Galuba, W., Rohrbach, M., & Kiela, D. (2022).  Flava: A foundational language and vision alignment model.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15638–15650.
  * Singh, Deng, & Ahn Singh, G., Deng, F., & Ahn, S. (2021).  Illiterate DALL-E Learns to Compose..
  * Sivic, Russell, Efros, Zisserman, & Freeman Sivic, J., Russell, B. C., Efros, A. A., Zisserman, A., & Freeman, W. T. (2005).  Discovering objects and their location in images.  In Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1, Vol. 1, pp. 370–377. IEEE.
  * Socher, Lin, Ng, & Manning Socher, R., Lin, C. C.-Y., Ng, A. Y., & Manning, C. D. (2011).  Parsing natural scenes and natural language with recursive neural networks.  In ICML.
  * Steels Steels, L. (2015).  The Talking Heads experiment: Origins of words and meanings, Vol. 1.  Language Science Press.
  * Storks, Gao, Thattai, & Tur Storks, S., Gao, Q., Thattai, G., & Tur, G. (2021).  Are We There Yet? Learning to Localize in Embodied Instruction Following.  In HAI @ AAAI 2021.
  * Suglia, Bisk, Konstas, Vergari, Bastianelli, Vanzo, & Lemon Suglia, A., Bisk, Y., Konstas, I., Vergari, A., Bastianelli, E., Vanzo, A., & Lemon, O. (2021).  An Empirical Study on the Generalization Power of Neural Representations Learned via Visual Guessing Games.  In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 2135–2144.
  * Suglia et al. Suglia, A., et al. (2022).  Visually grounded representation learning using language games for embodied AI.  Ph.D. thesis, Heriot-Watt University.
  * Suglia, Gao, Thomason, Thattai, & Sukhatme Suglia, A., Gao, Q., Thomason, J., Thattai, G., & Sukhatme, G. (2021).  Embodied bert: A transformer model for embodied, language-guided visual task completion..
  * Suglia, Konstas, Vanzo, Bastianelli, Elliott, Frank, & Lemon Suglia, A., Konstas, I., Vanzo, A., Bastianelli, E., Elliott, D., Frank, S., & Lemon, O. (2020).  CompGuessWhat?!: A Multi-task Evaluation Framework for Grounded Language Learning.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7625–7641 Online. Association for Computational Linguistics.
  * Suhr, Lewis, Yeh, & Artzi Suhr, A., Lewis, M., Yeh, J., & Artzi, Y. (2017).  A corpus of natural language for visual reasoning.  In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 217–223.
  * Suhr, Yan, Schluger, Yu, Khader, Mouallem, Zhang, & Artzi Suhr, A., Yan, C., Schluger, J., Yu, S., Khader, H., Mouallem, M., Zhang, I., & Artzi, Y. (2019a).  Executing Instructions in Situated Collaborative Interactions.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 2119–2130.
  * Suhr, Zhou, Zhang, Zhang, Bai, & Artzi Suhr, A., Zhou, S., Zhang, A., Zhang, I., Bai, H., & Artzi, Y. (2019b).  A Corpus for Reasoning about Natural Language Grounded in Photographs.  In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 6418–6428 Florence, Italy. Association for Computational Linguistics.
  * Tan & Bansal Tan, H., & Bansal, M. (2019a).  LXMERT: Learning cross-modality encoder representations from transformers.  In Empirical Methods in Natural Language Processing (EMNLP).
  * Tan & Bansal Tan, H., & Bansal, M. (2019b).  LXMERT: Learning Cross-Modality Encoder Representations from Transformers.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 5103–5114.
  * Tan & Bansal Tan, H., & Bansal, M. (2020).  Vokenization: Improving Language Understanding via Contextualized, Visually-Grounded Supervision.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 2066–2080.
  * Thomason, Murray, Cakmak, & Zettlemoyer Thomason, J., Murray, M., Cakmak, M., & Zettlemoyer, L. (2019a).  Vision-and-Dialog Navigation.  In Conference on Robot Learning (CoRL).
  * Thomason, Padmakumar, Sinapov, Walker, Jiang, Yedidsion, Hart, Stone, & Mooney Thomason, J., Padmakumar, A., Sinapov, J., Walker, N., Jiang, Y., Yedidsion, H., Hart, J., Stone, P., & Mooney, R. J. (2019b).  Improving Grounded Natural Language Understanding through Human-Robot Dialog.  In International Conference on Robotics and Automation (ICRA).
  * Thrush, Jiang, Bartolo, Singh, Williams, Kiela, & Ross Thrush, T., Jiang, R., Bartolo, M., Singh, A., Williams, A., Kiela, D., & Ross, C. (2022).  Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5238–5248.
  * Tokunaga, Iida, Terai, & Kuriyama Tokunaga, T., Iida, R., Terai, A., & Kuriyama, N. (2012).  The REX corpora: A collection of multimodal corpora of referring expressions in collaborative problem solving dialogues.  In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pp. 422–429.
  * Tomasello Tomasello, M. (2010).  Origins of human communication.  MIT press.
  * Tu, Lalwani, Gella, & He Tu, L., Lalwani, G., Gella, S., & He, H. (2020).  An empirical study on robustness to spurious correlations using pre-trained language models..  Vol. 8, pp. 621–633. MIT Press.
  * Vygotsky Vygotsky, L. S. (2012).  Thought and language.  MIT press.
  * Wang, Madhyastha, Figueiredo, Lala, & Specia Wang, J., Madhyastha, P., Figueiredo, J., Lala, C., & Specia, L. (2021).  Multisubs: A large-scale multimodal and multilingual dataset..
  * Wang, Wu, Chen, Li, Wang, & Wang Wang, X., Wu, J., Chen, J., Li, L., Wang, Y.-F., & Wang, W. Y. (2019).  Vatex: A large-scale, high-quality multilingual dataset for video-and-language research.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4581–4591.
  * Wang & Yao Wang, Y., & Yao, Q. (2019).  Few-shot Learning: A Survey..  Vol. abs/1904.05046.
  * Wei, Wang, Schuurmans, Bosma, Chi, Le, & Zhou Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., & Zhou, D. (2022).  Chain of thought prompting elicits reasoning in large language models..
  * Westera & Boleda Westera, M., & Boleda, G. (2019).  Don’t Blame Distributional Semantics if it can’t do Entailment..
  * Whitehead, Wu, Ji, Feris, & Saenko Whitehead, S., Wu, H., Ji, H., Feris, R., & Saenko, K. (2021).  Separating skills and concepts for novel visual question answering.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5632–5641.
  * Williams, Gopalan, Rhee, & Tellex Williams, E. C., Gopalan, N., Rhee, M., & Tellex, S. (2018).  Learning to parse natural language to grounded reward functions with weak supervision.  In 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1–7. IEEE.
  * Winograd Winograd, T. (1971).  Procedures as a representation for data in a computer program for understanding natural language.  Tech. rep., MASSACHUSETTS INST OF TECH CAMBRIDGE PROJECT MAC.
  * Wittgenstein, Anscombe, & Rhees Wittgenstein, L., Anscombe, G. E. M., & Rhees, R. (1953).  Philosophische Untersuchungen.(Philosophical investigations).  Basil Blackwell.
  * Wortsman, Ehsani, Rastegari, Farhadi, & Mottaghi Wortsman, M., Ehsani, K., Rastegari, M., Farhadi, A., & Mottaghi, R. (2019).  Learning to learn how to learn: Self-adaptive visual navigation using meta-learning.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6750–6759.
  * Xie, Lai, Doran, & Kadav Xie, N., Lai, F., Doran, D., & Kadav, A. (2018).  Visual Entailment Task for Visually-Grounded Language Learning..
  * Yan, Misra, Bennnett, Walsman, Bisk, & Artzi Yan, C., Misra, D., Bennnett, A., Walsman, A., Bisk, Y., & Artzi, Y. (2018).  CHALET: Cornell house agent learning environment..
  * Yang, Panagopoulou, Lyu, Zhang, Yatskar, & Callison-Burch Yang, Y., Panagopoulou, A., Lyu, Q., Zhang, L., Yatskar, M., & Callison-Burch, C. (2021).  Visual Goal-Step Inference using wikiHow..
  * Yao, Zhao, Yu, Du, Shafran, Narasimhan, & Cao Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022).  React: Synergizing reasoning and acting in language models..
  * Yi, Wu, Gan, Torralba, Kohli, & Tenenbaum Yi, K., Wu, J., Gan, C., Torralba, A., Kohli, P., & Tenenbaum, J. (2018).  Neural-symbolic vqa: Disentangling reasoning from vision and language understanding.  In Advances in Neural Information Processing Systems, pp. 1031–1042.
  * Young, Lai, Hodosh, & Hockenmaier Young, P., Lai, A., Hodosh, M., & Hockenmaier, J. (2014).  From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions..  Vol. 2, pp. 67–78. MIT Press.
  * Younger Younger, D. H. (1967).  Recognition and parsing of context-free languages in time n3..  Vol. 10, pp. 189–208. Elsevier.
  * Yu, Lian, Zhang, & Xu Yu, H., Lian, X., Zhang, H., & Xu, W. (2018a).  Guided Feature Transformation (GFT): A Neural Language Grounding Module for Embodied Agents.  In Conference on Robot Learning, pp. 81–98.
  * Yu, Zhang, & Xu Yu, H., Zhang, H., & Xu, W. (2018b).  Interactive grounded language acquisition and generalization in a 2d world..
  * Yu, Eshghi, & Lemon Yu, Y., Eshghi, A., & Lemon, O. (2017a).  Learning how to Learn: An Adaptive Dialogue Agent for Incrementally Learning Visually Grounded Word Meanings.  In Proceedings of the First Workshop on Language Grounding for Robotics, pp. 10–19.
  * Yu, Eshghi, Mills, & Lemon Yu, Y., Eshghi, A., Mills, G., & Lemon, O. (2017b).  The BURCHAK corpus: a Challenge Data Set for Interactive Learning of Visually Grounded Word Meanings.  In Proceedings of the Sixth Workshop on Vision and Language, pp. 1–10.
  * Zarrieß, Hough, Kennington, Manuvinakurike, DeVault, Fernández, & Schlangen Zarrieß, S., Hough, J., Kennington, C., Manuvinakurike, R., DeVault, D., Fernández, R., & Schlangen, D. (2016).  Pentoref: A corpus of spoken references in task-oriented dialogues.  In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pp. 125–131.
  * Zellers, Bisk, Farhadi, & Choi Zellers, R., Bisk, Y., Farhadi, A., & Choi, Y. (2019).  From recognition to cognition: Visual commonsense reasoning.  In Proceedings of IEEE CVPR.
  * Zellers, Holtzman, Peters, Mottaghi, Kembhavi, Farhadi, & Choi Zellers, R., Holtzman, A., Peters, M., Mottaghi, R., Kembhavi, A., Farhadi, A., & Choi, Y. (2021).  PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World..
  * Zellers, Lu, Lu, Yu, Zhao, Salehi, Kusupati, Hessel, Farhadi, & Choi Zellers, R., Lu, J., Lu, X., Yu, Y., Zhao, Y., Salehi, M., Kusupati, A., Hessel, J., Farhadi, A., & Choi, Y. (2022).  Merlot reserve: Neural script knowledge through vision and language and sound.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16375–16387.
  * Zellers, Lu, Hessel, Yu, Park, Cao, Farhadi, & Choi Zellers, R., Lu, X., Hessel, J., Yu, Y., Park, J. S., Cao, J., Farhadi, A., & Choi, Y. (2021).  MERLOT: Multimodal Neural Script Knowledge Models..
  * Zhang, Hu, Qiu, Shaw, & Sha Zhang, B., Hu, H., Qiu, L., Shaw, P., & Sha, F. (2021).  Visually Grounded Concept Composition..
  * Zhang, Yu, & Xu Zhang, H., Yu, H., & Xu, W. (2018).  Interactive Language Acquisition with One-shot Visual Concept Learning through a Conversational Game.  In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1, pp. 2609–2619.
  * Zhang, Li, Hu, Yang, Zhang, Wang, Choi, & Gao Zhang, P., Li, X., Hu, X., Yang, J., Zhang, L., Wang, L., Choi, Y., & Gao, J. (2021).  Vinvl: Revisiting visual representations in vision-language models.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5579–5588.
  * Zheng, Meng, Si, Fan, Xu, Zhou, Feng, & Wang Zheng, D., Meng, F., Si, Q., Fan, H., Xu, Z., Zhou, J., Feng, F., & Wang, X. (2022).  Spot the Difference: A Cooperative Object-Referring Game in Non-Perfectly Co-Observable Scene..
  * Zhong, Hanjie, Wang, Narasimhan, & Zettlemoyer Zhong, V., Hanjie, A. W., Wang, S., Narasimhan, K., & Zettlemoyer, L. (2021).  SILG: The Multi-domain Symbolic Interactive Language Grounding Benchmark..  Vol. 34, pp. 21505–21519.
  * Zhou, Palangi, Zhang, Hu, Corso, & Gao Zhou, L., Palangi, H., Zhang, L., Hu, H., Corso, J. J., & Gao, J. (2020).  Unified Vision-Language Pre-Training for Image Captioning and VQA..  In AAAI, pp. 13041–13049.
  * Zhu, Neubig, & Bisk Zhu, H., Neubig, G., & Bisk, Y. (2021).  Few-shot language coordination by modeling theory of mind.  In International Conference on Machine Learning, pp. 12901–12911. PMLR.
  * Zhu, Qi, Narayana, Sone, Basu, Wang, Wu, Eckstein, & Wang Zhu, W., Qi, Y., Narayana, P., Sone, K., Basu, S., Wang, X., Wu, Q., Eckstein, M., & Wang, W. Y. (2022).  Diagnosing Vision-and-Language Navigation: What Really Matters.  In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5981–5993 Seattle, United States. Association for Computational Linguistics.