--- There are two somewhat minor weakness: presentation and some missing related work. The main points in this paper can be understood with a bit of work, but there are lots of minor missing details and points of confusion. I've listed them roughly in order, with the most important first: * What factors varied in order to compute the error bars in figure 2? Were different random initializations used? Were different splits of the dataset used? How many samples do the error bars include? Do they indicate standard deviation or standard error? * L174: How exactly does the reactive baseline work? * L185: What does "without agent embeddings" mean precisely? * L201: More details about this metric are needed. I don't know exactly what is plotted on the y axis without reading the paper. Before looking into the details I'm not even sure whether higher or lower is good without looking into the details. (Does higher mean more information or does lower mean more information?) * Section 3: This would be much clearer if an example were used to illustrate the problem from the beginning of the section. * Will code be released? * L162: Since most experiments share perception between speaker and listener it would be much clearer to introduce this as a shared module and then present section 4.3 as a change to that norm. * L118: To what degree is this actually realized? * L84: It's not information content itself that will suffer, right? * L77: This is unnecessary and a bit distracting. * L144: Define M and N here. * L167: What is a "sqeuence of episodes" here? Are practice and evaluation the two types of this kind of sequence? Missing related work (seems very related, but does not negate this work's novelty): * Existing work has tried to model human minds, especially in robotics. It looks like [2] and [3] are good examples. The beginning of the related work in [1] has more references along these lines. This literature seems significantly different from what is proposed in this paper because the goals and settings are different. Only the high level motivation appears to be similar. Still, the literature seems significant enough (on brief inspection) to warrent a section in the related work. I'm not very familiar with this literature, so I'm not confident about how it relates to the current paper. [1]: Chandrasekaran, Arjun et al. âIt Takes Two to Tango: Towards Theory of AI's Mind.â CVPR 2017 [2]: Butterfield, Jesse et al. âModeling Aspects of Theory of Mind with Markov Random Fields.â International Journal of Social Robotics 1 (2009): 41-51. [3]: Warnier, Matthieu et al. âWhen the robot puts itself in your shoes. Managing and exploiting human and robot beliefs.â 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication (2012): 948-954. Suggestions --- * L216: It would be interesting to realize this by having the speaker interact with humans since the listeners are analogous to role humans take in the high level motivation. That would be a great addition to this or future work. Final Justification --- Clarity - This work could significantly improve its presentation and add more detail, but it currently is clear enough to get the main idea. Quality - Despite the missing details, the experiments seem to be measuring the right things and support very clear conclusions. Novelty - Lots of work uses reference games with multiple agents, but I'm not aware of attempts to specifically measure and model other agents' minds. Significance - The work is a useful step toward agents with a theory of mind because it presents interesting research directions that didn't exist before. Overall, this is a pretty good paper and should be accepted. Post-rebuttal Updates --- After reading the reviews and the rebuttal this paper seems like a clear accept. After discussion with R3 I think we all roughly agree. The rebuttal addressed all my concerns except the minor one listed below satisfactorily. There is one piece R3 and I touched on which is still missing. I asked about the relation to meta-learning and there was no response. More importantly, R3 asked about a comparison to a practice-stage only reward, which would show the importance of the meta-learning aspect of the reward. This was also not addressed satisfactorily, so it's still hard to understand the role of practice/evaluation stages in this work. This would be nice to have, but rest of the paper provides a valuable contribution without it. Though it's hard to tell how presentation and related work will ultimately be addressed in the final version, the rebuttal goes in the right direction so I'll increase my score as indicated in the Improvements section of my initial review.
* L167: What is a "sqeuence of episodes" here? Are practice and evaluation the two types of this kind of sequence? Missing related work (seems very related, but does not negate this work's novelty):