1.	The paper is motivated with using natural language feedback just as humans would provide while teaching a child. However, in addition to natural language feedback, the proposed feedback network also uses three additional pieces of information â which phrase is incorrect, what is the correct phrase, and what is the type of the mistake. Using these additional pieces is more than just natural language feedback. So I would like the authors to be clearer about this in introduction.
2.	The improvements of the proposed model over the RL without feedback model is not so high (row3 vs. row4 in table 6), in fact a bit worse for BLEU-1. So, I would like the authors to verify if the improvements are statistically significant.
3.	How much does the information about incorrect phrase / corrected phrase and the information about the type of the mistake help the feedback network? What is the performance without each of these two types of information and what is the performance with just the natural language feedback?
4.	In figure 1 caption, the paper mentions that in training the feedback network, along with the natural language feedback sentence, the phrase marked as incorrect by the annotator and the corrected phrase is also used. However, from equations 1-4, it is not clear where the information about incorrect phrase and corrected phrase is used. Also L175 and L176 are not clear. What do the authors mean by âas an exampleâ?
5.	L216-217: What is the rationale behind using cross entropy for first (P â floor(t/m)) phrases? How is the performance when using reinforcement algorithm for all phrases?
6.	L222: Why is the official test set of MSCOCO not used for reporting results?
7.	FBN results (table 5): can authors please throw light on why the performance degrades when using the additional information about missing/wrong/redundant?
8.	Table 6: can authors please clarify why the MLEC accuracy using ROUGE-L is so low? Is that a typo?
9.	Can authors discuss the failure cases of the proposed (RLF) network in order to guide future research?
10.	Other errors/typos:
a.	L190: complete -> completed
b.	L201, âWe use either â¦ feedback collectionâ: incorrect phrasing
c.	L218: multiply -> multiple
d.	L235: drop âbyâ
Post-rebuttal comments:
I agree that proper evaluation is critical. Hence I would like the authors to verify that the baseline results [33] are comparable and the proposed model is adding on top of that.
So, I would like to change my rating to marginally below acceptance threshold.
3. How much does the information about incorrect phrase / corrected phrase and the information about the type of the mistake help the feedback network? What is the performance without each of these two types of information and what is the performance with just the natural language feedback?