1) Instead of arguing that the MTL approach replaces the attention mechanism, I think the authors should investigate why attention did not work on MTL, and perhaps modify the attention mechanism so that it would not harm performance.
2) I think the authors should reference past seq2seq MTL work, such as [2] and [3]. The MTL work in [2] also worked on non-attention seq2seq models.
3) This paper only tested on one German historical text data set of 44 documents. It would be interesting if the authors can evaluate the same approach in another language or data set.
References: [1] Allen Schmaltz, Yoon Kim, Alexander M. Rush, and Stuart Shieber. 2016.
Sentence-level grammatical error identification as sequence-to-sequence correction. In Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications.
[2] Minh-Thang Luong, Ilya Sutskever, Quoc V. Le, Oriol Vinyals, and Lukasz Kaiser. Multi-task Sequence to Sequence Learning. ICLR’16. [3] Dong, Daxiang, Wu, Hua, He, Wei, Yu, Dianhai, and Wang, Haifeng.
Multi-task learning for multiple language translation. ACL'15 --------------------------- Here is my reply to the authors' rebuttal: I am keeping my review score of 3, which means I do not object to accepting the paper. However, I am not raising my score for 2 reasons: - the authors did not respond to my questions about other papers on seq2seq MTL, which also avoided using attention mechanism. So in terms of novelty, the main novelty lies in applying it to text normalization.
- it is always easier to show something (i.e. attention in seq2seq MTL) is not working, but the value would lie in finding out why it fails and changing the attention mechanism so that it works.
- it is always easier to show something (i.e. attention in seq2seq MTL) is not working, but the value would lie in finding out why it fails and changing the attention mechanism so that it works.