- Some parts of the paper (e.g., the first three paragraphs of the introduction) talk about NLG in general but the paper focuses on machine translation. Reranking methods, in particular, rely on good quality estimation models that exist for MT but may not exist for other tasks. Of course, MBR finetuning can be applied to other NLG tasks (e.g., summarization) but this paper does not touch this problem. I think the authors should either revise the writing and make the scope more clear from the beginning, or perform experiments in other NLG tasks. The contribution is mainly empirical and only validated for MT.
- The study is limited in terms of language coverage. They perform experiments on 2 language pairs only (mid & high resource), both for translation out of English. It’s not clear if the findings hold for lower resource languages and when translating into English. In particular, I’m expecting quality estimation models to be worse for low resource languages, which may end up affecting the quality of the translations produced with their method. Have you tried other languages? If so, what happens in that case?
- The related work discusses other methods for training NMT models beyond MLE (e.g., RL methods) but none of them is used as a baseline.
- The related work discusses other methods for training NMT models beyond MLE (e.g., RL methods) but none of them is used as a baseline.