- Different variants of the model achieve state-of-the-art performance across various data sets. However, the authors do provide an explanation for this (i.e. size of data set and text anonymization patterns).
- General Discussion: The paper describes an approach to text comprehension which uses gated attention modules to achieve state-of-the-art performance. Compared to previous attention mechanisms, the gated attention reader uses the query embedding and makes multiple passes (multi-hop architecture) over the document and applies multiplicative updates to the document token vectors before finally producing a classification output regarding the answer. This technique somewhat mirrors how humans solve text comprehension problems. Results show that the approach performs well on large data sets such as CNN and Daily Mail. For the CBT data set, some additional feature engineering is needed to achieve state-of-the-art performance. Overall, the paper is very well-written and model is novel and well-motivated.
Furthermore, the approach achieves state-of-the-art performance on several data sets. I had only minor issues with the evaluation. The experimental results section does not mention whether the improvements (e.g. in Table 3) are statistically significant and if so, which test was used and what was the p-value. Also I couldn't find an explanation for the performance on CBT-CN data set where the validation performance is superior to NSE but test performance is significantly worse.
- Different variants of the model achieve state-of-the-art performance across various data sets. However, the authors do provide an explanation for this (i.e. size of data set and text anonymization patterns).