1. Human evaluations were not performed. Given the weaknesses of SARI (Vásquez-Rodríguez et al. 2021) and FKGL (Tanprasert and Kauchak, 2021), the lack of human evaluations severely limits the potential impact of the results, combined with the variability in the results on different datasets.
2. While the authors explain the need to include text generation models into the framework of (Kumar et al., 2020), it is not clear as to why only the delete operation was retained from the framework, which used multiple edit operations (reordering, deletion, lexical simplification, etc.). Further, it is not clear how including those other operations will affect the quality and performance of the system.
3. ( minor) It is unclear how the authors arrived at the different components of the "scoring function," nor is it clear how they arrived at the different threshold values/ranges.
4. Finally, one might wonder that the performance gains on Newsela are due to a domain effect, given that the system was explicitly tuned for deletion operations (that abound in Newsela) and that performance is much lower on the ASSET test corpus. It is unclear how the system would generalize to new datasets with varying levels of complexity, and peripheral content.
1. Is there any reason why 'Gold Reference' was not reported for Newsela? It makes it hard to assess the performance of the existing system. 2. Similarly, is there a reason why the effect of linguistic acceptability was not analyzed (Table 3 and Section 4.6)?
3. It will be nice to see some examples of the system on actual texts (vs. other components & models).
4. What were the final thresholds that were used for the results? It will also be good for reproducibility if the authors can share the full set of hyperparameters as well.
3. It will be nice to see some examples of the system on actual texts (vs. other components & models).