- The experiment section is not informative to the readers, because it is stating obvious results such as Transformer models being more “capable of capturing content-specific information”.
- The case study section is also not very informative. The authors evaluate whether the model assigns bigger attention weights to the appropriate control token when generating relevant sentences. However, they only show a single sample as an illustration. Such a single sample illustration is very vulnerable to cherry-picking and biases. Moreover, interpreting bigger attention weights as being more important is somewhat controversial in the field. It would be more helpful to the readers if the authors show diverse quantitative analyses.
- The authors show how different control sequences affect the generated outputs with only 3 examples, which is extremely limited to see whether the model can be controlled.
- It will be interesting to drop the control sequences from the gold labels (or replace them with negative ones) one after one and see how the automatic metric changes. This will show whether the control tokens really do contribute to the model’s performance.
- The authors show how different control sequences affect the generated outputs with only 3 examples, which is extremely limited to see whether the model can be controlled.