1. A critical weakness of the paper is the lack of novelty and incremental nature of work. The paper addresses a particular problem of column operations in designing semantic parsers for Text-to-SQL. They design a new dataset which is a different train/test split of an existing dataset SQUALL. The other synthetic benchmark paper proposed is based on a single question template, "What was <column> in <year>?".
2. The paper assumes strong domain knowledge about the column types and assumes a domain developer first creates a set of templates based on column types. With the help of these column templates, I think many approaches (parsers) can easily solve the problem. For example, parsers utilizing the SQL grammar to generate the output SQL can use these templates to add new rules that can be used while generating the output. Few such works are 1. A Globally Normalized Neural Model for Semantic Parsing ACl 2021 2. TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation EMNP 2018 3. GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing, ICLR 2021.
1. It will good if the authors can learn the templates for schema expansion from source domain data.
2. Compare the proposed approach with methods which uses domain knowledge in the form of grammar. Comparing with below methods will show generality of ideas proposed in the paper in a much better way.
1. A Globally Normalized Neural Model for Semantic Parsing ACl 2021 2. TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation EMNP 2018 3. GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing, ICLR 2021.
1. A critical weakness of the paper is the lack of novelty and incremental nature of work. The paper addresses a particular problem of column operations in designing semantic parsers for Text-to-SQL. They design a new dataset which is a different train/test split of an existing dataset SQUALL. The other synthetic benchmark paper proposed is based on a single question template, "What was <column> in <year>?".