- Claims of being comparable to state of the art when the results on GeoQuery and ATIS do not support it. General Discussion: This is a sound work of research and could have future potential in the way semantic parsing for downstream applications is done. I was a little disappointed with the claims of “near-state-of-the-art accuracies” on ATIS and GeoQuery, which doesn’t seem to be the case (8 points difference from Liang et. al., 2011)). And I do not necessarily think that getting SOTA numbers should be the focus of the paper, it has its own significant contribution. I would like to see this paper at ACL provided the authors tone down their claims, in addition I have some questions for the authors.
- What do the authors mean by minimal intervention? Does it mean minimal human intervention, because that does not seem to be the case. Does it mean no intermediate representation? If so, the latter term should be used, being less ambiguous.
- Table 6: what is the breakdown of the score by correctness and incompleteness?
What % of incompleteness do these queries exhibit?
- What is expertise required from crowd-workers who produce the correct SQL queries? - It would be helpful to see some analysis of the 48% of user questions which could not be generated.
- Figure 3 is a little confusing, I could not follow the sharp dips in performance without paraphrasing around the 8th/9th stages. - Table 4 needs a little more clarification, what splits are used for obtaining the ATIS numbers?
I thank the authors for their response.
- Table 4 needs a little more clarification, what splits are used for obtaining the ATIS numbers? I thank the authors for their response.