The following are the questions I have and these are not necessarily 'reasons to reject'.
- I was looking for a comparison with the zero-shot chain of thought baseline which authors refer as ZOT (Kojima et al., 2022). The example selection method has a cost. Also, few shot experiments involve extra token usage cost than zero shot.
- Some of the numbers while comparing proposed method vs baselines seem to be pretty close. Wondering, if authors did any statistical significance test?
- A parallel field to explanation selection is prompt/instruction engineering, where we often change the zeroshot instruction. Another alternative is prompt-tuning via gradient descent. Wondering if authors have any thoughts regarding the tradeoff.
- Few shot examples has various types of example biases such as majority bias, recency bias etc. (http://proceedings.mlr.press/v139/zhao21c/zhao21c.pdf, https://aclanthology.org/2023.eacl-main.130/, https://aclanthology.org/2022.acl-long.556.pdf). Wondering if authors have any thought on how the robustness look like with the application of their method?
I am looking forward to hear answers to these questions from the authors.
- Some of the numbers while comparing proposed method vs baselines seem to be pretty close. Wondering, if authors did any statistical significance test?