The authors discuss two limitations: first, this paper focuses only on methods with explicit negatives. This is not a problem for me since it is okay for an analysis paper to focus on one type of methods. The second limitation is that the datasets used in the experiments are not fully realistic. This again is not an issue for me, since 1) the datasets used in this paper are variants of ImageNet and MNIST which are both realistic, and 2) fully realistic datasets will make it hard to control multiple aspects of variation with precision.
I agree with the authors' judgement that there is no immediate societal impact.
1) the datasets used in this paper are variants of ImageNet and MNIST which are both realistic, and