- **generalizability to other models**: the proposed framework is validated using gpt-4-turbo, a costly language model, which may compromise the applicability of the framework at scale. The paper could be further improved by showing how running the experiments using a cheaper model (e.g., gpt-4o) and/or open source models (e.g., Llama 3.1) would affect the obtained results.
- **generalizability of the results**: the conducted experiments are either too simple (simple synthetic regression setting with 4 variables) or include few data-model pairs (6 in Section 4.2, 36 in Section 4.3, and 10 for the human studies), raising questions about the generalizability of the proposed framework to more complex datasets.
- **lack of meaningful baselines**: despite mentioning various model criticism techniques in Section 2, the authors limit their comparisons to simple naive baselines. For example, the authors could compare with a chain-of-thought prompting approach.
- few insights about the generated and correctness of summary statistics: while the authors provide one example in Section 4.3, the paper could be further improved by adding additional insights and contrasting the proposed discrepancies with commonly discussed discrepancies in the literature (e.g., do these resemble the ones commonly found by humans?)
- **lack of meaningful baselines**: despite mentioning various model criticism techniques in Section 2, the authors limit their comparisons to simple naive baselines. For example, the authors could compare with a chain-of-thought prompting approach.