1. Limited Explanation of Failures: Although the paper provides examples of failure cases, it does not fully delve into the underlying causes or offer detailed solutions for these issues. While the authors acknowledge the problem of overconfidence in models such as GPT-4o with ReGuide, further exploration of these limitations could strengthen the study. There is still not a robust method to effectively introduce LVLMs to the OoD tasks.
2. Model-Specific Insights: The paper focuses on generic findings across models, but a deeper investigation into how specific models (e.g., GPT-4o vs. InternVL2) behave differently when ReGuide is applied could add nuance to the conclusions. For example, the differences in false positive rates (FPR) between models with and without ReGuide should be presented for a better comparison.
3. Scalability and Practicality: While the ReGuide method shows a promising direction, the computational overhead and API limitations mentioned in the paper could present challenges for practical, large-scale implementation. This issue is touched upon but not sufficiently addressed in terms of how ReGuide might be optimized for deployment at scale. Meanwhile, the inference cost analysis can further improve the paper's quality and inspire further work.
2. Model-Specific Insights: The paper focuses on generic findings across models, but a deeper investigation into how specific models (e.g., GPT-4o vs. InternVL2) behave differently when ReGuide is applied could add nuance to the conclusions. For example, the differences in false positive rates (FPR) between models with and without ReGuide should be presented for a better comparison.