Despite the idea being interesting, I have found some technical issues that weakened the overall soundness. I enumerate them as follows:
1. The assumption that generated URLs are always meaningfully related to the core content of the document from where the premises are to be fetched is not true by and large. It works for Wikipedia because the URLs are well-structured semantically.
2. LLMs generating URLs on Wikidata have a significantly higher probability of being linked with a valid URL because extensive entity linking has already been done. This, however, is not the case for many other web sources.
3. There are several URLs that are not named according to the premise entities. In that case, those sources will never be fetched.
4. How to resolve contradictory entailment from premises belonging to different sources?
5. There can be many sources that are themselves false (particularly for the open Internet and also in cases of unverified Wiki pages). So assuming the premises to be true may lead to incorrect RTE.
6. It is unclear how the prompt templates are designed, i.e., the rationale and methodology that would drive the demonstration example patterns in the few-shot cases.
7. A discussion on the prompt dataset (for the few-shot case) creation together with its source should be discussed.
8. The assumption that RTE (i.e. NLI) being true would imply that the hypothesis (fact/claim) is verified is bit tricky and may not always be true. A false statement can entail a hypothesis as well as its true version. Eg.:
$\textit{Apples come in many colors}$ $\implies$ $\textit{Apples can be blue (claim)}$.
$\textit{John was a follower of Jesus}$ $\implies$ $\textit{John was a Christian (claim)}$.
9. Line 253: What is citation threshold? I could not find the definition.
10. In the comparisons with the baselines and variants of LLM-Cite, what was the justification behind not keeping the model set fixed for all the experiments? I think this should be clear.
11. In sections 4.1 and 4.2, an analysis of why the verification models perform better on model-generated claims as compared to human-generated claims is very important to me. I could not find any adequate analysis for that.
12. The key success of LLM-Cite depends on the NLI model (given that at least one valid URL is generated that points to a valid premise). Hence, a discussion on the accuracy of SOTA NLI models (with citation) and the rationale behind choosing the Check Grounding NLI API and Gemini-1.5-Flash should be included.
7. A discussion on the prompt dataset (for the few-shot case) creation together with its source should be discussed.