1. Utilizing energy models to explain the fine-tuning of pre-trained models seems not to be essential. As per my understanding, the objective of the method in this paper as well as related methods ([1,2,3], etc.) is to reduce the difference in features extracted by the models before and after fine-tuning.
2. The authors claim that the text used is randomly generated, but it appears from the code in the supplementary material that tokens are sampled from the openai_imagenet_template. According to CAR-FT, using all templates as text input also yields good performance. What then is the significance of random token sampling in this scenario?
3. It is suggested that the authors provide a brief introduction to energy models in the related work section.
In Figure 1, it is not mentioned which points different learning rates in the left graph and different steps in the right graph correspond to.
[1] Context-aware robust fine-tuning.
[2] Fine-tuning can cripple your foundation model; preserving features may be the solution.
[3] Robust fine-tuning of zero-shot models.
3. It is suggested that the authors provide a brief introduction to energy models in the related work section. In Figure 1, it is not mentioned which points different learning rates in the left graph and different steps in the right graph correspond to. [1] Context-aware robust fine-tuning. [2] Fine-tuning can cripple your foundation model; preserving features may be the solution. [3] Robust fine-tuning of zero-shot models.