1. The novelty of this method appears somewhat constrained. Utilizing the first-order gradient for determining parameter importance is a common approach in pruning techniques applied to CNN, BERT, and ViT. This technique is well-established within the realm of model pruning. Considering in some instances this method even falls short of those achieved by SparseGPT (e.g., 2:4 for LLaMA-1 and LLaMA-2), I cannot say the first-order gradient in pruning LLMs might be a major contribution.
2. This paper lacks experiments on different LLM families. Conducting trials with models like OPT, BLOOM, or other alternatives could provide valuable insights into the method's applicability and generalizability across various LLM families.
3. The paper doesn't provide details regarding the latency of the pruned model. In a study centered on LLM compression, including latency metrics is crucial since such information is highly important to the readers to understand the efficiency of the pruned model.
2. This paper lacks experiments on different LLM families. Conducting trials with models like OPT, BLOOM, or other alternatives could provide valuable insights into the method's applicability and generalizability across various LLM families.