major concerns:
- This paper proposes a knowledge distillation framework from ANN to SNN. The only contribution is the loss function used here is different from previous work. This contribution is not sufficient for conferences like ICLR.
- It is hard to determine that the usage of this loss function is unique in the literature. In the domain of knowledge distillation in ANNs, there are numerous works studying different types of loss functions. Whether this loss function is original remains questioned.
- I find the design choice of simply averaging the time dimension in SNN featuremaps inappropriate. The method in this paper is not the real way to compute the similarity matrix. Instead, to calculate the real similarity matrix of SNNs, the authors should flatten the SNN activation to $[B, TCHW]$ and then compute the $B\times B$ covariance matrix. For detailed explanation, check [1].
- Apart from accuracy, there are not many insightful discoveries for readers to understand the specific mechanism of this loss function.
- The experiments are not validated on ImageNet, which largely weakens the empirical contribution of this paper.
minor concerns:
- " The second is that the neuron state of ANNs is represented in binary format but that of SNNs is represented in float format, leading to the precision mismatch between ANNs and SNNs", wrong value formats of SNNs and ANNs.
- The related work should be placed in the main text, rather than the appendix. There are many spaces left on the 9th page.
[1] Uncovering the Representation of Spiking Neural Networks Trained with Surrogate Gradient
- " The second is that the neuron state of ANNs is represented in binary format but that of SNNs is represented in float format, leading to the precision mismatch between ANNs and SNNs", wrong value formats of SNNs and ANNs.