__1. The relation between instance difficulty and training-inference consistency remains vague:__ This paper seems to try to decouple the concept of instance difficulty and training-inference consistency in current early exiting works. However, I don't think these two things are orthogonal and can be directly decoupled. Intuitively, according to the accuracy of the prediction, there are two main situations for training-inference inconsistency: the inconsistent exit makes the prediction during inference better than that during training, and vice versa. The first case is unlikely to occur. For the second case, considering instance difficulty reflects the decline in the prediction accuracy of the instances during training and inference, it may be regarded as the second case of training-inference consistency. Accordingly, I am still a little bit confused about the relation between instance difficulty and training-inference consistency after reading the paper. I would suggest that the authors calculate the token-level difficulty of the model before and after using the hash function, and perform more analysis on this basis. In fact, if the hash function is instance-level, the sentence-level difficulty of all baselines (including static and dynamic models) can be calculated, which will provide a more comprehensive and fair comparison.
__2. Lack of the analysis of the relation between instance-level consistency and token-level consistency:__ The core idea derived from the preliminary experiments is to enhance the instance-level consistency between training and inference, i.e., mapping semantically similar instances into the same exiting layer. However, the practical method introduces the consistency constrain at the token level. The paper doesn’t show that whether the token-level method can truely address or mitigate the inconsistency problem at the instance level. I would suggest the authors define metrics to reflect the instance-level and token-level consistency, and conduct an experiment to verify that whether they are correlated.
1. I didn’t follow the description of the sentence-level hash function from Line 324 to Line 328: If we use the sequence encoder (e.g., Sentence-BERT) as a hash function to directly map the instances to the exiting layer, why do we still need an internal classifier at that layer? And considering all the instances can be hashed by the pre-trained sequence encoder in advance before training (and early exiting), the appearance of label imbalance should not cause any actual harm? Why does it become a problem?
2. The paper addresses many times (Line 95-97, Line 308-310) that the consistency between training and inference can be easily satisfied due to the smoothness of neural models. I would suggest giving more explanations on this.
2. The paper addresses many times (Line 95-97, Line 308-310) that the consistency between training and inference can be easily satisfied due to the smoothness of neural models. I would suggest giving more explanations on this.