In lines 3-4, the statement is not necessarily true as there are also methods that focus on better distribution calibration during training (some of these ideas are mentioned later in the paper as well). I would recommend softening the message of this sentence to be more precise.
Language like 'Obviously' (line 47) and 'It is easy to see' (line 168) should generally be avoided in academic writing.
At the end of Sec. 2, it would be helpful to have a one or two sentence discussion to contextualize the proposed BATS method in the described related work.
Variations on the phrase 'We propose to rectify the features into the feature's typical set and then use these typical features to calculate the OOD score.' are repeated frequently throughout the paper. The λ
hyperparameter was not introduced in line 142 when it was first used.
The decreased BATS performance on Tiny-Imagenet OOD detection in Table 2 is not mentioned or discussed.
The paper needs to be proofread for typos. The following is a non-exhaustive list of the typos I found:
Line 2: 'which raises the attention on out-of-distribution (OOD) detection' is awkward phrasing.
Lines 74-75: 'large sufficiently' should read 'sufficiently large'.
Line 98: 'energe score' should read 'energy score'.
Line 109: 'is provable aligned' should read 'is provably aligned'.
Line 130: 'common-used layer' should read 'commonly used layer'.
Footnote 1: I did not grammatically understand the phrase: 'the pre-training outputs moving average estimators during iterations'. Maybe something like: 'The pre-trained model outputs moving average estimators at each iteration.'?
Line 184: 'a two-side rectified normal distribution' should read 'a two-sided rectified normal distribution'.
There should be a space between the abbreviation and the number in references (i.e., Fig. X, Table Y, Sec. Z).
Line 197: 'Fig.2 illustrate' should read 'Fig. 2 illustrates'.
Line 220: 'verse vice' should read 'vice versa'.
Line 225: 'Recent researches propose' should read 'Recent literature/work proposes'.
Line 235: 'models are standard pre-trained' is grammatically awkward. Maybe something like: 'models are pre-trained in a standard manner'?
Line 238: 'In specific,' should read 'Specifically,'.
Line 246: 'which cost more' should read 'which costs more'.
Line 252: 'The start learning rate' should read 'The starting learning rate'.
Line 284: 'Our BATS can reduce the variance that benefit the OOD detection but also introduce a bias.' should read 'Our proposed BATS method can reduce variance, which benefits OOD detection, but can also introduce a bias.'.
Line 285: 'Energy Score (The horizontal lines).' should read 'Energy Score (the horizontal lines)'.
Fig. 5: x-axis labels are missing. This is also the case for some figures in the appendix.
Sec. 6: 'Limitation and societal impact' should read 'Limitations and societal impact'.
Sec. 6: batch normalization is referred to in three different ways in the last paragraph (Batch Normalization, Batch-Norm, BN).
Line 693: 'our method surpass' should read 'our method surpasses'.
The references should be proofread (e.g., to ensure the year is not entered twice in a citation, the conference venue is listed instead of ArXiv when available, etc.).
The authors have done a good job in listing limitations of the BATS method. However, the addition of some potential negative societal implications would be helpful. For example, by truncating the features, certain biases in the data learned by the pre-trained model may be amplified. Furthermore, the process of truncation will inherently cause some information loss which may be crucial to model performance during deployment (building on the hyperparameter tuning discussion in the paper).
5: x-axis labels are missing. This is also the case for some figures in the appendix. Sec.