Weakness:
The idea is a bit simple -- which in of itself is not a true weakness. ResNet as an idea is not complicated at all. I find it disheartening that the paper did not really tell readers how to construct a white paper in section 3 (if I simply missed it, please let me know). However, the code in the supplementary materials helped. White paper is constructed as follow:
white_paper_gen = torch.ones(args.train_batch, 3, 32, 32)
It offers another way of constructing white paper, which is
white_paper_gen = 255 * np.ones((32, 32, 3), dtype=np.uint8)
white_paper_gen = Image.fromarray(white_paper_gen)
white_paper_gen = transforms.ToTensor()(white_paper_gen)
white_paper_gen = transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))(white_paper_gen)
The code states that either version works similarly and does not affect the performance. I wonder if there are other white papers as well, for example np.zeros((32, 32, 3)) -- most CNN models add explicit bias terms in their CNN kernel. Would a different white paper reveal different bias in the model? I don't think the paper answers this question or discusses it. 2. Section 4 "Is white paper training harmful to the model?" -- the evidences do not seem to support the claim. The evidences are 1). Only projection head (CNN layers) are affected but not classification head (FCN layer); 2). Parameter changes are small. None of these constitute as a direct support that the training is not "harmful" to the model. This point can simply be illustrated by the experimental results 3. Section 5.1 and 5.2 mainly build the narrative that WPA improves the test performance (generalization performance), but they are indirect evidence to support that WPA does in fact alleviate shortcut learning. Only Section 5.3 and Table 6 directly show whether WPA does what it's designed to do. A suggestion is to discuss the result of Section 5.3 more. 4. It would be interesting to try to explain why WPA works -- with np.ones input, what is the model predicting? Would any input serve as white paper? Figure 2 seems to suggest that Gaussian noise input does not work as well as WPA. Why? Authors spend lot of time showing WPA improves the test performance of the original model, but fails to provide useful insights on how WPA works -- this is particularly important because it can spark future research directions.
4. It would be interesting to try to explain why WPA works -- with np.ones input, what is the model predicting? Would any input serve as white paper? Figure 2 seems to suggest that Gaussian noise input does not work as well as WPA. Why? Authors spend lot of time showing WPA improves the test performance of the original model, but fails to provide useful insights on how WPA works -- this is particularly important because it can spark future research directions.