* It will help if the paper describes the possible alternate formulations for Confidence Diversity (CD). I am not asking for additional experiments here. It is difficult to what extra information is CD capturing on top of Predictive Uncertainty in the current form. Or why is entropy not a good measure of "amount of spreading of teacher predictions over the probability simplex among different (training) samples" (line 115). Note that line 113 did not clarify it for me. * I think the experimental section is somewhat weak. The paper considers three datasets and two models, but all the datasets are vision datasets and are small. Moreover, the baselines are sufficient. I would like to see at least a standard distillation baseline, trained with both soft and hard labels. We don't know the number of generations for the SD baseline as well. I understand the authors point out that parameter tuning is expensive. Still, they should consider ablations (and not parameter tuning) and show that their empirical gains are consistent across different parameters.
* It will help if the paper describes the possible alternate formulations for Confidence Diversity (CD). I am not asking for additional experiments here. It is difficult to what extra information is CD capturing on top of Predictive Uncertainty in the current form. Or why is entropy not a good measure of "amount of spreading of teacher predictions over the probability simplex among different (training) samples" (line 115). Note that line 113 did not clarify it for me.