, which justify the score: • The theoretical developments presented in the paper build on the Rademacher complexity, but ignore the conclusions drawn by Zhang et al. in Section 2.2 of their ICLR 2017 paper (Understanding deep learning requires rethinking generalization). • The theoretical developments build on the assumption that (i) there exists a lower bound, valid for any input, to the distance between the output of each pair of neurons, and (ii) the proposed diversity loss increases this lower bound. Those two assumptions are central to the theoretical developments, but are quite arguable. For example, a pair of neuron that is not activated by a sample, which is quite common, leads to a zero lower bound. • Experimental validation are not convincing. Only shallow networks are considered (2 or 3 layers), and the optimization strategy, including the grid search strategy for hyperparameters selection, is not described.
Minor issue: positioning with respect to related works is limited. For example, layer redundancy (which is the opposite of diversity) has been considered in the context of network pruning: https://openaccess.thecvf.com/content_CVPR_2019/papers/He_Filter_Pruning_via_Geometric_Median_for_Deep_Convolutional_Neural_Networks_CVPR_2019_paper.pdf
• Experimental validation are not convincing. Only shallow networks are considered (2 or 3 layers), and the optimization strategy, including the grid search strategy for hyperparameters selection, is not described. Minor issue: positioning with respect to related works is limited. For example, layer redundancy (which is the opposite of diversity) has been considered in the context of network pruning: https://openaccess.thecvf.com/content_CVPR_2019/papers/He_Filter_Pruning_via_Geometric_Median_for_Deep_Convolutional_Neural_Networks_CVPR_2019_paper.pdf