1. When introducing the theoretical results, we should make a detailed comparison with the existing cross-entropy loss results. The current writing method cannot reflect the advantages of square loss. 2. The synthetic experiment in a non-separable case seems to be a problem. Considering the nonlinear expression ability of neural networks, how to explain that the data distribution illustrated in Figure 1 is inseparable from the network model? 3. This paper presents that the loss functions like hinge loss don’t provide reliable information on the prediction confidence. In this regard, there is a lack of references to some relevant literature. [Gao, 2013] has given a detailed analysis of the advantages and disadvantages between the entire margin distribution and the minimum margin. Based on this, [Lyu, 2018] designed a square-type margin distribution loss to improve the generalization ability of DNN.
[Gao, 2013] W. Gao and Z.-H. Zhou. On the doubt about margin explanation of boosting. Artificial Intelligence 203:1-18 2013.
[Lyu, 2018] Shen-Huan Lyu, Lu Wang, and Zhi-Hua Zhou. Improving Generalization of Neural Networks by Leveraging Margin Distribution. http://arxiv.org/abs/1812.10761
2. The synthetic experiment in a non-separable case seems to be a problem. Considering the nonlinear expression ability of neural networks, how to explain that the data distribution illustrated in Figure 1 is inseparable from the network model?