We present the iterative two-pass decomposition flow to accelerate existing convolutional neural networks (CNNs).   The proposed rank selection algorithm can effectively determine the proper ranks of the target convolutional layers for the low rank approximation. Our two-pass CP-decomposition helps prevent from the instability problem. The iterative flow makes the decomposition of the deeper networks systematic. The experiment results shows that VGG16 can be accelerated with a 6.2x measured speedup while the accuracy drop remains only 1.2%.
 Deep learning has become of vital importance in a variety of artificial intelligence applications. Recently, convolutional neural networks (CNNs) have been widely applied to have the breakthrough in improving the recognition accuracy for challenging computer vision tasks such as image classification, localization, object detection, and so on BID19 ; BID11 ; BID20 ; BID3 ). However, those significant achievements using CNNs come with the cost of larger network size and higher computational complexity, which leads to an increasing difficulty for deploying to resource constrained edge devices, or even for the fast computation on the cloud servers. This paper addresses the acceleration of the existing CNN models to cope with such burden. The iterative two-pass decomposition flow has been presented to accelerate existing deep CNNs. Our two-pass decomposition effectively prevents from the CP instability. The Rank Selection algorithm provides the fine-grained rank configuration to achieve the target speedup while maintaining the accuracy. The experiment results show that VGG16 can be accelerated by 6.2 times with the accuracy drop of only 1.20% and the size reduction of 85%. In addition, ResNet50 can be speeded up by 1.35 times with the accuracy drop of 1.51% and the size reduction of 48%.The future works include the improvement of the grouping scheme and the decomposing order, and a smarter rank selection with non-linear fitness estimation. In addition, accelerating 1×1 convolutional layers will also be considered for the further improvement on the advanced CNNs.APPENDIX A FIG6 shows the initial fitnesses of the baseline rank configuration and fitness-based configuration. Note that the layer order is sorted by the fitnesses of the baseline rank configuration. Because there is only a slight difference between the sorting of the two configuration, the fitness-based grouping scheme is based on the sorting of the baseline rank configuration. TAB1
We present the iterative two-pass CP decomposition flow to effectively accelerate existing convolutional neural networks (CNNs). The paper proposes a novel workflow for acceleration and compression of CNNs and also proposes a way to determine the target rank of each layer given the target overall acceleration.  This paper addresses the problem of learning a low rank tensor filter operation for filtering layers in deep neural networks (DNNs). 