In this paper, we propose a novel regularization method, RotationOut, for neural networks. 
 Different from Dropout that handles each neuron/channel independently, RotationOut regards its input layer as an entire vector and introduces regularization by randomly rotating the vector. 
 RotationOut can also be used in convolutional layers and recurrent layers with a small modification.
 We further use a noise analysis method to interpret the difference between RotationOut and Dropout in co-adaptation reduction. 
 Using this method, we also show how to use RotationOut/Dropout together with Batch Normalization. 
 Extensive experiments in vision and language tasks are conducted to show the effectiveness of the proposed method. 
 Codes will be available.
We propose a regularization method for neural network and a noise analysis method This paper proposes a new regularization method to mitigate the overfitting issue of deep neural networks by rotating features with a random rotation matrix to reduce co-adaptation. This paper proposes a novel regularization method for training neural networks, which adds noise neurons in an inter-dependent fashion.