Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks