<p>It is not commonly used, though I found <a href="https://arxiv.org/pdf/1603.09025.pdf" rel="nofollow noreferrer">this paper</a> from 2017 shows a way to use batch normalization in the input-to-hidden and the hidden-to-hidden transformations trains faster and generalizes better on some problems.</p>
Is it normal to use batch normalization in RNN & LSTM?
<p>I know in regular neural nets people use batch norm before activation and it will reduce the reliance on good weight initialization. I wonder if it would do the same to RNN/lstm RNN when i use it. Does anyone have any experience with it?</p>