<p>The answer is Yes and No.</p>

<p>Why Yes, according to the paper <a href="https://arxiv.org/pdf/1607.06450.pdf" rel="nofollow noreferrer">layer normalization</a>, in section it clearly indicates the usage of BN in RNNs. </p>

<p>Why No? The distribution of output at each timestep has to be stored and calcualted to conduct BN. Imagine that you pad the sequence input so all examples have the same length, so if the predict case is longer than all training cases, at some time step you have no mean/std of the output distribution summarized from the SGD training procedure.</p>

<p>Meanwhile, at least in Keras, I believe the BN layer only consider the normalization in vertical direction, i.e., the sequence output. The horizontal direction, i.e., hidden_status, cell_status, are not normalized. Correct me if I an wrong here.</p>

<p>In multiple-layer RNNs, you may consider using layer normalization tricks.</p>
Is it normal to use batch normalization in RNN & LSTM?
<p>I know in regular neural nets people use batch norm before activation and it will reduce the reliance on good weight initialization. I wonder if it would do the same to RNN/lstm RNN when i use it. Does anyone have any experience with it?</p>