<p><strong>Update</strong>: the LayerNormalization implementation I was using was <em>inter-layer</em>, not <em>recurrent</em> as in the original paper; results with latter may prove superior.</p>

<hr>

<p><code>BatchNormalization</code> <em>can</em> work with LSTMs - the linked SO gives false advice; in fact, in my application of EEG classification, it dominated <code>LayerNormalization</code>. Now to your case:</p>

<ul>
<li><em>"Can I add it before <code>Conv1D</code>"</em>? Don't - instead, standardize your data beforehand, else you're employing an inferior variant to do the same thing</li>
<li>Try both: <code>BatchNormalization</code> before an activation, and after - apply to both <code>Conv1D</code> and <code>LSTM</code></li>
<li>If your model is exactly as you show it, <code>BN</code> after <code>LSTM</code> may be counterproductive per ability to introduce noise, which can confuse the classifier layer - but this is about being one layer before output, not <code>LSTM</code></li>
<li>If you aren't using stacked <code>LSTM</code> with <code>return_sequences=True</code> preceding <code>return_sequences=False</code>, you can place <code>Dropout</code> anywhere - before <code>LSTM</code>, after, or both</li>
<li><strong>Spatial Dropout</strong>: drop <em>units</em> / <em>channels</em> instead of random activations (see bottom); was shown more effective at reducing coadaptation in CNNs in paper by <a href="https://arxiv.org/pdf/1411.4280.pdf" rel="nofollow noreferrer">LeCun, et al</a>, w/ ideas applicable to RNNs. Can considerably increase convergence time, but also improve performance</li>
<li><code>recurrent_dropout</code> is still preferable to <code>Dropout</code> for <code>LSTM</code> - <em>however</em>, you can do both; just do not use with with <code>activation='relu'</code>, for which <code>LSTM</code> is unstable per a bug</li>
<li>For data of your dimensionality, any sort of <code>Pooling</code> is redundant and may harm performance; scarce data is better transformed via a non-linearity than simple averaging ops</li>
<li>I strongly recommend a <code>SqueezeExcite</code> block after your Conv; it's a form of self-attention - see <a href="https://arxiv.org/abs/1709.01507" rel="nofollow noreferrer">paper</a>; my implementation for 1D below</li>
<li>I also recommend trying <code>activation='selu'</code> with <code>AlphaDropout</code> and <code>'lecun_normal'</code> initialization, per paper <a href="https://arxiv.org/abs/1706.02515" rel="nofollow noreferrer">Self Normalizing Neural Networks</a></li>
<li><em>Disclaimer</em>: above advice may not apply to NLP and embed-like tasks</li>
</ul>

<p>Below is an example template you can use as a starting point; I also recommend the following SO's for further reading: <a href="https://stackoverflow.com/questions/48714407/rnn-regularization-which-component-to-regularize/58868383#58868383">Regularizing RNNs</a>, and <a href="https://stackoverflow.com/questions/59017288/how-to-visualize-rnn-lstm-gradients-in-keras-tensorflow/59017289#59017289">Visualizing RNN gradients</a></p>

<pre class="lang-py prettyprint-override"><code>from keras.layers import Input, Dense, LSTM, Conv1D, Activation
from keras.layers import AlphaDropout, BatchNormalization
from keras.layers import GlobalAveragePooling1D, Reshape, multiply
from keras.models import Model
import keras.backend as K
import numpy as np


def make_model(batch_shape):
    ipt = Input(batch_shape=batch_shape)
    x   = ConvBlock(ipt)
    x   = LSTM(16, return_sequences=False, recurrent_dropout=0.2)(x)
    # x   = BatchNormalization()(x)  # may or may not work well
    out = Dense(1, activation='relu')

    model = Model(ipt, out)
    model.compile('nadam', 'mse')
    return model

def make_data(batch_shape):  # toy data
    return (np.random.randn(*batch_shape),
            np.random.uniform(0, 2, (batch_shape[0], 1)))

batch_shape = (32, 21, 20)
model = make_model(batch_shape)
x, y  = make_data(batch_shape)

model.train_on_batch(x, y)
</code></pre>

<p><strong>Functions used</strong>:</p>

<pre class="lang-py prettyprint-override"><code>def ConvBlock(_input):  # cleaner code
    x   = Conv1D(filters=10, kernel_size=3, padding='causal', use_bias=False,
                 kernel_initializer='lecun_normal')(_input)
    x   = BatchNormalization(scale=False)(x)
    x   = Activation('selu')(x)
    x   = AlphaDropout(0.1)(x)
    out = SqueezeExcite(x)    
    return out

def SqueezeExcite(_input, r=4):  # r == "reduction factor"; see paper
    filters = K.int_shape(_input)[-1]

    se = GlobalAveragePooling1D()(_input)
    se = Reshape((1, filters))(se)
    se = Dense(filters//r, activation='relu',    use_bias=False,
               kernel_initializer='he_normal')(se)
    se = Dense(filters,    activation='sigmoid', use_bias=False, 
               kernel_initializer='he_normal')(se)
    return multiply([_input, se])
</code></pre>

<hr>

<p><strong>Spatial Dropout</strong>: pass <code>noise_shape = (batch_size, 1, channels)</code> to <code>Dropout</code> - has the effect below; see <a href="https://gist.github.com/OverLordGoldDragon/4d32dffd8f5d9123a886c21558da60a8" rel="nofollow noreferrer">Git gist</a> for code:</p>

<p><img src="https://i.stack.imgur.com/gqdNB.png" width="500"></p>
Batch normalization layer for CNN-LSTM
<p>Suppose that I have a model like this (this is a model for time series forecasting):</p>

<pre><code>ipt   = Input((data.shape[1] ,data.shape[2])) # 1
x     = Conv1D(filters = 10, kernel_size = 3, padding = 'causal', activation = 'relu')(ipt) # 2
x     = LSTM(15, return_sequences = False)(x) # 3
x = BatchNormalization()(x) # 4
out   = Dense(1, activation = 'relu')(x) # 5
</code></pre>

<p>Now I want to add batch normalization layer to this network. Considering the fact that <a href="https://stackoverflow.com/questions/45493384/is-it-normal-to-use-batch-normalization-in-rnn-lstm-rnn/45495331">batch normalization doesn't work with LSTM</a>, Can I add it before <code>Conv1D</code> layer? I think it's rational to have a batch normalization layer after <code>LSTM</code>.</p>

<p>Also, where can I add Dropout in this network? The same places? (after or before batch normalization?)</p>

<ul>
<li>What about adding <code>AveragePooling1D</code> between <code>Conv1D</code> and <code>LSTM</code>? Is it possible to add batch normalization between <code>Conv1D</code> and <code>AveragePooling1D</code> in this case without any effect on <code>LSTM</code> layer?</li>
</ul>