<p>There is a big difference between <code>tf.nn.batch_normalization</code> and <code>tf.layers.batch_normalization</code>. See <a href="https://stackoverflow.com/questions/48949318/what-is-the-difference-between-the-tensorflow-batch-normalization-implementation/48953548#48953548">my answer here</a>. So you have made the right choice by using the <code>layers</code> version. Now, on your questions:</p>

<ol>
<li><code>renorm_momentum</code> only has an effect is you use <a href="https://arxiv.org/abs/1702.03275" rel="nofollow noreferrer">batch renormalization</a> by setting the <code>renorm</code> argument to <code>True</code>. You can ignore this if using default batch normalization.</li>
<li>Short answer: You can literally copy that code snippet. Put it exactly where you would normally call <code>optimizer.minimize</code>.</li>
</ol>

<p>Long answer on 2.: Batch normalization has two "modes": Training and inference. During training, mean and variance of the current minibatch is used. During inference, this is not desirable (e.g. you might not even use batches as input, so there would be no minibatch statistics). For this reason, moving averages over minibatch means/variances are kept during training. These moving averages are then used for inference.<br>
By default, Tensorflow only executes what it needs to. Those moving averages are not needed for training, so they normally would never be executed/updated. The <code>tf.control_dependencies</code> context manager forces Tensorflow to do the updates every time it computes whatever is in the code block (in this case the cost). Since the cost certainly needs to be computed exactly one per training step, this is a good way of making sure the moving averages are updated.</p>

<p>The code example seems a bit arcane, but in context it would really just be (as an example):</p>

<pre><code>loss = ...
train_step = SomeOptimizer().minimize(loss)
with tf.Session() as sess:
    ....
</code></pre>

<p>becomes</p>

<pre><code>loss = ...
with tf.control_dependencies(tf.get_collection(tf.GraphKeys.UPDATE_OPS)):
    train_step = SomeOptimizer().minimize(loss)
with tf.Session() as sess:
    ....
</code></pre>

<p>Finally, keep in mind to use the correct <code>training</code> argument for batch normalization so that either minibatch statistics or moving averages are used as intended.</p>
Tensorflow batch normalization: difference momentum and renorm_momentum
<p>I want to replicate a network build with the lasagne-library in tensor flow. I'm having some trouble with the batch normalization. 
This is the lasagne documentation about the used batch normalization:
<a href="http://lasagne.readthedocs.io/en/latest/modules/layers/normalization.html?highlight=batchNorm" rel="nofollow noreferrer">http://lasagne.readthedocs.io/en/latest/modules/layers/normalization.html?highlight=batchNorm</a></p>

<p>In tensorflow I found two functions to normalize:</p>

<ol>
<li><a href="https://www.tensorflow.org/api_docs/python/tf/nn/batch_normalization" rel="nofollow noreferrer">https://www.tensorflow.org/api_docs/python/tf/nn/batch_normalization</a></li>
<li><a href="https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization" rel="nofollow noreferrer">https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization</a></li>
</ol>

<p>The first one is simpler but does not let me choose the alpha parameter from lasagne (Coefficient for the exponential moving average of batch-wise means and standard deviations computed during training). I tried using the second function, which has a lot more options, but there are two things I do not understand about it:</p>

<ol>
<li>I am not clear about the difference between momentum and renorm_momentum. If I have a alpha of 0.9 in the lasagne network, can I just set both tensorflow momentums to 0.9 and expect the same behaviour?</li>
<li>The tf documentation notes: </li>
</ol>

<p>when training, the moving_mean and moving_variance need to be updated. By default the update ops are placed in tf.GraphKeys.UPDATE_OPS, so they need to be added as a dependency to the train_op. For example:</p>

<pre><code>  update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
  with tf.control_dependencies(update_ops):
    train_op = optimizer.minimize(loss)
</code></pre>

<p>I do not really understand what is happening here and where I need to put something similar in my code. Can I just put this somewhere before I run the session? What parts of this code piece should I not copy literally but change depending on my code?</p>