title: How to add regularizations in TensorFlow?
body: <p>I found in many available neural network code implemented using TensorFlow that regularization terms are often implemented by manually adding an additional term to loss value.</p>
  
  <p>My questions are:</p>
  
  <ol>
  <li><p>Is there a more elegant or recommended way of regularization than doing it manually?</p></li>
  <li><p>I also find that <code>get_variable</code> has an argument <code>regularizer</code>. How should it be used? According to my observation, if we pass a regularizer to it (such as <code>tf.contrib.layers.l2_regularizer</code>, a tensor representing regularized term will be computed and added to a graph collection named <code>tf.GraphKeys.REGULARIZATOIN_LOSSES</code>. Will that collection be automatically used by TensorFlow (e.g. used by optimizers when training)? Or is it expected that I should use that collection by myself?</p></li>
  </ol>