title: L2 regularization of weights in Edward
body: <p>I'm trying to understand how can we use the regularization with <a href="http://edwardlib.org" rel="nofollow noreferrer">Edward models</a>. I'm still new to tensorflow (which is used as the backend of the Edward). Consider the model below,</p>
  
  <pre><code># prior
  w=Normal(loc=tf.zeros((d,c)),scale=tf.ones((d,c)))
  
  # likelihood
  y=Categorical(logits=tf.matmul(X,w))
  
  # posterior
  loc_qw = tf.get_variable("qw/loc", [d, c])
  scale_qw = tf.nn.softplus(tf.get_variable("qw/scale", [d, c]))
  qw = Normal(loc=loc_qw, scale=scale_qw)
  
  # inference 
  inference = ed.KLqp({w: qw, b: qb}, data={X:train_X, y:train_y})
  </code></pre>
  
  <p>I notice that Edward uses regularization losses in its <a href="https://github.com/blei-lab/edward/blob/27eefa0512a3936677dd61b38bcab39d6802e215/edward/inferences/klqp.py" rel="nofollow noreferrer">loss function</a>. 
  <code>loss = -(p_log_lik - kl_penalty - reg_penalty)</code></p>
  
  <p>However, I can't figure out how to apply the regularization losses to the Edward model. How can we add L1 or L2 regularization to the above model?</p>
  
  <p>Thanks!</p>