title: How to constrain a layer to be a probability matrix?
body: <p>I recently read this <a href="https://arxiv.org/pdf/1406.2080.pdf" rel="nofollow noreferrer">paper</a> which deals with noisy labels in convolutional neural networks.</p>
  
  <p>They model label noise by a probability transition matrix which forms a simple 
  constrained linear layer after the softmax output.</p>
  
  <p>So as an example we may have a 3-by-3 probability transition matrix (3 classes):</p>
  
  <p><a href="https://i.stack.imgur.com/3W9B7.jpg" rel="nofollow noreferrer">Example probability transition matrix. The sum of each column has to be 1.</a></p>
  
  <p>This matrix Q is basically trained in the same way as the rest of the network via backpropagation. But it needs to be constrained to be a probability matrix. Quote from the paper: </p>
  
  <blockquote>
    <p>After taking a gradient step with the Q and the model
    weights, we project Q back to the subspace of probability matrices because it represents conditional probabilities.</p>
  </blockquote>
  
  <p>Now I am wondering what is the best way to implement such a layer in tensorflow.
  I have some ideas but i'm not sure what could work or is best procedure.</p>
  
  <p>1) Hard code the constraint in the model before any training is done, something like:</p>
  
  <pre><code>    # ... build conv model without Q
      [...]
      # shape of y_conv (output CNN) assumed to be a [3,1] vector
      y_conv = tf.nn.softmax(y_conv, 0)
      # add linear layer representing Q, no bias
      W_Q = weight_variable([3, 3])
      # add constraint: columns are valid probability distribution
      W_Q = tf.nn.softmax(W_Q, 0)
      # output of model:
      Q_out = tf.matmul(W_Q, y_conv)
      # now compute loss, gradients and start training
  </code></pre>
  
  <p>2) Compute and apply gradients to the whole model (Q included), then apply constraint</p>
  
  <pre><code>    train_op = ...
      constraint_op = tf.assign(W_Q, tf.nn.softmax(W_Q,0))
      sess = tf.session()
      # compute and apply gradients in form of a train_op
      sess.run(train_op)
      sess.run(constraint_op)
  </code></pre>
  
  <p>I think the second approach is more related to the paper quote, but I am not sure to what extend external assignments confuse training. </p>
  
  <p>Or maybe my ideas are bananas. I hope you can give me some advice!</p>