title: how to keep the values of tensors in each epoch in one layer and pass it to Next epoch in tensorflow
body: <p>I have a general question.</p>
  <p>I am developing a new layer to incorporate into an autoencoder. To be more specific, the layer is something like the <code>KCompetitive</code> class over <a href="https://github.com/hugochan/KATE/blob/master/autoencoder/utils/keras_utils.py" rel="nofollow noreferrer">here</a>. What I want is that I need to save the output of this layer in a variable let's call it <code>previous_mat_values</code>, and then pass it to that same layer in the next epoch as well.</p>
  <p>To put it another way, I want to be able to save the output of this layer of <code>epoch 1</code> in one variable, and then in <code>epoch 2</code>, again use that same matrix.</p>
  <p>So the question arises what would be the values of this matrix in the first epoch because it does not have the output of that layer yet. We can initialize an array with the same shape of the weight matrix but with values of <code>0</code> I will do like this.</p>
  <pre><code>previous_mat_values = tf.zeros_like(weight_tensor)
  </code></pre>
  <p>So the steps are like this:</p>
  <ol>
  <li><p>In the first epoch, <code>previous_mat_values</code> and <code>weight_mat</code> will pass to the layer</p>
  <p>1.a at the end of the function of that layer, the final value of which we call it <code>modified_weight_mat</code> will save into the <code>previous_mat_values</code></p>
  <p><code>previous_mat_values = modified_weight_mat</code></p>
  </li>
  <li><p>In the second epoch, <code>previous_mat_values</code> and <code>weight_mat</code> will pass to the layer, however, <code>previous_mat_values</code> has the values saved in the first epoch.</p>
  </li>
  </ol>
  <p>I don't have any problem passing <code>weight_mat</code> and doing stuff related to that. The only issue here is that how we can save the values of <code>previous_mat_values</code> in each epoch and pass it to the next epoch.</p>
  <p>I was thinking to create a global tensor variable in the <code>class of that layer</code> and initialize it with zero, but I don't think it will help to keep the values of the previous epoch into the second epoch.</p>
  <p>Do you have any idea how can I implement this?</p>
  <p>Please let me know if my explanations are not clear.</p>
  <p><strong>Update 1:</strong></p>
  <p>This is the implementation of the layer:</p>
  <pre><code>class KCompetitive(Layer):
      '''Applies K-Competitive layer.
      # Arguments
      '''
      def __init__(self, topk, ctype, **kwargs):
          self.topk = topk
          self.ctype = ctype
          self.uses_learning_phase = True
          self.supports_masking = True
          super(KCompetitive, self).__init__(**kwargs)
  
      def call(self, x):
          if self.ctype == 'ksparse':
              return K.in_train_phase(self.kSparse(x, self.topk), x)
          elif self.ctype == 'kcomp':
              return K.in_train_phase(self.k_comp_tanh(x, self.topk), x)
          else:
              warnings.warn(&quot;Unknown ctype, using no competition.&quot;)
              return x
  
      def get_config(self):
          config = {'topk': self.topk, 'ctype': self.ctype}
          base_config = super(KCompetitive, self).get_config()
          return dict(list(base_config.items()) + list(config.items()))
  
      def k_comp_tanh(self, x, topk, factor=6.26):
          ###Some modification on x so now the x becomes 
          x= x+1
          res = x
          return res
  </code></pre>
  <p><strong>Update 2</strong></p>
  <p>For more clarification I will add this:</p>
  <p><strong>sample 1 of data:</strong></p>
  <pre><code>x_prev = zero
  mask = tf.greate(x, x_prev)   # x here related to sample 1
  x_modified = x[mask]
  x_prev = x_modified
  </code></pre>
  <p><strong>Sample 2 of data:</strong></p>
  <pre><code>mask = tf.greater(x, x_prev)   # x here related to sample 2  and 
  x_prev is from previous sample
  x_modified = x[mask]
  x_prev = x_modified
  </code></pre>