<p>There are a few things you need to address here:</p>

<ol>
<li><p>Is there a good reason not to just use larger batches? Are you trying to implement the <a href="https://arxiv.org/abs/1907.08610" rel="nofollow noreferrer">lookahead optimizer</a> or something?</p></li>
<li><p>You look like you're getting started with TensorFlow. Consider turning on eager execution with <code>tf.enable_eager_execution()</code>. TensorFlow 2.0 is coming soon, don't waste your time messing with <code>tf.Sessions</code>.</p></li>
<li><p>Variables are not differentiable. So accumulating the losses in a variable doesn't make any sense. </p></li>
<li><p>I would make a copy of all the model's variables, and accumulate new values there. Then, after <code>N</code> iterations assign those values back to the model. Something like:</p></li>
</ol>

<pre><code>model = tf.keras.Sequential(...)
vars = model.trainable_variables
weight_acc = [tf.Variable(var) for var in model.trainable_variables]
for n,(batch, label) in enumerate(dataset):
  with tf.GradientTape() as tape:
    pred = model(batch)
    loss = cal_loss(batch, label)
  grads = tape.gradients(loss, vars)
  for g, a in zip(grad, weight_acc):
    a.assign_add(learning_rate*g)
  if n%25 == 0:
    for a, v in zip(weight_acc, vars):
      v.assign_add(lookahead_fraction*(a-v)) 

</code></pre>
How to add after each iteration in tensorflow
<p>I am trying to achieve the following:</p>

<p>compute the losses in the previous 25 predictions and sum them before
computing the gradient. I have tried this:</p>

<pre><code>loss_summation=tf.Variable(0,dtype=tf.dtypes.float32,name="loss")
xentropy=tf.nn.sparse_softmax_cross_entropy_with_logits(labels=next_element[1],logits=logits2,name="xentropy")
loss=tf.math.reduce_sum(tf.reduce_mean(xentropy,name="loss"))
loss_summation=tf.assign(loss_summation,loss_summation+loss)
optimizer = tf.train.AdamOptimizer(learning_rate=self.learning_rate)
 gvs = optimizer.compute_gradients(loss_summation,[vars])
with tf.Session() as sess():
      for i in range(25):
                     b=sess.run([loss_summation])
</code></pre>

<p>However <code>optimizer.compute_gradients()</code> complains that
<code>None values not supported</code>. How can go around this ?</p>

<p>I am actually trying to implement the following function(feedforward of LSTM) in tensorflow to predict the next word given the previous ones</p>

<pre><code>def feedforward(self,x_s,hpre,targets,p_s):


    fts,its,gts,css,ots,output,inputs=[],[],[],[],[],[],[]
        losses=[]
        hprev=hpre
        hts=[hprev]
        loss=0
        losses=[]
        previous_state=p_s
        css.append(previous_state)
        for x,y in zip(x_s,targets):
            k=np.zeros((self.vocab_size,1))
            k[x]=1 
            M_c=np.row_stack((hprev,k))                      
            ft=self.sigmoid(np.dot(self.W1,M_c)+self.b1)
            fts.append(ft)
            it=self.sigmoid(np.dot(self.W2,M_c)+self.b2)
            its.append(it)
            gt=np.tanh(np.dot(self.W3,M_c)+self.b3)
            gts.append(gt)
            cs=(ft*previous_state)+(it*gt)
            previous_state=cs
            css.append(cs)
            ot=self.sigmoid(np.dot(self.W4,M_c)+self.b4)
            ots.append(ot)
            ht=ot*np.tanh(cs)
            hts.append(ht)
            yt=self.softmax(np.dot(self.W5,ht)+self.b5)
            hprev=ht
            output.append(yt)
            inputs.append(M_c)
            loss+=-np.log(yt[y])
            losses.append(loss)

        return fts,its,gts,css,ots,output,hts,loss,hts[-1],css[-1],inputs
</code></pre>

<p><code>x_s</code> is a list of integers representing words. </p>

<pre><code>x_s=[0,1,2,3,4,5,6,7,8....,24]
</code></pre>

<p>targets is the list of integers expected i.e if x_s=0 then next letter is 1</p>

<pre><code>targets=[1,2,3,4,5,6,7,8,9...,25]
</code></pre>

<p>The loss which is a summation of 25 losses is what will be minimized.</p>