title: Resume training with Adam optimizer in Keras
body: <p>My question is quite straightforward but I can't find a definite answer online (so far).</p>
  
  <p>I have saved the weights of a keras model trained with an adam optimizer after a defined number of epochs of training using:</p>
  
  <pre><code>callback = tf.keras.callbacks.ModelCheckpoint(filepath=path, save_weights_only=True)
  model.fit(X,y,callbacks=[callback])
  </code></pre>
  
  <p>When I resume training after closing my jupyter, can I simply use:</p>
  
  <pre><code>model.load_weights(path)
  </code></pre>
  
  <p>to continue training.</p>
  
  <p>Since Adam is dependent on the epoch number (such as in the case of learning rate decay), I would like to know the easiest way to resume training in the same conditions as before.</p>
  
  <p>Following ibarrond's answer, I have written a small custom callback.</p>
  
  <pre><code>optim = tf.keras.optimizers.Adam()
  model.compile(optimizer=optim, loss='categorical_crossentropy',metrics=['accuracy'])
  
  weight_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_path, save_weights_only=True, verbose=1, save_best_only=False)
  
  class optim_callback(tf.keras.callbacks.Callback):
      '''Custom callback to save optimiser state'''
  
            def on_epoch_end(self,epoch,logs=None):
                  optim_state = tf.keras.optimizers.Adam.get_config(optim)
                  with open(optim_state_pkl,'wb') as f_out:                  
                         pickle.dump(optim_state,f_out)
  
  model.fit(X,y,callbacks=[weight_callback,optim_callback()])
  </code></pre>
  
  <p>When I resume training:</p>
  
  <pre><code>model.load_weights(checkpoint_path)
  with open(optim_state_pkl,'rb') as f_out:                  
                      optim_state = pickle.load(f_out)
  tf.keras.optimizers.Adam.from_config(optim_state)
  </code></pre>
  
  <p>I would just like to check if this is correct. Many thanks again!!</p>
  
  <p>Addendum: On further reading of the default <a href="https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/Adam" rel="nofollow noreferrer">Keras implementation</a> of Adam and the <a href="https://arxiv.org/abs/1412.6980" rel="nofollow noreferrer">original Adam paper</a>, I believe that the default Adam is not dependent on epoch number but only on the iteration number. Therefore, this is unnecessary. However, the code may still be useful for anyone who wishes to keep track of other optimisers.</p>