<p>AdamOptimizer is using the <a href="https://arxiv.org/pdf/1412.6980v8.pdf" rel="nofollow noreferrer">Adam Optimizer</a> to update the learning rate. Its is an adaptive method compared to the gradient descent which maintains a single learning rate for all weight updates and the learning rate does not change.</p>

<p>Adam has the advantage over the GradientDescent of using the running average (momentum) of the gradients (mean) as well as the running average of the gradient squared.</p>

<p>There is no such thing as which one is the better to use, it is all dependent on your problem, network and data. But generally, Adam has proven itself to be leading and is one of the most commonly used within DL tasks, as it achieves better results and accuracy metrics.</p>
Adam's Optimizer and Gradient Descent
<p>I am trying to understand what is the difference between these Adam Optimizer and Gradient Descent Optimizer and which one is the best to use in which situation. I am looking into TF website, but if you know of place where these are explained in a better and easy to understand way, let me know?</p>