<blockquote>
  <p>The differences might somehow reflect the discussion whether learning
  rate decay is even needed when applying <a href="https://arxiv.org/abs/1412.6980" rel="nofollow noreferrer">Adam</a>.</p>
</blockquote>

<ol>
<li>Adam updates any parameter with an individual learning rate. This means that every parameter in the network have a specific learning rate associated.</li>
<li>The single learning rates for parameters is computed using the initial learning rate as upper limit. This means that every single learning rate can vary from 0 (no update) to the initial learning rate.</li>
<li>The learning rates adapt themselves during train steps, but if you want to be sure that every update step do not exceed an upper Limit you can than lower your initial (global) learning rate, using exponential decay. </li>
</ol>

<p>So these reasons show why there is a discussion whether learning rate decay with Adam is necessary after all.</p>
Why does the Keras implementation for the Adam optimizer have the decay argument and Tensorflow doesn't?
<p>Why does the Keras implementation for the Adam optimizer have the decay argument and Tensorflow doesn't? And what idea of this argument?</p>