<p>You're using the Adam optimizer (<a href="https://arxiv.org/abs/1412.6980" rel="nofollow noreferrer">https://arxiv.org/abs/1412.6980</a>) for optimization. Adam has two state variables to store statistics about the gradients which are the same size as the parameters (Algorithm 1), which is your two additional variables per parameter variable. The optimizer itself has a few hyperparameters, among them β<sub>1</sub> and β<sub>2</sub>, which I guess are in your case stored as variables.</p>
Why are there two additional variables, in the checkpoint, for each layer?
<p>I created a <em>convolutional neural network</em> with three convolutional layers and two fully connected layers. I used <code>tf.train.saver()</code> to save the variables. 
When I use <code>inspect_checkpoint.py</code> to check the variables saved in the checkpoint file. Why are there two additional variables saved for each layer, like <code>Adam_1</code> and <code>Adam</code>? Also, what are <code>beta1_power</code> and <code>beta2_power</code>? </p>

<pre><code>conv_layer1_b  (DT_FLOAT)  [32]

conv_layer1_w  (DT_FLOAT)  [1,16,1,32]

conv_layer1_b/Adam  (DT_FLOAT)  [32]

conv_layer1_w/Adam (DT_FLOAT) [1,16,1,32]

conv_layer1_w/Adam_1 (DT_FLOAT) [1,16,1,32]

conv_layer1_b/Adam_1 (DT_FLOAT) [32]

conv_layer3_w/Adam (DT_FLOAT) [1,16,64,64]

conv_layer3_w (DT_FLOAT) [1,16,64,64]

conv_layer3_b/Adam_1 (DT_FLOAT) [64]

conv_layer3_b (DT_FLOAT) [64]

conv_layer3_b/Adam (DT_FLOAT) [64]

conv_layer3_w/Adam_1 (DT_FLOAT) [1,16,64,64]

conv_layer2_w/Adam_1 (DT_FLOAT) [1,16,32,64]

conv_layer2_w/Adam (DT_FLOAT) [1,16,32,64]

conv_layer2_w (DT_FLOAT) [1,16,32,64]

conv_layer2_b/Adam_1 (DT_FLOAT) [64]

conv_layer2_b (DT_FLOAT) [64]

conv_layer2_b/Adam (DT_FLOAT) [64]

beta1_power (DT_FLOAT) []

beta2_power (DT_FLOAT) []

NN1_w (DT_FLOAT) [2432,512]

NN1_b (DT_FLOAT) [512]

NN1_w/Adam_1 (DT_FLOAT) [2432,512]

NN1_b/Adam_1 (DT_FLOAT) [512]

NN1_w/Adam (DT_FLOAT) [2432,512]

NN1_b/Adam (DT_FLOAT) [512]

NN2_w (DT_FLOAT) [512,2]

NN2_b (DT_FLOAT) [2]

NN2_w/Adam_1 (DT_FLOAT) [512,2]

NN2_b/Adam_1 (DT_FLOAT) [2]

NN2_w/Adam (DT_FLOAT) [512,2]

NN2_b/Adam (DT_FLOAT) [2]
</code></pre>