title: How to initialize the variables of the existing Keras Models?
body: <p>I would like to retrain Keras Model, Inception_v3, from scratch.</p>
  
  <p>The model is defined here:
  <a href="https://github.com/keras-team/keras-applications/blob/master/keras_applications/inception_v3.py" rel="nofollow noreferrer">https://github.com/keras-team/keras-applications/blob/master/keras_applications/inception_v3.py</a></p>
  
  <p>I read some posts,</p>
  
  <p>The listed solutions are:</p>
  
  <ol>
  <li><p>Freeze the layers (This is not what I want...)    </p>
  
  <pre><code>for layer in model.layers:
      layer.trainable = False
  </code></pre></li>
  <li><p><a href="https://stackoverflow.com/a/51727616/7748163">https://stackoverflow.com/a/51727616/7748163</a> </p></li>
  </ol>
  
  <p>Reset all layers by checking for initializers:</p>
  
  <pre><code>def reset_weights(model):
      session = K.get_session()
      for layer in model.layers: 
          if hasattr(layer, 'kernel_initializer'):
              layer.kernel_initializer.run(session=session)
          if hasattr(layer, 'bias_initializer'):
              layer.bias_initializer.run(session=session)     
  </code></pre>
  
  <ol start="3">
  <li><p>Use <code>tf.variables_initializer</code></p>
  
  <pre><code>  model = InceptionV3()
    for layer in model.layers:
        sess.run(tf.variables_initializer(layer.weights))
  </code></pre>
  
  <p>Reference: <a href="https://stackoverflow.com/a/56634827/7748163">https://stackoverflow.com/a/56634827/7748163</a></p></li>
  <li><p>The best one I think, but it raises an error.</p>
  
  <pre><code>sess = tf.Session()
  for layer in model.layers: 
       for v in layer.__dict__:
           v_arg = getattr(layer,v)
           if hasattr(v_arg,'initializer'):
               initializer_method = getattr(v_arg, 'initializer')
               initializer_method.run(session=sess)
               print('reinitializing layer {}.{}'.format(layer.name, v))
  </code></pre></li>
  </ol>
  
  <p><strong>However, none of them works for Inception_v3.</strong></p>
  
  <p>The error information is for BatchNorm layer:</p>
  
  <pre><code>tensorflow.python.framework.errors_impl.FailedPreconditionError: Error while reading resource variable batch_normalization_9/moving_mean from Container: localhost. This could mean that the variable was uninitialized. Not found: Resource localhost/batch_normalization_9/moving_mean/N10tensorflow3VarE does not exist.
       [[{{node batch_normalization_9_1/AssignMovingAvg/ReadVariableOp}}]]
       [[metrics_1/categorical_accuracy/Identity/_469]]
  </code></pre>
  
  <p>So, how to re-train the existing Keras Models, and initialize the variables? What is the best practice to re-train a model from Keras applications?</p>
  
  <p>Further discussion:</p>
  
  <p><a href="https://github.com/keras-team/keras/issues/341" rel="nofollow noreferrer">https://github.com/keras-team/keras/issues/341</a></p>