title: Creating a new Sequential model inside a for loop (using Keras)
body: <p>I want to try training my model with different hyperparameters, so I've setup a series of nested for loops to iterate through them.  </p>
  
  <pre><code>model = None
  batch_generator = None
  
  for sequence_length in all_sequence_length:
      for label_periods in all_label_periods:
          for num_layers in all_num_layers:
              for num_units in all_num_units:
                  loadFiles()
                  createmodel()
                  trainmodel()
  </code></pre>
  
  <p>The first iteration creates a model like this:</p>
  
  <pre><code>Layer (type)                 Output Shape              Param #
  =================================================================
  cu_dnnlstm_1 (CuDNNLSTM)     (None, 100, 75)           45300
  _________________________________________________________________
  dropout_1 (Dropout)          (None, 100, 75)           0
  _________________________________________________________________
  cu_dnnlstm_2 (CuDNNLSTM)     (None, 100, 75)           45600
  _________________________________________________________________
  dropout_2 (Dropout)          (None, 100, 75)           0
  _________________________________________________________________
  cu_dnnlstm_3 (CuDNNLSTM)     (None, 100, 75)           45600
  _________________________________________________________________
  dropout_3 (Dropout)          (None, 100, 75)           0
  _________________________________________________________________
  cu_dnnlstm_4 (CuDNNLSTM)     (None, 100, 75)           45600
  _________________________________________________________________
  dropout_4 (Dropout)          (None, 100, 75)           0
  _________________________________________________________________
  cu_dnnlstm_5 (CuDNNLSTM)     (None, 100, 75)           45600
  _________________________________________________________________
  dropout_5 (Dropout)          (None, 100, 75)           0
  _________________________________________________________________
  dense_1 (Dense)              (None, 3)                 228
  =================================================================
  </code></pre>
  
  <p>I then call <code>model.fit_generator()</code> to train the model and that executes fine. The model is then created again in the next loop iteration and the summary looks like this:</p>
  
  <pre><code>Layer (type)                 Output Shape              Param #
  =================================================================
  cu_dnnlstm_6 (CuDNNLSTM)     (None, 100, 75)           45300
  _________________________________________________________________
  dropout_6 (Dropout)          (None, 100, 75)           0
  _________________________________________________________________
  cu_dnnlstm_7 (CuDNNLSTM)     (None, 100, 75)           45600
  _________________________________________________________________
  dropout_7 (Dropout)          (None, 100, 75)           0
  _________________________________________________________________
  cu_dnnlstm_8 (CuDNNLSTM)     (None, 100, 75)           45600
  _________________________________________________________________
  dropout_8 (Dropout)          (None, 100, 75)           0
  _________________________________________________________________
  cu_dnnlstm_9 (CuDNNLSTM)     (None, 100, 75)           45600
  _________________________________________________________________
  dropout_9 (Dropout)          (None, 100, 75)           0
  _________________________________________________________________
  cu_dnnlstm_10 (CuDNNLSTM)     (None, 100, 75)           45600
  _________________________________________________________________
  dropout_10 (Dropout)          (None, 100, 75)           0
  _________________________________________________________________
  dense_2 (Dense)              (None, 3)                 228
  =================================================================
  </code></pre>
  
  <p>You'll see the layer IDs have incremented, which I'm surprised about, as I create a new Sequential model for the model variable, so I'd have expected the same summary as the first.</p>
  
  <p>When I call <code>model.fit_generator()</code> I get this error:</p>
  
  <blockquote>
    <p>InvalidArgumentError (see above for traceback): You must feed a value
    for placeholder tensor 'cu_dnnlstm_1_input' with dtype float and shape
    [?,100,74]</p>
  </blockquote>
  
  <p>You'll see it's expecting an input for <code>cu_dnnlstm_1_input</code>, which was the input on the model of the first iteration, not cu_dnnlstm_6 on the second model.  My code for creating the model is done in a function:</p>
  
  <pre><code>def createmodel():
  
      global model
  
      model = Sequential()
      model.add( CuDNNLSTM(units=num_units, return_sequences=True, input_shape=(sequence_length, features_size) ) )
  
      for _ in range(num_layers):
          model.add( Dropout(dropout_rate) )
          model.add( CuDNNLSTM(units=num_units, return_sequences=True) )
  
      model.add( Dropout(dropout_rate) )
      model.add( CuDNNLSTM(units=num_units, return_sequences=False) )
  
      model.add( Dropout(dropout_rate) )
      model.add( Dense(labels_size) )
  
      model.compile(loss='mean_absolute_error', optimizer='adam')
  
      model.summary()
  </code></pre>
  
  <p>The model is trained with this function:</p>
  
  <pre><code>def trainmodel():
  
      global model
  
      model.fit_generator(generator=batch_generator,
          epochs=num_epochs,
          steps_per_epoch=num_steps_per_epoch,
          validation_data=validation_data_tuple,
          callbacks=callbacks)
  </code></pre>
  
  <p>Can anyone spot my 'deliberate' mistake?</p>