title: Freeze layers with the multi_gpu_model in Keras
body: <p>I'm trying to fine-tune a modified InceptionV3 model in Keras.</p>
  
  <p>I follow the example "Fine-tune InceptionV3 on a new set of classes" on <a href="https://keras.io/applications/" rel="nofollow noreferrer">this page</a>.</p>
  
  <p>So I first trained the top dense layers that were added to the InceptionV3 base model with the following code:</p>
  
  <pre><code>model = Model(inputs=base_model.input, outputs=predictions)
  
  for layer in base_model.layers:
      layer.trainable = False
  
  parallel_model = multi_gpu_model(model, gpus=2)
  
  parallel_model.compile(optimizer='rmsprop', loss='categorical_crossentropy')
  
  history = parallel_model.fit_generator(generate_batches(path), steps_per_epoch = num_images/batch_size, epochs = num_epochs)
  </code></pre>
  
  <p>After that, I try to fine-tune the top 2 inception blocks from InceptionV3. And according to the example, what I should do is:</p>
  
  <pre><code>for layer in model.layers[:249]:
     layer.trainable = False
  for layer in model.layers[249:]:
     layer.trainable = True
  
  model.compile(optimizer=SGD(lr=0.0001, momentum=0.9), loss='categorical_crossentropy')
  
  model.fit_generator(...)
  </code></pre>
  
  <p>But I'm using the <code>multi_gpu_model</code>, so I don't know how to freeze the first 249 layers. </p>
  
  <p>I mean, if I freeze the layers in the no-gpu model (like the example), and use <code>parallel_model = multi_gpu_model(model, gpus=2)</code> to freeze the layers in the <code>parallel_model</code>, then the weights in the top dense layers that were just trained and contained in the <code>parallel_model</code> will be overwritten, right?</p>
  
  <p>On the other hand, I tried to directly use <code>for layer in parallel_model.layers[:249]: layer.trainable = False</code>, but when I checked the layers in the <code>parallel_model</code>, it showed:</p>
  
  <pre><code>for i, layer in enumerate(parallel_model.layers):
     print(i, layer.name)
  
  (0, 'input_1')
  (1, 'lambda_1')
  (2, 'lambda_2')
  (3, 'model_1')
  (4, 'dense_3')
  </code></pre>
  
  <p>So what are the 'lambda_1', 'lambda_2' and 'model_1' layers and why it only shows 5 layers in the <code>parallel_model</code>?</p>
  
  <p>More importantly, how to freeze the layers in the <code>parallel_model</code>?</p>