title: Training two networks simultaneously with keras
body: <p>Let's say one wants to train two slightly different networks simultaneously with the same dataset.
  A way to go is through the functional API: <a href="https://keras.io/getting-started/functional-api-guide/" rel="nofollow noreferrer">https://keras.io/getting-started/functional-api-guide/</a>.
  Here is how it would work on a simple architecture (just two multilayer dense networks with same input/output):</p>
  
  <pre><code>same_input = Input(shape=(n_feat, ), name='input')
  
  x = Dense(n_feat//2, kernel_initializer='uniform', activation='relu')(same_input)
  x = Dense(n_feat//128, kernel_initializer='uniform', activation='elu')(x)
  x = Dropout(dropout_val)(x)
  x = BatchNormalization()(x)
  x = Dense(n_feat//256, kernel_initializer='uniform', activation='relu')(x)
  x_out = Dense(out_dim, kernel_initializer='normal', name='output_x')(x)
  
  y = Dense(n_feat//2, kernel_initializer='uniform', activation='relu')(same_input)
  y = Dropout(dropout_val)(y)
  y = Dense(n_feat//128, kernel_initializer='uniform', activation='elu')(y)
  .... (some difference)
  y = Dropout(dropout_val)(y)
  y = BatchNormalization()(y)
  y_out = Dense(out_dim, kernel_initializer='uniform', name='output_y')(y)
  
  
  model = Model(inputs=same_input, 
               outputs=[x_out, y_out])
  
  model.compile(optimizer='rmsprop',
                loss={'output_x': 'mean_squared_error',
                     'output_y': 'mean_absolute_error'},
                loss_weights={'output_x': 1,
                              'output_y': 1.})
  
  model.fit({'input': X},
           {'output_x': Y, 'output_x': Y},
           epochs=fit_pars['num_ep'],
           batch_size=b_size,
           validation_split=valid_split,
           callbacks=callbacks,
           verbose=2)
  
  network_predict = np.array(model.predict(X_test, batch_size=b_size))
  </code></pre>
  
  <p>However, <strong>I am not absolutely sure whether this is correct, and whether there is a way to do it right on Keras.</strong> The total loss that is used to optimize the network will be the sum of the two losses, and that imposes an interference between the two networks.
  <strong>Can someone tell if this is right or wrong, and whether there is a way to train the networks independently yet concurrently?</strong></p>