title: Keras: Combining data generators to handle image + text
body: <p>I am working on a multilabel classification model where I am trying to combine two models, a CNN and a text-classifier into one model using Keras and train them together, like so:</p>
  
  <pre><code>#cnn_model is a vgg16 model
  
  #text_model looks as follows:
  ### takes the vectorized text as input
  text_model = Sequential()
  text_model .add(Dense(vec_size, input_shape=(vec_size,), name='aux_input'))
  
  ## merging both models
  merged = Merge([cnn_model, text_model], mode='concat')
  
  ### final_model takes the combined models and adds a sofmax classifier to it
  final_model = Sequential()
  final_model.add(merged)
  final_model.add(Dense(n_classes, activation='softmax'))
  </code></pre>
  
  <p>As such, I am working with an ImageDataGenerator to process the images and the respective labels.</p>
  
  <p>For the images I am using a custom helper function that reads images into the model via paths provided by pandas dataframes - one for training (df_train) and one for validation (df_validation). The dataframes also provide the final labels for the model in the "label_vec" column:</p>
  
  <pre><code># From https://github.com/keras-team/keras/issues/5152
  def flow_from_dataframe(img_data_gen, in_df, path_col, y_col, **dflow_args):
      base_dir = os.path.dirname(in_df[path_col].values[0])
      print('## Ignore next message from keras, values are replaced anyways')
      df_gen = img_data_gen.flow_from_directory(base_dir, class_mode = 'sparse', **dflow_args)
      df_gen.filenames = in_df[path_col].values
      df_gen.classes = numpy.stack(in_df[y_col].values)
      df_gen.samples = in_df.shape[0]
      df_gen.n = in_df.shape[0]
      df_gen._set_index_array()
      df_gen.directory = '' # since we have the full path
      print('Reinserting dataframe: {} images'.format(in_df.shape[0]))
      return df_gen 
  
  from keras.applications.vgg16 import preprocess_input
  
  train_datagen = keras.preprocessing.image.ImageDataGenerator(preprocessing_function=preprocess_input)                                                  horizontal_flip=True)
  validation_datagen = keras.preprocessing.image.ImageDataGenerator(preprocessing_function=preprocess_input)#rescale=1./255)
  
  train_generator = flow_from_dataframe(train_datagen, df_train,
                                                       path_col = 'filename',
                                                       y_col = 'label_vec', 
  
                                                      target_size=(224, 224), batch_size=128, shuffle=False)
  validation_generator = flow_from_dataframe(validation_datagen, df_validation,
                                                       path_col = 'filename',
                                                       y_col = 'label_vec', 
                                                           target_size=(224, 224), batch_size=64, shuffle=False)
  </code></pre>
  
  <p>Now I am trying to provide my one-hot-encoded text vectors (i.e. <code>[0,0,0,1,0,0]</code>) to the model, which are also stored in a pandas dataframe. </p>
  
  <p>Since my train_generator provides me with the image and label data, I am now looking for a solution to combine this generator with a generator which allows me to additionally feed the respective text-vector</p>