title: Param Tuning with Keras and Hyperas
body: <p>I have been working with a Python library called Hyperas which is a hyperopt/keras wrapper for tuning parameters in a Keras model. My question is about the output from Hyperas.</p>
  
  <p>I have read the documentation and source code, but cannot seem to figure out what the output means or how to interpret. It prints the following line after it is done optimizing:</p>
  
  <pre><code>{'batch_size': 3, 'optimizer': 1, 'l2': 0.7446290506725413, 'output_dim': 3, 'output_dim_1': 0, 'l2_1': 0.12090219120950985}
  </code></pre>
  
  <p>Why are there two dictionary values for output_dim, even though there is only one output_dim param in my code? How do I interpret everything elese?</p>
  
  <pre><code>def model(X_train, X_test, y_train, y_test, max_features, maxlen, class_weight_dict):
          model = Sequential()
          model.add(Embedding(max_features, output_dim = {{choice([32,64,128,256,512])}}, input_length=maxlen))
          model.add(LSTM({{choice([32,64,128,256,512])}},W_regularizer=l2({{uniform(0, 1)}})))
          model.add(Dropout({{uniform(0, 1)}}))
          model.add(Dense(138))
          model.add(Activation('softmax'))
  
          model.compile(loss='categorical_crossentropy',
                        optimizer={{choice(['rmsprop', 'adam', 'sgd'])}},
                        metrics=['accuracy'])
  
          early_stopping = EarlyStopping(monitor='val_loss', patience=4)
          checkpointer = ModelCheckpoint(filepath='keras_weights.hdf5',
                                         verbose=1,
                                         save_best_only=True)
  
          model.fit(X_train, y_train,
                    batch_size={{choice([32,16,64,128,256,512])}},
                    validation_data=(X_test, y_test),
                    nb_epoch=100,
                    class_weight=class_weight_dict,
                    callbacks=[early_stopping, checkpointer])
  
          score, acc = model.evaluate(X_test, y_test)
          print('Test score:', score)
          print('Test accuracy:', acc)
          return {'loss': -acc, 'status': STATUS_OK, 'model': model}
  
      if __name__ == '__main__':
          best_run, best_model = optim.minimize(model=model,
                                                data=data,
                                                algo=tpe.suggest,
                                                max_evals=10,
                                                trials=Trials())
          print(best_run)
          print(best_model)
  </code></pre>