title: I have machine learning data with binary features. How can I force an autoencoder to return binary data?
body: <p>I have a dataset of the following form: A series of M observations of N-dimensional data. In order to obtain latent factors from this data, I wish to make a single hidden-layer autoencoder trained on this data. Every dimension of a single observation is either a 0 or a 1. But the keras Model returns floats. Is there a way to add a layer to enforce a 0 or 1 as output?</p>
  
  <p>I tried using a simple keras Model to solve this problem. It claims good accuracy on the data, but when looking at the raw data it predicts the 0's correctly and often completely ignores the 1's.</p>
  
  <pre class="lang-py prettyprint-override"><code>n_nodes = 50
  input_1 = tf.keras.layers.Input(shape=(x_train.shape[1],))
  x = tf.keras.layers.Dense(n_nodes, activation='relu')(input_1)
  output_1 = tf.keras.layers.Dense(x_train.shape[1], activation='sigmoid')(x)
  
  model = tf.keras.models.Model(input_1, output_1)
  my_optimizer = tf.keras.optimizers.RMSprop()
  my_optimizer.lr = 0.002
  model.compile(optimizer=my_optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
  model.fit(x_train, y_train, epochs=10000)
  predictions = model.predict(x_test)
  </code></pre>
  
  <p>These observations I then validate by looking at all experiments and seeing if a large (>0.1) value is returned for the elements which are 1. The performance is very poor on the 1's.</p>
  
  <p>I have seen that the loss converges around 10000 epochs. However, the autoencoder fails to properly predict almost all 1's in the data set. Even when setting the width of the hidden layer to be identical to the dimensionality of the data (<code>n_nodes = x_train.shape[1]</code>) the autoencoder still gives bad performance, even worsening if I increase the width of the hidden layer.</p>