title: How does binary cross entropy loss work on autoencoders?
body: <p>I wrote a vanilla autoencoder using only <code>Dense</code> layer. 
  Below is my code:</p>
  
  
  
  <pre class="lang-py prettyprint-override"><code>iLayer = Input ((784,))
  layer1 = Dense(128, activation='relu' ) (iLayer)
  layer2 = Dense(64, activation='relu') (layer1)
  layer3 = Dense(28, activation ='relu') (layer2)
  layer4 = Dense(64, activation='relu') (layer3)
  layer5 = Dense(128, activation='relu' ) (layer4)
  layer6 = Dense(784, activation='softmax' ) (layer5)
  model = Model (iLayer, layer6)
  model.compile(loss='binary_crossentropy', optimizer='adam')
  
  (trainX, trainY), (testX, testY) =  mnist.load_data()
  print ("shape of the trainX", trainX.shape)
  trainX = trainX.reshape(trainX.shape[0], trainX.shape[1]* trainX.shape[2])
  print ("shape of the trainX", trainX.shape)
  model.fit (trainX, trainX, epochs=5, batch_size=100)
  </code></pre>
  
  <h2>Questions:</h2>
  
  <p>1) <code>softmax</code> provides probability distribution. Understood. This means, I would have a vector of 784 values with probability between 0 and 1. For example [ 0.02, 0.03..... upto 784 items], summing all 784 elements provides 1. </p>
  
  <p>2) I don't understand how the binary crossentropy works with these values. Binary cross entropy is for two values of output, right?</p>