title: Why binary_crossentropy and categorical_crossentropy give different performances for the same problem?
body: <p>I'm trying to train a CNN to categorize text by topic. When I use binary cross-entropy I get ~80% accuracy, with categorical cross-entropy I get ~50% accuracy.</p>
  
  <p>I don't understand why this is. It's a multiclass problem, doesn't that mean that I have to use categorical cross-entropy and that the results with binary cross-entropy are meaningless?</p>
  
  
  
  <pre class="lang-python prettyprint-override"><code>model.add(embedding_layer)
  model.add(Dropout(0.25))
  # convolution layers
  model.add(Conv1D(nb_filter=32,
                      filter_length=4,
                      border_mode='valid',
                      activation='relu'))
  model.add(MaxPooling1D(pool_length=2))
  # dense layers
  model.add(Flatten())
  model.add(Dense(256))
  model.add(Dropout(0.25))
  model.add(Activation('relu'))
  # output layer
  model.add(Dense(len(class_id_index)))
  model.add(Activation('softmax'))
  </code></pre>
  
  <p>Then I compile it either it like this using <code>categorical_crossentropy</code> as the loss function:</p>
  
  <pre class="lang-python prettyprint-override"><code>model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
  </code></pre>
  
  <p>or </p>
  
  <pre class="lang-python prettyprint-override"><code>model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
  </code></pre>
  
  <p>Intuitively it makes sense why I'd want to use categorical cross-entropy, I don't understand why I get good results with binary, and poor results with categorical.</p>