title: How to correctly implement cohen kappa metric in keras?
body: <p>I tried to implement cohen kappa metric for my project using custom metric guide provided by tensorflow for keras (<a href="https://www.tensorflow.org/guide/keras/train_and_evaluate#custom_metrics" rel="nofollow noreferrer">https://www.tensorflow.org/guide/keras/train_and_evaluate#custom_metrics</a>). </p>
  
  <p>After I trained my model and predicted output, I compared results with <code>sklearn.metrics.cohen_kappa_score</code> and it is not same. Can anyone tell how to resolve this problem? Here's the code:</p>
  
  <pre><code>class BinaryKappa(keras.metrics.Metric):
      """Stateful Metric to calculate kappa over all batches.
      Assumes predictions and targets of shape `(samples, 1)`.
      # Arguments
          name: String, name for the metric.
      """
  
      def __init__(self, name='glo_kappa', **kwargs):
          super(BinaryKappa, self).__init__(name=name, **kwargs)
          self.true_positives = self.add_weight(name='tp',initializer='zeros')
          self.true_negative = self.add_weight(name='tn',initializer='zeros')
          self.false_positives = self.add_weight(name='fp',initializer='zeros')
          self.false_negative = self.add_weight(name='fn',initializer='zeros')
  
      def reset_states(self):
          self.true_positives.assign(0.)
          self.true_negative.assign(0.)
          self.false_positives.assign(0.)
          self.false_negative.assign(0.)
  
      def update_state(self, y_true, y_pred, sample_weight=None):
          y_true = K.cast(y_true, 'int32')
          y_pred = K.cast(K.round(y_pred), 'int32')
          true_pos = K.cast(K.sum(y_pred * y_true),'int32')
          true_neg = K.cast(K.sum((1 - y_pred) * (1 - y_true)),'int32')
          false_pos = K.cast(K.sum(y_pred * (1 - y_true)),'int32')
          false_neg = K.cast(K.sum((1 - y_pred) * y_true),'int32')
  
          true_pos = K.cast(true_pos, "float32")
          true_neg = K.cast(true_neg, "float32")
          false_pos = K.cast(false_pos, "float32")
          false_neg = K.cast(false_neg, "float32")
  
          self.true_positives.assign_add(true_pos)
          self.true_negative.assign_add(true_neg)
          self.false_positives.assign_add(false_pos)
          self.false_negative.assign_add(false_neg)
  
      def result(self):
          sm = self.true_positives + self.true_negative + self.false_positives + self.false_negative
          obs_agree = (self.true_positives + self.true_negative) / sm
          poss_pos = (self.true_positives + self.false_negative) * (self.true_positives + self.false_positives) / (sm**2)
          poss_neg = (self.true_negative + self.false_negative) * (self.true_negative + self.false_positives) / (sm**2)
          poss_agree = poss_pos + poss_neg
          return (obs_agree - poss_agree) / (1 - poss_agree + K.epsilon())
  </code></pre>