title: How to create a metric in Keras that return a multi-dimensional value?
body: <p>I'm using keras to solve a multi-class problem. My data is very unbalanced, so I'm trying to create something similar to a confusion matrix. My dataset is very large, and saved as HDF5, so I use HDF5Matrix to fetch the X and Y, making scikit-learn confusion matrix irrelevant (as far as I know).
  I've seen it is possible to <a href="https://stackoverflow.com/questions/47079111/create-keras-callback-to-save-model-predictions-and-targets-for-each-batch-durin">save the predictions and true labels</a>, or <a href="https://stackoverflow.com/questions/47219780/output-error-rate-per-label-confusion-matrix">output the error per label</a>, however a more elegant solution would be to create a multi-dimensional metric that accumulates the (predicted,true) label pairs (sort of like a confusion matrix).
  I have used the following callback to try and peek into what's going on per batch / epoch:</p>
  
  <pre><code>from keras.callbacks import LambdaCallback
  batch_print_callback = LambdaCallback(on_batch_end=lambda batch, logs: 
  print(logs),on_epoch_end=lambda epoch, logs: print(logs))
  </code></pre>
  
  <p>but it only accumulates a single value (usually the average of sorts).</p>
  
  <p>I've also tried to see if it's possible to return the y_pred / y_true as following (to try and see if I can print a multi-dimensional value in the logs):</p>
  
  <pre><code>def pred(y_true, y_pred):
       return y_pred
  
  def true(y_true, y_pred):
      return y_true
  </code></pre>
  
  <p>However, it doesn't return a multi-dimensional value as I expected
  So basically, my question is, can I use keras to accumulate multi-dimensional metric?</p>