title: How to add custom metric to keras? (Percent Mean Absolute Error)
body: <p>I am trying to add <strong>percent mean absolute error</strong> (pmae) as a custom metric in keras. This is defined as (MAE divided by the mean absolute y-value multiplied by 100). I have tried:</p>
  
  <pre class="lang-py prettyprint-override"><code>def pmae(y_true,y_pred):
      return K.mean(K.abs(y_pred - y_true)) / K.mean(K.abs(y_true)) * 100
  ...
  model.compile(loss='mse', optimizer=Adam(),metrics=[pmae])
  </code></pre>
  
  <p>which runs, but the value is many orders of magnitude off (when I look at <code>model.history.history.pmae</code>)</p>
  
  <p>The working numpy version (on a test sample) is:</p>
  
  <pre class="lang-py prettyprint-override"><code>y_pred = model.predict(X_test)
  pmae = abs(y_pred - y_test).mean() / abs(y_true).mean() * 100
  </code></pre>
  
  <p>I've also tried adding <code>, axis=-1</code> to the <code>K.mean()</code> calls, with no improvement (as suggested in other stackoverflow answers). Does anyone know what's wrong?</p>
  
  <p><strong>Resources</strong></p>
  
  <ol>
  <li>The <a href="https://keras.io/metrics/#custom-metrics" rel="nofollow noreferrer">keras website</a> gives mean value of y as an example:</li>
  </ol>
  
  <pre class="lang-py prettyprint-override"><code>import keras.backend as K
  
  def mean_pred(y_true, y_pred):
      return K.mean(y_pred)
  
  model.compile(optimizer='rmsprop',
                loss='binary_crossentropy',
                metrics=['accuracy', mean_pred])
  </code></pre>
  
  <ol start="2">
  <li>Others have answered on stackoverflow about other custom metrics (eg <a href="https://stackoverflow.com/q/47582982/11918892">Keras custom RMSLE metric</a> and <a href="https://stackoverflow.com/questions/37657260/how-to-implement-custom-metric-in-keras">how to implement custom metric in keras?</a>), but the responses there haven't helped me with pmae calculation.</li>
  </ol>