title: How to calculate NME(Normalized Mean Error) between ground-truth and predicted landmarks when some of gt has no corresponding in predicted?
body: <p>I am trying to learn some facial landmark detection model, and notice that many of them use NME(Normalized Mean Error) as performance metric:</p>
  <p><a href="https://i.stack.imgur.com/IbSNn.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/IbSNn.jpg" alt="enter image description here" /></a></p>
  <p>The formula is straightforward, it calculate the l2 distance between ground-truth points and model prediction result, then divided it by a normalized factor, which vary from different dataset.</p>
  <p>However, when adopting this formula on some landmark detector that some one developed, i have to deal with this non-trivial situation, that is some detector may not able to generate enough number landmarks for some input image(might because of NMS/model inherited problem/image quality etc). Thus some of ground-truth points might not have their corresponding one in the prediction result.</p>
  <p>So how to solve this problem, should i just add such missing point result to &quot;failure result set&quot; and use FR to measure the model, and ignore them when doing the NME calculation?</p>