<p>According to this Oct 2018 paper: <a href="https://arxiv.org/pdf/1810.01109.pdf" rel="nofollow noreferrer">AI Benchmark: Running Deep Neural Networks
on Android Smartphones</a>, the NNAPI is defaults to the CPU path when no specific hardware and/or no drivers are available. Toward the end of the paper it notes that a number of devices have implementation issues.</p>
<p>As the paper's authors include representatives from Qualcomm, ARM, Huawei, MediaTek, and ETH Zurich, it is probably the most comprehensive overview of the state of machine learning on Android.</p>
<p>In Jan 2019 Google announced <a href="https://medium.com/tensorflow/tensorflow-lite-now-faster-with-mobile-gpus-developer-preview-e15797e6dee7?linkId=62443226" rel="nofollow noreferrer">TensorFlow Lite with GPU acceleration in developer preview</a> which will address some of the issues raised in the paper.</p>
<p><strong>2020 July Update:</strong></p>
<p>The researchers have a site at: <a href="http://ai-benchmark.com/" rel="nofollow noreferrer">http://ai-benchmark.com/</a></p>
<p>And have updated their paper in Oct 2019:
<a href="https://arxiv.org/pdf/1910.06663.pdf" rel="nofollow noreferrer">AI Benchmark: All About Deep Learning on Smartphones in 2019</a></p>
Use ML Kit with NNAPI
<p>I'm trying to use the new Google machine learning sdk, ML Kit, on an Android devices that run Android 9. 
From the official site:</p>

<blockquote>
  <p>ML Kit makes it easy to apply ML techniques in your apps by bringing
  Google's ML technologies, such as the Google Cloud Vision API,
  TensorFlow Lite, and the Android Neural Networks API together in a
  single SDK. Whether you need the power of cloud-based processing, the
  real-time capabilities of mobile-optimized on-device models, or the
  flexibility of custom TensorFlow Lite models, ML Kit makes it possible
  with just a few lines of code.</p>
</blockquote>

<p>I think it means that on a device with at least Android 8.1 (according to the documentation of nnapi) the SDK can uses NNAPI. But when I run the same app on a device with Android 7.1 (where nnapi is not supported) I obtain the same performance of the device that use Android 9 (and in theory the NNAPI). How i can use ML Kit with NNAPI? I am doing something wrong?
Link to documentation of mlkit: <a href="https://firebase.google.com/docs/ml-kit/" rel="nofollow noreferrer">https://firebase.google.com/docs/ml-kit/</a></p>