{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/23766","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/23766/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/23766/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/23766/events","html_url":"https://github.com/tensorflow/tensorflow/issues/23766","id":381090333,"node_id":"MDU6SXNzdWUzODEwOTAzMzM=","number":23766,"title":"TfLite performance is way worse comapred to Tensorflow mobile","user":{"login":"Noltibus","id":15614838,"node_id":"MDQ6VXNlcjE1NjE0ODM4","avatar_url":"https://avatars3.githubusercontent.com/u/15614838?v=4","gravatar_id":"","url":"https://api.github.com/users/Noltibus","html_url":"https://github.com/Noltibus","followers_url":"https://api.github.com/users/Noltibus/followers","following_url":"https://api.github.com/users/Noltibus/following{/other_user}","gists_url":"https://api.github.com/users/Noltibus/gists{/gist_id}","starred_url":"https://api.github.com/users/Noltibus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Noltibus/subscriptions","organizations_url":"https://api.github.com/users/Noltibus/orgs","repos_url":"https://api.github.com/users/Noltibus/repos","events_url":"https://api.github.com/users/Noltibus/events{/privacy}","received_events_url":"https://api.github.com/users/Noltibus/received_events","type":"User","site_admin":false},"labels":[{"id":750616506,"node_id":"MDU6TGFiZWw3NTA2MTY1MDY=","url":"https://api.github.com/repos/tensorflow/tensorflow/labels/comp:lite","name":"comp:lite","color":"0052cc","default":false}],"state":"open","locked":false,"assignee":{"login":"jdduke","id":479117,"node_id":"MDQ6VXNlcjQ3OTExNw==","avatar_url":"https://avatars2.githubusercontent.com/u/479117?v=4","gravatar_id":"","url":"https://api.github.com/users/jdduke","html_url":"https://github.com/jdduke","followers_url":"https://api.github.com/users/jdduke/followers","following_url":"https://api.github.com/users/jdduke/following{/other_user}","gists_url":"https://api.github.com/users/jdduke/gists{/gist_id}","starred_url":"https://api.github.com/users/jdduke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jdduke/subscriptions","organizations_url":"https://api.github.com/users/jdduke/orgs","repos_url":"https://api.github.com/users/jdduke/repos","events_url":"https://api.github.com/users/jdduke/events{/privacy}","received_events_url":"https://api.github.com/users/jdduke/received_events","type":"User","site_admin":false},"assignees":[{"login":"jdduke","id":479117,"node_id":"MDQ6VXNlcjQ3OTExNw==","avatar_url":"https://avatars2.githubusercontent.com/u/479117?v=4","gravatar_id":"","url":"https://api.github.com/users/jdduke","html_url":"https://github.com/jdduke","followers_url":"https://api.github.com/users/jdduke/followers","following_url":"https://api.github.com/users/jdduke/following{/other_user}","gists_url":"https://api.github.com/users/jdduke/gists{/gist_id}","starred_url":"https://api.github.com/users/jdduke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jdduke/subscriptions","organizations_url":"https://api.github.com/users/jdduke/orgs","repos_url":"https://api.github.com/users/jdduke/repos","events_url":"https://api.github.com/users/jdduke/events{/privacy}","received_events_url":"https://api.github.com/users/jdduke/received_events","type":"User","site_admin":false}],"milestone":null,"comments":9,"created_at":"2018-11-15T10:14:53Z","updated_at":"2018-12-27T11:13:43Z","closed_at":null,"author_association":"NONE","body":"**System information**\r\n- Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes\r\n- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10\r\n- Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: Samsung Galaxy S8\r\n- TensorFlow installed from (source or binary): binary\r\n- TensorFlow version (use command below): On PC: current tf-nightly (14.11.2018), on phone current org.tensorflow:tensorflow-lite:0.0.0-nightly\r\n- Python version: 3.6.6\r\n- Bazel version (if compiling from source): - \r\n- GCC/Compiler version (if compiling from source): - \r\n- CUDA/cuDNN version: - \r\n- GPU model and memory: CPU only\r\n\r\n**Describe the current behavior**\r\nI build a model with this code:\r\n```\r\nimport tensorflow as tf\r\nimport numpy as np\r\nfrom tensorflow.python.tools import freeze_graph\r\n\r\n\r\nclass AlexNet(object):\r\n    \"\"\"Implementation of the AlexNet.\"\"\"\r\n\r\n    def __init__(self, x, keep_prob, num_classes, skip_layer,\r\n                 weights_path='DEFAULT'):\r\n        \"\"\"Create the graph of the AlexNet model.\r\n        Args:\r\n            x: Placeholder for the input tensor.\r\n            keep_prob: Dropout probability.\r\n            num_classes: Number of classes in the dataset.\r\n            skip_layer: List of names of the layer, that get trained from\r\n                scratch\r\n            weights_path: Complete path to the pretrained weight file, if it\r\n                isn't in the same folder as this code\r\n        \"\"\"\r\n        # Parse input arguments into class variables\r\n        self.X = x\r\n        self.NUM_CLASSES = num_classes\r\n        self.KEEP_PROB = keep_prob\r\n        self.SKIP_LAYER = skip_layer\r\n\r\n        if weights_path == 'DEFAULT':\r\n            self.WEIGHTS_PATH = 'bvlc_alexnet.npy'\r\n        else:\r\n            self.WEIGHTS_PATH = weights_path\r\n\r\n        # Call the create function to build the computational graph of AlexNet\r\n        self.create()\r\n\r\n    def create(self):\r\n        \"\"\"Create the network graph.\"\"\"\r\n        # 1st Layer: Conv (w ReLu) -> Lrn -> Pool\r\n        conv1 = conv(self.X, 11, 11, 96, 4, 4, padding='VALID', name='conv1')\r\n        norm1 = lrn(conv1, 2, 2e-05, 0.75, name='norm1')\r\n        pool1 = max_pool(norm1, 3, 3, 2, 2, padding='VALID', name='pool1')\r\n\r\n        # 2nd Layer: Conv (w ReLu)  -> Lrn -> Pool with 2 groups\r\n        conv2 = conv(pool1, 5, 5, 256, 1, 1, groups=2, name='conv2')\r\n        norm2 = lrn(conv2, 2, 2e-05, 0.75, name='norm2')\r\n        pool2 = max_pool(norm2, 3, 3, 2, 2, padding='VALID', name='pool2')\r\n\r\n        # 3rd Layer: Conv (w ReLu)\r\n        self.conv3 = conv(pool2, 3, 3, 384, 1, 1, name='conv3')\r\n\r\n        # 4th Layer: Conv (w ReLu) splitted into two groups\r\n        conv4 = conv(self.conv3, 3, 3, 384, 1, 1, groups=2, name='conv4')\r\n\r\n        # 5th Layer: Conv (w ReLu) -> Pool splitted into two groups\r\n        conv5 = conv(conv4, 3, 3, 256, 1, 1, groups=2, name='conv5')\r\n        self.pool5 = max_pool(conv5, 3, 3, 2, 2, padding='VALID', name='pool5')\r\n\r\n        # 6th Layer: Flatten -> FC (w ReLu) -> Dropout\r\n        self.flattened = tf.reshape(self.pool5, [1, 9216], name='output')\r\n\r\n    def load_initial_weights(self, session):\r\n        \"\"\"Load weights from file into network.\r\n        As the weights from http://www.cs.toronto.edu/~guerzhoy/tf_alexnet/\r\n        come as a dict of lists (e.g. weights['conv1'] is a list) and not as\r\n        dict of dicts (e.g. weights['conv1'] is a dict with keys 'weights' &\r\n        'biases') we need a special load function\r\n        \"\"\"\r\n        # Load the weights into memory\r\n        weights_dict = np.load(self.WEIGHTS_PATH, encoding='bytes').item()\r\n\r\n        # Loop over all layer names stored in the weights dict\r\n        for op_name in weights_dict:\r\n\r\n            # Check if layer should be trained from scratch\r\n            if op_name not in self.SKIP_LAYER:\r\n\r\n                with tf.variable_scope(op_name, reuse=True):\r\n\r\n                    # Assign weights/biases to their corresponding tf variable\r\n                    for data in weights_dict[op_name]:\r\n\r\n                        # Biases\r\n                        if len(data.shape) == 1:\r\n                            var = tf.get_variable('biases', trainable=False)\r\n                            session.run(var.assign(data))\r\n\r\n                        # Weights\r\n                        else:\r\n                            var = tf.get_variable('weights', trainable=False)\r\n                            session.run(var.assign(data))\r\n\r\n\r\ndef conv(x, filter_height, filter_width, num_filters, stride_y, stride_x, name,\r\n         padding='SAME', groups=1):\r\n    \"\"\"Create a convolution layer.\r\n    Adapted from: https://github.com/ethereon/caffe-tensorflow\r\n    \"\"\"\r\n    # Get number of input channels\r\n    input_channels = int(x.get_shape()[-1])\r\n\r\n    # Create lambda function for the convolution\r\n    convolve = lambda i, k: tf.nn.conv2d(i, k,\r\n                                         strides=[1, stride_y, stride_x, 1],\r\n                                         padding=padding)\r\n\r\n    with tf.variable_scope(name) as scope:\r\n        # Create tf variables for the weights and biases of the conv layer\r\n        weights = tf.get_variable('weights', shape=[filter_height,\r\n                                                    filter_width,\r\n                                                    input_channels/groups,\r\n                                                    num_filters])\r\n        biases = tf.get_variable('biases', shape=[num_filters])\r\n\r\n    if groups == 1:\r\n        conv = convolve(x, weights)\r\n\r\n    # In the cases of multiple groups, split inputs & weights and\r\n    else:\r\n        # Split input and weights and convolve them separately\r\n        input_groups = tf.split(axis=3, num_or_size_splits=groups, value=x)\r\n        weight_groups = tf.split(axis=3, num_or_size_splits=groups,\r\n                                 value=weights)\r\n        output_groups = [convolve(i, k) for i, k in zip(input_groups, weight_groups)]\r\n\r\n        # Concat the convolved output together again\r\n        conv = tf.concat(axis=3, values=output_groups)\r\n\r\n    # Add biases\r\n    bias = tf.reshape(tf.nn.bias_add(conv, biases), tf.shape(conv))\r\n\r\n    # Apply relu function\r\n    relu = tf.nn.relu(bias, name=scope.name)\r\n\r\n    return relu\r\n\r\n\r\ndef fc(x, num_in, num_out, name, relu=True):\r\n    \"\"\"Create a fully connected layer.\"\"\"\r\n    with tf.variable_scope(name) as scope:\r\n\r\n        # Create tf variables for the weights and biases\r\n        weights = tf.get_variable('weights', shape=[num_in, num_out],\r\n                                  trainable=True)\r\n        biases = tf.get_variable('biases', [num_out], trainable=True)\r\n\r\n        # Matrix multiply weights and inputs and add bias\r\n        act = tf.nn.xw_plus_b(x, weights, biases, name=scope.name)\r\n\r\n    if relu:\r\n        # Apply ReLu non linearity\r\n        relu = tf.nn.relu(act)\r\n        return relu\r\n    else:\r\n        return act\r\n\r\n\r\ndef max_pool(x, filter_height, filter_width, stride_y, stride_x, name,\r\n             padding='SAME'):\r\n    \"\"\"Create a max pooling layer.\"\"\"\r\n    return tf.nn.max_pool(x, ksize=[1, filter_height, filter_width, 1],\r\n                          strides=[1, stride_y, stride_x, 1],\r\n                          padding=padding, name=name)\r\n\r\n\r\ndef lrn(x, radius, alpha, beta, name, bias=1.0):\r\n    \"\"\"Create a local response normalization layer.\"\"\"\r\n    return tf.nn.local_response_normalization(x, depth_radius=radius,\r\n                                              alpha=alpha, beta=beta,\r\n                                              bias=bias, name=name)\r\n\r\n\r\ndef dropout(x, keep_prob):\r\n    \"\"\"Create a dropout layer.\"\"\"\r\n    return tf.nn.dropout(x, keep_prob)\r\n\r\n\r\n\r\n\r\ng = tf.Graph()\r\nwith g.as_default():\r\n    # Initialize all variables\r\n    x = tf.placeholder(tf.float32, [1, 227, 227, 3])\r\n    y = tf.placeholder(tf.float32, [1, 1000])\r\n    keep_prob = tf.placeholder(tf.float32)\r\n    alex_net = AlexNet(x, keep_prob, 1000, [\"fc6\", \"fc7\", \"fc8\"])\r\n```\r\n\r\nThen I froze the model. In the first try I converted it to TFlite directly, in a second try I set the `converter.post_training_quantize = True` option in the conversion process to quantize my model and in a second try I optimized my model for inference before converting and quantizing.\r\nThe code for the tflite conversion is the following:\r\n```\r\nimport tensorflow as tf\r\n\r\ngraph_def_file = \"alex_frozen_optimized.pb\"\r\ninput_arrays = [\"Placeholder\"]\r\noutput_arrays = [\"output\"]\r\n\r\nconverter = tf.contrib.lite.TFLiteConverter.from_frozen_graph(\r\n    graph_def_file, input_arrays, output_arrays, input_shapes={\"Placeholder\" : [1, 227, 227, 3]})\r\nconverter.post_training_quantize = True\r\ntflite_model = converter.convert()\r\nopen(\"save_path/alex_frozen_optimized_quantized.tflite\", \"wb\").write(tflite_model)\r\n```\r\nThe code for optimizing for inference was the following terminal call: `python -m tensorflow.python.tools.optimize_for_inference --input alex_frozen.pb --output alex_frozen_optimized.pb --input_names=Placeholder --output_names=output`\r\n\r\nAll my models, including the plain, unconverted model is uploaded here for testing: [models.zip](https://www.dropbox.com/s/gyu9cfkn3yn07oc/models.zip?dl=0)\r\n\r\n**Describe the expected behavior**\r\nI deployed those models to my phone. I get runtime of around 290 ms for the unconverted model, while having runtimes of about 420 ms for the quantized tflite, the optimized and quantized tflite and the plain tflite model. That can't be right, can it?\r\n\r\n**Code to reproduce the issue**\r\nThe code for running inference on the phone for the tflite models is the following:\r\n```\r\nimport android.content.res.AssetFileDescriptor;\r\nimport android.content.res.AssetManager;\r\nimport android.graphics.Bitmap;\r\nimport android.os.Trace;\r\nimport java.io.FileInputStream;\r\nimport java.io.IOException;\r\nimport java.nio.ByteBuffer;\r\nimport java.nio.ByteOrder;\r\nimport java.nio.MappedByteBuffer;\r\nimport java.nio.channels.FileChannel;\r\nimport one.realnote.app.PpmItem;\r\nimport one.realnote.app.Util;\r\nimport org.tensorflow.lite.Interpreter;\r\nimport org.tensorflow.lite.Interpreter.Options;\r\n\r\npublic class AlexFeatureExtractionLite {\r\n\r\n  private static final String TAG = \"TFLiteAPIAlex\";\r\n\r\n  // Only return this many results.\r\n  private boolean isModelQuantized;\r\n  // Float model\r\n  // Number of threads in the java app\r\n  private static final int NUM_THREADS = 4;\r\n  // Config values.\r\n  private int inputSize;\r\n  // Pre-allocated buffers.\r\n  private float[][] outputVector;\r\n  private int[] intValues;\r\n\r\n  private static PpmItem ppm_tfRun = PpmItem.createDebugItem(\"TFA\", \"runAlex\", String.class);\r\n\r\n  private ByteBuffer imgData;\r\n  private Interpreter tfLite;\r\n\r\n  /**\r\n   * Memory-map the model file in Assets.\r\n   */\r\n  private static MappedByteBuffer loadModelFile(AssetManager assets)\r\n      throws IOException {\r\n    AssetFileDescriptor fileDescriptor = assets.openFd(\"tf/alex_frozen_optimized_quantized.tflite\");\r\n    FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());\r\n    FileChannel fileChannel = inputStream.getChannel();\r\n    long startOffset = fileDescriptor.getStartOffset();\r\n    long declaredLength = fileDescriptor.getDeclaredLength();\r\n    return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);\r\n  }\r\n\r\n  /**\r\n   * Initializes a native TensorFlow session for classifying images.\r\n   *\r\n   * @param assetManager The asset manager to be used to load assets.\r\n   * @param inputSize The size of image input\r\n   * @param isQuantized Boolean representing model is quantized or not\r\n   */\r\n  public static AlexFeatureExtractionLite create(final AssetManager assetManager,\r\n      final int inputSize, final boolean isQuantized) {\r\n    final AlexFeatureExtractionLite d = new AlexFeatureExtractionLite();\r\n\r\n    d.inputSize = inputSize;\r\n    Interpreter.Options options = new Options();\r\n    options.setUseNNAPI(false);\r\n    options.setAllowFp16PrecisionForFp32(false);\r\n    options.setNumThreads(4);\r\n    try {\r\n      d.tfLite = new Interpreter(loadModelFile(assetManager), options);\r\n    } catch (Exception e) {\r\n      Util.logError(e);\r\n      throw new RuntimeException(e);\r\n    }\r\n\r\n    d.isModelQuantized = isQuantized;\r\n    // Pre-allocate buffers.\r\n    int numBytesPerChannel;\r\n    if (isQuantized) {\r\n      numBytesPerChannel = 1; // Quantized\r\n    } else {\r\n      numBytesPerChannel = 4; // Floating point\r\n    }\r\n    d.imgData = ByteBuffer.allocateDirect(1 * d.inputSize * d.inputSize * 3 * numBytesPerChannel);\r\n    d.imgData.order(ByteOrder.nativeOrder());\r\n    d.intValues = new int[d.inputSize * d.inputSize];\r\n    d.outputVector = new float[1][9216];\r\n\r\n    //d.tfLite.setNumThreads(NUM_THREADS);\r\n    return d;\r\n  }\r\n\r\n  private AlexFeatureExtractionLite() {\r\n  }\r\n\r\n\r\n  public float[] calcFeatureVector(final Bitmap bitmap) {\r\n    // Log this method so that it can be analyzed with systrace.\r\n    Trace.beginSection(\"recognizeImage\");\r\n\r\n    Trace.beginSection(\"preprocessBitmap\");\r\n    // Preprocess the image data from 0-255 int to normalized float based\r\n    // on the provided parameters.\r\n    bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0, bitmap.getWidth(), bitmap.getHeight());\r\n\r\n    imgData.rewind();\r\n    for (int i = 0; i < inputSize; ++i) {\r\n      for (int j = 0; j < inputSize; ++j) {\r\n        int pixelValue = intValues[i * inputSize + j];\r\n        if (isModelQuantized) {\r\n          // Quantized model\r\n          imgData.put((byte) ((pixelValue >> 16) & 0xFF));\r\n          imgData.put((byte) ((pixelValue >> 8) & 0xFF));\r\n          imgData.put((byte) (pixelValue & 0xFF));\r\n        } else { // Float model\r\n          imgData.putFloat((((pixelValue >> 16) & 0xFF) - 104f));\r\n          imgData.putFloat((((pixelValue >> 8) & 0xFF) - 117f));\r\n          imgData.putFloat(((pixelValue & 0xFF) - 124f));\r\n        }\r\n      }\r\n    }\r\n    Trace.endSection(); // preprocessBitmap\r\n\r\n    // Copy the input data into TensorFlow.\r\n    Trace.beginSection(\"feed\");\r\n    outputVector = new float[1][9216];\r\n\r\n    Trace.endSection();\r\n\r\n    // Run the inference call.\r\n    Trace.beginSection(\"run\");\r\n    ppm_tfRun.start();\r\n    tfLite.run(imgData, outputVector);\r\n    ppm_tfRun.stop();\r\n    Trace.endSection();\r\n\r\n    Trace.endSection(); // \"recognizeImage\"\r\n    return outputVector[0];\r\n  }\r\n}\r\n```\r\n\r\nThe code to run inference for the unconverted model is:\r\n```\r\nimport android.content.res.AssetManager;\r\nimport android.graphics.Bitmap;\r\nimport android.os.Trace;\r\nimport one.realnote.app.PpmItem;\r\nimport org.tensorflow.Graph;\r\nimport org.tensorflow.Operation;\r\nimport org.tensorflow.contrib.android.TensorFlowInferenceInterface;\r\n\r\npublic class AlexFeatureExtraction {\r\n\r\n    private static final String TAG = \"AlexFeatureExtraction\";\r\n\r\n    // Config values.\r\n    private String inputName;\r\n    private int inputSize;\r\n\r\n    // Pre-allocated buffers.\r\n    private float[] outputVector;\r\n\r\n    private int[] intValues;\r\n    private float[] floatValues;\r\n\r\n    private TensorFlowInferenceInterface inferenceInterface;\r\n\r\n    public static AlexFeatureExtraction create(\r\n        final AssetManager assetManager) {\r\n\r\n        final AlexFeatureExtraction d = new AlexFeatureExtraction();\r\n\r\n\r\n        final String modelFilename = \"file:///android_asset/tf/alex.pb\";\r\n        final int inputSize = 227;\r\n\r\n        d.inferenceInterface = new TensorFlowInferenceInterface(assetManager, modelFilename);\r\n\r\n        final Graph g = d.inferenceInterface.graph();\r\n\r\n        d.inputName = \"Placeholder\"; // ssd: \"image_tensor\";\r\n\r\n        // The inputName node has a shape of [N, H, W, C], where\r\n        // N is the batch size\r\n        // H = W are the height and width\r\n        // C is the number of channels (3 for our purposes - RGB)\r\n        final Operation inputOp = g.operation(d.inputName);\r\n        if (inputOp == null) {\r\n            throw new RuntimeException(\"Failed to find input Node '\" + d.inputName + \"'\");\r\n        }\r\n        d.inputSize = inputSize;\r\n        // The outputScoresName node has a shape of [N, NumLocations], where N\r\n        // is the batch size.\r\n\r\n        // ssd: final Operation outputOp1 = g.operation(\"detection_scores\");\r\n        final Operation outputOp1 = g.operation(\"output\");\r\n        if (outputOp1 == null) {\r\n            throw new RuntimeException(\"Failed to find output Node 'detection_scores'\");\r\n        }\r\n\r\n        // Pre-allocate buffers.\r\n        d.outputVector = new float[9216];\r\n\r\n        d.intValues = new int[d.inputSize * d.inputSize];\r\n        d.floatValues = new float[d.inputSize * d.inputSize * 3];\r\n\r\n        return d;\r\n    }\r\n\r\n\r\n    private AlexFeatureExtraction() {\r\n    }\r\n\r\n\r\n    private static PpmItem ppm_tfFeed = PpmItem.createDebugItem(\"TFA\", \"feed\", String.class);\r\n    private static PpmItem ppm_tfRun = PpmItem.createDebugItem(\"TFA\", \"runAlex\", String.class);\r\n\r\n\r\n    public float[] calcFeatureVector(final Bitmap bitmap) {\r\n        // Log this method so that it can be analyzed with systrace.\r\n        Trace.beginSection(\"recognizeImage\");\r\n\r\n        Trace.beginSection(\"preprocessBitmap\");\r\n        // Preprocess the image data from 0-255 int to normalized float based\r\n        // on the provided parameters.\r\n        bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0, bitmap.getWidth(), bitmap.getHeight());\r\n\r\n        for (int i = 0; i < intValues.length; ++i) {\r\n            floatValues[i * 3 + 2] = (float) (intValues[i] & 0xFF) - 124f;\r\n            floatValues[i * 3 + 1] = (float) ((intValues[i] >> 8) & 0xFF) - 117f;\r\n            floatValues[i * 3 + 0] = (float) ((intValues[i] >> 16) & 0xFF) -104f;\r\n        }\r\n\r\n        Trace.endSection(); // preprocessBitmap\r\n\r\n        // Copy the input data into TensorFlow.\r\n        Trace.beginSection(\"feed\");\r\n        ppm_tfFeed.start();\r\n        inferenceInterface.feed(inputName, floatValues, 1, inputSize, inputSize, 3);\r\n        ppm_tfFeed.stop();\r\n        Trace.endSection();\r\n\r\n        // Run the inference call.\r\n        Trace.beginSection(\"run\");\r\n        ppm_tfRun.start();\r\n        inferenceInterface.run(new String[]{\"output\"}, false);\r\n        ppm_tfRun.stop();\r\n        Trace.endSection();\r\n\r\n        // Copy the output Tensor back into the output array.\r\n        Trace.beginSection(\"fetch\");\r\n\r\n\r\n        inferenceInterface.fetch(\"output\", outputVector);\r\n\r\n        Trace.endSection();\r\n\r\n        Trace.endSection(); // \"recognizeImage\"\r\n\r\n        return outputVector;\r\n    }\r\n\r\n    public void close() {\r\n        inferenceInterface.close();\r\n    }\r\n}\r\n```\r\n\r\n**Other info / logs**\r\nOutput behavior is as expected for all models. Activating the useNNAPI option doesn't really do anything performance wise aswell as setting the number of threads.\r\n"},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/450131578","html_url":"https://github.com/tensorflow/tensorflow/issues/23766#issuecomment-450131578","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/23766","id":450131578,"node_id":"MDEyOklzc3VlQ29tbWVudDQ1MDEzMTU3OA==","user":{"login":"Noltibus","id":15614838,"node_id":"MDQ6VXNlcjE1NjE0ODM4","avatar_url":"https://avatars3.githubusercontent.com/u/15614838?v=4","gravatar_id":"","url":"https://api.github.com/users/Noltibus","html_url":"https://github.com/Noltibus","followers_url":"https://api.github.com/users/Noltibus/followers","following_url":"https://api.github.com/users/Noltibus/following{/other_user}","gists_url":"https://api.github.com/users/Noltibus/gists{/gist_id}","starred_url":"https://api.github.com/users/Noltibus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Noltibus/subscriptions","organizations_url":"https://api.github.com/users/Noltibus/orgs","repos_url":"https://api.github.com/users/Noltibus/repos","events_url":"https://api.github.com/users/Noltibus/events{/privacy}","received_events_url":"https://api.github.com/users/Noltibus/received_events","type":"User","site_admin":false},"created_at":"2018-12-27T11:13:43Z","updated_at":"2018-12-27T11:13:43Z","author_association":"NONE","body":"Any word on this? It seems, that using the normal, not the nightly dependency of tflite now runs the model, but still way slower than the not-lite model"}}
{"actor":{"display_login":"Noltibus"}}