title: tflite quantization how to change the input dtype
body: <p><em>see possible solution at the end of the post</em></p>
  
  <hr>
  
  <p>I am trying to fully quantize the keras-vggface model from <a href="https://github.com/rcmalli/keras-vggface" rel="nofollow noreferrer">rcmalli</a> to run on an NPU. The model is a Keras model (not tf.keras).</p>
  
  <p>When using TF 1.15 for quantization with:</p>
  
  <pre><code>print(tf.version.VERSION)
  num_calibration_steps=5
  
  converter = tf.lite.TFLiteConverter.from_keras_model_file('path_to_model.h5')        
  
  #converter.post_training_quantize = True  # This only makes the weight in8 but does not initialize model quantization  
  def representative_dataset_gen():
      for _ in range(num_calibration_steps):
          pfad='path_to_image(s)'
          img=cv2.imread(pfad)
          # Get sample input data as a numpy array in a method of your choosing.
          yield [img]
  converter.representative_dataset = representative_dataset_gen
  tflite_quant_model = converter.convert()
  open("quantized_model", "wb").write(tflite_quant_model)
  </code></pre>
  
  <p>The model is converted but as I need full int8 quantization, I add:</p>
  
  <pre><code>converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
  converter.inference_input_type = tf.int8  # or tf.uint8
  converter.inference_output_type = tf.int8  # or tf.uint8
  </code></pre>
  
  <p>This error message appears: </p>
  
  <blockquote>
    <p>ValueError: Cannot set tensor: Got value of type UINT8 but expected type FLOAT32 for input 0, name: input_1 </p>
  </blockquote>
  
  <p>clearly, the input of the model still requires float32.</p>
  
  <p><strong>Questions</strong>:</p>
  
  <ol>
  <li>Do I have to adapt the quantization method that the input dtype is changed ? or</li>
  <li>Do I have to change the input layer of the model to dtype int8 beforehand? </li>
  <li>Or is that actually reporting that the model is not actually quantized?</li>
  </ol>
  
  <p>If 1 or 2 is the answer, would you also have a best practice tip for me? </p>
  
  <hr>
  
  <p><strong>Addition</strong>:</p>
  
  <p>Using :</p>
  
  <pre><code>h5_path = 'my_model.h5'
  model = keras.models.load_model(h5_path)
  model.save(os.getcwd() +'/modelTF2') 
  </code></pre>
  
  <p>to save the h5 as pb with TF 2.2 and then using <code>converter=tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)</code> </p>
  
  <p>as TF 2.x tflite takes floats, and convert them to uint8s <a href="https://github.com/google-coral/edgetpu/issues/13#issuecomment-543758317" rel="nofollow noreferrer">internally</a> . I thought that could be a solution. Unfortunately, this error message appears:</p>
  
  <blockquote>
    <p>tf.lite.TFLiteConverter.from_keras_model giving 'str' object has no attribute 'call'</p>
  </blockquote>
  
  <p>Apparently TF2.x cannot handle pure keras models. </p>
  
  <p>using <code>tf.compat.v1.lite.TFLiteConverter.from_keras_model_file()</code> to solve this error just repeats the error from above, as we are back again at "TF 1.15" level.</p>
  
  <hr>
  
  <p><strong>Addition 2</strong></p>
  
  <p>Another solution is to transfer the keras model to tf.keras manually. I will look into that if there is no other solution.</p>
  
  <hr>
  
  <p><strong>Regarding the comment of Meghna Natraj</strong></p>
  
  <p>To recreate the model <strong>(using TF 1.13.x)</strong> just:</p>
  
  <p><code>pip install git+https://github.com/rcmalli/keras-vggface.git</code></p>
  
  <p>and</p>
  
  <pre><code>from keras_vggface.vggface import VGGFace   
  pretrained_model = VGGFace(model='resnet50', include_top=False, input_shape=(224, 224, 3), pooling='avg')  # pooling: None, avg or max
  pretrained_model.summary()
  pretrained_model.save("my_model.h5") #using h5 extension
  </code></pre>
  
  <p>The input layer is connected. Too bad, that looked like a good/easy fix.
  <a href="https://i.stack.imgur.com/zhWtU.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/zhWtU.png" alt="summary of the first few layers of the VGGface model"></a></p>
  
  <hr>
  
  <p><strong>Possible Solution</strong></p>
  
  <p>It seems to work using TF 1.15.3 I used 1.15.0 beforehand. I will check if I did something else different by accident.</p>