title: Cannot convert from a fine-tuned GPT-2 model to a Tensorflow Lite model
body: <p>I've fine tuned a distilgpt2 model using my own text using <code>run_language_modeling.py</code> and its working fine after training and <code>run_generation.py</code> script produces the expected results.</p>
  <p>Now I want to convert this to a Tensorflow Lite model and did so by using the following</p>
  <pre><code>from transformers import *
  
  CHECKPOINT_PATH = '/content/drive/My Drive/gpt2_finetuned_models/checkpoint-2500'
  
  model = GPT2LMHeadModel.from_pretrained(&quot;distilgpt2&quot;)
  model.save_pretrained(CHECKPOINT_PATH)
  model = TFGPT2LMHeadModel.from_pretrained(CHECKPOINT_PATH, from_pt=True) 
  </code></pre>
  <p>But I dont think I'm doing this right as after conversion, when I write</p>
  <pre><code>print(model.inputs)
  print(model.outputs)
  </code></pre>
  <p>I get</p>
  <pre><code>None
  None
  </code></pre>
  <p>But I still went ahead with the TFLite conversion using :</p>
  <pre><code>import tensorflow as tf
  
  input_spec = tf.TensorSpec([1, 64], tf.int32)
  model._set_inputs(input_spec, training=False)
  
  converter = tf.lite.TFLiteConverter.from_keras_model(model)
  
  # FP16 quantization:
  converter.optimizations = [tf.lite.Optimize.DEFAULT]
  converter.target_spec.supported_types = [tf.float16]
  
  tflite_model = converter.convert()
  
  open(&quot;/content/gpt2-fp16.tflite&quot;, &quot;wb&quot;).write(tflite_model)
  </code></pre>
  <p>But does not work and when using the generated <code>tflite</code> model I get the error:</p>
  <blockquote>
  <p>tensorflow/lite/kernels/kernel_util.cc:249 d1 == d2 || d1 == 1 || d2 == 1 was not true.</p>
  </blockquote>
  <p>Which I'm sure has something to to with my model not converting properly and getting <code>None</code> for input/output.</p>
  <p>Does anyone have any idea how to fix this?</p>
  <p>Thanks</p>