title: How to get input data shape when model is in predicting phase?
body: <p>I tried to build Seq2Seq model by using TensorFlow 2.0.<br />
  It is not for NLP, but for predict timeseries data.</p>
  <p>Since Seq2Seq model behave differently in training and predicting phase, I needed to write call function based on the phase.</p>
  <p>In training phase, the decoder takes shifted timeseries data padded with zero.
  (e.g. encoder input: [1,2,3,4], decoder input: [0,1,2,3])</p>
  <p>And in predicting phase, the decoder takes single zero tensor at first, and it takes its output as input until it produces same length of input timeseries data (e.g. decoder input: 1) [0] -- (decoder predict 1) --&gt; [1] -- ... --&gt; final output : [0,1,2,3])</p>
  <p>Therefore, the call function in the predicting phase (<code>training=False</code>), it gets the input shape of encoder and makes a <code>tf.zeros</code> tensor based on the shape, and it goes to decoder's input.</p>
  <p>Here is some part of the class of model.</p>
  <pre class="lang-py prettyprint-override"><code>class Seq2Seq(Model):
      def __init__(self, **kwargs):
              self.encoder = layers.LSTM(5, return_state=True, input_shape=(None, 1))
              self.decoder = layers.LSTM(5, return_state=True, return_sequences=True,
                                         input_shape=(None, 1))
              self.dense = layers.Dense(1)
  
      def call(self, inputs, training=False, mask=None, **kwargs):
          decoded = None
  
          if training:
              encoder_input, decoder_input = inputs[0], inputs[1]
              encoded, state_h, state_c = self.encoder(encoder_input)
              decoded, _, _ = self.decoder(decoder_input, initial_state=[state_h, state_c])
              decoded = self.dense(decoded)
          else:
              encoder_input = inputs
              batch, time_step, features = tf.shape(encoder_input)
              encoded, state_h, state_c = self.encoder(encoder_input)
              decoder_input = tf.zeros(shape=[batch, 1, features])
              decoded = None
              for i in range(time_step):
                  decoder_input, state_h, state_c = self.decoder(decoder_input, initial_state=[state_h, state_c])
                  decoder_input = self.dense(decoder_input)
                  if decoded is not None:
                      decoded = tf.concat([decoded, decoder_input], axis=1)
                  else:
                      decoded = tf.identity(decoder_input)
          return decoded
  
  </code></pre>
  <p>In training phase, <code>inputs</code> has two input data. One is the <code>encoder_input</code> has shape <code>[batch_size, time_step, features]</code> and, another is the <code>decoder_input</code> has same shape as <code>encoder_input</code>.
  But, in predicting phase, <code>inputs</code> is just same as the <code>encoder_input</code> of the training's.</p>
  <p>Training worked well, but in validation, it produces error.</p>
  <pre class="lang-py prettyprint-override"><code>model.fit(train_dataset, epochs=10, validation_data=(seq_test_data, seq_test_data))
  # &gt;&gt;&gt; train_dataset.element_spec
  # (TensorSpec(shape=(2, None, 140, 1), dtype=tf.float32, name=None),
  # TensorSpec(shape=(None, 140, 1), dtype=tf.float32, name=None))
  # &gt;&gt;&gt; seq_test_data.shape
  # TensorShape([1000, 140, 1])
  </code></pre>
  <pre><code>Epoch 1/10
  5/5 [==============================] - ETA: 0s - MSE: 0.1487Traceback (most recent call last):
    File &quot;/home/dbadmin/anaconda3/lib/python3.7/site-packages/IPython/core/interactiveshell.py&quot;, line 3417, in run_code
      exec(code_obj, self.user_global_ns, self.user_ns)
    File &quot;&lt;ipython-input-2-91afa9c067da&gt;&quot;, line 1, in &lt;module&gt;
      runfile('/home/dbadmin/woosung/DWT-AutoEncoder-Comparison/source/autoencoder_example_using_ecg_data.py', wdir='/home/dbadmin/woosung/DWT-AutoEncoder-Comparison/source')
    File &quot;/home/dbadmin/.pycharm_helpers/pydev/_pydev_bundle/pydev_umd.py&quot;, line 197, in runfile
      pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
    File &quot;/home/dbadmin/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py&quot;, line 18, in execfile
      exec(compile(contents+&quot;\n&quot;, file, 'exec'), glob, loc)
    File &quot;/home/dbadmin/woosung/DWT-AutoEncoder-Comparison/source/autoencoder_example_using_ecg_data.py&quot;, line 112, in &lt;module&gt;
      history = ae_rnn_model.fit(train_dataset, epochs=10, validation_data=(seq_test_data, seq_test_data))
    File &quot;/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py&quot;, line 108, in _method_wrapper
      return method(self, *args, **kwargs)
    File &quot;/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py&quot;, line 1133, in fit
      return_dict=True)
    File &quot;/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py&quot;, line 108, in _method_wrapper
      return method(self, *args, **kwargs)
    File &quot;/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py&quot;, line 1379, in evaluate
      tmp_logs = test_function(iterator)
    File &quot;/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py&quot;, line 780, in __call__
      result = self._call(*args, **kwds)
    File &quot;/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py&quot;, line 823, in _call
      self._initialize(args, kwds, add_initializers_to=initializers)
    File &quot;/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py&quot;, line 697, in _initialize
      *args, **kwds))
    File &quot;/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/function.py&quot;, line 2855, in _get_concrete_function_internal_garbage_collected
      graph_function, _, _ = self._maybe_define_function(args, kwargs)
    File &quot;/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/function.py&quot;, line 3213, in _maybe_define_function
      graph_function = self._create_graph_function(args, kwargs)
    File &quot;/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/function.py&quot;, line 3075, in _create_graph_function
      capture_by_value=self._capture_by_value),
    File &quot;/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py&quot;, line 986, in func_graph_from_py_func
      func_outputs = python_func(*func_args, **func_kwargs)
    File &quot;/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py&quot;, line 600, in wrapped_fn
      return weak_wrapped_fn().__wrapped__(*args, **kwds)
    File &quot;/home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py&quot;, line 973, in wrapper
      raise e.ag_error_metadata.to_exception(e)
  tensorflow.python.framework.errors_impl.OperatorNotAllowedInGraphError: in user code:
      /home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py:1224 test_function  *
          return step_function(self, iterator)
      /home/dbadmin/woosung/DWT-AutoEncoder-Comparison/source/autoencoder.py:57 call  *
          batch, time_step, features = tf.shape(encoder_input)
      /home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/ops.py:503 __iter__
          self._disallow_iteration()
      /home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/ops.py:496 _disallow_iteration
          self._disallow_when_autograph_enabled(&quot;iterating over `tf.Tensor`&quot;)
      /home/dbadmin/anaconda3/lib/python3.7/site-packages/tensorflow/python/framework/ops.py:474 _disallow_when_autograph_enabled
          &quot; indicate you are trying to use an unsupported feature.&quot;.format(task))
      OperatorNotAllowedInGraphError: iterating over `tf.Tensor` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.
  </code></pre>
  <p>I guess the error comes in the reason of that it is not possible to get exact input shape until I put explicit data.</p>
  <p>However, still not clear how to fix it.</p>