title: How to change dimension of input during TensorFlow import_graph_def
body: <p>My scenario:</p>
  
  <ol>
  <li>Define an RNN model structure and train it using an input with fixed batch size and sequence length.</li>
  <li>Freeze the model (i.e. converting all trainable variables into constants) producing a <code>GraphDef</code> containing everything one needs to use the model at test-time (via <code>tf.graph_util.convert_variables_to_constants</code>).</li>
  <li>Import the <code>GraphDef</code> via <code>tf.import_graph_def</code> and replace the input using the <code>input_map</code> argument. The new input needs to have arbitrary batch size and sequence length.</li>
  </ol>
  
  <p>The problem: All of the above works until I pass in an input to the test-time graph that uses a batch size or sequence length that differs from the original sizes used at training-time. At that point I get an error like this:</p>
  
  <pre><code>InvalidArgumentError (see above for traceback): ConcatOp : Dimensions of inputs should match: shape[0] = [1,5] vs. shape[1] = [2,7]
       [[Node: import/rnn/while/basic_rnn_cell/basic_rnn_cell_1/concat = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/cpu:0"](import/rnn/while/TensorArrayReadV3, import/rnn/while/Identity_2, import/rnn/while/basic_rnn_cell/basic_rnn_cell_1/concat/axis)]]
  </code></pre>
  
  <p>To illustrate and reproduce the problem, please consider the following minimal examples.</p>
  
  <ul>
  <li><code>v1</code>: a graph is created with arbitrary batch size and sequence length. This works fine but unfortunately I must use a fixed batch size and sequence length at training-time and must use an arbitrary batch size and sequence length at test-time so I can't use this simple approach.</li>
  <li><code>v2a</code>: we simulate creating the training-time graph with fixed batch size (2) and sequence length (3) and freeze the graph.</li>
  <li><code>v2ba</code>: we demonstrate that loading the frozen model in unchanged still produces the same results.</li>
  <li><code>v2bb</code>: we demonstrate that loading the frozen model in with a replaced input that still uses a fixed batch size and sequence length still produces the same results.</li>
  <li><code>v2bc</code>: we demonstrate that loading the frozen model in with a replaced input that uses arbitrary batch size and sequence length still produces the same results, <em>as long as the input is shaped according to the original batch size and sequence length</em>. It works with <code>data</code> but fails with <code>data2</code> -- the only difference being the batch size of the former is 2 and the batch size of the latter is 1.</li>
  </ul>
  
  <p><strong>Is it possible to change an RNN graph via the <code>input_map</code> argument to <code>tf.import_graph_def</code> such that the input no longer has a fixed batch size and sequence length?</strong></p>
  
  <p>The following code works with TensorFlow 1.1 RC2 and may work with TensorFlow 1.0.</p>
  
  <pre><code>import numpy
  import tensorflow as tf
  from tensorflow import graph_util as tf_graph_util
  from tensorflow.contrib import rnn as tfc_rnn
  
  
  def v1(data):
      with tf.Graph().as_default():
          tf.set_random_seed(1)
          x = tf.placeholder(tf.float32, shape=(None, None, 5))
          _, s = tf.nn.dynamic_rnn(tfc_rnn.BasicRNNCell(7), x, dtype=tf.float32)
  
          with tf.Session() as session:
              session.run(tf.global_variables_initializer())
              print session.run(s, feed_dict={x: data})
  
  
  def v2a():
      with tf.Graph().as_default():
          tf.set_random_seed(1)
          x = tf.placeholder(tf.float32, shape=(2, 3, 5), name="x")
          _, s = tf.nn.dynamic_rnn(tfc_rnn.BasicRNNCell(7), x, dtype=tf.float32)
  
          with tf.Session() as session:
              session.run(tf.global_variables_initializer())
              return tf_graph_util.convert_variables_to_constants(
                  session, session.graph_def, [s.op.name]), s.name
  
  
  def v2ba((graph_def, s_name), data):
      with tf.Graph().as_default():
          x, s = tf.import_graph_def(graph_def,
                                     return_elements=["x:0", s_name])
  
          with tf.Session() as session:
              print '2ba', session.run(s, feed_dict={x: data})
  
  
  def v2bb((graph_def, s_name), data):
      with tf.Graph().as_default():
          x = tf.placeholder(tf.float32, shape=(2, 3, 5))
          [s] = tf.import_graph_def(graph_def, input_map={"x:0": x},
                                    return_elements=[s_name])
  
          with tf.Session() as session:
              print '2bb', session.run(s, feed_dict={x: data})
  
  
  def v2bc((graph_def, s_name), data):
      with tf.Graph().as_default():
          x = tf.placeholder(tf.float32, shape=(None, None, 5))
          [s] = tf.import_graph_def(graph_def, input_map={"x:0": x},
                                    return_elements=[s_name])
  
          with tf.Session() as session:
              print '2bc', session.run(s, feed_dict={x: data})
  
  
  def main():
      data1 = numpy.random.random_sample((2, 3, 5))
      data2 = numpy.random.random_sample((1, 3, 5))
      v1(data1)
      model = v2a()
      v2ba(model, data1)
      v2bb(model, data1)
      v2bc(model, data1)
      v2bc(model, data2)
  
  
  if __name__ == "__main__":
      main()
  </code></pre>