title: Running inference with a loaded tensorflow model in C++
body: <p>My tensorflow model for this example is simple and has been successfully loaded. My question is, to run inference, what do I pass the tf::Session? How do I create the feedDict mentioned <a href="https://stackoverflow.com/questions/35508866/tensorflow-different-ways-to-export-and-run-graph-in-c/43639305#43639305">here</a> and what are my "outputOps"?</p>
  
  <p>Here's the model in python:</p>
  
  <pre><code>model = tf.keras.Sequential([
  tf.keras.layers.Convolution2D(filters, (5, 5), input_shape=(BoardDepth, BoardLength, BoardLength), data_format='channels_first', name='Input'),
  tf.keras.layers.Activation('relu', name='Relu'),
  tf.keras.layers.MaxPooling2D(pool_size=(2,2), name='Pool'),
  tf.keras.layers.Flatten(name="Flatten"),
  tf.keras.layers.Dense(BoardSize, activation='softmax', name='Output'),
  ])
  
  optimizer = tf.train.AdamOptimizer(learning_rate=0.0018)
  
  model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy']) 
  </code></pre>
  
  <p>Here's the method used to train the model which uses a generator to feed images in:</p>
  
  <pre><code>earlyExit = keras.callbacks.EarlyStopping(monitor='val_acc', min_delta=0, patience=0)
  
  # Train the model
  history = model.fit_generator(generator=gen.generator(),
                  steps_per_epoch=gen.stepsPerEpoch(),
                  validation_data=valGen.generator(),
                  validation_steps=valGen.stepsPerEpoch(),
                  epochs=numEpochs, 
                  verbose=2, workers=1, callbacks=[earlyExit])
  </code></pre>
  
  <p>And here's the method used to save the model:</p>
  
  <pre><code>K.set_learning_phase(0)
  with tf.Session() as sess:
      sess.run(tf.global_variables_initializer())
      saver = tf.train.Saver(tf.trainable_variables())
      saver.save(sess, './models/test/latestModel')
  </code></pre>
  
  <p>In case you didn't follow the link above inference is supposed to be done with something like this:</p>
  
  <pre><code>// and run the inference to your liking
  auto feedDict = ...
  auto outputOps = ...
  std::vector&lt;tensorflow::Tensor&gt; outputTensors;
  status = session-&gt;Run(feedDict, outputOps, {}, &amp;outputTensors);
  </code></pre>
  
  <p>I've tried following the example <a href="https://github.com/PatWie/tensorflow_inference/blob/master/inference/cc/inference_cc.cc" rel="nofollow noreferrer">here</a> but cannot seem to figure out what I need to pass as outputOps. Do I need to only pass the name of the output node, or do I need to pass every single operation in the model? The different configurations I've tried so far haven't worked. </p>
  
  <p>My code so far looks something like this:</p>
  
  <pre><code>auto shape = tf::TensorShape({ BoardDepth, BoardSize, BoardSize});
  
  // Placeholder until we're using actual input data
  float inData[inputSize] = { 0.0 };
  
  tf::Tensor input(tf::DT_FLOAT, shape);
  std::copy_n(inData, inputSize, input.flat&lt;float&gt;().data());
  
  std::vector&lt;std::pair&lt;std::string, tf::Tensor&gt;&gt; inputs = {
      { "Input/Conv2D", input },
  };
  
  std::vector&lt;std::string&gt; outOps = { "Output/bias:0"};
  
  std::vector&lt;tf::Tensor&gt; outputs;
  
  status = session-&gt;Run(inputs, outOps, {}, &amp;outputs);
  </code></pre>
  
  <p>The input/output names are just set to names I could find that don't throw errors. I really don't know what I'm supposed to be setting them to in order for the model to function as it was trained. </p>