title: How do you make TensorFlow + Keras fast with a TFRecord dataset?
body: <p><strong>What is an example of how to use a TensorFlow TFRecord with a Keras Model and tf.session.run() while keeping the dataset in tensors w/ queue runners?</strong></p>
  
  <p>Below is a snippet that works but it needs the following improvements:</p>
  
  <ul>
  <li>Use the <a href="https://keras.io/models/model/" rel="nofollow noreferrer">Model API</a></li>
  <li>specify an Input()</li>
  <li>Load a dataset from a TFRecord</li>
  <li>Run through a dataset in parallel (such as with a queuerunner)</li>
  </ul>
  
  <p>Here is the snippet, there are several TODO lines indicating what is needed:</p>
  
  
  
  <pre class="lang-python prettyprint-override"><code>from keras.models import Model
  import tensorflow as tf
  from keras import backend as K
  from keras.layers import Dense, Input
  from keras.objectives import categorical_crossentropy
  from tensorflow.examples.tutorials.mnist import input_data
  
  sess = tf.Session()
  K.set_session(sess)
  
  # Can this be done more efficiently than placeholders w/ TFRecords?
  img = tf.placeholder(tf.float32, shape=(None, 784))
  labels = tf.placeholder(tf.float32, shape=(None, 10))
  
  # TODO: Use Input() 
  x = Dense(128, activation='relu')(img)
  x = Dense(128, activation='relu')(x)
  preds = Dense(10, activation='softmax')(x)
  # TODO: Construct model = Model(input=inputs, output=preds)
  
  loss = tf.reduce_mean(categorical_crossentropy(labels, preds))
  
  # TODO: handle TFRecord data, is it the same?
  mnist_data = input_data.read_data_sets('MNIST_data', one_hot=True)
  
  train_step = tf.train.GradientDescentOptimizer(0.5).minimize(loss)
  
  sess.run(tf.global_variables_initializer())
  
  # TODO remove default, add queuerunner
  with sess.as_default():
      for i in range(1000):
          batch = mnist_data.train.next_batch(50)
          train_step.run(feed_dict={img: batch[0],
                                    labels: batch[1]})
      print(loss.eval(feed_dict={img:    mnist_data.test.images, 
                                 labels: mnist_data.test.labels}))
  </code></pre>
  
  <p><strong>Why is this question relevant?</strong></p>
  
  <ul>
  <li>For high performance training without going back to python
  
  <ul>
  <li>no <a href="https://stackoverflow.com/questions/36026892/how-can-i-convert-tfrecords-into-numpy-arrays">TFRecord to numpy</a> to tensor conversions</li>
  </ul></li>
  <li><a href="https://github.com/fchollet/keras/issues/5358" rel="nofollow noreferrer">Keras will soon be part of tensorflow</a></li>
  <li>Demonstrate how Keras Model() classes can accept tensors for input data correctly.</li>
  </ul>
  
  <p><strong>Here is some starter information for a semantic segmentation problem example:</strong></p>
  
  <ul>
  <li>example unet Keras model <a href="https://github.com/ahundt/tf-image-segmentation/blob/cd7df53e6f59ef9e3dff1ed0119e12922ae98a3a/tf_image_segmentation/models/unet.py" rel="nofollow noreferrer">unet.py</a>, happens to be for semantic segmentation.</li>
  <li><a href="https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html" rel="nofollow noreferrer">Keras + Tensorflow Blog Post</a></li>
  <li>An <a href="https://github.com/ahundt/tf-image-segmentation/blob/cd7df53e6f59ef9e3dff1ed0119e12922ae98a3a/tf_image_segmentation/recipes/pascal_voc/FCNs/densenet_fcn.py#L78" rel="nofollow noreferrer">attempt at running the unet model a tf session with TFRecords and a Keras model</a> (not working)</li>
  <li>Code to create the TFRecords: <a href="https://github.com/ahundt/tf-image-segmentation/blob/cd7df53e6f59ef9e3dff1ed0119e12922ae98a3a/tf_image_segmentation/utils/tf_records.py" rel="nofollow noreferrer">tf_records.py</a></li>
  <li>An attempt at running the unet model a tf session with TFRecords and a Keras model is in <a href="https://github.com/ahundt/tf-image-segmentation/blob/cd7df53e6f59ef9e3dff1ed0119e12922ae98a3a/tf_image_segmentation/recipes/pascal_voc/FCNs/densenet_fcn.py#L78" rel="nofollow noreferrer">densenet_fcn.py</a> (not working)</li>
  </ul>