title: How to process single training file in parallel
body: <p>I have a file <code>train.csv</code> that contains paths to images and their labels. ie:</p>
  
  <pre><code>img1.jpg 3
  img2.jpg 1
  ...
  </code></pre>
  
  <p>After going through the <a href="https://www.tensorflow.org/versions/r0.7/how_tos/reading_data/index.html" rel="nofollow">reading data tutorial</a> I came up with some code to go through each image, resize it and apply distortions:</p>
  
  <pre><code>def apply_distortions(resized_image):
      # do a bunch of tf.image distortion...
      return float_image
  
  def processing(filename):
      file_contents = tf.read_file(filename)
      image = tf.image.decode_jpeg(file_contents, channels=3)
      resized_image = tf.image.resize_images(image, 299, 299)
      distorted_image = apply_distortions(resized_image)
      return distorted_image
  
  def parse_csv(filename_queue):
      line_reader = tf.TextLineReader()
      key, line = line_reader.read(filename_queue)
      filename, label = tf.decode_csv(line,     # line_batch or line (depending if you want to batch)
                                 record_defaults=[tf.constant([],dtype=tf.string),
                                                  tf.constant([],dtype=tf.int32)],
                                 field_delim=' ')
      processed_image = processing(filename)
      return processed_image, label
  </code></pre>
  
  <p>The problem now is that I'm confused how to do these operations across the file in parallel. The documentation suggests either using <code>tf.train.batch_join</code> or <code>tf.train.batch</code> with num_threads=N.</p>
  
  <p>I first tried following the example code using <code>tf.train.batch_join</code> but this seems to be intended for processing multiple files in parallel. In my case however I just have 1 file. </p>
  
  <pre><code>filename_queue = tf.train.string_input_producer(["train.txt"], num_epochs=1, shuffle=True)    
  example_list = [parse_csv(filename_queue) for _ in range(8)]
  example_batch, label_batch = tf.train.batch_join(example_list, batch_size)
  </code></pre>
  
  <p>I also tried setting <code>tf.train.batch([example, label], batch_size, num_threads=8)</code> but its not clear to me if this is doing the right thing (although I can see more cpu cores in use)</p>
  
  <pre><code>filename_queue = tf.train.string_input_producer(["train.txt"], num_epochs=1, shuffle=True)
  example, label = parse_csv(filename_queue)
  example_batch, label_batch = tf.train.batch([example, label], batch_size, num_threads=8)
  </code></pre>
  
  <p>Here is my code for executing the graph:</p>
  
  <pre><code>sess.run(tf.initialize_all_variables())
  coord = tf.train.Coordinator()
  threads = tf.train.start_queue_runners(sess,coord)
  try:
      while not coord.should_stop():
          X, Y = sess.run([example_batch, label_batch])
          # Now run a training step
  except tf.errors.OutOfRangeError:
      print('Done training -- epoch limit reached')
  finally:
      # When done, ask the threads to stop.
      coord.request_stop()
  coord.join(threads)
  sess.close()
  </code></pre>
  
  <p>Whats the best way to process this file in parallel?</p>