title: How to use dataset.shard in tensorflow?
body: <p>Recently I am looking into the dataset API in Tensorflow, and there is a method <code>dataset.shard()</code> which is for distributed computations.</p>
  
  <p>This is what's stated in Tensorflow's documentation:</p>
  
  <pre><code>Creates a Dataset that includes only 1/num_shards of this dataset.
  
  d = tf.data.TFRecordDataset(FLAGS.input_file)
  d = d.shard(FLAGS.num_workers, FLAGS.worker_index)
  d = d.repeat(FLAGS.num_epochs)
  d = d.shuffle(FLAGS.shuffle_buffer_size)
  d = d.map(parser_fn, num_parallel_calls=FLAGS.num_map_threads)
  </code></pre>
  
  <p>This method is said to return a portion of the original dataset. If I have two workers, am I supposed to do:</p>
  
  <pre><code>d_0 = d.shard(FLAGS.num_workers, worker_0)
  d_1 = d.shard(FLAGS.num_workers, worker_1)
  ......
  iterator_0 = d_0.make_initializable_iterator()
  iterator_1 = d_1.make_initializable_iterator()
  
  for worker_id in workers:
      with tf.device(worker_id):
          if worker_id == 0:
              data = iterator_0.get_next()
          else:
              data = iterator_1.get_next()
          ......
  </code></pre>
  
  <p>Because the documentation did not specify how to make subsequent calls, I am a bit confused here.</p>
  
  <p>Thanks!</p>