title: Tensorflow dataset questions about .shuffle, .batch and .repeat
body: <p>I had a question about the use of batch, repeat and shuffle with tf.Dataset.</p>
  
  <p>It is not clear to me exactly how repeat and shuffle are used. I understand that <code>.batch</code> will dictate how many training examples will undergo stochastic gradient descent, the uses of <code>.repeat</code> and <code>.shuffle</code> are still not clear to me.</p>
  
  <p><strong>First Question</strong></p>
  
  <p>Even after reviewing <a href="https://stackoverflow.com/questions/50437234/tensorflow-dataset-shuffle-then-batch-or-batch-then-shuffle">here</a> and <a href="https://stackoverflow.com/questions/53514495/what-does-batch-repeat-and-shuffle-do-with-tensorflow-dataset">here</a>, <code>.repeat</code> is used to reiterate over the dataset once a <code>tf.errors.OutOfRangeError</code> is thrown. Therefore, in my code does that mean I no longer have to implement:</p>
  
  <pre><code>try:
      while True:
          _ = sess.run(self.optimizer)
  
  except tf.errors.OutOfRangeError:
          pass
  </code></pre>
  
  <p>because <code>.repeat</code> will automatically repeat the dataset once it is exhausted? When does it stop? or will it never stop and you just have to exit out of the while True loop once a certain number of batches (say 1000) have passed?</p>
  
  <p><strong>Second Question</strong></p>
  
  <p>Secondly, the use <code>.shuffle</code> makes no sense to me. Does <code>.shuffle.batch()</code> imply that I have, say, 100,000 samples, put 1000 randomly in a buffer with <code>.shuffle</code>, then batch say, 100 of them with <code>.batch()</code>. From my understanding the next batch will use 999 of those samples and place 1 new one in the buffer. So if my samples have no order to them, then <code>.shuffle</code> should be avoided all together? And if <code>.batch</code> is used, it would still batch 100 from those 999+1 in the buffer?</p>
  
  <p><strong>Third Question</strong></p>
  
  <p>And lastly, if I am using a separate <code>td.dataset</code> object for testing, what order of <code>.shuffle.batch()</code> should I consider? Right now I use:</p>
  
  <pre><code>sess.run(self.test_init)
  try:
      while True:
          accuracy_batch = sess.run(self.accuracy)
  
  except tf.errors.OutOfRangeError:
      pass
  </code></pre>
  
  <p>With:</p>
  
  <pre><code>test_data = self.test_dataset.shuffle(self.batch_size).batch(self.batch_size)
  </code></pre>
  
  <p>I have over 110,000 training examples at my disposal, so self.batch_size will set the number of samples I want to use to test my accuracy. So, if I wanted to just test on the whole test dataset I wouldn't use <code>.batch</code>? But since I have it iterating over the whole dataset with <code>while True</code>, it makes no difference? With the use of <code>.shuffle</code> I noticed my accuracies changed, but without it they were very similar. This makes me think <code>.shuffle</code>is randomizing the batch and may be reusing training examples?</p>