title: Tensorflow speed-up prediction
body: <p>I have a problem regarding prediction performance. What I do is I repeatedly call <code>test_predictions</code> op in Python loop and put all its return values into the list. The code looks like this:</p>
  
  <pre><code>predictions = []
  for _ in trange(args.num_batches):
      predictions.extend(sess.run(model.test_predictions))
  </code></pre>
  
  <p>When I look at performance statistics for more than 2/3 of time my GPU card is idle, probably because of continual switching between Python and TF code. I cannot make batch size bigger, because it won't fit in memory. Is there any better solution I can implement?</p>