title: How to make best use of GPU for TensorFlow Estimators?
body: <p>I was using Tensorflow(CPU version) for my Deep Learning Model. Specifically using DNNRegressor Estimator for training, with given set of parameters (network structure, hidden layers, alpha etc.) Though I was able to reduce the loss, but model took very large time for learning (approx 3 days.) and time it was taking was 9 sec per 100th step.</p>
  
  <p><a href="https://i.stack.imgur.com/7SyG4.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/7SyG4.png" alt="enter image description here"></a></p>
  
  <p>I came accross this article :- 
  <a href="https://medium.com/towards-data-science/how-to-traine-tensorflow-models-79426dabd304" rel="nofollow noreferrer">https://medium.com/towards-data-science/how-to-traine-tensorflow-models-79426dabd304</a>
  and found that GPU's can be more faster to learn. So i took p2.xlarge gpu from AWS (single core GPU)<a href="https://i.stack.imgur.com/Lw3R6.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Lw3R6.png" alt="enter image description here"></a> with 4(vCPU), 12(ECU) and 61 (MiB). </p>
  
  <p>But the learning rate is same at 9 sec per 100th step. I m using same code I used for Estimators on CPU, because I read that Estimators use GPU on their own. Here is my "nvidia-smi" command output.
  <a href="https://i.stack.imgur.com/73zdK.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/73zdK.png" alt="enter image description here"></a></p>
  
  <ul>
  <li>It is showing that GPU Memory being used, but my Volatile GPU-Util is at 1%. Not able to figure out, what I am missing out. Is it designed to work same or I m missing something, because global steps per sec is same for both CPU and GPU implementation of Estimators.</li>
  <li>Do I have to explicitly change something in DNNRegressor Estimator Code?</li>
  </ul>