title: Low gpu usage of tensorflow in Windows
body: <p>I recently installed tensorflow-gpu for Windows 10, and I followed all of the steps, including CUDA 9.0 and cudadnn. But computation speed doesn't increase dramatically after I run the code.</p>
  
  <pre><code>import numpy as np
  import tensorflow as tf
  tf.set_random_seed(777)
  
  origin_data=np.load("train-imgs.npz")['arr_0']
  answer_data=np.load("train-labels.npz")['arr_0']
  
  d2_origin_data,answer_2d=[],[]
  
  for i in range(len(origin_data)):
      d2_origin_data.append(origin_data[i].flatten())
  
  answer_data=np.reshape(answer_data,(-1,1))
  
  num_classes=10
  
  input_feed=tf.placeholder(tf.float32,[None,len(d2_origin_data[0])])
  answer_feed=tf.placeholder(tf.int32,[None,1])
  w=tf.Variable(tf.random_normal([len(d2_origin_data[0]),num_classes]))
  b=tf.Variable(tf.random_normal([num_classes]))
  
  with tf.device('/gpu:1'):
      our_answer=tf.nn.softmax(tf.matmul(input_feed,w*0.001)+b)
      vec_answer=tf.one_hot(answer_feed,num_classes) 
      vec_answer=tf.reshape(vec_answer,[-1,num_classes]) #reduce dimension
      cost=tf.reduce_meantf.reduce_sum(vec_answer*tf.log(our_answer),axis=1))
  train=tf.train.AdamOptimizer(learning_rate=0.1).minimize(cost)
  
  sess = tf.Session(config=tf.ConfigProto( allow_soft_placement=True, log_device_placement=True))
  sess.run(tf.global_variables_initializer())
  
  for i in range(1000):        
      cost_val,acc,_=sess.run([cost,accuracy,train],feed_dict{input_feed:d2_origin_data,answer_feed:answer_data})
      if i%50==0:
          print(i,acc,cost_val)
  </code></pre>
  
  <p><strong>GPU usage:</strong></p>
  
  <p><img src="https://i.stack.imgur.com/8xebh.png" alt="gpu usage"></p>
  
  <p><strong>CPU usage:</strong></p>
  
  <p><img src="https://i.stack.imgur.com/pTFpk.png" alt="cpu usage"></p>
  
  <p>I used 62.8mb 'train-imgs.npz' file and 160kb 'train-labels.npz' file. 
  As shown above photes, gpu utilisation fluctuates and doesn't exceeds 90%. I think training step is still low as I run the code with GPU because of low GPU utilisation.</p>
  
  <p>However, when I run the cnn code which requires more complex calculation my laptop uses GPU over 60% consistently.
  I knew that GPU usage depends on networks size, batch size and preprocessing. How should I do in order to increase GPU usage over 90% consistently?</p>