title: How to visualize images in tensorboard using tf.summary.image() in [batch, channels, width, height] (NCHW) order?
body: <p>First, I want to reshape the 2-D to 4-D tensor using <code>tf.reshape()</code>.<br>
  I thought <code>tf.reshape()</code> will transform<br>
  <code>[batch, array]</code> -> <code>[batch, width, height, channels]</code> (NHWC) order<br>
  but in practice it transformed<br>
  <code>[batch, array]</code> -> <code>[batch, channels, width, height]</code> (NCHW) order</p>
  
  <p>Example:</p>
  
  <pre><code>a = np.array([[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16],[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]])
  print(a.shape)
  
  # [batch_size, channels, height, width]
  b = sess.run(tf.reshape(a, shape=[2, 3, 4, 4]))
  
  # [batch_size, height, width, channels]
  c = sess.run(tf.reshape(a, shape=[2, 4, 4, 3]))
  
  print(b)
  print('*******')
  print(c)
  </code></pre>
  
  <p>The result was:</p>
  
  <pre><code>(2, 48)
  [[[[ 1  2  3  4]
     [ 5  6  7  8]
     [ 9 10 11 12]
     [13 14 15 16]]
  
    [[ 1  2  3  4]
     [ 5  6  7  8]
     [ 9 10 11 12]
     [13 14 15 16]]
  
    [[ 1  2  3  4]
     [ 5  6  7  8]
     [ 9 10 11 12]
     [13 14 15 16]]]
  
  
   [[[ 1  2  3  4]
     [ 5  6  7  8]
     [ 9 10 11 12]
     [13 14 15 16]]
  
    [[ 1  2  3  4]
     [ 5  6  7  8]
     [ 9 10 11 12]
     [13 14 15 16]]
  
    [[ 1  2  3  4]
     [ 5  6  7  8]
     [ 9 10 11 12]
     [13 14 15 16]]]]
  *******
  [[[[ 1  2  3]
     [ 4  5  6]
     [ 7  8  9]
     [10 11 12]]
  
    [[13 14 15]
     [16  1  2]
     [ 3  4  5]
     [ 6  7  8]]
  
    [[ 9 10 11]
     [12 13 14]
     [15 16  1]
     [ 2  3  4]]
  
    [[ 5  6  7]
     [ 8  9 10]
     [11 12 13]
     [14 15 16]]]
  
  
   [[[ 1  2  3]
     [ 4  5  6]
     [ 7  8  9]
     [10 11 12]]
  
    [[13 14 15]
     [16  1  2]
     [ 3  4  5]
     [ 6  7  8]]
  
    [[ 9 10 11]
     [12 13 14]
     [15 16  1]
     [ 2  3  4]]
  
    [[ 5  6  7]
     [ 8  9 10]
     [11 12 13]
     [14 15 16]]]]
  </code></pre>
  
  <p>So, I changed <code>data_format='channels_first'</code> for conv and pooling layers to using the reshaped tensor in NCHW order. In fact, the training was good. <em><code>--verbose:</code></em> it gave better result, as mention by @mrry in <a href="https://stackoverflow.com/questions/41396970/tensorflow-3-channel-order-of-color-inputs/47704862#47704862">here</a>, and I think it could be understandable because NCHW is the default order of cuDNN.</p>
  
  <p>However, I can not add image to summary using <code>tf.summary.image()</code>, <a href="https://www.tensorflow.org/api_docs/python/tf/summary/image" rel="nofollow noreferrer">which is documented here</a>, because the required tensor shape should be in <code>[batch_size, height, width, channels]</code> order.  </p>
  
  <p>Moreover, if I train and visualize the input images in <code>[batch, width, height, channels]</code> order, it represented incorrect images.
  <a href="https://i.stack.imgur.com/e5zUu.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/e5zUu.png" alt="enter image description here"></a><br>
  And I worth to mention that, the training result was not as good as using <code>[batch, channels, width, height]</code> order.</p>
  
  <p>There are several questions:<br>
  <strong>1. Why</strong> <code>tf.reshape()</code> transform <code>[batch , array]</code> -> (NCHW) order instead of (NHWC) order ? I tested with both tf CPU and GPU, same result. I also used np.reshape(), also same result. <em>(This's why I think I could misunderstand something here)</em><br>
  <strong>2. How</strong> can I visualize images in tensorboard using tf.summary.image() in (NCHW) order? (<strong>question #2 solved</strong> using advice from <strong>@Maosi Chen</strong>. Thanks)<br>
  <a href="https://i.stack.imgur.com/g7Ks0.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/g7Ks0.png" alt="enter image description here"></a></p>
  
  <p>I've trained the model on <em>GPU (version 1.4)</em>, the images are from <em>CIFAR-10</em> dataset.<br>
  Thanks    </p>