title: tf.conv2d gaussian kernel convolution gives strange border effect in Tensorflow
body: <p>Currently my code is succeeding in visualizing the deeper layers of a network via activation maximization. However, in order to get a more interpretable image, I'm experimenting with different regularization methods. currently I'm regularizing via a Gaussian convolution. See Understanding Neural Networks Through Deep Visualization by Yosinski . </p>
  
  <p>To do this I've added a Gaussian loss to my loss function. I'm using Python &amp; Tensorflow. The Gaussian loss is calculated by (each iteration) subtracting a blurred image from the current image, and thereby steering the network towards producing a more blurry final image.</p>
  
  <p>First a Gaussian kernel is made of size 4x4.
  Then, I perform a convolution with this kernel for each color channel through tf.conv2d with the code: (gauss_var is the gaussian kernel with dimension [4, 4, 1, 1])</p>
  
  <pre><code># unstack 3 channel image
  [tR, tG, tB] = tf.unstack(input_image, num=3, axis=3)
  # give each one a fourth dimension in order to use it in conv2d
  tR = tf.expand_dims(tR, 3)
  tG = tf.expand_dims(tG, 3)
  tB = tf.expand_dims(tB, 3)
  
  #convolve each input image with the gaussian filter
  tR_gauss = tf.nn.conv2d(tR, gauss_var, strides=[1, 1, 1, 1], padding='SAME')
  tG_gauss = tf.nn.conv2d(tG, gauss_var, strides=[1, 1, 1, 1], padding='SAME')
  tB_gauss = tf.nn.conv2d(tB, gauss_var, strides=[1, 1, 1, 1], padding='SAME')
  </code></pre>
  
  <p>I calculate the difference by doing:</p>
  
  <pre><code># calculate difference
  R_diff = tf.subtract(tR, tR_gauss)
  G_diff = tf.subtract(tR, tG_gauss)
  B_diff = tf.subtract(tR, tB_gauss)
  </code></pre>
  
  <p>And make it into one number:</p>
  
  <pre><code>total_diff = tf.add_n([R_diff, G_diff, B_diff])
  gaussian_loss = tf.reduce_sum(total_diff)
  </code></pre>
  
  <p>The problem is that the resulting image always shows bars at the borders, and is colored blueish. <a href="https://i.stack.imgur.com/wyvok.png" rel="nofollow noreferrer">This</a> is an over-exaggerated example of a final image. </p>
  
  <p>I'm pretty sure this bordering effect has something to do with conv2d, but I don't know how to change it. So far I've tried using different kernel sizes, and although the borders change, they still remain. Changing padding from 'SAME' to 'VALID' results in different output dimensions which is also problematic. Any ideas on how to solve this?</p>
  
  <p>Thanks in advance!
  Cheers,    </p>