title: Tensorflow: why is multiplication 20x faster than subtraction in this case
body: <p>As in title, why is multiplication much faster than subtraction in this example?</p>
  
  <p>In this example we have M=784 inputs and N=100 outputs, so matrix multiplication requires MxN scalar multiplications and (M-1)xN additions, while subtraction requires only MxN scalar subtractions.</p>
  
  <p>Is it because software and hardware is heavily optimized towards multiplication and addition?</p>
  
  <p>Or is it because of all the shape brodcasting that is done here and tensorflow is possibly inefficient at that? </p>
  
  <p>All the computation is done on CPU, so this might be another lead.</p>
  
  <p>Code is as following:</p>
  
  <pre><code>input = tf.placeholder(tf.float32, [None, 784])
  
  W = tf.Variable(tf.truncated_normal(shape=[784, 100], mean=0., stddev=.5))
  
  sub = tf.expand_dims(input, axis=2)-tf.expand_dims(W1, axis=0)
  mul = tf.matmul(input*W)
  
  sess = tf.InteractiveSession()
  tf.global_variables_initializer().run()
  
  start = timer()
  sess.run(diff, {input: somedata}) #somedata has shape (100, 784)
  timer() - start #yields around 0.15s after averaging multiple tries
  
  start = timer()
  sess.run(mul, {input: somedata})
  timer() - start #yields around 0.008s after averaging
  </code></pre>