title: Evaluating Tensorflow operation is very slow in a loop
body: <p>I'm trying to learn tensorflow by coding up some simple problems: I was trying to find the value of pi using a direct sampling Monte Carlo method.</p>
  
  <p>The run time is much longer than I thought it would be when using a <code>for loop</code> to do this. I've seen other posts about similar things and I've tried to follow the solutions, but I think I still must be doing something wrong.</p>
  
  <p>Attached below is my code:</p>
  
  <pre><code>import tensorflow as tf
  import numpy as np
  import time
  
  n_trials = 50000
  
  tf.reset_default_graph()
  
  
  x = tf.random_uniform(shape=(), name='x')
  y = tf.random_uniform(shape=(), name='y')
  r = tf.sqrt(x**2 + y**2)
  
  hit = tf.Variable(0, name='hit')
  
  # perform the monte carlo step
  is_inside = tf.cast(tf.less(r, 1), tf.int32)
  hit_op = hit.assign_add(is_inside) 
  
  with tf.Session() as sess:
      init_op = tf.global_variables_initializer()
      sess.run(init_op)
  
      # Make sure no new nodes are added to the graph
      sess.graph.finalize()
  
      start = time.time()   
  
      # Run monte carlo trials  -- This is very slow
      for _ in range(n_trials):
          sess.run(hit_op)
  
      hits = hit.eval()
      print("Pi is {}".format(4*hits/n_trials))
      print("Tensorflow operation took {:.2f} s".format((time.time()-start)))
  
  &gt;&gt;&gt; Pi is 3.15208
  &gt;&gt;&gt; Tensorflow operation took 8.98 s
  </code></pre>
  
  <p>In comparison, doing a <code>for loop</code> type solution in numpy is an order of magnitude faster</p>
  
  <pre><code>start = time.time()   
  hits = [ 1 if np.sqrt(np.sum(np.square(np.random.uniform(size=2)))) &lt; 1 else 0 for _ in range(n_trials) ]
  a = 0
  for hit in hits:
      a+=hit
  print("numpy operation took {:.2f} s".format((time.time()-start)))
  print("Pi is {}".format(4*a/n_trials))
  
  &gt;&gt;&gt; Pi is 3.14032
  &gt;&gt;&gt; numpy operation took 0.75 s
  </code></pre>
  
  <p>Attached below is a plot of the difference in overall executioin times for various numbers of trials. </p>
  
  <p><a href="https://i.stack.imgur.com/brUjQ.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/brUjQ.png" alt="enter image description here"></a>  </p>
  
  <p>Please note: my question is not about "how to perform this task the fastest", I recognize there are much more effective ways of calculating Pi. I've only used this as a benchmarking tool to check the performance of tensorflow against something I'm familiar with (numpy). </p>