title: Linear Regression With Manual Gradient Computation
body: <p>I understand <strong>Code 1</strong> is the code for the Linear Regression using <code>tf.train.GradientDescentOptimizer</code> which belong to TensorFlow library(black box).<br></p>
  
  <p><strong>Code 2</strong>  is a code example to do the same thing without <code>GradientDescentOptimizer</code>. 
  is the code without the black box.<br></p>
  
  <p>I want to add bias (<code># hypothesis = X * W + b</code>) in Code 2. In this case, how the code(gradient, descent, update, etc) should be?</p>
  
  <p>Code 1</p>
  
  <pre><code>import tensorflow as tf
  
  x_train = [1, 2, 3]
  y_train = [1, 2, 3]
  
  X = tf.placeholder(tf.float32)
  Y = tf.placeholder(tf.float32)
  W = tf.Variable(5.)
  b = tf.Variable(5.)
  hypothesis = X * W + b
  cost = tf.reduce_mean(tf.square(hypothesis - Y))
  learning_rate = 0.1
  
  optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)
  gvs = optimizer.compute_gradients(cost, [W, b])
  apply_gradients = optimizer.apply_gradients(gvs)
  
  sess = tf.Session()
  sess.run(tf.global_variables_initializer())
  for step in range(21):
      gradient_val, cost_val, _ = sess.run(
          [gvs, cost, apply_gradients], feed_dict={X: x_train, Y: y_train})
      print("%3d Cost: %10s, W': %10s, W: %10s, b': %10s, b: %10s" %
            (step, round(cost_val, 5),
             round(gradient_val[0][0] * learning_rate, 5), round(gradient_val[0][1], 5),
             round(gradient_val[1][0] * learning_rate, 5), round(gradient_val[1][1], 5)))
  </code></pre>
  
  <p>Code2</p>
  
  <pre><code>import tensorflow as tf
  
  x_train = [1, 2, 3]
  y_train = [1, 2, 3]
  
  X = tf.placeholder(tf.float32)
  Y = tf.placeholder(tf.float32)
  W = tf.Variable(5.)
  # b = tf.Variable(5.)  # Bias
  hypothesis = X * W
  # hypothesis = X * W + b
  cost = tf.reduce_mean(tf.square(hypothesis - Y))
  learning_rate = 0.1
  
  gradient = tf.reduce_mean((W * X - Y) * X) * 2
  descent = W - learning_rate * gradient
  update = tf.assign(W, descent)
  
  sess = tf.Session()
  sess.run(tf.global_variables_initializer())
  print(sess.run(W))
  for step in range(21):
      gradient_val, update_val, cost_val = sess.run(
          [gradient, update, cost], feed_dict={X: x_train, Y: y_train})
      print(step, gradient_val * learning_rate, update_val, cost_val)
  </code></pre>