title: My regression network is not learning
body: <p>I have my data tested in the simplest regression network with tensorflow, and I dont get valid learning. Here is my code:</p>
  
  <pre><code>import tensorflow as tf
  
  with tf.variable_scope("Inputs"):
      tf_x = tf.placeholder(tf.float32, xs.shape, name="x")
      tf_y = tf.placeholder(tf.float32, ys.shape, name="y")
  
  with tf.variable_scope("Net"):
      l1 = tf.layers.dense(tf_x, 3, tf.nn.relu)
      weights = l1.graph.get_tensor_by_name(os.path.split(l1.name)[0] + "/kernel:0")
      output = tf.layers.dense(l1, 1)
      tf.summary.histogram("h_out", l1)
      tf.summary.histogram("pred", output)
      tf.summary.histogram("weights", weights)
  
  loss = tf.losses.mean_squared_error(tf_y, output, scope="loss")
  optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.05)
  train_op = optimizer.minimize(loss)
  tf.summary.scalar("loss", loss)
  sess = tf.Session()
  sess.run(tf.global_variables_initializer())
  
  writer = tf.summary.FileWriter("./logs", sess.graph)
  merge_op = tf.summary.merge_all()
  
  for step in range(1000):
      _, l, pred, result = sess.run([train_op, loss, output, merge_op], {tf_x:xs, tf_y:ys})
      writer.add_summary(result, step)
      if step % 100 == 0:
          print (l)
  </code></pre>
  
  <p>My data is :</p>
  
  <pre><code>x:   
       2.120
       1.860
       2.310
       2.060
       2.520
       1.770
       1.450
       1.420
       2.250
       1.930
       2.550
       2.050
       2.250
       2.570
       1.790
       2.380
       2.570
       1.850
       2.740
       1.830
       2.360
       2.460
  
  
  y:
    27.00000
    57.00000
    98.00000
   267.00000
    59.00000
   142.00000
   110.00000
   135.00000
    91.00000
   119.00000
    62.00000
    40.00000
   166.00000
   116.00000
   335.00000
    39.00000
    67.00000
    48.00000
    35.00000
    33.00000
    48.00000
    35.00000
  </code></pre>
  
  <p>Here is my loss and weights:</p>
  
  <p><a href="https://i.stack.imgur.com/Rr25V.png" rel="nofollow noreferrer">loss</a> <a href="https://i.stack.imgur.com/TynfE.png" rel="nofollow noreferrer">weights</a></p>