title: Binary classifier always returns 0.5
body: <p>I am training a classifier that takes a RGB input (so three 0 to 255 values) and returns whether black or white (0 or 1) font would fit best with that colour. After training, my classifier always returns 0.5 (or there about) and never gets any more accurate than that.</p>
  
  <p>The code is below:</p>
  
  <pre><code>import tensorflow as tf
  import numpy as np
  from tqdm import tqdm
  
  print('Creating Datasets:')
  
  x_train = []
  y_train = []
  
  for i in tqdm(range(10000)):
      x_train.append([np.random.uniform(0, 255), np.random.uniform(0, 255), np.random.uniform(0, 255)])
  
  for elem in tqdm(x_train):
      if (((elem[0] + elem[1] + elem[2]) / 3) / 255) &gt; 0.5:
          y_train.append(0)
      else:
          y_train.append(1)
  
  x_train = np.array(x_train)
  y_train = np.array(y_train)
  
  graph = tf.Graph()
  
  with graph.as_default():
  
      x = tf.placeholder(tf.float32)
      y = tf.placeholder(tf.float32)
  
      w_1 = tf.Variable(tf.random_normal([3, 10], stddev=1.0), tf.float32)
      b_1 = tf.Variable(tf.random_normal([10]), tf.float32)
      l_1 = tf.sigmoid(tf.matmul(x, w_1) + b_1)
  
      w_2 = tf.Variable(tf.random_normal([10, 10], stddev=1.0), tf.float32)
      b_2 = tf.Variable(tf.random_normal([10]), tf.float32)
      l_2 = tf.sigmoid(tf.matmul(l_1, w_2) + b_2)
  
      w_3 = tf.Variable(tf.random_normal([10, 5], stddev=1.0), tf.float32)
      b_3 = tf.Variable(tf.random_normal([5]), tf.float32)
      l_3 = tf.sigmoid(tf.matmul(l_2, w_3) + b_3)
  
      w_4 = tf.Variable(tf.random_normal([5, 1], stddev=1.0), tf.float32)
      b_4 = tf.Variable(tf.random_normal([1]), tf.float32)
      y_ = tf.sigmoid(tf.matmul(l_3, w_4) + b_4)
  
      loss = tf.reduce_mean(tf.squared_difference(y, y_))
  
      optimizer = tf.train.AdadeltaOptimizer().minimize(loss)
  
      with tf.Session() as sess:
  
          sess.run(tf.global_variables_initializer())
  
          print('Training:')
  
          for step in tqdm(range(5000)):
              index = np.random.randint(0, len(x_train) - 129)
              feed_dict = {x : x_train[index:index+128], y : y_train[index:index+128]}
              sess.run(optimizer, feed_dict=feed_dict)
              if step % 1000 == 0:
                  print(sess.run([loss], feed_dict=feed_dict))
  
          while True:
              inp1 = int(input(''))
              inp2 = int(input(''))
              inp3 = int(input(''))
              print(sess.run(y_, feed_dict={x : [[inp1, inp2, inp3]]}))
  </code></pre>
  
  <p>As you can see, I start by importing the modules I will be using. Next I generate my input x dataset and desired output y dataset. The x_train dataset consists of 10000 random RGB values, while the y_train dataset consists of 0's and 1's, with a 1 corresponding to an RGB value with a mean lower than 128 and a 0 corresponding to an RGB value with a mean higher than 128 (this ensures bright backgrounds get dark font and vice versa).</p>
  
  <p>My neural net is admittedly overly complex (or so i assume), but as far as I am aware it is a pretty standard feed forward net, with an Adadelta optimiser and the default learning rate.</p>
  
  <p>The training of the net is normal as far as my limited knowledge informs me, but nonetheless the model always spits out 0.5.</p>
  
  <p>The last block of code allows the user to input values and see what they turn into when passed to the neural net.</p>
  
  <p>I have messed around with different activation functions, losses, methods of initialising biases etc. But to no avail. Some times when I tinker with the code, the model always returns 1 or 0 respectively, but this is still just as inaccurate as being indecisive and returning 0.5 over and over. I have not been able to find a suitable solution to my problem online. Any advice or suggestions are welcome.   </p>
  
  <p>Edit:</p>
  
  <p>The loss, weights, biases and the output don't change much over the course of training (the weights and biases only change by hundredths and thousandths every 1000 iterations, and the loss fluctuates around 0.3). Also, the output sometimes varies f depending on the input (as you would expect), but other times is constant. One run of the program lead to constant 0.7's as output, while another always returned 0.5 apart from very near to zero, where it returned 0.3 or 0.4 type values. Neither of the aforementioned are the desired output. What should happen is that (255, 255, 255) should map to 0 and (0, 0, 0) should map to 1 and (128, 128, 128) should map to either 1 or 0, as in the middle the font colour doesn't really matter.</p>