title: Train Error with CNN in Tensorflow
body: <p>I'm trying to set up a CNN to classify images with two possible outputs and I'm using Tensorflow to do this. I followed the tutorials and then moved on to adapting the CNN they had used to my problem, but it hasn't worked out very well. </p>
  
  <p>The first thing I changed was how the images for my problem were loaded in. I have another script that writes the location of all the images along with their expected output (separated by a space). I used some code I found elsewhere to load in the images, which gives out the batches (label_batch and image_batch). However, as this format is different from the tutorial, I don't know how to do the training loop. I've tried a variety of things from taking indexes of the batches to running sess.run() there and tried a bunch of things I found online, but so far nothing has helped.</p>
  
  <p>Sorry if this is something very simple, I'm fairly new to this and just starting to feel my way through.</p>
  
  <p>My Code:</p>
  
  <pre><code>#imports tensorflow
  import tensorflow as tf
  sess = tf.InteractiveSession()
  
  #weight generation with small amount of noise in normal dist &amp; slight + bias bc ReLU nuerons
  def weight_variable(shape):
    initial = tf.truncated_normal(shape, stddev=0.1)
    return tf.Variable(initial)
  
  #bias generation with small amount of noise in normal dist &amp; slight + bias bc ReLU nuerons
  def bias_variable(shape):
    initial = tf.constant(0.1, shape=shape)
    return tf.Variable(initial)
  
  #creates conv layer with stride 1 and 0 padding
  def conv2d(x, W):
    return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
  
  #creates max_pool layer thats 2x2
  def max_pool_2x2(x):
    return tf.nn.max_pool(x, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')    
  
  #reads image file names and respective labels
  def read_labeled_image_list(image_list_file):
      f = open(image_list_file, 'r')
      filenames = []
      labels = []
      for line in f:
      filename, label = line[:-1].split(' ')
      filenames.append(filename)
      labels.append(int(label))
      return filenames, labels
  
  #image name to image 
  def read_images_from_disk(input_queue):
      label = input_queue[1]
      file_contents = tf.read_file(str(input_queue[0]))
      example = tf.image.decode_png(file_contents, channels=1)
      return example, label
  
  # Reads paths of images together with their labels
  image_list, label_list = read_labeled_image_list("images.txt")
  
  images = tf.convert_to_tensor(image_list)
  labels = tf.convert_to_tensor(label_list)
  
  # Makes an input queue
  input_queue = tf.train.slice_input_producer([images, labels], shuffle=True)
  
  image, label = read_images_from_disk(input_queue)
  image.set_shape([28,28,1])
  
  #Image and Label Batching
  image_batch, label_batch = tf.train.batch([image, label],batch_size=50, allow_smaller_final_batch = True)
  
  #placeholder define vars?
  x = tf.placeholder(tf.float32, shape=[None, 784])
  y_ = tf.placeholder(tf.float32, shape=[None, 2])
  
  #conv layer 1, 5x5 patch with 32 features
  W_conv1 = weight_variable([5, 5, 1, 32])
  b_conv1 = bias_variable([32])
  
  #4D tensor, 2 and 3 is w and h, 4th is color channels
  x_image = tf.reshape(x, [-1,28,28,1])
  
  #sets up forward for x_image
  h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
  h_pool1 = max_pool_2x2(h_conv1)
  
  #second conv layer, 64 feature extraction
  W_conv2 = weight_variable([5, 5, 32, 64])
  b_conv2 = bias_variable([64])
  h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
  h_pool2 = max_pool_2x2(h_conv2)
  
  #converts from feature to 1024
  W_fc1 = weight_variable([7 * 7 * 64, 1024])
  b_fc1 = bias_variable([1024])
  h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64])
  h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)
  
  #prevents overfitting, disabled during testing (todo: potentially     remove for us, is complex)
  keep_prob = tf.placeholder(tf.float32)
  h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)
  
  #softmax layer with 10 outputs
  W_fc2 = weight_variable([1024, 2])
  b_fc2 = bias_variable([2])
  y_conv=tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)
  
  
  #training: ADAM optimizer with overfitting help and logging every 100th     iteration
  cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_conv), reduction_indices=[1]))
  train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
  correct_prediction = tf.equal(tf.argmax(y_conv,1), tf.argmax(y_,1))
  accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
  sess.run(tf.initialize_all_variables())
  
  for i in range(20000):
    #What do I do here!?!??!
    #imgs, lbls = sess.run([image_batch, label_batch])
    #imgs = image_batch[i]
    #lbls = label_batch[i]
    if i%100 == 0:
      train_accuracy = accuracy.eval(feed_dict={ x:imgs, y_: lbls, keep_prob: 1.0})
      print("step %d, training accuracy %g"%(i, train_accuracy))
    train_step.run(feed_dict={x: imgs, y_: lbls, keep_prob: 0.5})
  
  #prints final accuracy, to be updated
  print("test accuracy %g"%accuracy.eval(feed_dict={
      x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))
  </code></pre>
  
  <p>Update:
  <a href="https://stackoverflow.com/questions/37454932/tensorflow-train-step-feed-incorrect">Tensorflow, train_step feed incorrect</a>
  I found this and attempted to implement the second answer but I get this error:</p>
  
  <pre><code>W tensorflow/core/framework/op_kernel.cc:936] Unimplemented: File system scheme Tensor("input_producer/Gather not implemented
  </code></pre>
  
  <p>When I tried the first solution, I got this: </p>
  
  <p>Traceback (most recent call last):</p>
  
  <pre><code>  File "Tester.py", line 77, in &lt;module&gt;
      h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
    File "Tester.py", line 22, in conv2d
       return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
    TypeError: DataType uint8 for attr 'T' not in list of allowed values: float16, float32, float64
  </code></pre>
  
  <p>Update 2
  So I realized x and y have to be equal to image_batch and label_batch respectively and it worked once I casted it to float32 using tf.cast(image_batch, tf.float32). However, now the train line still fails with this printed twice in a row:</p>
  
  <pre><code> W tensorflow/core/framework/op_kernel.cc:936] Unimplemented: File system scheme Tensor("input_producer/Gather not implemented
  </code></pre>