title: Tensor Flow neural network model predicting unrelated data also
body: <h2>Neural Network Predicting unrelated data</h2>
  
  <p>My problem is There are 3 paths the object can follow Left, Right and Zigzag. I was training the neural network on one path suppose Left. And when I test the same neural network model in Right data it predicts really well but it shouldn't do that.
  The function create_train_model(...) is for training the neural network and getting the weights and biases of the model. And function evaluate_model(...) is to predict next point given a set of previous points. </p>
  
  <pre><code>def create_train_model(Xtrain, ytrain):
      tf.reset_default_graph()
  
      _, num_inputs = np.shape(Xtrain)
      _, num_outputs = np.shape(ytrain)
  
      keep_prob = 0.9
  
      weights = {
          'h1': tf.Variable(tf.random_normal([num_inputs, n_hidden_1], 0, 0.1)),
          'h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2], 0, 0.1)),
          'out': tf.Variable(tf.random_normal([n_hidden_2, num_outputs], 0, 0.1))
      }
  
      biases = {
          'b1': tf.Variable(tf.random_normal([n_hidden_1], 0, 0.1)),
          'b2': tf.Variable(tf.random_normal([n_hidden_2], 0, 0.1)),
          'out': tf.Variable(tf.random_normal([num_outputs], 0, 0.1))
      }
  
      X = tf.placeholder(dtype=tf.float32, shape=(None, num_inputs),  name='X')
      y = tf.placeholder(dtype=tf.float32, shape=(None, num_outputs), name='y')
  
      # Hidden layer with RELU activation
      layer_1 = tf.add(tf.matmul(X, weights['h1']), biases['b1'])
      layer_1 = tf.nn.relu(layer_1)
  
      layer_2 = tf.add(tf.matmul(layer_1, weights['h2']), biases['b2'])
      layer_2 = tf.nn.relu(layer_2)
  
      # Output layer with linear activation
      out_layer = tf.matmul(layer_2, weights['out']) + biases['out']
      cost = tf.reduce_sum(tf.square(out_layer - y))
      # # cost = tf.losses.huber_loss(y, out_layer)
      # # cost = tf.losses.mean_squared_error(y, out_layer)
      # cost = tf.losses.hinge_loss(y, out_layer)
  
  
      optimizer = tf.train.AdamOptimizer(learning_rate=0.001).minimize(cost)
  
      # Initialize variables and run session
      init = tf.global_variables_initializer()
  
      with tf.Session() as sess:
          sess.run(init)
  
          for i in range(epochs):
              _, c = sess.run([optimizer, cost],
                                 feed_dict={X: Xtrain, y: ytrain})
  
          weights = sess.run(weights)
          biases = sess.run(biases)
      sess.close()
  
      return weights, biases
  
  def evaluate_model(w, b, X, Xtest):
      weights = {
          'h1': tf.Variable(w['h1']),
          'h2': tf.Variable(w['h2']),
          'out': tf.Variable(w['out'])
      }
  
      biases = {
          'b1': tf.Variable(b['b1']),
          'b2': tf.Variable(b['b2']),
          'out': tf.Variable(b['out'])
      }
  
      layer_1 = tf.add(tf.matmul(X, weights['h1']), biases['b1'])
      layer_1 = tf.nn.relu(layer_1)
  
      layer_2 = tf.add(tf.matmul(layer_1, weights['h2']), biases['b2'])
      layer_2 = tf.nn.relu(layer_2)
  
      out_layer = tf.matmul(layer_2, weights['out']) + biases['out']
  
      init = tf.global_variables_initializer()
      with tf.Session() as sess:
          sess.run(init)
          print(sess.run(weights))
  
          print(sess.run(biases))
  
          pred = sess.run(out_layer, feed_dict={X: Xtest})
  
      return pred
  </code></pre>
  
  <p>The data has (x,y) coordinate in each time sequences 1,2,3,...N, with labels. I obtained the data for left by looking the label index and corresponding coordinates for that label. And time shifting them to get features for Neural Network. For 4 timeshift, I will have 8 features data and the output of the neural network is the next point in that motion.
  Please help, I am stuck in this for really long I tried everything with no progress.</p>
  
  <p>To Train the model:</p>
  
  <pre><code>    time_history = 4
      Train_Left, Train_Waggle, Train_Right = seperate_class_data(xy_train, time_history)
  
      Train_Right_X = Train_Right[0]
      Train_Right_Y = Train_Right[1]
  
  
      Train_Left_X = Train_Left[0]
      Train_Left_Y = Train_Left[1]
  
      Train_Waggle_X = Train_Waggle[0]
      Train_Waggle_Y = Train_Waggle[1]
  
      num_inputs = np.shape(Train_Left_X)[1]
      num_outputs = np.shape(Train_Left_Y)[1]
  
  
      print("Training Left Model")
      weights_left, biases_left = create_train_model(Train_Left_X, Train_Left_Y)
      weights_waggle, biases_waggle = create_train_model(Train_Waggle_X, Train_Waggle_Y)
  </code></pre>
  
  <p>To Evaluate The Model:</p>
  
  <pre><code>    X = tf.placeholder(dtype=tf.float32, shape=(None, num_inputs),  name='X')
  pred_left = evaluate_model(weights_left, biases_left, X, Test_Left_X)
  pred_waggle = evaluate_model(weights_waggle, biases_waggle, X, Test_Waggle_X)
  </code></pre>