title: How to use numpy array inputs in tensorflow RNN
body: <p>I just curious on how how to generate a sequence, batches and or epochs to feed into a tensor flow model, a multi_layer RNN graph from a numpy array. Originally numpy array was generated from pandas dataset and a Sklearn split below.</p>
  
  <p>From Numpy to Pandas</p>
  
  <pre><code>#define features and labels using X, Y from a numpy array
   X = Input_Output_Matrix.iloc[:, 0:3].values
   y = np.around(Input_Output_Matrix.iloc[:, 3], decimals=1).values
  
  # Splitting the dataset into the Training set and Test set
    from sklearn.cross_validation import train_test_split
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, 
          random_state = 0)   
  </code></pre>
  
  <p>Note: Very important</p>
  
  <pre><code>y_train.shape
  </code></pre>
  
  <p>Out[37]:
  (6721, 100)</p>
  
  <pre><code>X_train.shape
  </code></pre>
  
  <p>Out[38]:
  (6721, 3)</p>
  
  <p>Now the shape of </p>
  
  <h1>Scaling the features to speed up the model</h1>
  
  <pre><code> from sklearn.preprocessing import StandardScaler
   sc = StandardScaler()
   X = sc.fit_transform(X)
  
  
  res = tf.one_hot(indices=y, depth=100)
  with tf.Session() as sess:
       y = sess.run(res)
  </code></pre>
  
  <p>In order to generate the configuration parameters.</p>
  
  <pre><code># Configuration is wrapped in one object for easy tracking and passing.
    class RNNConfig():
         input_size = X_train.shape[1]
         output_size = y_train.shape[1]
         num_steps = 100
         lstm_size = y_train.shape[0]//100
         num_layers = 4
         keep_prob = 0.8
         batch_size = 100
         init_learning_rate = 0.001
         learning_rate_decay = 0.99
         init_epoch = 5
         max_epoch = 5000
  
  DEFAULT_CONFIG = RNNConfig()
  </code></pre>
  
  <p>The input parameters used for the config are actually based on the shape of the numpy array, lets say input_size = 3 for 3 inputs and output_size = 100 derived from the out puts from the one hot encoding i.e depth equals 100.</p>
  
  <pre><code> #one hot encoding to generate 10 columns for the labels
  
   res = tf.one_hot(indices=y, depth=100)
  
  with tf.Session() as sess:
  y = sess.run(res)
  
  with multi_lstm_graph.as_default():
  
      x_data = tf.placeholder(tf.float32, [None, DEFAULT_CONFIG.num_steps, 
                            DEFAULT_CONFIG.input_size])
  
      y_label = tf.placeholder(tf.float32, [None, DEFAULT_CONFIG.num_steps, 
                            DEFAULT_CONFIG.output_size])
  
      learning_rate = tf.placeholder(tf.float32, None)
  
      def _create_one_cell():
          lstm_cell = tf.contrib.rnn.LSTMCell(config.lstm_size, 
                                             state_is_tuple=True)
  
          if config.keep_prob &lt; 1.0:
              lstm_cell = tf.contrib.rnn.DropoutWrapper(lstm_cell, 
                                output_keep_prob=config.keep_prob)
          return lstm_cell               
      cell = tf.contrib.rnn.MultiRNNCell([_create_one_cell() for _ in 
                 range(config.num_layers)], state_is_tuple=True) if 
                        config.num_layers &gt; 1 else _create_one_cell()
  
      val, _ = tf.nn.dynamic_rnn(cell, x_data, dtype=tf.float32)
  
      val = tf.transpose(val, [1, 0, 2])
  
          last = tf.gather(val, int(val.get_shape()[0]) - 1)
  
          weight = tf.Variable(tf.truncated_normal([config.lstm_size, 
                    config.input_size]))
  
          bias = tf.Variable(tf.constant(0.01, shape=[config.input_size]))
  
          y_pred = tf.matmul(last, weight) + bias
  </code></pre>
  
  <p>For the graph features</p>
  
  <p>The tensor flow features are as listed below,
          # Now for the training session</p>
  
  <pre><code>        loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits
                   (logits=y_pred, labels=y_label), name="graph_loss")
          optimizer = tf.train.AdamOptimizer(learning_rate)
          minimize = optimizer.minimize(loss )
          tf.summary.scalar("loss_mse", loss)
  </code></pre>
  
  <p>Lastly for the training session  </p>
  
  <pre><code>with tf.Session(graph=Multilayer_RNN_Graph_Cell) as sess:    
  
      tf.global_variables_initializer().run()
  </code></pre>
  
  <p>Configuration parameters</p>
  
  <pre><code>    learning_rates_to_use = [config.init_learning_rate*
                             (config.learning_rate_decay ** max(
                      float(i + 1 -config.init_epoch), 0.0)) for i in 
                                           range(config.max_epoch)]
  
      test_data_feed = {inputs: X_test, targets: X_test, learning_rate: 0.0}
  </code></pre>
  
  <p>Here is how I try to Iterating over the Epochs.</p>
  
  <pre><code>        for epoch_step in range(DEFAULT_CONFIG.max_epoch):
          current_lr = learning_rates_to_use[epoch_step]
  </code></pre>
  
  <p>Here are my batches again based on the shape of the input array, specifically the number of features.</p>
  
  <pre><code>        for _ in range(int(X_train.shape[0]/config.batch_size)):
              rand_index = np.random.choice(len(X_train), 
                                   size=config.batch_size)
              batch_X = X_train[rand_index].reshape((1, config.num_steps, 
                                   config.input_size))
              #indexing of 1_D np.array
              batch_y = y_train[rand_index].reshape((1, config.num_steps, 
                               config.output_size)) 
  
              '''Each loop below completes one epoch training.'''
              train_data_feed = {inputs: batch_X,
                                  targets: batch_y,
                                  learning_rate: 0}    
  
              '''Each loop below completes one epoch training.'''              
              train_loss, _ = sess.run([loss, minimize], train_data_feed)
              cost_history = np.append(cost_history, train_loss)                
  
              '''results of the Session'''
  
              print('Epoch', epoch, 'completed out of', hm_epochs,'loss:', 
                          cost_history)  
  
              '''In order to test for Model Accuracy '''
  
          if epoch_step%10 == 0:                
              test_loss, _pred, _summary = sess.run([loss, prediction, 
                               merged_summary], test_data_feed)
              assert len(_pred) == len(y_test)
              print ("Epoch %d [%f]:" % (epoch_step, current_lr), test_loss) 
  </code></pre>
  
  <p>Now for my out put. I get the below error. I have particular issue with the logits_size=[1,3] which I don't know how it was generated. It does not relater to either of the matrices( Input matrix, X_train or output, matrix, y_train.). My question is this how do I match the logits_size to the labels_size=[100,100].</p>
  
  <p>Thanks in advance</p>
  
  <pre><code>---------------------------------------------------------------------------
  InvalidArgumentError                      Traceback (most recent call last)
  C:\Users\MAULIDI\Anaconda3\lib\site-packages\tensorflow\python\client\session.py in _do_call(self, fn, *args)
     1326     try:
  -&gt; 1327       return fn(*args)
     1328     except errors.OpError as e:
  
  C:\Users\MAULIDI\Anaconda3\lib\site-packages\tensorflow\python\client\session.py in _run_fn(session, feed_dict, fetch_list, target_list, options, run_metadata)
     1305                                    feed_dict, fetch_list, target_list,
  -&gt; 1306                                    status, run_metadata)
     1307 
  
  C:\Users\MAULIDI\Anaconda3\lib\contextlib.py in __exit__(self, type, value, traceback)
       87             try:
  ---&gt; 88                 next(self.gen)
       89             except StopIteration:
  
  C:\Users\MAULIDI\Anaconda3\lib\site-packages\tensorflow\python\framework\errors_impl.py in raise_exception_on_not_ok_status()
      465           compat.as_text(pywrap_tensorflow.TF_Message(status)),
  --&gt; 466           pywrap_tensorflow.TF_GetCode(status))
      467   finally:
  
  InvalidArgumentError: logits and labels must be same size: logits_size=[1,3] labels_size=[100,100]
       [[Node: train/SoftmaxCrossEntropyWithLogits = SoftmaxCrossEntropyWithLogits[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](train/Reshape, train/Reshape_1)]]
  
  During handling of the above exception, another exception occurred: 
  </code></pre>