title: How to shape and train multicolumn input and multicolumn output (many to many) with RNN LSTM model in TensorFlow?
body: <p>I am facing a problem with training an LSTM model with multicolumn input output. My code is below:</p>
  <pre><code>time_step = 60
  
  #Create a data structure with n-time steps
  X = []
  y = []
  for i in range(time_step + 1, len(training_set_scaled)):
      X.append(training_set_scaled[i-time_step-1:i-1, 0:len(training_set.columns)]) #take all columns into the set
      y.append(training_set_scaled[i, 0:len(training_set.columns)]) #take all columns into the set
  X_train_arr, y_train_arr = np.array(X), np.array(y)
  print(X_train_arr.shape) #(2494, 60, 5)
  print(y_train_arr.shape) #(2494, 5)
  
  #Split data
  X_train_splitted = X_train_arr[:split]
  y_train_splitted = y_train_arr[:split]
  X_test_splitted = X_train_arr[split:]
  y_test_splitted = y_train_arr[split:]
  
  #Initialize the RNN
  model = Sequential()
  
  #Add the LSTM layers and some dropout regularization
  model.add(LSTM(units= 50, activation = 'relu', return_sequences = True, input_shape = (X_train_arr.shape[1], X_train_arr.shape[2]))) #time_step/columns
  model.add(Dropout(0.2))
  model.add(LSTM(units= 40, activation = 'relu', return_sequences = True))
  model.add(Dropout(0.2))
  model.add(LSTM(units= 80, activation = 'relu', return_sequences = True))
  model.add(Dropout(0.2))
  
  #Add the output layer.
  model.add(Dense(units = 1))
  
  #Compile the RNN
  model.compile(optimizer='adam', loss = 'mean_squared_error')
  
  #Fit to the training set
  model.fit(X_train_splitted, y_train_splitted, epochs=3, batch_size=32)
  </code></pre>
  <p>The idea is to train the model with 60 steps back from <code>i</code> and having 5 column target in <code>i</code>:</p>
  <pre><code>for i in range(time_step + 1, len(training_set_scaled)):
      X.append(training_set_scaled[i-time_step-1:i-1, 0:len(training_set.columns)]) #take all columns into the set
      y.append(training_set_scaled[i, 0:len(training_set.columns)]) #take all columns into the set
  </code></pre>
  <p>So my x-train (feed) and y-train (targets) are:</p>
  <pre><code>X_train_arr, y_train_arr = np.array(X), np.array(y)
  print(X_train_arr.shape) #(2494, 60, 5)
  print(y_train_arr.shape) #(2494, 5)
  </code></pre>
  <p>Unfortunately, when fitting the model:</p>
  <pre><code>model.fit(X_train_splitted, y_train_splitted, epochs=3, batch_size=32)
  </code></pre>
  <p>I am getting an error:</p>
  <blockquote>
  <p>Dimensions must be equal, but are 60 and 5 for '{{node
  mean_squared_error/SquaredDifference}} =
  SquaredDifference[T=DT_FLOAT](mean_squared_error/remove_squeezable_dimensions/Squeeze,
  IteratorGetNext:1)' with input shapes: [?,60], [?,5].</p>
  </blockquote>
  <p>I understand that <code>X_train_arr</code> and <code>y_train_arr</code> need to be the same. BUT when testing with case below, everyting is fine:</p>
  <pre><code>X_train_arr, y_train_arr = np.array(X), np.array(y)
      print(X_train_arr.shape) #(2494, 60, 5)
      print(y_train_arr.shape) #(2494, 1)
  </code></pre>
  <p>Idea of having <code>print(y_train_arr.shape) #(2494, 5)</code> is to be able to predict n-steps into the future, where each iteration of prediction generates new entire row of the data with 5 columns values.</p>