<p>First you need some placeholders to put your training data (one batch)</p>

<pre><code>x_input = tf.placeholder(tf.float32, [batch_size, truncated_series_length, 1])
y_output = tf.placeholder(tf.float32, [batch_size, truncated_series_length, 1])
</code></pre>

<p>A LSTM need a state, which consists of two components, the hidden state and the cell state, very good guide here: <a href="https://arxiv.org/pdf/1506.00019.pdf" rel="nofollow">https://arxiv.org/pdf/1506.00019.pdf</a>. For every layer in the LSTM you have one cell state and one hidden state.</p>

<p>The problem is that Tensorflow stores this in a LSTMStateTuple which you can not send into placeholder. So you need to store it in a Tensor, and then unpack it into a tuple:</p>

<pre><code>state_placeholder = tf.placeholder(tf.float32, [num_layers, 2, batch_size, state_size])

l = tf.unpack(state_placeholder, axis=0)
rnn_tuple_state = tuple(
    [tf.nn.rnn_cell.LSTMStateTuple(l[idx][0], l[idx][1])
     for idx in range(num_layers)]
)
</code></pre>

<p>Then you can use the built-in Tensorflow API to create the stacked LSTM layer.</p>

<pre><code>cell = tf.nn.rnn_cell.LSTMCell(state_size, state_is_tuple=True)
cell = tf.nn.rnn_cell.MultiRNNCell([cell]*num_layers, state_is_tuple=True)
outputs, state = tf.nn.dynamic_rnn(cell, x_input, initial_state=rnn_tuple_state)
</code></pre>

<p>From here you continue with the outputs to calculate logits and then a loss with respect to the <code>y_inputs</code>. </p>

<p>Then you run each batch with the <code>sess.run</code>-command, with truncated backpropagation (good explanation here <a href="http://r2rt.com/styles-of-truncated-backpropagation.html" rel="nofollow">http://r2rt.com/styles-of-truncated-backpropagation.html</a>)</p>

<pre><code> init_state = np.zeros((num_layers, 2, batch_size, state_size))

...current_state... = sess.run([...state...], feed_dict={x_input:batch_in, state_placeholder:current_state ...})
current_state = np.array(current_state)
</code></pre>

<p>You will have to convert the state to a <code>numpy</code> array before feeding it again.</p>

<p>Perhaps it is better to use a librarly like Tflearn or Keras instead?</p>
how to stack LSTM layers using TensorFlow
<p>what I have is the following, which I believe is a network with one hidden LSTM layer:</p>

<pre><code># Parameters
learning rate = 0.001
training_iters = 100000
batch_size = 128
display_step = 10

# Network Parameters
n_input = 13
n_steps = 10
n_hidden = 512
n_classes = 13

# tf Graph input
x = tf.placeholder("float", [None, n_steps, n_input])
y = tf.placeholder("float", [None, n_classes])

# Define weights
weights = {
    'out' : tf.Variable(tf.random_normal([n_hidden, n_classes]))
}
biases = {
    'out' : tf.Variable(tf.random_normal([n_classes]))
}
</code></pre>

<p>However, I am trying to build an LSTM network using TensorFlow to predict power consumption.  I have been looking around to find a good example, but I could not find any model with 2 hidden LSTM layers. Here's the model that I would like to build:</p>

<p>1 input layer, 
1 output layer, 
2 hidden LSTM layers(with 512 neurons in each), 
time step(sequence length): 10</p>

<p>Could anyone guide me to build this using TensorFlow? ( from defining weights, building input shape, training, predicting, use of optimizer or cost function, etc), any help would be much appreciated.</p>

<p>Thank you so much in advance!</p>