title: In Keras, what exactly am I configuring when I create a stateful `LSTM` layer with N `units`?
body: <p>The first arguments in a normal <code>Dense</code> layer is also <code>units</code>, and is the number of neurons/nodes in that layer. A standard LSTM unit however looks like the following:</p>
  
  <p><a href="https://i.stack.imgur.com/aTDpS.png" rel="noreferrer"><img src="https://i.stack.imgur.com/aTDpS.png" alt="enter image description here"></a></p>
  
  <p>(This is a reworked version of "<a href="http://colah.github.io/posts/2015-08-Understanding-LSTMs/" rel="noreferrer">Understanding LSTM Networks</a>")</p>
  
  <p>In Keras, when I create an LSTM object like this <code>LSTM(units=N, ...)</code>, am I actually creating <code>N</code> of these LSTM units? Or is it the size of the "Neural Network" layers inside the LSTM unit, i.e., the <code>W</code>'s in the formulas? Or is it something else?</p>
  
  <p>For context, I'm working based on <a href="https://github.com/fchollet/keras/blob/master/examples/stateful_lstm.py" rel="noreferrer">this example code</a>.</p>
  
  <p>The following is the documentation: <a href="https://keras.io/layers/recurrent/" rel="noreferrer">https://keras.io/layers/recurrent/</a></p>
  
  <p>It says:</p>
  
  <blockquote>
    <p>units: Positive integer, dimensionality of the output space.</p>
  </blockquote>
  
  <p>It makes me think it is the number of outputs from the Keras LSTM "layer" object. Meaning the next layer will have <code>N</code> inputs. Does that mean there actually exists <code>N</code> of these LSTM units in the LSTM layer, or maybe that that exactly <em>one</em> LSTM unit is run for <code>N</code> iterations outputting <code>N</code> of these <code>h[t]</code> values, from, say, <code>h[t-N]</code> up to <code>h[t]</code>?</p>
  
  <p>If it only defines the number of outputs, does that mean the input still can be, say, just <em>one</em>, or do we have to manually create lagging input variables <code>x[t-N]</code> to <code>x[t]</code>, one for each LSTM unit defined by the <code>units=N</code> argument?</p>
  
  <p>As I'm writing this it occurs to me what the argument <code>return_sequences</code> does. If set to <code>True</code> all the <code>N</code> outputs are passed forward to the next layer, while if it is set to <code>False</code> it only passes the last <code>h[t]</code> output to the next layer. Am I right?</p>