<p>Since your data is temporal, I would recommend using a model specifically intended for processing temporal data. As you mention, <a href="https://keras.io/layers/recurrent/#lstm" rel="nofollow noreferrer">LSTM</a> is quite popular but Keras also has implementation of <a href="https://keras.io/layers/recurrent/#gru" rel="nofollow noreferrer">GRU</a> and you can also try <a href="https://github.com/philipperemy/keras-tcn" rel="nofollow noreferrer">Temporal Convolution Networks (TCNs)</a> which use simple causal convolutions and avoid the complicated memory/gating structures of LSTM and GRU and have been shown to be more effective on some problems in <a href="https://arxiv.org/abs/1803.01271" rel="nofollow noreferrer">this paper</a>.</p>

<p>You will be looking for many-to-one temporal structure since you are taking an input sequence and predicting the next timestep. See <a href="https://stackoverflow.com/questions/43034960/many-to-one-and-many-to-many-lstm-examples-in-keras">this post</a> for help on implementing that with LSTMs. A key takeaway is there is an argument on the Keras temporal models of <code>return_sequences</code>, this for you should be set to <code>False</code>. The temporal models process the time dimension for you and, in the case of LSTMs, capture temporal dependencies by maintaining an internal memory. TCNs achieve similar behavior by performing 1-D convolutions, but causally in the sense that information from the past cannot leak into the future.</p>

<p>I would recommend starting with LSTM as you will find the most resources on blogs and SO questions about using them, and then you can try other models if you're not getting the results you want. I do not recommend using only dense layers, as they are not going to handle temporal relations properly, and I would also disagree with @Solvalou regarding 2D convolutions, because you are mixing temporal and spatial dimensions which will more likely just confuse your network. If you do convolutions, the causal 1-D convolutions of TCN should give you what you want.</p>
Keras, sequential, and timeseries: should we flatten or not?
<p>I try to build a simple NN for timeseries analysis. So far I add only Dense layers (but be welcome to comment about LSTM etc. if this is what you prefer). </p>

<p>My input is in the usual format {samples, time steps, features}, let's say {1000, 100, 3} and I want a single-step output. So far I cannot understand <strong>whether I should flatten the data, and where</strong>. </p>

<p>The results change if I don't flatten, if I do before the last layer, and if I do before the first layer. But I have no way to tell yet if any of these is the correct one. </p>

<p>A good discussion can be found under <a href="https://stackoverflow.com/questions/43237124/what-is-the-role-of-flatten-in-keras">this question</a>. However, please note that I am specifically interested in <strong>timeseries</strong>. So, I wonder if flattening before the first layer might in some way remove the info needed for time-dependence... </p>