<p>Three years later, we have what seems to be the start of solutions for this type of problem: sparse transformers.</p>
<p>See</p>
<p><a href="https://arxiv.org/abs/1904.10509" rel="nofollow noreferrer">https://arxiv.org/abs/1904.10509</a></p>
<p><a href="https://openai.com/blog/sparse-transformer/" rel="nofollow noreferrer">https://openai.com/blog/sparse-transformer/</a></p>
How to handle extremely long LSTM sequence length?
<p>I have some data that is sampled at at a very high rate (on the order of hundreds of times per second). This results in a sequence length that is huge (~90,000 samples) on average for any given instance. This entire sequence has a single label. I am trying to use an LSTM neural network to classify new sequences as one of these labels (multiclass classification). </p>

<p>However, using an LSTM with a such a large sequence length results in a network that is quite large.</p>

<p>What are some methods to effectively 'chunk' these sequences so that I could reduce the sequence length of the neural network, yet still maintain the information captured in the entire instance?  </p>