<p>Personally, I have not gone that in depth with using LSTMs to go into the level of depth that you are trying to attain but I do have some suggestions. </p>

<p>One solution to your problem, which you mentioned above, could be to simply analyze different pieces of the document by splitting your document into smaller pieces and analyzing them that way. You'll probably have to be creative.</p>

<p>Another solution, that I think might be of interest of you is to uses a Tree LSTM model in order to get the level to depth. <a href="https://arxiv.org/pdf/1503.00075.pdf" rel="nofollow noreferrer">Here's the link to the paper</a> Using the Tree model you could feed in individual characters or words on the lowest level and then feed it upward to higher levels of abstraction. Again, I am not completely familiar with the model, so don't take my word on it, but it could be a possible solution.</p>
Practical advice on dealing with very long inputs using LSTM model?
<p>I built a character-level LSTM model on text data, but ultimately I'm looking to apply this model on very long text documents (such as a novel) where it's important to understand contextual information, such as where in the novel it's in.  </p>

<p>For these large-scale NLP tasks, is the data usually cut into smaller pieces and concatenated with metadata - such as position within the document, detected topic, etc. - to be fed into the model?  Or are there more elegant techniques?</p>