<p>In addition to chunking data and passing it to BERT, check the following new approaches.</p>
<p>There are new researches for long document analysis. As you've asked for Bert a similar pre-trained transformer <a href="https://github.com/allenai/longformer" rel="nofollow noreferrer">Longformer</a> has recently been made available from ALLEN NLP (<a href="https://arxiv.org/abs/2004.05150" rel="nofollow noreferrer">https://arxiv.org/abs/2004.05150</a>). Check out this link for the paper.</p>
<p>The related work section also mentions some previous work on long sequences. Google them too. I'll suggest at least go through Transformer XL (<a href="https://arxiv.org/abs/1901.02860" rel="nofollow noreferrer">https://arxiv.org/abs/1901.02860</a>). As far I know it was one of the initial models for long sequences, so would be good to use it as a foundation before moving into 'Longformers'.</p>
How to use Bert for long text classification?
<p>We know that BERT has a max length limit of tokens = 512, So if an article has a length of much bigger than 512, such as 10000 tokens in text
How can BERT be used?</p>