<p>It highly depends on your dataset and is part of the data scientist's job to find which model is more suitable for a particular task in terms of selected performance metric, training cost, model complexity etc.</p>

<p>When you work on the problem you will probably test all of the above models and compare them. Which one of them to choose first? Andrew Ng in <a href="https://www.deeplearning.ai/machine-learning-yearning/" rel="nofollow noreferrer">"Machine Learning Yearning"</a> suggest starting with simple model so you can quickly iterate and test your idea, data preprocessing pipeline etc.</p>

<blockquote>
  <p>Don’t start off trying to design and build the perfect system.
  Instead, build and train a basic system quickly—perhaps in just a few
  days</p>
</blockquote>

<p>According to this suggestion, you can start with a simpler model such as <a href="https://arxiv.org/abs/1801.06146" rel="nofollow noreferrer">ULMFiT</a> as a baseline, verify your ideas and then move on to more complex models and see how they can improve your results.</p>

<p>Note that modern NLP models contain a large number of parameters and it is difficult to train them from scratch without a large dataset. That's why you may want to use <em>transfer learning</em>: you can download pre-trained model and use it as a basis and fine-tune it to your task-specific dataset to achieve better performance and reduce training time.</p>
Which model (GPT2, BERT, XLNet and etc) would you use for a text classification task? Why?
<p>I'm trying to train a model for a sentence classification task. The input is a sentence (a vector of integers) and the output is a label (0 or 1). I've seen some articles here and there about using Bert and GPT2 for text classification tasks. However, I'm not sure which one should I pick to start with. Which of these recent models in NLP such as original Transformer model, Bert, GPT2, XLNet would you use to start with? And why? I'd rather to implement in Tensorflow, but I'm flexible to go for PyTorch too. 
Thanks!</p>