title: What is the loss function used in Trainer from the Transformers library of Hugging Face?
body: <p>What is the loss function used in Trainer from the Transformers library of Hugging Face?</p>
  <p>I am trying to fine tine a BERT model using the <strong>Trainer class</strong> from the Transformers library of Hugging Face.</p>
  <p>In their <a href="https://huggingface.co/docs/transformers/main_classes/trainer" rel="nofollow noreferrer">documentation</a>, they mention that one can specify a customized loss function by overriding the <code>compute_loss</code> method in the class. However, if I do not do the method override and use the Trainer to fine tine a BERT model directly for sentiment classification, what is the default loss function being use? Is it the categorical crossentropy? Thanks!</p>