title: PyTorch BERT TypeError: forward() got an unexpected keyword argument 'labels'
body: <p>Training a BERT model using PyTorch transformers (following the tutorial <a href="https://mccormickml.com/2019/07/22/BERT-fine-tuning/" rel="noreferrer">here</a>).</p>
  
  <p>Following statement in the tutorial</p>
  
  <pre><code>loss = model(b_input_ids, token_type_ids=None, attention_mask=b_input_mask, labels=b_labels)
  </code></pre>
  
  <p>leads to</p>
  
  <pre><code>TypeError: forward() got an unexpected keyword argument 'labels'
  </code></pre>
  
  <p>Here is the full error,</p>
  
  <pre><code>TypeError                                 Traceback (most recent call last)
  &lt;ipython-input-53-56aa2f57dcaf&gt; in &lt;module&gt;
       26         optimizer.zero_grad()
       27         # Forward pass
  ---&gt; 28         loss = model(b_input_ids, token_type_ids=None, attention_mask=b_input_mask, labels=b_labels)
       29         train_loss_set.append(loss.item())
       30         # Backward pass
  
  ~/anaconda3/envs/systreviewclassifi/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
      539             result = self._slow_forward(*input, **kwargs)
      540         else:
  --&gt; 541             result = self.forward(*input, **kwargs)
      542         for hook in self._forward_hooks.values():
      543             hook_result = hook(self, input, result)
  
  TypeError: forward() got an unexpected keyword argument 'labels'
  </code></pre>
  
  <p>I cant seem to figure out what kind of argument the forward() function expects.</p>
  
  <p>There is a similar problem <a href="https://github.com/allenai/allennlp/issues/2528" rel="noreferrer">here</a>, but I still do not get what the solution is.</p>
  
  <p>System information:</p>
  
  <ul>
  <li>OS: Ubuntu 16.04 LTS</li>
  <li>Python version: 3.6.x</li>
  <li>Torch version: 1.3.0</li>
  <li>Torch Vision version: 0.4.1</li>
  <li>PyTorch transformers version: 1.2.0</li>
  </ul>