title: I try to use BertForSequenceClassification for binary sentiment analysis task, but during the evaluating process, all the logits all the same
body: <p>It seems all the things all right during the training process, and every 10 step i try to evaluate the model by using dev_data_set, the model give every the sample of the batch the same logits.</p>
  <p>This is my code:</p>
  <pre><code>&lt;!-- language: lang-python --&gt;
  
         def train(self):
          params = [{'params' : self.model.parameters()}]
          optimizer = torch.optim.Adam(params, lr=self.args.lr, weight_decay=self.args.weight_decay)
          my_lr_scheduler = torch.optim.lr_scheduler.ExponentialLR(optimizer=optimizer, gamma=self.args.decay_rate)
          best_eval_loss = 1e9
          for epoch in range(int(self.args.epochs)):
              total_train_loss = 0
              step = 0
              train_batch_num = 0
              for batch_index, batch in tqdm(enumerate(self.train_loader)):
                  self.model.train()
                  step += 1
                  train_batch_num += 1
                  # optimizer.zero_grad()
                  sources = list(batch[0])
                  label = batch[1]
              
                  inputs = self.tokenizer(sources,return_tensors='pt', padding=True).to(self.device)
                  labels = torch.tensor(label).to(self.device)
                  labels = labels.squeeze(0)
  
                  logits = self.model(**inputs, labels=labels).logits
  
                  loss = self.model(**inputs, labels=labels).loss
                  
                  # logits = outputs.logits
                  # loss = self.loss(logits , labels)
  
                  # print('train loss per step:', loss.item())
                  total_train_loss += loss.item()
  
                  optimizer.zero_grad()
                  loss.backward()
                  optimizer.step()
                  
              
                  if step % 10 == 0:
                      total_dev_loss = 0
                      self.model.eval()
                      dev_num = 0
                      with torch.no_grad():
                          for batch_index, batch in tqdm(enumerate(self.dev_loader)):
                              sources = list(batch[0])
                              label = batch[1]
                              dev_num += 1
                              
  
                              inputs = self.tokenizer(sources,return_tensors='pt', padding=True).to(self.device)
                              labels = torch.tensor(label).to(self.device)
                              labels = labels.squeeze(0)
  
                              logits = self.model(**inputs, labels=labels).logits
  
                              loss = self.model(**inputs, labels=labels).loss
  
  </code></pre>
  <p>During the training process:
  the logits are:
  <strong>training logits:  tensor([[-0.3385, -1.0671],
  [-0.1526, -1.0956]], device='cuda:0', grad_fn=)</strong> . It seems all right.</p>
  <p>But after pass the line :  <strong>if step % 10 == 0:</strong>,
  no matter what the data is, for example:</p>
  <p><a href="https://i.stack.imgur.com/eAVId.png" rel="nofollow noreferrer">enter image description here</a></p>
  <p>or
  <a href="https://i.stack.imgur.com/DDdpz.png" rel="nofollow noreferrer">enter image description here</a>
  the eval logits become the same:</p>
  <ul>
  <li><p>eval logits:  tensor([[-0.4534, -0.9629],
  [-0.4533, -0.9628]], device='cuda:0')</p>
  </li>
  <li><p>eval logits:  tensor([[-0.4534, -0.9629],
  [-0.4534, -0.9630]], device='cuda:0')</p>
  </li>
  </ul>