title: How can i have more free memory on pytorch?
body: <p>I'm having a issue when training my model on my GPU. The free space i have is very small compared to the total capacity.</p>
  <p>PS : I'm using a EC2 instance to run my code and my database is composed of 1920x1080 images</p>
  <pre><code>(GPU 0; 14.76 GiB total capacity; 1.51 GiB already allocated; 497.75 MiB free; 1.53 GiB reserved in total by PyTorch)
  </code></pre>
  <p>Another thing is that the free memory seems to grow with the batch-size i use.
  For example :</p>
  <p>-for <code>batch_size = 4</code> I get :  <code> (GPU 0; 14.76 GiB total capacity; 1.87 GiB already allocated; 123.75 MiB free; 1.90 GiB reserved in total by PyTorch)</code></p>
  <p>-for <code>batch_size = 8</code> I get :  <code> (GPU 0; 14.76 GiB total capacity; 1.87 GiB already allocated; 797.75 MiB free; 1.90 GiB reserved in total by PyTorch)</code></p>
  <p>And here you have my main :</p>
  <pre><code>device = Trainer.define_working_device()
  
      train_data_dir = '../../asset/train'
      test_data_dir = '../../asset/test'
      batch_size = 4
  
      pre_trained_model = Resnet_model.load_pretrained_resnet18()
  
      pre_trained_model = pre_trained_model.to(device)
  
      train_data = Load_function.load_and_normalized_data(train_data_dir, batch_size)
      test_data = Load_function.load_and_normalized_data(test_data_dir, batch_size)
  
      torch.cuda.empty_cache()
  
      pre_trained_model = Trainer.train_model(pre_trained_model, train_data, test_data, device, 25)
  </code></pre>
  <p>And here you have the function i used :</p>
  <pre><code>def train_model(model, train_data, test_data, device, num_epochs):
      plt.ion()
      since = time.time()
  
      criterion = nn.CrossEntropyLoss()
      optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)
      scheduler = lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)
  
      train_loss_history = []
      train_accuracy_history = []
  
      test_loss_history = []
      test_accuracy_history = []
  
      best_acc = 0.0
  
      for epoch in range(num_epochs):
          print('Epoch {}/{}'.format(epoch, num_epochs - 1))
          print('-' * 10)
  
          train_running_loss = 0.0
          train_running_corrects = 0
  
          test_running_loss = 0.0
          test_running_corrects = 0.0
  
          train_data_size = len(train_data)
          test_data_size = len(test_data)
  
          # Iterate over train data.
          for i, data_train in enumerate(train_data, 0):
              model.train()
              inputs, labels = data_train
  
              inputs = inputs.to(device)
              labels = labels.to(device)
  
              # zero the parameter gradients
              optimizer.zero_grad()
  
              outputs = model(inputs)
  
              _, train_preds = torch.max(outputs, 1)
              loss = criterion(outputs, labels)
  
              # backward + optimize only if in training phase
              loss.backward()
              optimizer.step()
  
              # statistics
              train_running_loss += loss.item() * inputs.size(0)
              train_running_corrects += torch.sum(train_preds == labels.data)
  
          del inputs
          del labels
  
          for i, data_test in enumerate(test_data, 0):
              model.eval()
              test_inputs, test_labels = data_test
  
              test_inputs = test_inputs.to(device)
              test_labels = test_labels.to(device)
  
              # zero the parameter gradients
              optimizer.zero_grad()
  
              outputs = model(test_inputs)
  
              _, test_preds = torch.max(outputs, 1)
              loss = criterion(outputs, test_labels)
  
              # statistics
              test_running_loss += loss.item() * test_inputs.size(0)
              test_running_corrects += torch.sum(test_preds == test_labels.data)
  
          del test_inputs
          del test_labels
  
          scheduler.step()
  
          train_epoch_loss = train_running_loss / train_data_size
          train_epoch_acc = train_running_corrects.double() / train_data_size
  
          test_epoch_loss = test_running_loss / test_data_size
          test_epoch_acc = test_running_corrects.double() / test_data_size
  
          train_loss_history.append(train_epoch_loss)
          train_accuracy_history.append(train_epoch_acc)
  
          test_loss_history.append(test_epoch_loss)
          test_accuracy_history.append(test_epoch_acc)
  
          &quot;&quot;&quot;
          plt.close()
  
          plt.plot(test_loss_history)
          plt.plot(train_loss_history)
          plt.ylabel('Loss')
          plt.show()
  
          plt.plot(test_accuracy_history)
          plt.plot(train_accuracy_history)
          plt.ylabel('Accuracy')
          plt.show()
  &quot;&quot;&quot;
          print('Train data : Loss: {:.4f} Acc: {:.4f}'.format(train_epoch_loss, train_epoch_acc))
          print('-' * 5)
          print('Test data : Loss: {:.4f} Acc: {:.4f}'.format(test_epoch_loss, test_epoch_acc))
  
      plt.imsave('../results/accuracy.png', test_accuracy_history)
      plt.imsave('../results/loss.png', test_loss_history)
  
      time_elapsed = time.time() - since
      print('Training complete in {:.0f}m {:.0f}s'.format(
          time_elapsed // 60, time_elapsed % 60))
      print('Best val Acc: {:4f}'.format(best_acc))
  
      print('Finished Training')
      return model
  </code></pre>
  <pre><code>def define_working_device():
      logging.info(torch.__version__)
  
      if torch.cuda.is_available():
          device = torch.device(&quot;cuda:0&quot;)  # you can continue going on here, like cuda:1 cuda:2....etc.
          print(device)
          print(&quot;Running on the GPU&quot;)
          logging.info(&quot;Running on the GPU&quot;)
          return device
      else:
          device = torch.device(&quot;cpu&quot;)
          print(&quot;Running on the CPU&quot;)
          logging.info(&quot;Running on the CPU&quot;)
          return device
  </code></pre>
  <pre><code>def load_pretrained_resnet18():
      net = models.resnet18(pretrained=True)
      logging.info(&quot;Model successfully loaded&quot;)
      num_features = net.fc.in_features
      net.fc = nn.Linear(num_features, 2)
      return net
  </code></pre>
  <p>If someone can help me, thanks !!</p>