title: simple CNN model training using CIFAR-10 dataset STUCK at a low accuracy
body: <p>Hi i have just learned to implement NN models in pytorch through the udacity course and thus created a simple model with a a few CNN and FC layers. after much struggle i got the model to work. but it seems that it is stuck at the same loss even after repeated executions. i dont know where i am going wrong. Must be some logical error which i cant see.
  Here is the code. </p>
  
  <p><strong>model</strong></p>
  
  <pre><code>class cifar_clasify(nn.Module):
  
      def __init__(self):
  
          super().__init__()
  
          self.conv1 = nn.Conv2d(3,16,3)
          self.BNorm1 = nn.BatchNorm2d(16)
          self.conv2 = nn.Conv2d(16,32,3)
          self.BNorm2 = nn.BatchNorm2d(32)
          self.fc1 = nn.Linear(32*6*6,256)
          self.fc2 = nn.Linear(256,512)
          self.fc3 = nn.Linear(512,10)
  
          self.drop = nn.Dropout(p =0.2)
  
      def forward(self,x):
  
  
          out = self.conv1(x)
          out = F.relu(out)
          #print(out.shape)
          out = F.max_pool2d(out,2)
          out = self.BNorm1(out)
          #print(out.shape)
  
          out = self.conv2(out)
          out = F.relu(out)
          #print(out.shape)
          out = F.max_pool2d(out,2)
          out = self.BNorm2(out)
          #print(out.shape)
          out = out.view(out.shape[0],-1)
  
          out = self.fc1(out)
          out = self.drop(F.relu(out))
          out = self.fc2(out)
          out = self.drop(F.relu(out))
          final = F.log_softmax(F.relu(self.fc3(out)) , dim = 1)
  
          return final
  </code></pre>
  
  <p><strong>training code</strong></p>
  
  <pre><code>device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
  print(device)
  
  model = cifar_clasify()
  criterion = nn.NLLLoss()
  optimizer = optim.Adam(model.parameters(), lr =0.03)
  
  epoch =2 
  step = 2
  running_loss = 0
  accuracy = 0 
  print_every = 5
  
  model.to(device)
  
  
  for e in range(epoch):
  
      for inputs,label_ in zip(train_X,train_labels):
  
          step +=1
          inputs = inputs.view((-1,3,32,32))
          inputs,label_ = inputs.to(device) , label_.to(device)
  
          #inputs.cuda()
          #label.cuda()
  
          optimizer.zero_grad()
  
          logps = model.forward(inputs)
  
          loss = criterion(logps , label_.reshape(1))
  
          loss.backward()
  
          optimizer.step()
  
          running_loss += loss.item()
  
  
          if step % print_every == 0:
              test_loss = 0 
              accuracy = 0
              model.eval()
              with torch.no_grad():
                  for testx, lab in zip(test_X , test_labels):
  
                      testx = testx.view((-1,3,32,32))
  
                      testx,lab = testx.to(device) , lab.to(device)
                      lab = lab.reshape(1)
                      logps = model.forward(testx)
                      batch_loss = criterion(logps , lab)
                      #print(batch_loss.item())
                      test_loss += batch_loss.item()
  
                      ps = torch.exp(logps)
  
                      top_p , topclass = ps.topk(1,dim = 1)
  
                      equals = topclass == lab.view(*topclass.shape)
  
                      accuracy += torch.mean(torch.mean(equals.type(torch.FloatTensor))).item()
              print(f"Epoch {e+1}/{epoch}.. "
                    f"Train loss: {running_loss/print_every:.3f}.. "
                    f"Test loss: {test_loss/len(test_X):.3f}.. "
                    f"Test accuracy: {accuracy/len(test_X):.3f}")
              running_loss = 0
              model.train()
  </code></pre>
  
  <p>Here is the result which i had to stop as it was not improving :</p>
  
  <pre><code>Epoch 1/2.. Train loss: 1.396.. Test loss: 5.288.. Test accuracy: 0.104
  step =  5
  Epoch 1/2.. Train loss: 3.038.. Test loss: 2.303.. Test accuracy: 0.104
  step =  10
  Epoch 1/2.. Train loss: 2.303.. Test loss: 2.303.. Test accuracy: 0.104
  step =  15
  Epoch 1/2.. Train loss: 2.669.. Test loss: 2.318.. Test accuracy: 0.105
  step =  20
  Epoch 1/2.. Train loss: 3.652.. Test loss: 2.303.. Test accuracy: 0.104
  step =  25
  Epoch 1/2.. Train loss: 2.303.. Test loss: 2.303.. Test accuracy: 0.104
  step =  30
  Epoch 1/2.. Train loss: 2.303.. Test loss: 2.303.. Test accuracy: 0.104
  step =  35
  Epoch 1/2.. Train loss: 2.303.. Test loss: 2.303.. Test accuracy: 0.104
  step =  40
  Epoch 1/2.. Train loss: 2.303.. Test loss: 2.303.. Test accuracy: 0.104
  step =  45
  Epoch 1/2.. Train loss: 2.303.. Test loss: 2.303.. Test accuracy: 0.104
  step =  50
  Epoch 1/2.. Train loss: 2.303.. Test loss: 2.303.. Test accuracy: 0.104
  step =  55
  </code></pre>
  
  <p>Here is the code if you want any other information:</p>
  
  <p><a href="https://colab.research.google.com/drive/1HPims3PDuwEPV-dDBmMdHxXCkO1sMkXj" rel="nofollow noreferrer">Simple CNN for CIFAR 10 classification in google colab</a></p>