title: Pytorch: weights not updating, using dual network
body: <p>I hope you can help. I am designing a controller for a system, where the controller NN is trained using an imported NN I made, which is the state space representation of the system.</p>
  <p>But it seems the weights are not updating on each epoch, I have attached the code below:</p>
  <p>This is my defined controller:</p>
  <pre class="lang-py prettyprint-override"><code>class Net(nn.Module):
      
      def __init__(self, n1, n2, n3):
          super().__init__()
          self.fc1 = nn.Linear((10 * 2), n1)
          self.fc2 = nn.Linear(n1, n2)
          self.fc3 = nn.Linear(n2, n3)
          self.fc4 = nn.Linear(n3, 1)
      
      def forward(self, x):
          x = F.gelu(self.fc1(x))
          x = F.gelu(self.fc2(x))
          x = F.gelu(self.fc3(x))
          x = F.gelu(self.fc4(x))
          return x
  </code></pre>
  <p>below is my training loop:</p>
  <pre class="lang-py prettyprint-override"><code>LR = 0.001
  w_decay = 0.005
  optimizer = optim.SGD(dpc.parameters(), lr=LR, weight_decay=w_decay)
  EPOCHS = 50
  batch = 2
  print(‘Training network…’)
  dpc.train(mode=True)
  
  for epoch in range(EPOCHS):
      for dd in range(0, len(x_train) - batch):
          X = x_train[dd:dd + batch]
          optimizer.zero_grad()
          output = dpc(X)
          input_ssm = X.detach().numpy()
          input_ssm = input_ssm.tolist()
          input_ssm[0].append(output.detach().numpy().tolist()[0][0])
          input_ssm[1].append(output.detach().numpy().tolist()[1][0])
          input_ssm = np.array(input_ssm)
          input_ssm = torch.Tensor(input_ssm)
          y = ss_model(input_ssm)
          y = (y*(max_a[1][0] - min_a[1][0])) + min_a[1][0]
          desired = np.array([[0], [0]])
          desired = torch.Tensor(desired)
          loss = (F.mse_loss(y, desired))
          combined_loss += loss
          loss.backward()
          optimizer.step()
  </code></pre>
  <p>Many thanks.</p>