title: PyTorch: Extract learned weights correctly
body: <p>I am trying to extract the weights from a linear layer, but they do not appear to change, although error is dropping monotonously (i.e. training is happening). Printing the weights' sum, nothing happens because it stays constant: </p>
  
  <blockquote>
    <p>np.sum(model.fc2.weight.data.numpy())</p>
  </blockquote>
  
  <p>Here are the code snippets:</p>
  
  <pre><code>def train(epochs):
      model.train()
      for epoch in range(1, epochs+1):
          # Train on train set
          print(np.sum(model.fc2.weight.data.numpy()))
          for batch_idx, (data, target) in enumerate(train_loader):
              data, target = Variable(data), Variable(data)
              optimizer.zero_grad()
              output = model(data)
              loss = criterion(output, target)
              loss.backward()
              optimizer.step()
  </code></pre>
  
  <p>and</p>
  
  <pre><code># Define model
  class Net(nn.Module):
      def __init__(self):
          super(Net, self).__init__()
          # an affine operation: y = Wx + b
          self.fc1 = nn.Linear(100, 80, bias=False)
          init.normal(self.fc1.weight, mean=0, std=1)
          self.fc2 = nn.Linear(80, 87)
          self.fc3 = nn.Linear(87, 94)
          self.fc4 = nn.Linear(94, 100)
  
      def forward(self, x):
          x = self.fc1(x)
          x = F.relu(self.fc2(x))
          x = F.relu(self.fc3(x))
          x = F.relu(self.fc4(x))
          return x
  </code></pre>
  
  <p>Maybe I am looking on the wrong parameters, although I checked the docs. Thanks for your help!</p>