{"action":"created","issue":{"url":"https://api.github.com/repos/pytorch/pytorch/issues/3358","repository_url":"https://api.github.com/repos/pytorch/pytorch","labels_url":"https://api.github.com/repos/pytorch/pytorch/issues/3358/labels{/name}","comments_url":"https://api.github.com/repos/pytorch/pytorch/issues/3358/comments","events_url":"https://api.github.com/repos/pytorch/pytorch/issues/3358/events","html_url":"https://github.com/pytorch/pytorch/issues/3358","id":269432966,"node_id":"MDU6SXNzdWUyNjk0MzI5NjY=","number":3358,"title":"Weights won't update during backpropogation","user":{"login":"Mickey-Livesgood","id":31745473,"node_id":"MDQ6VXNlcjMxNzQ1NDcz","avatar_url":"https://avatars2.githubusercontent.com/u/31745473?v=4","gravatar_id":"","url":"https://api.github.com/users/Mickey-Livesgood","html_url":"https://github.com/Mickey-Livesgood","followers_url":"https://api.github.com/users/Mickey-Livesgood/followers","following_url":"https://api.github.com/users/Mickey-Livesgood/following{/other_user}","gists_url":"https://api.github.com/users/Mickey-Livesgood/gists{/gist_id}","starred_url":"https://api.github.com/users/Mickey-Livesgood/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Mickey-Livesgood/subscriptions","organizations_url":"https://api.github.com/users/Mickey-Livesgood/orgs","repos_url":"https://api.github.com/users/Mickey-Livesgood/repos","events_url":"https://api.github.com/users/Mickey-Livesgood/events{/privacy}","received_events_url":"https://api.github.com/users/Mickey-Livesgood/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2017-10-29T20:42:52Z","updated_at":"2019-04-16T11:25:13Z","closed_at":"2017-11-01T13:35:20Z","author_association":"NONE","body":"I have updated my model to use a dataset to train it in batches. Before I did that (and didn't use mini-batches) the model worked great and the backpropogation was training my network, getting it to reach high accuracy).\r\nHowever, it seems now that the optimizer.step() does not update my weights. I'm not sure what the problem is. I tried to change the learning rate, the loss function and the optimization function but nothing work.\r\n\r\nThis is the code of my model (removed irrelevant parts):\r\n```python\r\n\r\ntrain_file_name = \"train.csv\"\r\n\r\nclass DataSet(Dataset):\r\n    \r\n    def __init__(self, file_name):\r\n        xy = np.loadtxt(file_name, dtype=np.float32, skiprows=1, delimiter=\",\")\r\n        #xy= pd.read_csv(train_file_name, encoding = \"ISO-8859-1\")\r\n        self.len = xy.shape[0]\r\n        self.x_data = torch.from_numpy(xy[:,3:303])\r\n        self.y_data = torch.from_numpy(xy[:,1:3])\r\n        \r\n    def __getitem__(self,index):\r\n        return self.x_data[index], self.y_data[index]\r\n    \r\n    def __len__(self):\r\n        return self.len\r\n\r\nD_in, H, D_out = 300, 300, 2\r\n\r\nclass model(torch.nn.Module):\r\n    \r\n    def __init__(self, D_in, H, D_out):\r\n        super(model, self).__init__()\r\n        ## Initiate two linear levels with the dimensions given to the object\r\n        self.linear1 = torch.nn.Linear(D_in, H)\r\n        self.linear2 = torch.nn.Linear(H, H)\r\n        self.linear3 = torch.nn.Linear(H,H)\r\n        self.linear4 = torch.nn.Linear(H, H)\r\n        self.linear5 = torch.nn.Linear(H, D_out)\r\n        \r\n    def forward(self, x):\r\n        func = torch.nn.Softmax()\r\n        z1 = self.linear1(x).clamp(min=0) # relu\r\n        z2 = self.linear2(z1).clamp(min=0) # relu\r\n        z3 = self.linear3(z2).clamp(min=0) # relu\r\n        z4 = self.linear4(z3).clamp(min=0) # relu\r\n        z5 = func(self.linear5(z4))\r\n        return z5\r\n \r\n\r\nnet = model(D_in, H, D_out)\r\n\r\n#criterion = torch.nn.MSELoss(size_average = False)\r\ncriterion = torch.nn.BCELoss(size_average = False)\r\noptimizer = torch.optim.SGD(net.parameters(), lr=0.00002)\r\n\r\n\r\ntrain_data_set = DataSet(train_file_name)\r\ntrain_loader = DataLoader(dataset=train_data_set, batch_size = 128, shuffle = True)\r\n\r\nfor epoch in range(500):\r\n    # go in batches of 32\r\n    for i, data in enumerate(train_loader, 0):\r\n        inputs, labels = data\r\n        # wrap the input in a Variable\r\n        inputs, labels = Variable(inputs), Variable(labels)\r\n        #print(inputs, labels)\r\n        # forward pass\r\n        y_pred = net(inputs)\r\n        \r\n        # compute the loss\r\n        loss = criterion(y_pred, labels)\r\n        print(epoch, i, loss.data[0])\r\n        \r\n        # backpropogation\r\n        optimizer.zero_grad()\r\n        loss.backward()\r\n        optimizer.step()\r\n        \r\n```\r\n\r\nI would love some help on this issue. I'm guessing it has to do with the Dataset and batch loop that I added. Thanks!"},"comment":{"url":"https://api.github.com/repos/pytorch/pytorch/issues/comments/483620067","html_url":"https://github.com/pytorch/pytorch/issues/3358#issuecomment-483620067","issue_url":"https://api.github.com/repos/pytorch/pytorch/issues/3358","id":483620067,"node_id":"MDEyOklzc3VlQ29tbWVudDQ4MzYyMDA2Nw==","user":{"login":"VDeamoV","id":35023950,"node_id":"MDQ6VXNlcjM1MDIzOTUw","avatar_url":"https://avatars3.githubusercontent.com/u/35023950?v=4","gravatar_id":"","url":"https://api.github.com/users/VDeamoV","html_url":"https://github.com/VDeamoV","followers_url":"https://api.github.com/users/VDeamoV/followers","following_url":"https://api.github.com/users/VDeamoV/following{/other_user}","gists_url":"https://api.github.com/users/VDeamoV/gists{/gist_id}","starred_url":"https://api.github.com/users/VDeamoV/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/VDeamoV/subscriptions","organizations_url":"https://api.github.com/users/VDeamoV/orgs","repos_url":"https://api.github.com/users/VDeamoV/repos","events_url":"https://api.github.com/users/VDeamoV/events{/privacy}","received_events_url":"https://api.github.com/users/VDeamoV/received_events","type":"User","site_admin":false},"created_at":"2019-04-16T11:25:13Z","updated_at":"2019-04-16T11:25:13Z","author_association":"NONE","body":"> > @IssamLaradji I run your for loop to print the param.grad but got the following error\r\n> > ```\r\n> > print(param.grad.data.sum())\r\n> > AttributeError: 'NoneType' object has no attribute 'data'\r\n> > ```\r\n> > any idea?\r\n> > Thanks.\r\n> > add your model name, instead of net.\r\n> > loss.backward()\r\n> \r\n> for param in .parameters():\r\n> print(param.grad.data.sum())\r\n> \r\n> # start debugger\r\n> import pdb; pdb.set_trace()\r\n> \r\n> optimizer.step()\r\n\r\nHi there,\r\nI have the same issue, `print(param)` can print the params normally, but it didn't contained grad.\r\n\r\nmy optim's code is below\r\n\r\n```python\r\ntorch.optim.Adam([{'params': model.parameters()}, {'params': model_2.parameters()}], \r\n                    lr=opt.learning_rate)\r\n```"}}
{"actor":{"display_login":"VDeamoV"}}