{"action":"created","issue":{"url":"https://api.github.com/repos/facebook/fb.resnet.torch/issues/141","repository_url":"https://api.github.com/repos/facebook/fb.resnet.torch","labels_url":"https://api.github.com/repos/facebook/fb.resnet.torch/issues/141/labels{/name}","comments_url":"https://api.github.com/repos/facebook/fb.resnet.torch/issues/141/comments","events_url":"https://api.github.com/repos/facebook/fb.resnet.torch/issues/141/events","html_url":"https://github.com/facebook/fb.resnet.torch/issues/141","id":194478648,"number":141,"title":"Memory leak while saving the checkpoint","user":{"login":"erogol","id":1402048,"avatar_url":"https://avatars.githubusercontent.com/u/1402048?v=3","gravatar_id":"","url":"https://api.github.com/users/erogol","html_url":"https://github.com/erogol","followers_url":"https://api.github.com/users/erogol/followers","following_url":"https://api.github.com/users/erogol/following{/other_user}","gists_url":"https://api.github.com/users/erogol/gists{/gist_id}","starred_url":"https://api.github.com/users/erogol/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/erogol/subscriptions","organizations_url":"https://api.github.com/users/erogol/orgs","repos_url":"https://api.github.com/users/erogol/repos","events_url":"https://api.github.com/users/erogol/events{/privacy}","received_events_url":"https://api.github.com/users/erogol/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2016-12-09T00:03:28Z","updated_at":"2016-12-20T09:28:35Z","closed_at":null,"body":"This code leads to memory bloat after I update torch. Any run ends up killed after couple of epochs but fine without checkpoint function. Particularly deepcopy function causes the problem.\r\n\r\n```\r\nfunction checkpoint.save(epoch, model, optimState, isBestModel, opt)\r\n   -- don't save the DataParallelTable for easier loading on other machines\r\n   if torch.type(model) == 'nn.DataParallelTable' then\r\n      model = model:get(1)\r\n   end\r\n\r\n   -- create a clean copy on the CPU without modifying the original network\r\n   model = deepCopy(model):float():clearState()\r\n\r\n   local modelFile = 'model_' .. epoch .. '.t7'\r\n   local optimFile = 'optimState_' .. epoch .. '.t7'\r\n\r\n   if opt.checkpoint == 'true' then\r\n      model:clearState()\r\n      torch.save(paths.concat(opt.save, modelFile), model)\r\n      torch.save(paths.concat(opt.save, optimFile), optimState)\r\n      torch.save('latest.t7', {\r\n         epoch = epoch,\r\n         modelFile = modelFile,\r\n         optimFile = optimFile,\r\n      })\r\n   end\r\n   if isBestModel then\r\n      model:clearState()\r\n      torch.save(paths.concat(opt.save, 'model_best.t7'), model)\r\n      torch.save(paths.concat(opt.save, 'model_best_optimState.t7'), optimState)\r\n   end\r\nend\r\n```\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/facebook/fb.resnet.torch/issues/comments/268195704","html_url":"https://github.com/facebook/fb.resnet.torch/issues/141#issuecomment-268195704","issue_url":"https://api.github.com/repos/facebook/fb.resnet.torch/issues/141","id":268195704,"user":{"login":"erogol","id":1402048,"avatar_url":"https://avatars.githubusercontent.com/u/1402048?v=3","gravatar_id":"","url":"https://api.github.com/users/erogol","html_url":"https://github.com/erogol","followers_url":"https://api.github.com/users/erogol/followers","following_url":"https://api.github.com/users/erogol/following{/other_user}","gists_url":"https://api.github.com/users/erogol/gists{/gist_id}","starred_url":"https://api.github.com/users/erogol/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/erogol/subscriptions","organizations_url":"https://api.github.com/users/erogol/orgs","repos_url":"https://api.github.com/users/erogol/repos","events_url":"https://api.github.com/users/erogol/events{/privacy}","received_events_url":"https://api.github.com/users/erogol/received_events","type":"User","site_admin":false},"created_at":"2016-12-20T09:28:35Z","updated_at":"2016-12-20T09:28:35Z","body":"If you disable ``` model = deepCopy(model):float():clearState()``` in checkpoints.lua, it works but I don't know its side effects."}}
{"actor":{"display_login":"erogol"}}