{"action":"created","issue":{"url":"https://api.github.com/repos/soumith/imagenet-multiGPU.torch/issues/32","labels_url":"https://api.github.com/repos/soumith/imagenet-multiGPU.torch/issues/32/labels{/name}","comments_url":"https://api.github.com/repos/soumith/imagenet-multiGPU.torch/issues/32/comments","events_url":"https://api.github.com/repos/soumith/imagenet-multiGPU.torch/issues/32/events","html_url":"https://github.com/soumith/imagenet-multiGPU.torch/issues/32","id":125226911,"number":32,"title":"Changing GPU device id to >1 gives an error.","user":{"login":"vislab2013","id":3234935,"avatar_url":"https://avatars.githubusercontent.com/u/3234935?v=3","gravatar_id":"","url":"https://api.github.com/users/vislab2013","html_url":"https://github.com/vislab2013","followers_url":"https://api.github.com/users/vislab2013/followers","following_url":"https://api.github.com/users/vislab2013/following{/other_user}","gists_url":"https://api.github.com/users/vislab2013/gists{/gist_id}","starred_url":"https://api.github.com/users/vislab2013/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vislab2013/subscriptions","organizations_url":"https://api.github.com/users/vislab2013/orgs","repos_url":"https://api.github.com/users/vislab2013/repos","events_url":"https://api.github.com/users/vislab2013/events{/privacy}","received_events_url":"https://api.github.com/users/vislab2013/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-01-06T17:48:08Z","updated_at":"2016-01-27T21:41:26Z","closed_at":null,"body":"Getting this bug when changing the GPU device id from 1 to 2 or 3 (I have 3 GPUs on the same machine). https://github.com/soumith/imagenet-multiGPU.torch/blob/master/opts.lua#L28\r\nLeaving it to 1 works fine. \r\n\r\n\r\n(with GPU=2, nGPU =2)\r\n==> doing epoch on training data:\r\n==> online epoch # 1\r\nDebugging session completed (traced 3 instructions).\r\n/home/mf/Toolkits/torch/install/bin/luajit: /home/mf/Toolkits/torch/install/share/lua/5.1/nn/Module.lua:70: Assertion `THCudaTensor_checkGPU(state, 1, self_)' failed.  at /home/mf/Toolkits/torch/extra/cutorch/lib/THC/THCTensorMath.cu:30\r\nstack traceback:\r\n    [C]: in function 'zero'\r\n    /home/mf/Toolkits/torch/install/share/lua/5.1/nn/Module.lua:70: in function 'zeroGradParameters'\r\n    ...mf/Toolkits/torch/install/share/lua/5.1/nn/Container.lua:30: in function 'func'\r\n    ...mf/Toolkits/torch/install/share/lua/5.1/nn/Container.lua:25: in function 'applyToModules'\r\n    ...mf/Toolkits/torch/install/share/lua/5.1/nn/Container.lua:30: in function 'zeroGradParameters'\r\n    ...mf/Toolkits/torch/install/share/lua/5.1/nn/Container.lua:30: in function 'func'\r\n    ...mf/Toolkits/torch/install/share/lua/5.1/nn/Container.lua:25: in function 'applyToModules'\r\n    ...mf/Toolkits/torch/install/share/lua/5.1/nn/Container.lua:30: in function 'zeroGradParameters'\r\n    ...s/torch/install/share/lua/5.1/cunn/DataParallelTable.lua:458: in function 'zeroGradParameters'\r\n    ...mf/Toolkits/torch/install/share/lua/5.1/nn/Container.lua:30: in function 'func'\r\n    ...mf/Toolkits/torch/install/share/lua/5.1/nn/Container.lua:25: in function 'applyToModules'\r\n    ...mf/Toolkits/torch/install/share/lua/5.1/nn/Container.lua:30: in function 'zeroGradParameters'\r\n    /home/mf/Toolkits/Codigo/imagenet-multiGPU.torch/train.lua:167: in function 'opfunc'\r\n    /home/mf/Toolkits/torch/install/share/lua/5.1/optim/sgd.lua:44: in function 'sgd'\r\n    /home/mf/Toolkits/Codigo/imagenet-multiGPU.torch/train.lua:174: in function 'f2'\r\n    /home/mf/Toolkits/Codigo/imagenet-multiGPU.torch/data.lua:36: in function 'addjob'\r\n    /home/mf/Toolkits/Codigo/imagenet-multiGPU.torch/train.lua:97: in function 'train'\r\n    /home/mf/Toolkits/Codigo/imagenet-multiGPU.torch/main.lua:45: in main chunk\r\n    [C]: in function 'dofile'\r\n    ...kits/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk\r\n    [C]: at 0x00406670\r\nProgram completed in 28.99 seconds (pid: 6719).\r\n\r\n\r\n(with GPU=2, nGPU =1)\r\n==> doing epoch on training data:\r\n==> online epoch # 1\r\n/home/mf/Toolkits/torch/install/bin/luajit: /home/mf/Toolkits/torch/install/share/lua/5.1/nn/THNN.lua:177: Assertion `THCudaTensor_checkGPU( state, 4, input, target, output, total_weight )' failed.  at /home/mf/Toolkits/torch/extra/cunn/lib/THCUNN/ClassNLLCriterion.cu:123\r\nstack traceback:\r\n\t[C]: in function 'v'\r\n\t/home/mf/Toolkits/torch/install/share/lua/5.1/nn/THNN.lua:177: in function 'ClassNLLCriterion_updateOutput'\r\n\t...its/torch/install/share/lua/5.1/nn/ClassNLLCriterion.lua:41: in function 'forward'\r\n\t/home/mf/Toolkits/Codigo/imagenet-multiGPU.torch/train.lua:169: in function 'opfunc'\r\n\t/home/mf/Toolkits/torch/install/share/lua/5.1/optim/sgd.lua:44: in function 'sgd'\r\n\t/home/mf/Toolkits/Codigo/imagenet-multiGPU.torch/train.lua:174: in function 'f2'\r\n\t/home/mf/Toolkits/Codigo/imagenet-multiGPU.torch/data.lua:36: in function 'addjob'\r\n\t/home/mf/Toolkits/Codigo/imagenet-multiGPU.torch/train.lua:97: in function 'train'\r\n\t/home/mf/Toolkits/Codigo/imagenet-multiGPU.torch/main.lua:45: in main chunk\r\n\t[C]: in function 'dofile'\r\n\t...kits/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk\r\n\t[C]: at 0x00406670\r\nProgram completed in 4.10 seconds (pid: 11801).\r\n\r\n\r\nNOTE: torch is updated and cloned the master-branch repo. "},"comment":{"url":"https://api.github.com/repos/soumith/imagenet-multiGPU.torch/issues/comments/175870263","html_url":"https://github.com/soumith/imagenet-multiGPU.torch/issues/32#issuecomment-175870263","issue_url":"https://api.github.com/repos/soumith/imagenet-multiGPU.torch/issues/32","id":175870263,"user":{"login":"WendyShang","id":4813789,"avatar_url":"https://avatars.githubusercontent.com/u/4813789?v=3","gravatar_id":"","url":"https://api.github.com/users/WendyShang","html_url":"https://github.com/WendyShang","followers_url":"https://api.github.com/users/WendyShang/followers","following_url":"https://api.github.com/users/WendyShang/following{/other_user}","gists_url":"https://api.github.com/users/WendyShang/gists{/gist_id}","starred_url":"https://api.github.com/users/WendyShang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/WendyShang/subscriptions","organizations_url":"https://api.github.com/users/WendyShang/orgs","repos_url":"https://api.github.com/users/WendyShang/repos","events_url":"https://api.github.com/users/WendyShang/events{/privacy}","received_events_url":"https://api.github.com/users/WendyShang/received_events","type":"User","site_admin":false},"created_at":"2016-01-27T21:41:26Z","updated_at":"2016-01-27T21:41:26Z","body":"I run into the same problem and CUDA_VISIBLE_DEVICES solves it. thanks. \r\ne.g.: CUDA_VISIBLE_DEVICES=2 th main.lua "}}