{"action":"created","issue":{"url":"https://api.github.com/repos/chainer/chainercv/issues/567","repository_url":"https://api.github.com/repos/chainer/chainercv","labels_url":"https://api.github.com/repos/chainer/chainercv/issues/567/labels{/name}","comments_url":"https://api.github.com/repos/chainer/chainercv/issues/567/comments","events_url":"https://api.github.com/repos/chainer/chainercv/issues/567/events","html_url":"https://github.com/chainer/chainercv/issues/567","id":313973406,"number":567,"title":"Training on SSD terminated due to 'thrust::system::system_error'","user":{"login":"leewaymay","id":17137405,"avatar_url":"https://avatars1.githubusercontent.com/u/17137405?v=4","gravatar_id":"","url":"https://api.github.com/users/leewaymay","html_url":"https://github.com/leewaymay","followers_url":"https://api.github.com/users/leewaymay/followers","following_url":"https://api.github.com/users/leewaymay/following{/other_user}","gists_url":"https://api.github.com/users/leewaymay/gists{/gist_id}","starred_url":"https://api.github.com/users/leewaymay/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/leewaymay/subscriptions","organizations_url":"https://api.github.com/users/leewaymay/orgs","repos_url":"https://api.github.com/users/leewaymay/repos","events_url":"https://api.github.com/users/leewaymay/events{/privacy}","received_events_url":"https://api.github.com/users/leewaymay/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"Hakuyume","id":3014172,"avatar_url":"https://avatars3.githubusercontent.com/u/3014172?v=4","gravatar_id":"","url":"https://api.github.com/users/Hakuyume","html_url":"https://github.com/Hakuyume","followers_url":"https://api.github.com/users/Hakuyume/followers","following_url":"https://api.github.com/users/Hakuyume/following{/other_user}","gists_url":"https://api.github.com/users/Hakuyume/gists{/gist_id}","starred_url":"https://api.github.com/users/Hakuyume/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Hakuyume/subscriptions","organizations_url":"https://api.github.com/users/Hakuyume/orgs","repos_url":"https://api.github.com/users/Hakuyume/repos","events_url":"https://api.github.com/users/Hakuyume/events{/privacy}","received_events_url":"https://api.github.com/users/Hakuyume/received_events","type":"User","site_admin":false},"assignees":[{"login":"Hakuyume","id":3014172,"avatar_url":"https://avatars3.githubusercontent.com/u/3014172?v=4","gravatar_id":"","url":"https://api.github.com/users/Hakuyume","html_url":"https://github.com/Hakuyume","followers_url":"https://api.github.com/users/Hakuyume/followers","following_url":"https://api.github.com/users/Hakuyume/following{/other_user}","gists_url":"https://api.github.com/users/Hakuyume/gists{/gist_id}","starred_url":"https://api.github.com/users/Hakuyume/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Hakuyume/subscriptions","organizations_url":"https://api.github.com/users/Hakuyume/orgs","repos_url":"https://api.github.com/users/Hakuyume/repos","events_url":"https://api.github.com/users/Hakuyume/events{/privacy}","received_events_url":"https://api.github.com/users/Hakuyume/received_events","type":"User","site_admin":false}],"milestone":null,"comments":2,"created_at":"2018-04-13T05:26:12Z","updated_at":"2018-05-28T00:29:41Z","closed_at":null,"author_association":"NONE","body":"I encountered an issue when I was using my own dataset to train SSD. And I have the following error after I trained the first 20 iterations.\r\n`terminate called after throwing an instance of 'thrust::system::system_error'`\r\n`what():  merge_sort: failed to synchronize: an illegal memory access was encountered`\r\nI used the same dataset to train Faster-rCNN and it has no problem.\r\nI trained with the VOC dataset and the training code works.\r\nI checked the `cupy.sort()` method, and I can successfully call this method.\r\n`asd = np.array([1, 23, 343, 3424, 134])\r\nasd = cupy.array(asd)\r\ncupy.sort(asd)\r\n`\r\nThe output is \r\n`array([   1,   23,  134,  343, 3424])`\r\n\r\nI don't know what cause this issue. Could anyone help me?"},"comment":{"url":"https://api.github.com/repos/chainer/chainercv/issues/comments/392391477","html_url":"https://github.com/chainer/chainercv/issues/567#issuecomment-392391477","issue_url":"https://api.github.com/repos/chainer/chainercv/issues/567","id":392391477,"user":{"login":"doiken23","id":33407879,"avatar_url":"https://avatars0.githubusercontent.com/u/33407879?v=4","gravatar_id":"","url":"https://api.github.com/users/doiken23","html_url":"https://github.com/doiken23","followers_url":"https://api.github.com/users/doiken23/followers","following_url":"https://api.github.com/users/doiken23/following{/other_user}","gists_url":"https://api.github.com/users/doiken23/gists{/gist_id}","starred_url":"https://api.github.com/users/doiken23/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/doiken23/subscriptions","organizations_url":"https://api.github.com/users/doiken23/orgs","repos_url":"https://api.github.com/users/doiken23/repos","events_url":"https://api.github.com/users/doiken23/events{/privacy}","received_events_url":"https://api.github.com/users/doiken23/received_events","type":"User","site_admin":false},"created_at":"2018-05-28T00:29:41Z","updated_at":"2018-05-28T00:29:41Z","author_association":"NONE","body":"I have encountered same problem.\r\nI could train with VOCDataset, but I got  `'thrust::system::system_error'`  when I use my own dataset.\r\nIs there any possible cause?\r\n\r\nBest regards."}}
{"actor":{"display_login":"doiken23"}}