{"action":"created","issue":{"url":"https://api.github.com/repos/yueatsprograms/Stochastic_Depth/issues/2","repository_url":"https://api.github.com/repos/yueatsprograms/Stochastic_Depth","labels_url":"https://api.github.com/repos/yueatsprograms/Stochastic_Depth/issues/2/labels{/name}","comments_url":"https://api.github.com/repos/yueatsprograms/Stochastic_Depth/issues/2/comments","events_url":"https://api.github.com/repos/yueatsprograms/Stochastic_Depth/issues/2/events","html_url":"https://github.com/yueatsprograms/Stochastic_Depth/issues/2","id":146183433,"number":2,"title":"The val error does not decrease.","user":{"login":"wlw208dzy","id":11946386,"avatar_url":"https://avatars.githubusercontent.com/u/11946386?v=3","gravatar_id":"","url":"https://api.github.com/users/wlw208dzy","html_url":"https://github.com/wlw208dzy","followers_url":"https://api.github.com/users/wlw208dzy/followers","following_url":"https://api.github.com/users/wlw208dzy/following{/other_user}","gists_url":"https://api.github.com/users/wlw208dzy/gists{/gist_id}","starred_url":"https://api.github.com/users/wlw208dzy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wlw208dzy/subscriptions","organizations_url":"https://api.github.com/users/wlw208dzy/orgs","repos_url":"https://api.github.com/users/wlw208dzy/repos","events_url":"https://api.github.com/users/wlw208dzy/events{/privacy}","received_events_url":"https://api.github.com/users/wlw208dzy/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":11,"created_at":"2016-04-06T04:25:37Z","updated_at":"2016-09-30T06:58:22Z","closed_at":"2016-04-11T15:52:09Z","body":"Thanks for releasing the code. I am using one TitanX to reproduce the results of Stochastic Depth for cifar10, but the val error does not decrease since epoch 1, as shown below:\r\n  resultFolder : \"results-cifar10/\"\r\n  device : 2\r\n  dataset : \"cifar10\"\r\n  N : 18\r\n  batchSize : 128\r\n  augmentation : true\r\n  maxEpochs : 500\r\n  deathRate : 0.5\r\n  deathMode : \"lin_decay\"\r\n}\r\nTraining set size:\t\t45000\t\r\nValidation set size:\t\t5000\t\r\nTest set size:\t\t\t10000\t\r\nBuilding model...\t\r\nTraining...\r\nEpoch\tValid. err\tTest err\tTraining time\t\r\nEpoch 1:\t89.74%\t\t90.00%\t\t51s\t\r\nEpoch 2:\t90.02%\t\t90.00%\t\t51s\t\r\nEpoch 3:\t89.74%\t\t90.00%\t\t51s\t\r\nEpoch 4:\t89.74%\t\t90.00%\t\t51s\t\r\nEpoch 5:\t89.74%\t\t90.00%\t\t51s\t\r\nEpoch 6:\t89.74%\t\t90.00%\t\t51s\t\r\nEpoch 7:\t89.74%\t\t90.00%\t\t51s\t\r\nEpoch 8:\t89.74%\t\t90.00%\t\t51s\t\r\nEpoch 9:\t89.74%\t\t90.00%\t\t51s\t\r\nEpoch 10:\t89.74%\t\t90.00%\t\t51s\t\r\nEpoch 11:\t89.74%\t\t90.00%\t\t51s\t\r\nEpoch 12:\t89.74%\t\t90.00%\t\t51s\t\r\nEpoch 13:\t89.74%\t\t90.00%\t\t51s\t\r\nEpoch 14:\t89.74%\t\t90.00%\t\t51s\r\nI am wonderring if any configuration is wrong."},"comment":{"url":"https://api.github.com/repos/yueatsprograms/Stochastic_Depth/issues/comments/250673674","html_url":"https://github.com/yueatsprograms/Stochastic_Depth/issues/2#issuecomment-250673674","issue_url":"https://api.github.com/repos/yueatsprograms/Stochastic_Depth/issues/2","id":250673674,"user":{"login":"ishann","id":14039739,"avatar_url":"https://avatars.githubusercontent.com/u/14039739?v=3","gravatar_id":"","url":"https://api.github.com/users/ishann","html_url":"https://github.com/ishann","followers_url":"https://api.github.com/users/ishann/followers","following_url":"https://api.github.com/users/ishann/following{/other_user}","gists_url":"https://api.github.com/users/ishann/gists{/gist_id}","starred_url":"https://api.github.com/users/ishann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ishann/subscriptions","organizations_url":"https://api.github.com/users/ishann/orgs","repos_url":"https://api.github.com/users/ishann/repos","events_url":"https://api.github.com/users/ishann/events{/privacy}","received_events_url":"https://api.github.com/users/ishann/received_events","type":"User","site_admin":false},"created_at":"2016-09-30T06:58:22Z","updated_at":"2016-09-30T06:58:22Z","body":"I experienced the same issue with a 110-layer Resnet on CIFAR-10. Xavier's suggestion fixed the non-convergence. However, the Constant Depth model did not converge properly. Final test error is 7.37%. I've attached a plot of the errors obtained from running the code.\r\n[errorResnet110Cifar10.pdf](https://github.com/yueatsprograms/Stochastic_Depth/files/502290/errorResnet110Cifar10.pdf)\r\n\r\n\r\nCould any one suggest what I could try to ensure robust/consisten convergence as reported in the Stochastic Depth paper, as well as the first ResNet paper?"}}
{"actor":{"display_login":"ishann"}}