{"action":"created","issue":{"url":"https://api.github.com/repos/Lasagne/Recipes/issues/38","labels_url":"https://api.github.com/repos/Lasagne/Recipes/issues/38/labels{/name}","comments_url":"https://api.github.com/repos/Lasagne/Recipes/issues/38/comments","events_url":"https://api.github.com/repos/Lasagne/Recipes/issues/38/events","html_url":"https://github.com/Lasagne/Recipes/pull/38","id":123930288,"number":38,"title":"Code for reproducing cifar-10 examples in \"Deep Residual Learni…","user":{"login":"auduno","id":1588066,"avatar_url":"https://avatars.githubusercontent.com/u/1588066?v=3","gravatar_id":"","url":"https://api.github.com/users/auduno","html_url":"https://github.com/auduno","followers_url":"https://api.github.com/users/auduno/followers","following_url":"https://api.github.com/users/auduno/following{/other_user}","gists_url":"https://api.github.com/users/auduno/gists{/gist_id}","starred_url":"https://api.github.com/users/auduno/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/auduno/subscriptions","organizations_url":"https://api.github.com/users/auduno/orgs","repos_url":"https://api.github.com/users/auduno/repos","events_url":"https://api.github.com/users/auduno/events{/privacy}","received_events_url":"https://api.github.com/users/auduno/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":16,"created_at":"2015-12-26T13:45:58Z","updated_at":"2016-01-08T20:37:42Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/Lasagne/Recipes/pulls/38","html_url":"https://github.com/Lasagne/Recipes/pull/38","diff_url":"https://github.com/Lasagne/Recipes/pull/38.diff","patch_url":"https://github.com/Lasagne/Recipes/pull/38.patch"},"body":"Here's the code for reproducing the cifar-10 examples in \"Deep Residual Learning for Image Recognition\". The code is based on the MNIST example, feel free to reformat it. Note that it also depends on batch-normalization code in PR #467.\r\n\r\nTraining a 32-layer network (n=5) with learning parameters similar to the descriptions in the paper, I got validation error 6.88%, which actually is slightly better than the error in the paper. I wanted to try a 56-layer network as well, but currently this fails with error \"RuntimeError: maximum recursion depth exceeded while calling a Python object\".\r\n\r\nThere still seems to be some differences between this model and the one used in the paper, as my model seems to have slightly more parameters than what is described in the paper, e.g. the 1202-layer version has 19.6M parameters versus 19.4M in paper. This model also seems to learn a bit slower and more unstably than what I see in figure 6 in the paper, though the final accuracy is similar. It's actually not entirely clear to me what an iteration is in the paper, since they say \"trained with a minibatch size of 128 on two GPUs\", so I don't know if an iteration is equal to one or two minibatches of size 128. I've assumed an iteration is a single minibatch of 128, but two minibatches of 128 would make learning speed more similar.\r\n\r\nLet me know if I should upload the weights for the trained 32-layer model as well, and of course if you discover any discrepancies between this model and what is described in the paper."},"comment":{"url":"https://api.github.com/repos/Lasagne/Recipes/issues/comments/170117708","html_url":"https://github.com/Lasagne/Recipes/pull/38#issuecomment-170117708","issue_url":"https://api.github.com/repos/Lasagne/Recipes/issues/38","id":170117708,"user":{"login":"erfannoury","id":1921165,"avatar_url":"https://avatars.githubusercontent.com/u/1921165?v=3","gravatar_id":"","url":"https://api.github.com/users/erfannoury","html_url":"https://github.com/erfannoury","followers_url":"https://api.github.com/users/erfannoury/followers","following_url":"https://api.github.com/users/erfannoury/following{/other_user}","gists_url":"https://api.github.com/users/erfannoury/gists{/gist_id}","starred_url":"https://api.github.com/users/erfannoury/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/erfannoury/subscriptions","organizations_url":"https://api.github.com/users/erfannoury/orgs","repos_url":"https://api.github.com/users/erfannoury/repos","events_url":"https://api.github.com/users/erfannoury/events{/privacy}","received_events_url":"https://api.github.com/users/erfannoury/received_events","type":"User","site_admin":false},"created_at":"2016-01-08T20:37:42Z","updated_at":"2016-01-08T20:37:42Z","body":"Hi,\r\nI tried running this code (with `Conv2DDNNLayer` replaced by the vanilla `Conv2DLayer`), but unfortunately I get lots of complaints from Theano. Lots of errors like this:\r\n```\r\nERROR (theano.gof.opt): Optimization failure due to: LocalOptGroup(local_abstractconv_cudnn,local_conv_dnn,local_abstractconv_gemm,local_abstractconv_gradinputs_gemm,local_abstractconv_gradweight_gemm,local_conv_gemm)\r\n(...)\r\nValueError: shape must be given if subsample != (1, 1) or border_mode == \"half\"\r\n\r\nNotImplementedError: AbstractConv2d theano optimization failed. Did you exclude both \"conv_dnn\" and \"conv_gemm\" from the optimizer?\r\nApply node that caused the error: AbstractConv2d{border_mode='half', subsample=(1, 1), filter_flip=True, imshp=(None, 128, 112, 112), kshp=(128, 128, 3, 3)}(GpuElemwise{Composite{((i0 * i1) + (i2 * Abs(i1)))}}[(0, 1)].0, W)\r\n```\r\n\r\nWhat is the problem? It's weird since I have been using the `Conv2DLayer` without any issues."}}