{"action":"created","issue":{"url":"https://api.github.com/repos/liuzhuang13/DenseNetCaffe/issues/2","repository_url":"https://api.github.com/repos/liuzhuang13/DenseNetCaffe","labels_url":"https://api.github.com/repos/liuzhuang13/DenseNetCaffe/issues/2/labels{/name}","comments_url":"https://api.github.com/repos/liuzhuang13/DenseNetCaffe/issues/2/comments","events_url":"https://api.github.com/repos/liuzhuang13/DenseNetCaffe/issues/2/events","html_url":"https://github.com/liuzhuang13/DenseNetCaffe/issues/2","id":175984332,"number":2,"title":"Batch normalization with or without learned offset","user":{"login":"f0k","id":629706,"avatar_url":"https://avatars.githubusercontent.com/u/629706?v=3","gravatar_id":"","url":"https://api.github.com/users/f0k","html_url":"https://github.com/f0k","followers_url":"https://api.github.com/users/f0k/followers","following_url":"https://api.github.com/users/f0k/following{/other_user}","gists_url":"https://api.github.com/users/f0k/gists{/gist_id}","starred_url":"https://api.github.com/users/f0k/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/f0k/subscriptions","organizations_url":"https://api.github.com/users/f0k/orgs","repos_url":"https://api.github.com/users/f0k/repos","events_url":"https://api.github.com/users/f0k/events{/privacy}","received_events_url":"https://api.github.com/users/f0k/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-09-09T11:07:15Z","updated_at":"2016-09-09T18:08:13Z","closed_at":null,"body":"Nice paper! I just have a minor detail question for reimplementing it.\r\n\r\nIn https://github.com/liuzhuang13/DenseNetCaffe/blob/master/make_densenet.py#L8, you use:\r\n`scale = L.Scale(batch_norm, bias_term=False, ...)`\r\nThis would correspond to batch normalization with learned `gamma`, but without `beta`.\r\nIn https://github.com/liuzhuang13/DenseNet/blob/master/densenet.lua#L28, you use:\r\n`convFactory:add(cudnn.SpatialBatchNormalization(nChannels))`\r\nThis includes a learnable `beta`. So I think the Caffe code needs to be adapted to match the Torch implementation.\r\n\r\nOn a side note, the convolutions (both in Caffe and Torch, if I see correctly) all have a bias term, but that will be rendered meaningless by the following batch normalization."},"comment":{"url":"https://api.github.com/repos/liuzhuang13/DenseNetCaffe/issues/comments/245993083","html_url":"https://github.com/liuzhuang13/DenseNetCaffe/issues/2#issuecomment-245993083","issue_url":"https://api.github.com/repos/liuzhuang13/DenseNetCaffe/issues/2","id":245993083,"user":{"login":"f0k","id":629706,"avatar_url":"https://avatars.githubusercontent.com/u/629706?v=3","gravatar_id":"","url":"https://api.github.com/users/f0k","html_url":"https://github.com/f0k","followers_url":"https://api.github.com/users/f0k/followers","following_url":"https://api.github.com/users/f0k/following{/other_user}","gists_url":"https://api.github.com/users/f0k/gists{/gist_id}","starred_url":"https://api.github.com/users/f0k/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/f0k/subscriptions","organizations_url":"https://api.github.com/users/f0k/orgs","repos_url":"https://api.github.com/users/f0k/repos","events_url":"https://api.github.com/users/f0k/events{/privacy}","received_events_url":"https://api.github.com/users/f0k/received_events","type":"User","site_admin":false},"created_at":"2016-09-09T18:08:13Z","updated_at":"2016-09-09T18:08:13Z","body":"> https://github.com/liuzhuang13/DenseNet/blob/master/densenet.lua#L87\r\n\r\n:+1: Didn't see this (I don't know Torch, didn't expect a way to change this globally). You should probably still add `bias_term=False` in the Caffe model then.\r\n\r\nI've written a Lasagne implementation for the CIFAR-10 experiments: https://github.com/Lasagne/Recipes/pull/84\r\nI noticed that your 40-layer network has 41 layers with trainable weights (40 convolutions, one dense layer). And I'm getting more parameters than you (1.2M instead of 1.0M for the smallest network) -- maybe I'm doing something wrong. Let's see if I can reproduce the results. Cheers!"}}
{"actor":{"display_login":"f0k"}}