{"action":"created","issue":{"url":"https://api.github.com/repos/heuritech/convnets-keras/issues/6","repository_url":"https://api.github.com/repos/heuritech/convnets-keras","labels_url":"https://api.github.com/repos/heuritech/convnets-keras/issues/6/labels{/name}","comments_url":"https://api.github.com/repos/heuritech/convnets-keras/issues/6/comments","events_url":"https://api.github.com/repos/heuritech/convnets-keras/issues/6/events","html_url":"https://github.com/heuritech/convnets-keras/pull/6","id":154185035,"number":6,"title":"Divide alpha by n in LRN layer","user":{"login":"AlexanderFabisch","id":869592,"avatar_url":"https://avatars.githubusercontent.com/u/869592?v=3","gravatar_id":"","url":"https://api.github.com/users/AlexanderFabisch","html_url":"https://github.com/AlexanderFabisch","followers_url":"https://api.github.com/users/AlexanderFabisch/followers","following_url":"https://api.github.com/users/AlexanderFabisch/following{/other_user}","gists_url":"https://api.github.com/users/AlexanderFabisch/gists{/gist_id}","starred_url":"https://api.github.com/users/AlexanderFabisch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AlexanderFabisch/subscriptions","organizations_url":"https://api.github.com/users/AlexanderFabisch/orgs","repos_url":"https://api.github.com/users/AlexanderFabisch/repos","events_url":"https://api.github.com/users/AlexanderFabisch/events{/privacy}","received_events_url":"https://api.github.com/users/AlexanderFabisch/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-05-11T08:15:04Z","updated_at":"2016-05-11T13:36:22Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/heuritech/convnets-keras/pulls/6","html_url":"https://github.com/heuritech/convnets-keras/pull/6","diff_url":"https://github.com/heuritech/convnets-keras/pull/6.diff","patch_url":"https://github.com/heuritech/convnets-keras/pull/6.patch"},"body":"Hi,\r\n\r\nyour implementation of the LRN layer seems to be based on [this implementation](https://github.com/MarcBS/keras/blob/master/keras/caffe/extra_layers.py) of the local response normalization layer. It contains a bug in comparison to Caffe: alpha is not divided by n as it [should be](https://github.com/BVLC/caffe/blob/master/src/caffe/layers/lrn_layer.cpp#L120). That would be fixed with this pull request."},"comment":{"url":"https://api.github.com/repos/heuritech/convnets-keras/issues/comments/218460932","html_url":"https://github.com/heuritech/convnets-keras/pull/6#issuecomment-218460932","issue_url":"https://api.github.com/repos/heuritech/convnets-keras/issues/6","id":218460932,"user":{"login":"leonardblier","id":10939049,"avatar_url":"https://avatars.githubusercontent.com/u/10939049?v=3","gravatar_id":"","url":"https://api.github.com/users/leonardblier","html_url":"https://github.com/leonardblier","followers_url":"https://api.github.com/users/leonardblier/followers","following_url":"https://api.github.com/users/leonardblier/following{/other_user}","gists_url":"https://api.github.com/users/leonardblier/gists{/gist_id}","starred_url":"https://api.github.com/users/leonardblier/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/leonardblier/subscriptions","organizations_url":"https://api.github.com/users/leonardblier/orgs","repos_url":"https://api.github.com/users/leonardblier/repos","events_url":"https://api.github.com/users/leonardblier/events{/privacy}","received_events_url":"https://api.github.com/users/leonardblier/received_events","type":"User","site_admin":false},"created_at":"2016-05-11T13:36:22Z","updated_at":"2016-05-11T13:36:22Z","body":"Hi,\r\n\r\nThank you for pointing this out. \r\nIn fact, we reproduced this implementation https://github.com/uoguelph-mlrg/theano_alexnet/\r\nThere are using this method for the CrossChannelNormalization : https://github.com/lisa-lab/pylearn2/blob/master/pylearn2/expr/normalize.py\r\n```python\r\nclass CrossChannelNormalization(object):\r\n    def __init__(self, alpha = 1e-4, k=2, beta=0.75, n=5):\r\n        self.__dict__.update(locals())\r\n        del self.self\r\n        if n % 2 == 0:\r\n            raise NotImplementedError(\"Only works with odd n for now\")\r\n\r\n    def __call__(self, c01b):\r\n        half = self.n // 2\r\n        sq = T.sqr(c01b)\r\n        ch, r, c, b = c01b.shape\r\n        extra_channels = T.alloc(0., ch + 2*half, r, c, b)\r\n        sq = T.set_subtensor(extra_channels[half:half+ch,:,:,:], sq)\r\n        scale = self.k\r\n        for i in xrange(self.n):\r\n            scale += self.alpha * sq[i:i+ch,:,:,:]\r\n        scale = scale ** self.beta\r\n        return c01b / scale\r\n```\r\nI don't think that in in this implementation, alpha is divided by n. Do you agree ?\r\nHave you tested it with the normalization ? Are the results better ?"}}