{"action":"created","issue":{"url":"https://api.github.com/repos/VigneshSrinivasan10/interprettensor/issues/8","repository_url":"https://api.github.com/repos/VigneshSrinivasan10/interprettensor","labels_url":"https://api.github.com/repos/VigneshSrinivasan10/interprettensor/issues/8/labels{/name}","comments_url":"https://api.github.com/repos/VigneshSrinivasan10/interprettensor/issues/8/comments","events_url":"https://api.github.com/repos/VigneshSrinivasan10/interprettensor/issues/8/events","html_url":"https://github.com/VigneshSrinivasan10/interprettensor/issues/8","id":311243872,"number":8,"title":"What is difference lrp algorithm between here and heatmapping.org tutorial?","user":{"login":"segnam","id":22655166,"avatar_url":"https://avatars3.githubusercontent.com/u/22655166?v=4","gravatar_id":"","url":"https://api.github.com/users/segnam","html_url":"https://github.com/segnam","followers_url":"https://api.github.com/users/segnam/followers","following_url":"https://api.github.com/users/segnam/following{/other_user}","gists_url":"https://api.github.com/users/segnam/gists{/gist_id}","starred_url":"https://api.github.com/users/segnam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/segnam/subscriptions","organizations_url":"https://api.github.com/users/segnam/orgs","repos_url":"https://api.github.com/users/segnam/repos","events_url":"https://api.github.com/users/segnam/events{/privacy}","received_events_url":"https://api.github.com/users/segnam/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-04-04T14:03:47Z","updated_at":"2018-04-06T08:11:45Z","closed_at":null,"author_association":"NONE","body":"Hi, I'm studying about interpretting the deep learning and followed your code to implement the lrp.\r\n\r\nBut there is a memory problem of your code that extract_patches() function consumes a lot of memory and time in case large network such as 3D convolution.\r\n\r\nSo i followed the code in heatmapping.org tutorial to implement the code and it works in my network.\r\n\r\nI thought that it works, but i found that the result is different between tutorial and your code when i tested in small network.\r\n```\r\n\r\nclass FirstConvolution(modules.Convolution):\r\n    def relprop(self,R):\r\n        iself = copy.deepcopy(self); iself.B *= 0\r\n        nself = copy.deepcopy(self); nself.B *= 0; nself.W = numpy.minimum(0,nself.W)\r\n        pself = copy.deepcopy(self); pself.B *= 0; pself.W = numpy.maximum(0,pself.W)\r\n        X,L,H = self.X,self.X*0+utils.lowest,self.X*0+utils.highest\r\n\r\n        Z = iself.forward(X)-pself.forward(L)-nself.forward(H)+1e-9; S = R/Z\r\n        R = X*iself.gradprop(S)-L*pself.gradprop(S)-H*nself.gradprop(S)\r\n        return R\r\n\r\nclass NextConvolutionAlphaBeta(modules.Convolution,object):\r\n\r\n    def __init__(self,name,alpha):\r\n        super(self.__class__, self).__init__(name)\r\n        self.alpha = alpha\r\n        self.beta  = alpha-1\r\n        \r\n    def relprop(self,R):\r\n        pself = copy.deepcopy(self); pself.B *= 0; pself.W = numpy.maximum( 1e-9,pself.W)\r\n        nself = copy.deepcopy(self); nself.B *= 0; nself.W = numpy.minimum(-1e-9,nself.W)\r\n\r\n        X = self.X+1e-9\r\n        ZA = pself.forward(X); SA =  self.alpha*R/ZA\r\n        ZB = nself.forward(X); SB = -self.beta *R/ZB\r\n        R = X*(pself.gradprop(SA)+nself.gradprop(SB))\r\n        return R\r\n\r\n```\r\n\r\n\r\nI'm sure that i followed this tutorial and also implemented same in maxpooling.\r\n\r\nwhat is the difference? please help me"},"comment":{"url":"https://api.github.com/repos/VigneshSrinivasan10/interprettensor/issues/comments/379181267","html_url":"https://github.com/VigneshSrinivasan10/interprettensor/issues/8#issuecomment-379181267","issue_url":"https://api.github.com/repos/VigneshSrinivasan10/interprettensor/issues/8","id":379181267,"user":{"login":"sebastian-lapuschkin","id":17984624,"avatar_url":"https://avatars1.githubusercontent.com/u/17984624?v=4","gravatar_id":"","url":"https://api.github.com/users/sebastian-lapuschkin","html_url":"https://github.com/sebastian-lapuschkin","followers_url":"https://api.github.com/users/sebastian-lapuschkin/followers","following_url":"https://api.github.com/users/sebastian-lapuschkin/following{/other_user}","gists_url":"https://api.github.com/users/sebastian-lapuschkin/gists{/gist_id}","starred_url":"https://api.github.com/users/sebastian-lapuschkin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sebastian-lapuschkin/subscriptions","organizations_url":"https://api.github.com/users/sebastian-lapuschkin/orgs","repos_url":"https://api.github.com/users/sebastian-lapuschkin/repos","events_url":"https://api.github.com/users/sebastian-lapuschkin/events{/privacy}","received_events_url":"https://api.github.com/users/sebastian-lapuschkin/received_events","type":"User","site_admin":false},"created_at":"2018-04-06T08:11:45Z","updated_at":"2018-04-06T08:11:45Z","author_association":"COLLABORATOR","body":"short answer:\r\n\r\nthe tutorial on heatmapping.org covers Deep Taylor Decomposition (DTD) from [this paper](https://www.sciencedirect.com/science/article/pii/S0031320316303582?via%3Dihub),\r\n\r\nwhich is a bit more constricted in its assumptions, e.g. the inputs of a layer are always >= 0, the bias is negative, etc. Thus, the decompositions deviate slightly and do not neccesarily work with networks not fulfilling those additional assumptions.\r\n\r\nDTD is a more specialized sibling of LRP, following the theory of Taylor approximation-based relevance decomposition for each and every neuron in a NN.\r\n"}}
{"actor":{"display_login":"sebastian-lapuschkin"}}