{"action":"opened","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/3235","repository_url":"https://api.github.com/repos/fchollet/keras","labels_url":"https://api.github.com/repos/fchollet/keras/issues/3235/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/3235/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/3235/events","html_url":"https://github.com/fchollet/keras/issues/3235","id":165931275,"number":3235,"title":"Proposal: A more dynamic way to incorporate losses","user":{"login":"berleon","id":1197132,"avatar_url":"https://avatars.githubusercontent.com/u/1197132?v=3","gravatar_id":"","url":"https://api.github.com/users/berleon","html_url":"https://github.com/berleon","followers_url":"https://api.github.com/users/berleon/followers","following_url":"https://api.github.com/users/berleon/following{/other_user}","gists_url":"https://api.github.com/users/berleon/gists{/gist_id}","starred_url":"https://api.github.com/users/berleon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/berleon/subscriptions","organizations_url":"https://api.github.com/users/berleon/orgs","repos_url":"https://api.github.com/users/berleon/repos","events_url":"https://api.github.com/users/berleon/events{/privacy}","received_events_url":"https://api.github.com/users/berleon/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-07-16T14:23:52Z","updated_at":"2016-07-16T14:23:52Z","closed_at":null,"body":"# Proposal: A more dynamic way to incorporate losses\r\n\r\nCurrently, there a two ways to control the loss of a model:\r\n1. Through `model.compile('adam', {'output': 'mse'})`\r\n2. Regularizers\r\n\r\n\r\nProblems with the current solution:\r\n* the `ActivityRegularization` layer is not shareable by multiple models.  The regularizer sums up the [regularization loss of all outputs](https://github.com/fchollet/keras/blob/master/keras/regularizers.py#L103). If the outputs originate from different models with different inputs, the regularization loss cannot be computed. See this issue #2804.\r\n  The expected behavior would be:\r\n    * `ActivityRegularization` layer is called n times for model A and m times for a different model B.\r\n    * Model A receives the loss of the activity regularization from the n calls.\r\n    * Model B receives the loss from the m different times.\r\n\r\n* It is unnecessarily complicated to build models that incorporate loss between two network values, rather than a network value to a given label. In unsupervised models, a reconstruction loss is often measured between two internal network values. For example, the ladder networks have a reconstruction loss for every layer.  Currently, one would implement this by adding some regularization, but this would make the whole model not shareable.\r\n\r\nI propose to add a `compute_loss` method to the `Layer` class. This would enable every layer\r\nto influence the final model loss. The loss would be propagated through the model graph similarly to the masks and shapes.\r\n\r\nThe new `Layer.compute_loss` method:\r\n\r\n```\r\nclass Layer:\r\n    def compute_loss(input, output, masks=None, output_mask=None):\r\n        \"\"\"\r\n        The returned loss  is added to the final model loss. If the layer is\r\n        called multiple times, the loss  is also added multiple times\r\n        to the final loss of the model.\r\n        \"\"\"\r\n        return 0\r\n    [..]\r\n```\r\n\r\nExample implementation of the `compute_loss` method:\r\n```\r\nclass MSE(Layer):\r\n    def output_shape_for():\r\n        return (1,)\r\n\r\n    def call(input, masks=None):\r\n        assert(len(input) == 2)\r\n        x, y = input\r\n        return K.mean(K.square(x - y), axis=-1)\r\n\r\n    def compute_loss(input, output, masks=None, output_mask=None):\r\n        return output\r\n```\r\n\r\nUsage of `MSE` in a simple Autoencoder:\r\n```\r\nX = get_data()\r\ninput = Input(shape=(20,))\r\nx = Dense(10)(input)\r\nreconstruction = Dense(20)(x)\r\nloss = MSE(input, reconstruction)\r\n\r\nm = Model(input=input, output=[x, y], loss=loss)\r\n\r\nm.compile('adam')\r\n\r\n# will minimize the difference between `input` and `reconstruction`\r\nm.fit(X)\r\n\r\n# returns the representation x and the reconstruction y\r\nm.predict(X)\r\n```\r\n\r\nNotice the new `loss` argument of the `Model` constructor.  It allows one to separate models outputs from losses.  In the `Model.compile` function the `loss` argument would become optional.\r\n\r\n\r\n## Implementation\r\n\r\nI do not have that much experience with the inner working of keras. I came up with this implementation idea from reading the source code and I hope it makes sense.\r\n\r\nAdd a `_keras_loss` attribute to every keras tensor as done for `_keras_shape` etc. In the `_keras_loss` attribute, we store a dictionary of a marker of the layer (layer name, layer id, node position, tensor position) to the actual loss tensor.  Every time `Node.create_node` is called, we merge the `_keras_loss` dictionaries of the input tensors and add the return value of `outbound_layer.loss(..)` to the dictionary.  The `_keras_loss` attribute of all `output_tensors` is set to the new dictionary.\r\n\r\nIn the `Model.compile` method, we add the losses in the `_keras_loss` attribute of the `output` and `loss` tensors to the final loss.\r\n\r\nThe `Container.run_internal_graph` method would handle the loss similar to the masks and shapes. It would return an additional `output_losses` list.\r\n\r\nThe weight regularizers will not be changed.\r\n\r\nThe only breaking change is the additional output of `Container.run_internal_graph`. And this would only break code if one calls this method directly.\r\n\r\n## Review ActivityRegularization\r\n\r\nLets review how this approach solves the problem if a `ActivityRegularization` layer is shared between multiple models.\r\n\r\n```\r\nclass ActivityRegularization(Layer):\r\n    [..]\r\n    def compute_loss(input, output, masks=None, output_mask=None):\r\n        regularized_loss = K.sum(K.abs(input)) * self.l1\r\n        regularized_loss += K.sum(K.square(input)) * self.l2\r\n        return K.in_train_phase(regularized_loss, loss)\r\n    def call(input, mask=None):\r\n        return input\r\n\r\ninput1 = Input(shape=20)\r\nx1 = Dense(10)(input1)\r\n\r\ninput2 = Input(shape=20)\r\nx2 = Dense(10)(input2)\r\n\r\nactivity_reg = ActivityRegularization(l1=0.01)\r\n\r\n# this will invoke the ActivityRegularization.compute_loss method with input=x1\r\ny1 = activity_reg(x1)\r\n# y1._keras_loss  will now contain an item of (unique marker for activity_reg, loss on x1)\r\n\r\n# this will also invoke the ActivityRegularization.compute_loss method but with input=x2\r\ny2 = activity_reg(x2)\r\n# y2._keras_loss  will now contain an item of (unique marker for activity_reg, loss on x2)\r\n\r\n# looks up y1._keras_loss to extract the additional loss.\r\nModel(x1, y1)\r\n\r\n# looks up y2._keras_loss to extract the additional loss.\r\nModel(x2, y2)\r\n\r\n# This model will add the loss from both y1 and y2 to the final loss.\r\nModel([x1, x2], [y1, y2])\r\n```\r\n\r\n\r\n## Review ladder networks\r\n\r\n```\r\nclass Ladder(Layer):\r\n    def __init__(self, forward_layer, reconstruction_layer):\r\n        self.forward_layer = forward_layer\r\n        self.reconstruction_layer = reconstruction_layer\r\n\r\n    def compute_loss(input, output, masks=None, output_mask=None):\r\n        reconstruction = self.reconstruction_layer(output)\r\n        return K.mean(K.square(reconstruction - input), axis=-1)\r\n\r\n    def call(input, mask=None):\r\n        return self.forward_layer(input, mask)\r\n\r\ninput =  Input(shape=(20,))\r\nx = Ladder(Dense(10), Dense(10))(input)\r\nx = Ladder(Dense(10), Dense(10))(x)\r\noutput = Dense(1)(x)\r\n\r\nm = Model(input, output)\r\nm.compile('adam', 'mse')\r\nm.fit(X, y)\r\n```\r\n\r\nThe loss of the `Ladder` layers propagates through the `_keras_loss` dictionary to the `Model`.\r\n\r\n## Open questions\r\n\r\n* Is this the right way to implement it?\r\n* How are metrics handled?  `<layer_name>: <loss from layer>`? Should it be\r\n  customizable?\r\n\r\n\r\n## Conclusion\r\n\r\nThis change would make `ActivityRegularization` shareable and furthermore simplify the construction of many unsupervised models.\r\n\r\nFor my personal work, I have to call an activity regularization multiple times. Therefore I will start to implement a basic version of this proposal."}}
{"actor":{"display_login":"berleon"}}