{"action":"created","issue":{"url":"https://api.github.com/repos/keras-team/keras/issues/9561","repository_url":"https://api.github.com/repos/keras-team/keras","labels_url":"https://api.github.com/repos/keras-team/keras/issues/9561/labels{/name}","comments_url":"https://api.github.com/repos/keras-team/keras/issues/9561/comments","events_url":"https://api.github.com/repos/keras-team/keras/issues/9561/events","html_url":"https://github.com/keras-team/keras/issues/9561","id":302274301,"number":9561,"title":"Custom Embedding layer with Dropout","user":{"login":"riadsouissi","id":32774199,"avatar_url":"https://avatars2.githubusercontent.com/u/32774199?v=4","gravatar_id":"","url":"https://api.github.com/users/riadsouissi","html_url":"https://github.com/riadsouissi","followers_url":"https://api.github.com/users/riadsouissi/followers","following_url":"https://api.github.com/users/riadsouissi/following{/other_user}","gists_url":"https://api.github.com/users/riadsouissi/gists{/gist_id}","starred_url":"https://api.github.com/users/riadsouissi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/riadsouissi/subscriptions","organizations_url":"https://api.github.com/users/riadsouissi/orgs","repos_url":"https://api.github.com/users/riadsouissi/repos","events_url":"https://api.github.com/users/riadsouissi/events{/privacy}","received_events_url":"https://api.github.com/users/riadsouissi/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-03-05T12:02:36Z","updated_at":"2018-03-13T16:11:29Z","closed_at":null,"author_association":"NONE","body":"Hi,\r\nI wanted to implemented a custom dropout in the embedding layer (I am not dropping from the input, instead I am dropping entire words from the embedding dictionary).\r\n\r\nI am using call(self, inputs, training) function to override embedding and I need to make sure the dropout is not used during inference.\r\nWhat happens is that Keras does not pass training=True during the training phase neither passes False during inference (which considering I am using TF backend makes sense since the graph is static).\r\nBut I can see that LSTM/GRU layers use the same call() function.\r\nSo my question, how do I instruct Keras to pass training mode ? I tried looking at the source code, and honestly, couldn't find a solution...\r\n\r\nThis is my code:\r\n```\r\nclass EmbeddingDropout(Embedding):\r\n    def __init__(self, input_dim, output_dim, word_dropout, **kwargs):\r\n        self.word_dropout = word_dropout #kwargs.get(\"word_dropout\",0)\r\n        super(EmbeddingDropout, self).__init__(input_dim, output_dim, **kwargs)\r\n\r\n    def call(self, inputs, training=None):\r\n        print(\"EmbeddingDropout: training=\",training)\r\n        if K.dtype(inputs) != 'int32':\r\n            inputs = K.cast(inputs, 'int32')\r\n            \r\n        if 0 < self.word_dropout < 1:\r\n            _ones = _generate_dropout_ones(self.embeddings, 1) #K.shape(self.embeddings)[-1])\r\n            _dropout_mask = _generate_dropout_mask(_ones,self.word_dropout,training=True)\r\n            _embeddings_with_dropout = self.embeddings * _dropout_mask        \r\n            \r\n        out = K.gather(_embeddings_with_dropout, inputs)\r\n        return out\r\n\r\n    def get_config(self):\r\n        config = {'word_dropout': self.word_dropout}\r\n        base_config = super(EmbeddingDropout, self).get_config()\r\n        return dict(list(base_config.items()) + list(config.items()))\r\n```\r\n\r\nThank you!"},"comment":{"url":"https://api.github.com/repos/keras-team/keras/issues/comments/372722422","html_url":"https://github.com/keras-team/keras/issues/9561#issuecomment-372722422","issue_url":"https://api.github.com/repos/keras-team/keras/issues/9561","id":372722422,"user":{"login":"riadsouissi","id":32774199,"avatar_url":"https://avatars2.githubusercontent.com/u/32774199?v=4","gravatar_id":"","url":"https://api.github.com/users/riadsouissi","html_url":"https://github.com/riadsouissi","followers_url":"https://api.github.com/users/riadsouissi/followers","following_url":"https://api.github.com/users/riadsouissi/following{/other_user}","gists_url":"https://api.github.com/users/riadsouissi/gists{/gist_id}","starred_url":"https://api.github.com/users/riadsouissi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/riadsouissi/subscriptions","organizations_url":"https://api.github.com/users/riadsouissi/orgs","repos_url":"https://api.github.com/users/riadsouissi/repos","events_url":"https://api.github.com/users/riadsouissi/events{/privacy}","received_events_url":"https://api.github.com/users/riadsouissi/received_events","type":"User","site_admin":false},"created_at":"2018-03-13T16:11:29Z","updated_at":"2018-03-13T16:11:29Z","author_association":"NONE","body":"Thanks. But it seems K.in_train_phase also expects training parameter from the call() function. My problem is that when call() for my custom embedding is called, it never passes training, so I cannot figure out if I am in inference phase or not. "}}
{"actor":{"display_login":"riadsouissi"}}