{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/2373","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/2373/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/2373/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/2373/events","html_url":"https://github.com/tensorflow/tensorflow/issues/2373","id":154915680,"number":2373,"title":"Embedding lookup table doesn't mask padding value","user":{"login":"aleSuglia","id":1479733,"avatar_url":"https://avatars.githubusercontent.com/u/1479733?v=3","gravatar_id":"","url":"https://api.github.com/users/aleSuglia","html_url":"https://github.com/aleSuglia","followers_url":"https://api.github.com/users/aleSuglia/followers","following_url":"https://api.github.com/users/aleSuglia/following{/other_user}","gists_url":"https://api.github.com/users/aleSuglia/gists{/gist_id}","starred_url":"https://api.github.com/users/aleSuglia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aleSuglia/subscriptions","organizations_url":"https://api.github.com/users/aleSuglia/orgs","repos_url":"https://api.github.com/users/aleSuglia/repos","events_url":"https://api.github.com/users/aleSuglia/events{/privacy}","received_events_url":"https://api.github.com/users/aleSuglia/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-05-15T16:00:27Z","updated_at":"2016-06-07T16:37:01Z","closed_at":"2016-06-07T16:37:01Z","body":"Hi there,\r\n\r\nI'm using an *embedding_lookup* operation in order to generate dense vector representations for each token in my document which are feed to a convolutional neural network (the network architecture is similar to the one in a [WildML article](http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/)). \r\n\r\nEverything works correctly but when I pad my document inserting a padding value in it, the embedding lookup generates a vector for this token too. I think that this approach could alterate the results in the classification task. What I want to achieve is something similar to what Torch [LookupTableMaskZero](https://github.com/Element-Research/rnn#rnn.LookupTableMaskZerol) does. \r\n\r\n1) Is correct what I want to do?\r\n2) Is already implemented something like this? \r\n3) If not, how can I mask the padding value in order to prevent the generation of the corresponding vector for it? \r\n\r\nThank you in advance, \r\nAlessandro"},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/224338835","html_url":"https://github.com/tensorflow/tensorflow/issues/2373#issuecomment-224338835","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/2373","id":224338835,"user":{"login":"girving","id":70511,"avatar_url":"https://avatars.githubusercontent.com/u/70511?v=3","gravatar_id":"","url":"https://api.github.com/users/girving","html_url":"https://github.com/girving","followers_url":"https://api.github.com/users/girving/followers","following_url":"https://api.github.com/users/girving/following{/other_user}","gists_url":"https://api.github.com/users/girving/gists{/gist_id}","starred_url":"https://api.github.com/users/girving/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/girving/subscriptions","organizations_url":"https://api.github.com/users/girving/orgs","repos_url":"https://api.github.com/users/girving/repos","events_url":"https://api.github.com/users/girving/events{/privacy}","received_events_url":"https://api.github.com/users/girving/received_events","type":"User","site_admin":false},"created_at":"2016-06-07T16:37:01Z","updated_at":"2016-06-07T16:37:01Z","body":"I think the sample code doesn't consider the issue because it doesn't affect neural network performance.  If you're seeing significant differences between zero padding an new-token padding, I would use a mask as @LittleYUYU suggests.  `embedding_lookup` doesn't handle this by itself since we try to keep the semantics of each op simple and focused."}}
{"actor":{"display_login":"girving"}}