{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/models/issues/3847","repository_url":"https://api.github.com/repos/tensorflow/models","labels_url":"https://api.github.com/repos/tensorflow/models/issues/3847/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/models/issues/3847/comments","events_url":"https://api.github.com/repos/tensorflow/models/issues/3847/events","html_url":"https://github.com/tensorflow/models/issues/3847","id":310690399,"number":3847,"title":"Exploring the use of 'costs' and 'cost'","user":{"login":"sk-g","id":22529552,"avatar_url":"https://avatars3.githubusercontent.com/u/22529552?v=4","gravatar_id":"","url":"https://api.github.com/users/sk-g","html_url":"https://github.com/sk-g","followers_url":"https://api.github.com/users/sk-g/followers","following_url":"https://api.github.com/users/sk-g/following{/other_user}","gists_url":"https://api.github.com/users/sk-g/gists{/gist_id}","starred_url":"https://api.github.com/users/sk-g/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sk-g/subscriptions","organizations_url":"https://api.github.com/users/sk-g/orgs","repos_url":"https://api.github.com/users/sk-g/repos","events_url":"https://api.github.com/users/sk-g/events{/privacy}","received_events_url":"https://api.github.com/users/sk-g/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":{"login":"nealwu","id":726075,"avatar_url":"https://avatars3.githubusercontent.com/u/726075?v=4","gravatar_id":"","url":"https://api.github.com/users/nealwu","html_url":"https://github.com/nealwu","followers_url":"https://api.github.com/users/nealwu/followers","following_url":"https://api.github.com/users/nealwu/following{/other_user}","gists_url":"https://api.github.com/users/nealwu/gists{/gist_id}","starred_url":"https://api.github.com/users/nealwu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nealwu/subscriptions","organizations_url":"https://api.github.com/users/nealwu/orgs","repos_url":"https://api.github.com/users/nealwu/repos","events_url":"https://api.github.com/users/nealwu/events{/privacy}","received_events_url":"https://api.github.com/users/nealwu/received_events","type":"User","site_admin":false},"assignees":[{"login":"nealwu","id":726075,"avatar_url":"https://avatars3.githubusercontent.com/u/726075?v=4","gravatar_id":"","url":"https://api.github.com/users/nealwu","html_url":"https://github.com/nealwu","followers_url":"https://api.github.com/users/nealwu/followers","following_url":"https://api.github.com/users/nealwu/following{/other_user}","gists_url":"https://api.github.com/users/nealwu/gists{/gist_id}","starred_url":"https://api.github.com/users/nealwu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nealwu/subscriptions","organizations_url":"https://api.github.com/users/nealwu/orgs","repos_url":"https://api.github.com/users/nealwu/repos","events_url":"https://api.github.com/users/nealwu/events{/privacy}","received_events_url":"https://api.github.com/users/nealwu/received_events","type":"User","site_admin":false},{"login":"bitfort","id":1093424,"avatar_url":"https://avatars3.githubusercontent.com/u/1093424?v=4","gravatar_id":"","url":"https://api.github.com/users/bitfort","html_url":"https://github.com/bitfort","followers_url":"https://api.github.com/users/bitfort/followers","following_url":"https://api.github.com/users/bitfort/following{/other_user}","gists_url":"https://api.github.com/users/bitfort/gists{/gist_id}","starred_url":"https://api.github.com/users/bitfort/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bitfort/subscriptions","organizations_url":"https://api.github.com/users/bitfort/orgs","repos_url":"https://api.github.com/users/bitfort/repos","events_url":"https://api.github.com/users/bitfort/events{/privacy}","received_events_url":"https://api.github.com/users/bitfort/received_events","type":"User","site_admin":false}],"milestone":null,"comments":2,"created_at":"2018-04-03T04:28:56Z","updated_at":"2018-04-07T04:34:23Z","closed_at":"2018-04-06T23:24:59Z","author_association":"NONE","body":"https://github.com/tensorflow/models/blob/a7aa25d3ba235c95f6346dbeadfc3a8df7b656b7/tutorials/rnn/ptb/ptb_word_lm.py#L412\r\n\r\nI see that with this approach we can compute per word perplexity. \r\n\r\nI was wondering if it is possible to get per sentence perplexity.  As far as I think, to compute per sentence perplexity (at least in test/inference mode), we have to set batch_size = 1 and sum costs till '<EOS>'.\r\n\r\nBut, assuming all sentences were padded to be of same size we can however do batch_size > 1 in which case we can compute average (per batch) sentence perplexity.\r\n\r\nComputing per sentence perplexity is not generic but can have many applications when modeling an LM for anomaly detection over sentences. Could anyone shed light on how to do this without breaking the code too much? \r\n\r\nTo be honest, the 'cost' can be summed over all words in current batch in which case the average would be over (sum of costs)//(batch_size x max length)"},"comment":{"url":"https://api.github.com/repos/tensorflow/models/issues/comments/379431564","html_url":"https://github.com/tensorflow/models/issues/3847#issuecomment-379431564","issue_url":"https://api.github.com/repos/tensorflow/models/issues/3847","id":379431564,"user":{"login":"sk-g","id":22529552,"avatar_url":"https://avatars3.githubusercontent.com/u/22529552?v=4","gravatar_id":"","url":"https://api.github.com/users/sk-g","html_url":"https://github.com/sk-g","followers_url":"https://api.github.com/users/sk-g/followers","following_url":"https://api.github.com/users/sk-g/following{/other_user}","gists_url":"https://api.github.com/users/sk-g/gists{/gist_id}","starred_url":"https://api.github.com/users/sk-g/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sk-g/subscriptions","organizations_url":"https://api.github.com/users/sk-g/orgs","repos_url":"https://api.github.com/users/sk-g/repos","events_url":"https://api.github.com/users/sk-g/events{/privacy}","received_events_url":"https://api.github.com/users/sk-g/received_events","type":"User","site_admin":false},"created_at":"2018-04-07T04:34:23Z","updated_at":"2018-04-07T04:34:23Z","author_association":"NONE","body":"Hi @nealwu, thanks for the suggestion. By mentioning this I hoped to have it added as an additional feature."}}
{"actor":{"display_login":"sk-g"}}