{"action":"closed","issue":{"url":"https://api.github.com/repos/mila-udem/blocks-examples/issues/71","labels_url":"https://api.github.com/repos/mila-udem/blocks-examples/issues/71/labels{/name}","comments_url":"https://api.github.com/repos/mila-udem/blocks-examples/issues/71/comments","events_url":"https://api.github.com/repos/mila-udem/blocks-examples/issues/71/events","html_url":"https://github.com/mila-udem/blocks-examples/issues/71","id":125440069,"number":71,"title":"Can not reach groundhog performance","user":{"login":"critias","id":3457153,"avatar_url":"https://avatars.githubusercontent.com/u/3457153?v=3","gravatar_id":"","url":"https://api.github.com/users/critias","html_url":"https://github.com/critias","followers_url":"https://api.github.com/users/critias/followers","following_url":"https://api.github.com/users/critias/following{/other_user}","gists_url":"https://api.github.com/users/critias/gists{/gist_id}","starred_url":"https://api.github.com/users/critias/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/critias/subscriptions","organizations_url":"https://api.github.com/users/critias/orgs","repos_url":"https://api.github.com/users/critias/repos","events_url":"https://api.github.com/users/critias/events{/privacy}","received_events_url":"https://api.github.com/users/critias/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2016-01-07T17:09:41Z","updated_at":"2016-01-21T18:04:19Z","closed_at":"2016-01-21T18:04:19Z","body":"Hi,\r\nwe are having a hard time to reproduce the results we got with GroundHog using Blocks.\r\nGiven the exact same training data, vocabulary, test set and settings we are 3 Bleu points behind GroundHog on a German to English translation task. We tried many different setups and number of iterations, but we can't reach it.  \r\nThe GroundHog translation costs also seem to have a higher correlation between good and bad sentences then blocks. e.g.:\r\n\r\n\"vielen Dank .\" translated to \"thank you .\"\r\na perfect translation and a common phrase which should have a low cost.\r\nGroundHog cost: 0.000250929\r\nBlocks cost: 0.357417 \r\n\r\n\"fliegende Katze .\" is translated to \"fly away , cat .\" not wrong but kind of a strange/unusual sentence.\r\nGroundHog: 0.280177\r\nBlocks cost: 0.267061\r\n\r\nBlocks gives \"thank you .\" a higher cost to then \"fly away , cat .\" which seems strange to me. I take this as a hint that the problem is mainly related to the model and not to the search. The last comment here:\r\nhttps://github.com/kyunghyuncho/NMT/issues/21\r\nseems to have the same issue. Has there been any progress on this?\r\nAny tips where the Blocks computation graph differs from the GroundHog graph (it's to large to just look at it an see a difference)? Or other hints what the problem could be?\r\n\r\nThanks,\r\n"}}