{"action":"created","issue":{"url":"https://api.github.com/repos/ZhenYangIACAS/NMT_GAN/issues/11","repository_url":"https://api.github.com/repos/ZhenYangIACAS/NMT_GAN","labels_url":"https://api.github.com/repos/ZhenYangIACAS/NMT_GAN/issues/11/labels{/name}","comments_url":"https://api.github.com/repos/ZhenYangIACAS/NMT_GAN/issues/11/comments","events_url":"https://api.github.com/repos/ZhenYangIACAS/NMT_GAN/issues/11/events","html_url":"https://github.com/ZhenYangIACAS/NMT_GAN/issues/11","id":320068504,"number":11,"title":"Clarification on files in config_discriminator.yaml","user":{"login":"kellymarchisio","id":3612904,"avatar_url":"https://avatars0.githubusercontent.com/u/3612904?v=4","gravatar_id":"","url":"https://api.github.com/users/kellymarchisio","html_url":"https://github.com/kellymarchisio","followers_url":"https://api.github.com/users/kellymarchisio/followers","following_url":"https://api.github.com/users/kellymarchisio/following{/other_user}","gists_url":"https://api.github.com/users/kellymarchisio/gists{/gist_id}","starred_url":"https://api.github.com/users/kellymarchisio/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kellymarchisio/subscriptions","organizations_url":"https://api.github.com/users/kellymarchisio/orgs","repos_url":"https://api.github.com/users/kellymarchisio/repos","events_url":"https://api.github.com/users/kellymarchisio/events{/privacy}","received_events_url":"https://api.github.com/users/kellymarchisio/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-05-03T20:28:48Z","updated_at":"2018-05-13T08:33:04Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"I'm hoping for clarification on the files passed into config_discriminator.yaml. \r\n\r\nAs I understand it:\r\n- dis_positive_data and dis_source_data are 1 million random lines from the original target (o_dst) and source training files (o_src), manually padded with <PAD> to length 50 (I wrote my own script to do this). These files are created using the same files used in generator pretraining. Must we pad the sentences ourselves as I have?\r\n- dis_negative_data (negative_bpe_u8.txt) is the output from sh generate_sample.sh predicted using o_src\r\n- dis_src_vocab and dis_dst_vocab are BPE vocabulary lists corresponding to the vocabulary of all BPEs used in o_dst and o_src. The vocabulary lists are created by running o_dst and o_src through vocab.py\r\n  - alternatively, since I am running the en-de translation, should I expect to use vocab.bpe.32000 for src_vocab and dst_vocab instead of separate vocab files with 38831/34432 words, as in the demo config?\r\n\r\nIs my understanding correct? Can you please provide clarification on how the files in the discriminator pretraining are created?\r\n\r\nFor context, I am trying to solve an issue where when running the discriminator pretraining, my loss falls to 2-3, but the accuracy always oscillates around 0.5 - even after 700K steps.  \r\n"},"comment":{"url":"https://api.github.com/repos/ZhenYangIACAS/NMT_GAN/issues/comments/388610459","html_url":"https://github.com/ZhenYangIACAS/NMT_GAN/issues/11#issuecomment-388610459","issue_url":"https://api.github.com/repos/ZhenYangIACAS/NMT_GAN/issues/11","id":388610459,"user":{"login":"kellymarchisio","id":3612904,"avatar_url":"https://avatars0.githubusercontent.com/u/3612904?v=4","gravatar_id":"","url":"https://api.github.com/users/kellymarchisio","html_url":"https://github.com/kellymarchisio","followers_url":"https://api.github.com/users/kellymarchisio/followers","following_url":"https://api.github.com/users/kellymarchisio/following{/other_user}","gists_url":"https://api.github.com/users/kellymarchisio/gists{/gist_id}","starred_url":"https://api.github.com/users/kellymarchisio/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kellymarchisio/subscriptions","organizations_url":"https://api.github.com/users/kellymarchisio/orgs","repos_url":"https://api.github.com/users/kellymarchisio/repos","events_url":"https://api.github.com/users/kellymarchisio/events{/privacy}","received_events_url":"https://api.github.com/users/kellymarchisio/received_events","type":"User","site_admin":false},"created_at":"2018-05-13T08:33:04Z","updated_at":"2018-05-13T08:33:04Z","author_association":"CONTRIBUTOR","body":"@ZhenYangIACAS Thanks very much for your response. After fixing some errors, I also achieve accuracy 0.70 after 2 epochs.  I notice though that the accuracy bounces around quite significantly as seen here:\r\n- testing the accuracy on the evaluation sets when epoch 1, samples 3400000\r\n- the total accuracy in evaluation is 0.714286\r\n- testing the accuracy on the evaluation sets when epoch 1, samples 3410000\r\n- the total accuracy in evaluation is 0.708494\r\n- testing the accuracy on the evaluation sets when epoch 1, samples 3420000\r\n- the total accuracy in evaluation is 0.722008\r\n- testing the accuracy on the evaluation sets when epoch 1, samples 3430000\r\n- the total accuracy in evaluation is 0.735521\r\n- testing the accuracy on the evaluation sets when epoch 1, samples 3440000\r\n- the total accuracy in evaluation is 0.700772\r\n\r\nIs this expected, or a bug?\r\n\r\nI also notice that loss alternates between very high values and lower values at the beginning of training:\r\n- epoch 0, samples 100, loss 8.257384, accuracy 0.510000 BatchTime 21.912569\r\n- epoch 0, samples 200, loss 109.485771, accuracy 0.500000 BatchTime 1.326053\r\n- epoch 0, samples 300, loss 26.595387, accuracy 0.500000 BatchTime 1.261572\r\n- epoch 0, samples 400, loss 109.342659, accuracy 0.500000 BatchTime 1.244808\r\n- epoch 0, samples 500, loss 24.455862, accuracy 0.500000 BatchTime 1.179417\r\n- epoch 0, samples 600, loss 101.686081, accuracy 0.500000 BatchTime 1.192621\r\n\r\nIs this also expected, and what might cause this behavior? I would expect loss to monotonically decrease.\r\n\r\nThanks very much for releasing this code base - I've enjoyed working with it."}}
{"actor":{"display_login":"kellymarchisio"}}