{"action":"created","issue":{"url":"https://api.github.com/repos/r9y9/deepvoice3_pytorch/issues/43","repository_url":"https://api.github.com/repos/r9y9/deepvoice3_pytorch","labels_url":"https://api.github.com/repos/r9y9/deepvoice3_pytorch/issues/43/labels{/name}","comments_url":"https://api.github.com/repos/r9y9/deepvoice3_pytorch/issues/43/comments","events_url":"https://api.github.com/repos/r9y9/deepvoice3_pytorch/issues/43/events","html_url":"https://github.com/r9y9/deepvoice3_pytorch/issues/43","id":297795407,"number":43,"title":"Issue training with DeepVoice3 model with LJSpeech Data","user":{"login":"timbrucks","id":17724266,"avatar_url":"https://avatars1.githubusercontent.com/u/17724266?v=4","gravatar_id":"","url":"https://api.github.com/users/timbrucks","html_url":"https://github.com/timbrucks","followers_url":"https://api.github.com/users/timbrucks/followers","following_url":"https://api.github.com/users/timbrucks/following{/other_user}","gists_url":"https://api.github.com/users/timbrucks/gists{/gist_id}","starred_url":"https://api.github.com/users/timbrucks/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/timbrucks/subscriptions","organizations_url":"https://api.github.com/users/timbrucks/orgs","repos_url":"https://api.github.com/users/timbrucks/repos","events_url":"https://api.github.com/users/timbrucks/events{/privacy}","received_events_url":"https://api.github.com/users/timbrucks/received_events","type":"User","site_admin":false},"labels":[{"id":736393883,"url":"https://api.github.com/repos/r9y9/deepvoice3_pytorch/labels/bug","name":"bug","color":"ee0701","default":true}],"state":"open","locked":false,"assignee":{"login":"r9y9","id":1220272,"avatar_url":"https://avatars1.githubusercontent.com/u/1220272?v=4","gravatar_id":"","url":"https://api.github.com/users/r9y9","html_url":"https://github.com/r9y9","followers_url":"https://api.github.com/users/r9y9/followers","following_url":"https://api.github.com/users/r9y9/following{/other_user}","gists_url":"https://api.github.com/users/r9y9/gists{/gist_id}","starred_url":"https://api.github.com/users/r9y9/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/r9y9/subscriptions","organizations_url":"https://api.github.com/users/r9y9/orgs","repos_url":"https://api.github.com/users/r9y9/repos","events_url":"https://api.github.com/users/r9y9/events{/privacy}","received_events_url":"https://api.github.com/users/r9y9/received_events","type":"User","site_admin":false},"assignees":[{"login":"r9y9","id":1220272,"avatar_url":"https://avatars1.githubusercontent.com/u/1220272?v=4","gravatar_id":"","url":"https://api.github.com/users/r9y9","html_url":"https://github.com/r9y9","followers_url":"https://api.github.com/users/r9y9/followers","following_url":"https://api.github.com/users/r9y9/following{/other_user}","gists_url":"https://api.github.com/users/r9y9/gists{/gist_id}","starred_url":"https://api.github.com/users/r9y9/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/r9y9/subscriptions","organizations_url":"https://api.github.com/users/r9y9/orgs","repos_url":"https://api.github.com/users/r9y9/repos","events_url":"https://api.github.com/users/r9y9/events{/privacy}","received_events_url":"https://api.github.com/users/r9y9/received_events","type":"User","site_admin":false}],"milestone":null,"comments":15,"created_at":"2018-02-16T14:05:34Z","updated_at":"2018-02-24T15:02:32Z","closed_at":null,"author_association":"NONE","body":"Thanks for your excellent implementation of Deep Voice 3. I am attempting to retrain a DeepVoice3 model using the LJSpeech data. My interest in training a new model is that I want to make some small model parameter changes in order to enable fine-tuning using some Spanish data that I have.\r\n\r\nAs a first step I tried to retrain the baseline model and I have run into some issues.\r\n\r\nWith my installation, I have been able to successfully synthesize using the pre-trained DeepVoice3 model with git commit 4357976 as your instructions indicate. That synthesized audio sounds very much like the samples linked from the instructions page.\r\n\r\nHowever, I am trying to train now with the **latest git commit** (commit 48d1014, dated Feb 7). I am using the LJSpeech data set downloaded from the link you provided. I have run the pre-processing and training steps as indicated in your instructions. I am using the default preset parameters for deepvoice3_ljspeech.\r\n\r\nI have let the training process run for a while. When I synthesize using the checkpoint saved at 210K iterations, the alignment is bad and the audio is very robotic and mostly unintelligible.\r\n\r\n![0_checkpoint_step000210000_alignment](https://user-images.githubusercontent.com/17724266/36311054-9e087ea8-12ef-11e8-8f1d-dbbb91e3de90.png)\r\n\r\nWhen I synthesize using the checkpoint saved at 700K iterations, the alignment is better (but not great); the audio is improved but still robotic and choppy.\r\n\r\n![0_checkpoint_step000700000_alignment](https://user-images.githubusercontent.com/17724266/36311069-ac429698-12ef-11e8-868f-05e1d016c419.png)\r\n\r\nI can post the synthesized wav files via dropbox if you are interested. I expected to have good alignment and audio at 210K iterations as that is what the pretrained model used. \r\n\r\n\r\nAny ideas what has changed between git commits 4357976 and 48d1014 that could have caused this issue? When I diff the two commits, I see some changes in audio.py, some places where support for multi-voice has been added, and some other changes I do not yet understand. There are some additions to hparams.py, but I only noticed one difference: in the current commit, masked_loss_weight defaults to 0.5, but in the prior commit the default was 0.0.\r\n\r\n\r\nI have just started a new training run with masked_loss_weight set to 0.0. In the meantime, do you have thoughts on anything else that might be causing the issues I am seeing?"},"comment":{"url":"https://api.github.com/repos/r9y9/deepvoice3_pytorch/issues/comments/368234214","html_url":"https://github.com/r9y9/deepvoice3_pytorch/issues/43#issuecomment-368234214","issue_url":"https://api.github.com/repos/r9y9/deepvoice3_pytorch/issues/43","id":368234214,"user":{"login":"r9y9","id":1220272,"avatar_url":"https://avatars1.githubusercontent.com/u/1220272?v=4","gravatar_id":"","url":"https://api.github.com/users/r9y9","html_url":"https://github.com/r9y9","followers_url":"https://api.github.com/users/r9y9/followers","following_url":"https://api.github.com/users/r9y9/following{/other_user}","gists_url":"https://api.github.com/users/r9y9/gists{/gist_id}","starred_url":"https://api.github.com/users/r9y9/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/r9y9/subscriptions","organizations_url":"https://api.github.com/users/r9y9/orgs","repos_url":"https://api.github.com/users/r9y9/repos","events_url":"https://api.github.com/users/r9y9/events{/privacy}","received_events_url":"https://api.github.com/users/r9y9/received_events","type":"User","site_admin":false},"created_at":"2018-02-24T15:02:32Z","updated_at":"2018-02-24T15:02:32Z","author_association":"OWNER","body":"OK, now I can reproduce. Looking into it..."}}
{"actor":{"display_login":"r9y9"}}