{"action":"created","issue":{"url":"https://api.github.com/repos/ibab/tensorflow-wavenet/issues/65","repository_url":"https://api.github.com/repos/ibab/tensorflow-wavenet","labels_url":"https://api.github.com/repos/ibab/tensorflow-wavenet/issues/65/labels{/name}","comments_url":"https://api.github.com/repos/ibab/tensorflow-wavenet/issues/65/comments","events_url":"https://api.github.com/repos/ibab/tensorflow-wavenet/issues/65/events","html_url":"https://github.com/ibab/tensorflow-wavenet/issues/65","id":178320740,"number":65,"title":"add regularization, dropout and batch norm?","user":{"login":"r-zemblys","id":8914323,"avatar_url":"https://avatars.githubusercontent.com/u/8914323?v=3","gravatar_id":"","url":"https://api.github.com/users/r-zemblys","html_url":"https://github.com/r-zemblys","followers_url":"https://api.github.com/users/r-zemblys/followers","following_url":"https://api.github.com/users/r-zemblys/following{/other_user}","gists_url":"https://api.github.com/users/r-zemblys/gists{/gist_id}","starred_url":"https://api.github.com/users/r-zemblys/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/r-zemblys/subscriptions","organizations_url":"https://api.github.com/users/r-zemblys/orgs","repos_url":"https://api.github.com/users/r-zemblys/repos","events_url":"https://api.github.com/users/r-zemblys/events{/privacy}","received_events_url":"https://api.github.com/users/r-zemblys/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":8,"created_at":"2016-09-21T11:34:17Z","updated_at":"2016-09-22T12:29:26Z","closed_at":null,"body":"Has anybody got loss lower than ~2? Tried couple of configurations (default, 3 and 4 stacks of 10 dilation layers), but loss does not get lower, suggesting the network is not learning anymore. \r\n\r\nAlso, there is what happened happened after ~30k steps:\r\n![training](https://cloud.githubusercontent.com/assets/8914323/18709029/99d32c48-8006-11e6-8a33-ecd8388afde7.png)\r\nI believe this is the same problem as reported in [#30](https://github.com/ibab/tensorflow-wavenet/issues/30). There is what happens with weights:\r\n![weights](https://cloud.githubusercontent.com/assets/8914323/18709214/abe1ffbc-8007-11e6-9454-fabca3cfc3e5.png)\r\n\r\nNow running the same network with l2 norm regularization added.\r\n\r\nAnd one more note: training just stops after 44256 steps (already happened twice) without any warnings or errors, despite of num_steps=50000\r\n"},"comment":{"url":"https://api.github.com/repos/ibab/tensorflow-wavenet/issues/comments/248889871","html_url":"https://github.com/ibab/tensorflow-wavenet/issues/65#issuecomment-248889871","issue_url":"https://api.github.com/repos/ibab/tensorflow-wavenet/issues/65","id":248889871,"user":{"login":"r-zemblys","id":8914323,"avatar_url":"https://avatars.githubusercontent.com/u/8914323?v=3","gravatar_id":"","url":"https://api.github.com/users/r-zemblys","html_url":"https://github.com/r-zemblys","followers_url":"https://api.github.com/users/r-zemblys/followers","following_url":"https://api.github.com/users/r-zemblys/following{/other_user}","gists_url":"https://api.github.com/users/r-zemblys/gists{/gist_id}","starred_url":"https://api.github.com/users/r-zemblys/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/r-zemblys/subscriptions","organizations_url":"https://api.github.com/users/r-zemblys/orgs","repos_url":"https://api.github.com/users/r-zemblys/repos","events_url":"https://api.github.com/users/r-zemblys/events{/privacy}","received_events_url":"https://api.github.com/users/r-zemblys/received_events","type":"User","site_admin":false},"created_at":"2016-09-22T12:29:26Z","updated_at":"2016-09-22T12:29:26Z","body":"Forgot to add. This is configuration I've used:\r\n```\r\n\r\n{\r\n    \"filter_width\": 2,\r\n    \"quantization_steps\": 256,\r\n    \"sample_rate\": 16000,\r\n    \"dilations\": [1, 2, 4, 8, 16, 32, 64, 128, 256, 512,\r\n                  1, 2, 4, 8, 16, 32, 64, 128, 256, 512,\r\n                  1, 2, 4, 8, 16, 32, 64, 128, 256, 512],\r\n    \"residual_channels\": 32,\r\n    \"dilation_channels\": 16,\r\n    \"use_biases\": false\r\n}\r\n```\r\nBut as I've mention in the beginning, there is no difference (at least in loss curve) if using default configuration."}}
{"actor":{"display_login":"r-zemblys"}}