{"action":"created","issue":{"url":"https://api.github.com/repos/zhongkaifu/RNNSharp/issues/6","repository_url":"https://api.github.com/repos/zhongkaifu/RNNSharp","labels_url":"https://api.github.com/repos/zhongkaifu/RNNSharp/issues/6/labels{/name}","comments_url":"https://api.github.com/repos/zhongkaifu/RNNSharp/issues/6/comments","events_url":"https://api.github.com/repos/zhongkaifu/RNNSharp/issues/6/events","html_url":"https://github.com/zhongkaifu/RNNSharp/issues/6","id":133887560,"number":6,"title":"Not converging due to learning rate alpha","user":{"login":"bratao","id":1090152,"avatar_url":"https://avatars.githubusercontent.com/u/1090152?v=3","gravatar_id":"","url":"https://api.github.com/users/bratao","html_url":"https://github.com/bratao","followers_url":"https://api.github.com/users/bratao/followers","following_url":"https://api.github.com/users/bratao/following{/other_user}","gists_url":"https://api.github.com/users/bratao/gists{/gist_id}","starred_url":"https://api.github.com/users/bratao/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bratao/subscriptions","organizations_url":"https://api.github.com/users/bratao/orgs","repos_url":"https://api.github.com/users/bratao/repos","events_url":"https://api.github.com/users/bratao/events{/privacy}","received_events_url":"https://api.github.com/users/bratao/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2016-02-16T05:09:07Z","updated_at":"2016-02-16T06:01:03Z","closed_at":null,"body":"Hello,\r\nTesting RNNSharp I was unable to make the model converge. No matter what setting I used.\r\nI changed two places and got the model to finally converge:\r\n\r\n```\r\nif (ppl >= lastPPL && lastAlpha != rnn.LearningRate)\r\n {\r\n  //Although we reduce alpha value, we still cannot get better result.\r\n```\r\nI changed it to break only after we tried to lower alpha 8 times and failed to get a improvement.\r\n\r\nI also changed this fragment:\r\n`rnn.LearningRate = rnn.LearningRate / 2.0f;`\r\n\r\nTo a decrease in a lower rate, in my case, 1.4.\r\n\r\nDo any of those changes make sense ? \r\n\r\nDo you think that a smarter learning rate annealing could improve RNNSharp, or am I looking at the wrong place ??\r\n\r\nThanks !!\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/zhongkaifu/RNNSharp/issues/comments/184533993","html_url":"https://github.com/zhongkaifu/RNNSharp/issues/6#issuecomment-184533993","issue_url":"https://api.github.com/repos/zhongkaifu/RNNSharp/issues/6","id":184533993,"user":{"login":"zhongkaifu","id":3347714,"avatar_url":"https://avatars.githubusercontent.com/u/3347714?v=3","gravatar_id":"","url":"https://api.github.com/users/zhongkaifu","html_url":"https://github.com/zhongkaifu","followers_url":"https://api.github.com/users/zhongkaifu/followers","following_url":"https://api.github.com/users/zhongkaifu/following{/other_user}","gists_url":"https://api.github.com/users/zhongkaifu/gists{/gist_id}","starred_url":"https://api.github.com/users/zhongkaifu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zhongkaifu/subscriptions","organizations_url":"https://api.github.com/users/zhongkaifu/orgs","repos_url":"https://api.github.com/users/zhongkaifu/repos","events_url":"https://api.github.com/users/zhongkaifu/events{/privacy}","received_events_url":"https://api.github.com/users/zhongkaifu/received_events","type":"User","site_admin":false},"created_at":"2016-02-16T06:01:03Z","updated_at":"2016-02-16T06:01:03Z","body":"In addition, I have fixed the RNNSharp crashing bug when word embedding feature isn't enabled for SimpleRNN. You can get it by syncing the latest code from code base."}}