{"action":"created","issue":{"url":"https://api.github.com/repos/BrainJS/brain.js/issues/333","repository_url":"https://api.github.com/repos/BrainJS/brain.js","labels_url":"https://api.github.com/repos/BrainJS/brain.js/issues/333/labels{/name}","comments_url":"https://api.github.com/repos/BrainJS/brain.js/issues/333/comments","events_url":"https://api.github.com/repos/BrainJS/brain.js/issues/333/events","html_url":"https://github.com/BrainJS/brain.js/issues/333","id":395046856,"node_id":"MDU6SXNzdWUzOTUwNDY4NTY=","number":333,"title":"Failure to resume training","user":{"login":"Prentillion","id":39955720,"node_id":"MDQ6VXNlcjM5OTU1NzIw","avatar_url":"https://avatars1.githubusercontent.com/u/39955720?v=4","gravatar_id":"","url":"https://api.github.com/users/Prentillion","html_url":"https://github.com/Prentillion","followers_url":"https://api.github.com/users/Prentillion/followers","following_url":"https://api.github.com/users/Prentillion/following{/other_user}","gists_url":"https://api.github.com/users/Prentillion/gists{/gist_id}","starred_url":"https://api.github.com/users/Prentillion/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Prentillion/subscriptions","organizations_url":"https://api.github.com/users/Prentillion/orgs","repos_url":"https://api.github.com/users/Prentillion/repos","events_url":"https://api.github.com/users/Prentillion/events{/privacy}","received_events_url":"https://api.github.com/users/Prentillion/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2019-01-01T01:09:19Z","updated_at":"2019-01-01T12:35:11Z","closed_at":null,"author_association":"NONE","body":"## *What* is wrong?\r\nNeural Network doesn't seem to lose old training progress when training is continued with more data\r\n\r\n## *Where* does it happen?\r\nTraining a neural network then continuing to train it later.\r\nSame result produced when loaded from JSON, new training significantly overrides old training\r\n\r\n```javascript\r\nnet.train([\r\n...\r\n]);\r\n\r\nnet.train([\r\n...\r\n])\r\n```\r\n\r\n## *How* do we replicate the issue?\r\nExample with an XOR gate\r\n```javascript\r\nconst net = new brain.NeuralNetwork();\r\n\r\nnet.train([\r\n    {input: [0, 0], output: [0]},\r\n    {input: [0, 1], output: [1]},\r\n    {input: [1, 0], output: [1]}\r\n]);\r\n\r\nnet.train({input: [1, 1], output: [0]});\r\n\r\nconsole.log(net.run([1, 0]));\r\n```\r\n\r\nCurrent result:\r\n```\r\n[ 0.03540751338005066 ]\r\n```\r\nExpected result:\r\n```\r\nExpect to be near to 1\r\n```\r\n## *How* important is this (1-5)?\r\n3\r\n\r\n## Expected behavior (i.e. solution)\r\n<!-- What do you think should have happened? -->\r\nContinuing to train the neural net should not completely override what training has been done already.\r\n\r\nIt can be fixed by training with the whole data set on the second time but this obviously takes a lot longer to compute\r\n```javascript\r\nconst net = new brain.NeuralNetwork();\r\n\r\nnet.train([\r\n    {input: [0, 0], output: [0]},\r\n    {input: [0, 1], output: [1]},\r\n    {input: [1, 0], output: [1]}\r\n]);\r\n\r\nnet.train([\r\n    {input: [0, 0], output: [0]},\r\n    {input: [0, 1], output: [1]},\r\n    {input: [1, 0], output: [1]},\r\n    {input: [1, 1], output: [0]}\r\n]);\r\n\r\nconsole.log(net.run([1, 0]));\r\n```\r\n\r\nOutput:\r\n```\r\n[ 0.9319882392883301 ]\r\n```\r\n\r\n## Other Comments\r\nLog error also starts significantly high on second round of training\r\n\r\nInital training of XOR:\r\n```\r\niterations: 10, training error: 0.2304970590057908\r\niterations: 20, training error: 0.22787304424770718\r\niterations: 30, training error: 0.22734293034487388\r\n...\r\niterations: 1010, training error: 0.005272660033630898\r\niterations: 1020, training error: 0.0051577028514087075\r\niterations: 1030, training error: 0.0050470909409198\r\n```\r\n\r\nContinued training\r\n```\r\niterations: 10, training error: 0.9645061463300095\r\niterations: 20, training error: 0.9593827602339076\r\niterations: 30, training error: 0.9525140087905726\r\n...\r\niterations: 150, training error: 0.007853373593889934\r\niterations: 160, training error: 0.006529269205814148\r\niterations: 170, training error: 0.00557478249546689\r\n```"},"comment":{"url":"https://api.github.com/repos/BrainJS/brain.js/issues/comments/450726293","html_url":"https://github.com/BrainJS/brain.js/issues/333#issuecomment-450726293","issue_url":"https://api.github.com/repos/BrainJS/brain.js/issues/333","id":450726293,"node_id":"MDEyOklzc3VlQ29tbWVudDQ1MDcyNjI5Mw==","user":{"login":"geokontop","id":43925925,"node_id":"MDQ6VXNlcjQzOTI1OTI1","avatar_url":"https://avatars2.githubusercontent.com/u/43925925?v=4","gravatar_id":"","url":"https://api.github.com/users/geokontop","html_url":"https://github.com/geokontop","followers_url":"https://api.github.com/users/geokontop/followers","following_url":"https://api.github.com/users/geokontop/following{/other_user}","gists_url":"https://api.github.com/users/geokontop/gists{/gist_id}","starred_url":"https://api.github.com/users/geokontop/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/geokontop/subscriptions","organizations_url":"https://api.github.com/users/geokontop/orgs","repos_url":"https://api.github.com/users/geokontop/repos","events_url":"https://api.github.com/users/geokontop/events{/privacy}","received_events_url":"https://api.github.com/users/geokontop/received_events","type":"User","site_admin":false},"created_at":"2019-01-01T12:35:11Z","updated_at":"2019-01-01T12:35:11Z","author_association":"CONTRIBUTOR","body":"I think that you should't expect that kind of partial training behaviour.\r\n\r\nYou are asking the network to train itself and that is what it does. It readjusts taking into account only the new training set. The previous training serves as initialization, which is quite insignificant. Its influence faints with each iteration if the validity of the old dataset statements is not reaffirmed. As a result, the newly adjusted network does not comply to the previous training set. \r\n\r\nThis is the expected behaviour. Training is overall, it overides previous trainings.  You have to provide, each time, the hole training set. :-)"}}
{"actor":{"display_login":"geokontop"}}