{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/1971","repository_url":"https://api.github.com/repos/fchollet/keras","labels_url":"https://api.github.com/repos/fchollet/keras/issues/1971/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/1971/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/1971/events","html_url":"https://github.com/fchollet/keras/issues/1971","id":140857615,"number":1971,"title":"Accuracy decreases as epoch increases","user":{"login":"anik786","id":5897460,"avatar_url":"https://avatars.githubusercontent.com/u/5897460?v=3","gravatar_id":"","url":"https://api.github.com/users/anik786","html_url":"https://github.com/anik786","followers_url":"https://api.github.com/users/anik786/followers","following_url":"https://api.github.com/users/anik786/following{/other_user}","gists_url":"https://api.github.com/users/anik786/gists{/gist_id}","starred_url":"https://api.github.com/users/anik786/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anik786/subscriptions","organizations_url":"https://api.github.com/users/anik786/orgs","repos_url":"https://api.github.com/users/anik786/repos","events_url":"https://api.github.com/users/anik786/events{/privacy}","received_events_url":"https://api.github.com/users/anik786/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-03-15T03:01:47Z","updated_at":"2016-03-15T22:51:34Z","closed_at":null,"body":"When running my neural network and fitting it like so:\r\n\r\n`model.fit(x, t, batch_size=256, nb_epoch=100, verbose=2, validation_split=0.1, show_accuracy=True)`\r\n\r\nI have found that as the number of epochs increases, there are times where the validation accuracy actually decreases.\r\n\r\nFor example at epoch 12 I got:\r\n\r\n```\r\nEpoch 12/100\r\n4s - loss: 0.1026 - acc: 0.9667 - val_loss: 0.1384 - val_acc: 0.9733\r\n```\r\n\r\nBut by the end I git:\r\n```\r\nEpoch 95/100\r\n3s - loss: 4.6988e-04 - acc: 1.0000 - val_loss: 0.1290 - val_acc: 0.9600\r\nEpoch 96/100\r\n2s - loss: 5.7437e-04 - acc: 1.0000 - val_loss: 0.1321 - val_acc: 0.9600\r\nEpoch 97/100\r\n1s - loss: 6.3242e-04 - acc: 1.0000 - val_loss: 0.1312 - val_acc: 0.9600\r\nEpoch 98/100\r\n1s - loss: 5.3643e-04 - acc: 1.0000 - val_loss: 0.1322 - val_acc: 0.9600\r\nEpoch 99/100\r\n2s - loss: 4.2413e-04 - acc: 1.0000 - val_loss: 0.1326 - val_acc: 0.9600\r\nEpoch 100/100\r\n1s - loss: 4.8201e-04 - acc: 1.0000 - val_loss: 0.1295 - val_acc: 0.9600\r\n```\r\n\r\nIs this supposed to happen. Why?\r\nAlso will the final network have an accuracy of 0.96 or 0.9733."},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/197058372","html_url":"https://github.com/fchollet/keras/issues/1971#issuecomment-197058372","issue_url":"https://api.github.com/repos/fchollet/keras/issues/1971","id":197058372,"user":{"login":"tboquet","id":5553901,"avatar_url":"https://avatars.githubusercontent.com/u/5553901?v=3","gravatar_id":"","url":"https://api.github.com/users/tboquet","html_url":"https://github.com/tboquet","followers_url":"https://api.github.com/users/tboquet/followers","following_url":"https://api.github.com/users/tboquet/following{/other_user}","gists_url":"https://api.github.com/users/tboquet/gists{/gist_id}","starred_url":"https://api.github.com/users/tboquet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tboquet/subscriptions","organizations_url":"https://api.github.com/users/tboquet/orgs","repos_url":"https://api.github.com/users/tboquet/repos","events_url":"https://api.github.com/users/tboquet/events{/privacy}","received_events_url":"https://api.github.com/users/tboquet/received_events","type":"User","site_admin":false},"created_at":"2016-03-15T22:51:34Z","updated_at":"2016-03-15T22:51:34Z","body":"This behavior is closely related to the bias-variance trade-off. You could take a look [here](http://research.microsoft.com/en-us/um/people/cmbishop/prml/slides/prml-slides-3.pdf) from slide 17 to 25.\r\n\r\nYou could also use a `ModelCheckpoint` callback to save your weights. You will be able to select the epoch based on the results and to reload the weights. You could find an example [here](http://keras.io/callbacks/#example-model-checkpoints)."}}