{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/2768","repository_url":"https://api.github.com/repos/fchollet/keras","labels_url":"https://api.github.com/repos/fchollet/keras/issues/2768/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/2768/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/2768/events","html_url":"https://github.com/fchollet/keras/issues/2768","id":156070539,"number":2768,"title":"model.fit() does not return the best model","user":{"login":"DSA101","id":9289572,"avatar_url":"https://avatars.githubusercontent.com/u/9289572?v=3","gravatar_id":"","url":"https://api.github.com/users/DSA101","html_url":"https://github.com/DSA101","followers_url":"https://api.github.com/users/DSA101/followers","following_url":"https://api.github.com/users/DSA101/following{/other_user}","gists_url":"https://api.github.com/users/DSA101/gists{/gist_id}","starred_url":"https://api.github.com/users/DSA101/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DSA101/subscriptions","organizations_url":"https://api.github.com/users/DSA101/orgs","repos_url":"https://api.github.com/users/DSA101/repos","events_url":"https://api.github.com/users/DSA101/events{/privacy}","received_events_url":"https://api.github.com/users/DSA101/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2016-05-20T23:49:22Z","updated_at":"2016-05-24T05:17:18Z","closed_at":null,"body":"Unless  you are using ModelCheckpoint callback with save_best_only parameter, model.fit() returns not the best model encountered during the training (i.e. the one with the lowest validation loss or with highest validation accuracy), but rather the model whatever it happens to be at the last epoch (which cannot be counted on to have lowest loss or highest accuracy). \r\n\r\nSince many users, esp beginners are unaware of this callback, by default their results are always not the best possible with the current parameters and the accuracy of their network is unnecessarily negatively impacted, despite that a better model has likely been already encountered during the already performed training pass.\r\n\r\nProposed change: model.fit() should return the best model encountered during training by default, or if it will negatively impact performance, at least provide a parameter to do so.\r\n\r\nThe current workaround is the following:\r\nmodel = Sequential()\r\n...\r\nmodel.compile(loss='mse', optimizer=opt)\r\ncheckpointer = ModelCheckpoint(filepath=\"weights.hdf5\", verbose=1, save_best_only=True)\r\nhist = model.fit(..., callbacks=[checkpointer])\r\nmodel.load_weights('weights.hdf5')\r\npredicted = model.predict(X_test_mat)\r\n\r\n- [X] Check that you are up-to-date with the master branch of Keras. You can update with:\r\npip install git+git://github.com/fchollet/keras.git --upgrade --no-deps\r\n\r\n- [X] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:\r\npip install git+git://github.com/Theano/Theano.git --upgrade --no-deps\r\n\r\n- [X] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short)."},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/221168829","html_url":"https://github.com/fchollet/keras/issues/2768#issuecomment-221168829","issue_url":"https://api.github.com/repos/fchollet/keras/issues/2768","id":221168829,"user":{"login":"joelthchao","id":10974532,"avatar_url":"https://avatars.githubusercontent.com/u/10974532?v=3","gravatar_id":"","url":"https://api.github.com/users/joelthchao","html_url":"https://github.com/joelthchao","followers_url":"https://api.github.com/users/joelthchao/followers","following_url":"https://api.github.com/users/joelthchao/following{/other_user}","gists_url":"https://api.github.com/users/joelthchao/gists{/gist_id}","starred_url":"https://api.github.com/users/joelthchao/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joelthchao/subscriptions","organizations_url":"https://api.github.com/users/joelthchao/orgs","repos_url":"https://api.github.com/users/joelthchao/repos","events_url":"https://api.github.com/users/joelthchao/events{/privacy}","received_events_url":"https://api.github.com/users/joelthchao/received_events","type":"User","site_admin":false},"created_at":"2016-05-24T05:17:18Z","updated_at":"2016-05-24T05:17:18Z","body":"> What would be a down side of such change?\r\n\r\nThe behaviors of `fit` in other ML framework are the same as Keras. Modifying it might produce unexpected result for most of the users. "}}
{"actor":{"display_login":"joelthchao"}}