{"action":"created","issue":{"url":"https://api.github.com/repos/dmlc/xgboost/issues/3055","repository_url":"https://api.github.com/repos/dmlc/xgboost","labels_url":"https://api.github.com/repos/dmlc/xgboost/issues/3055/labels{/name}","comments_url":"https://api.github.com/repos/dmlc/xgboost/issues/3055/comments","events_url":"https://api.github.com/repos/dmlc/xgboost/issues/3055/events","html_url":"https://github.com/dmlc/xgboost/issues/3055","id":290289250,"number":3055,"title":"Is it possible to update a model with new data without retraining the model from scratch?","user":{"login":"antontarasenko","id":6628679,"avatar_url":"https://avatars3.githubusercontent.com/u/6628679?v=4","gravatar_id":"","url":"https://api.github.com/users/antontarasenko","html_url":"https://github.com/antontarasenko","followers_url":"https://api.github.com/users/antontarasenko/followers","following_url":"https://api.github.com/users/antontarasenko/following{/other_user}","gists_url":"https://api.github.com/users/antontarasenko/gists{/gist_id}","starred_url":"https://api.github.com/users/antontarasenko/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/antontarasenko/subscriptions","organizations_url":"https://api.github.com/users/antontarasenko/orgs","repos_url":"https://api.github.com/users/antontarasenko/repos","events_url":"https://api.github.com/users/antontarasenko/events{/privacy}","received_events_url":"https://api.github.com/users/antontarasenko/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":14,"created_at":"2018-01-21T17:20:39Z","updated_at":"2018-04-21T05:40:24Z","closed_at":null,"author_association":"NONE","body":"I need to update a model with new data without retraining the model from scratch. That is, incremental training for the cases when not all the data available right away. \r\n\r\nThis problem is similar to the \"can't fit data in memory\" problem, which was raised before in #56, #163, #244. But it's been 2-3 years ago and I see some changes in available parameters `process_type` and `updater`. The [FAQ suggests](http://xgboost.readthedocs.io/en/latest/faq.html#i-have-a-big-dataset) using external memory via `cacheprefix`. But this assumes I have all the data ready.\r\n\r\nThe [solution](https://gist.github.com/ylogx/53fef94cc61d6a3e9b3eb900482f41e0) in #1686 uses several iterations over the entire data.\r\n\r\nAnother related issue is #2970, in particular https://github.com/dmlc/xgboost/issues/2970#issuecomment-354684604. I tried `'process_type': 'update'` but it throws the initial error mentioned in that issue. Without it, the model gives inconsistent results.\r\n\r\nI tried various combinations of parameters for `train` in Python. And `train` keeps making the model from scratch or something else. Here're [the examples](https://stackoverflow.com/questions/48366379/reduce-error-rates-from-incremental-training-in-xgboost-in-python).\r\n\r\nIn a nutshell, this is what works (sometimes) and needs a feedback from more experienced members of the community:\r\n\r\n```py\r\nprint('Full')\r\nbst_full = xgb.train(dtrain=train, params=params)\r\nprint(mean_squared_error(y_true=y_test, y_pred=bst_full.predict(test)))\r\n\r\nprint('Subset 1')\r\nbst_1 = xgb.train(dtrain=train_1, params=params)\r\nprint(mean_squared_error(y_true=y_test, y_pred=bst_1.predict(test)))\r\n\r\nprint('Subset 2')\r\nbst_2 = xgb.train(dtrain=train_2, params=params)\r\nprint(mean_squared_error(y_true=y_test, y_pred=bst_2.predict(test)))\r\n\r\nprint('Subset 1 updated with subset 2')\r\nbst_1u2 = xgb.train(dtrain=train_1, params=params)\r\nbst_1u2 = xgb.train(dtrain=train_2, params=params, xgb_model=bst_1u2)\r\nprint(mean_squared_error(y_true=y_test, y_pred=bst_1u2.predict(test)))\r\n```\r\n\r\nHere I'm looking to minimize the difference between the first and the fourth models. But it's keep jumping up and down. Even with equalling total boosting rounds in both methods.\r\n\r\nIs there a canonical way to update models with newly arriving data alone?\r\n\r\n### Environment\r\n\r\n* Python: 3.6\r\n* `xgboost`: 0.7.post3\r\n\r\n### Similar issues\r\n\r\n* Continue training in a dynamic training data environment #1225\r\n* incremental learning, partial_fit like sklearn? #1686\r\n* [xgboost4j-spark] Incremental training #1859\r\n* Retrain Xgboost Model #2495\r\n* \"xgb_model\" parameter in train() doesn't increment learning correctly #2707\r\n\r\nContributors saying new-data training was impossible at the time of writing:\r\n\r\n* https://github.com/dmlc/xgboost/issues/2495#issuecomment-317240566\r\n* https://github.com/dmlc/xgboost/issues/56#issuecomment-53962722"},"comment":{"url":"https://api.github.com/repos/dmlc/xgboost/issues/comments/383269612","html_url":"https://github.com/dmlc/xgboost/issues/3055#issuecomment-383269612","issue_url":"https://api.github.com/repos/dmlc/xgboost/issues/3055","id":383269612,"user":{"login":"benyaminelc90","id":25047555,"avatar_url":"https://avatars0.githubusercontent.com/u/25047555?v=4","gravatar_id":"","url":"https://api.github.com/users/benyaminelc90","html_url":"https://github.com/benyaminelc90","followers_url":"https://api.github.com/users/benyaminelc90/followers","following_url":"https://api.github.com/users/benyaminelc90/following{/other_user}","gists_url":"https://api.github.com/users/benyaminelc90/gists{/gist_id}","starred_url":"https://api.github.com/users/benyaminelc90/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/benyaminelc90/subscriptions","organizations_url":"https://api.github.com/users/benyaminelc90/orgs","repos_url":"https://api.github.com/users/benyaminelc90/repos","events_url":"https://api.github.com/users/benyaminelc90/events{/privacy}","received_events_url":"https://api.github.com/users/benyaminelc90/received_events","type":"User","site_admin":false},"created_at":"2018-04-21T05:40:24Z","updated_at":"2018-04-21T05:40:24Z","author_association":"NONE","body":"I also need to use incremental learning. I've read all links that have been mentioned above. However, I'm confused. \r\nFinally, is there any version of XGBoost to retrain a trained xgb model based on new received data point or batch of data? \r\nI've found below links that addressed this issue before the date of this post. Don't they work? Can't we do incremental learning with them? What's the problem with them?\r\n\r\nhttps://github.com/dmlc/xgboost/issues/1686\r\nhttps://github.com/dmlc/xgboost/issues/484\r\nhttps://stackoverflow.com/questions/38079853/how-can-i-implement-incremental-training-for-xgboost?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa\r\nhttps://github.com/dmlc/xgboost/issues/2495\r\n\r\n"}}
{"actor":{"display_login":"benyaminelc90"}}