{"action":"created","issue":{"url":"https://api.github.com/repos/dmlc/xgboost/issues/799","repository_url":"https://api.github.com/repos/dmlc/xgboost","labels_url":"https://api.github.com/repos/dmlc/xgboost/issues/799/labels{/name}","comments_url":"https://api.github.com/repos/dmlc/xgboost/issues/799/comments","events_url":"https://api.github.com/repos/dmlc/xgboost/issues/799/events","html_url":"https://github.com/dmlc/xgboost/issues/799","id":130563835,"number":799,"title":"base_score default","user":{"login":"ronmexico2718","id":16491289,"avatar_url":"https://avatars.githubusercontent.com/u/16491289?v=3","gravatar_id":"","url":"https://api.github.com/users/ronmexico2718","html_url":"https://github.com/ronmexico2718","followers_url":"https://api.github.com/users/ronmexico2718/followers","following_url":"https://api.github.com/users/ronmexico2718/following{/other_user}","gists_url":"https://api.github.com/users/ronmexico2718/gists{/gist_id}","starred_url":"https://api.github.com/users/ronmexico2718/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ronmexico2718/subscriptions","organizations_url":"https://api.github.com/users/ronmexico2718/orgs","repos_url":"https://api.github.com/users/ronmexico2718/repos","events_url":"https://api.github.com/users/ronmexico2718/events{/privacy}","received_events_url":"https://api.github.com/users/ronmexico2718/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2016-02-02T03:43:39Z","updated_at":"2016-02-09T09:00:28Z","closed_at":null,"body":"At least in the R package, it seems that the base_score defaults to 0.5.  For a classification problem, this seems to be a reasonable choice.  However, this strikes me as a bizarre choice for a regression problem.  Would it make more sense to set base_score=mean(label) by default?  This choice might even be an improvement over the default in both regression and classification settings.\r\n\r\nIf the learning rate/shrinkage were set to 1, then the choice of base_score would be irrelevant as the resulting model would just compensate for any changes in the mean.  However, essentially all reasonable implementations of boosted trees use learning rate much smaller than 1.  Thus, in principle, it would seem that the choice of base_score would affect the final model.\r\n\r\nI understand that it is extremely straightforward to set the base_score manually, but if there is a way to pick a better default, it may be worth implementing.\r\n\r\nAny thoughts?"},"comment":{"url":"https://api.github.com/repos/dmlc/xgboost/issues/comments/181768076","html_url":"https://github.com/dmlc/xgboost/issues/799#issuecomment-181768076","issue_url":"https://api.github.com/repos/dmlc/xgboost/issues/799","id":181768076,"user":{"login":"gugatr0n1c","id":17110787,"avatar_url":"https://avatars.githubusercontent.com/u/17110787?v=3","gravatar_id":"","url":"https://api.github.com/users/gugatr0n1c","html_url":"https://github.com/gugatr0n1c","followers_url":"https://api.github.com/users/gugatr0n1c/followers","following_url":"https://api.github.com/users/gugatr0n1c/following{/other_user}","gists_url":"https://api.github.com/users/gugatr0n1c/gists{/gist_id}","starred_url":"https://api.github.com/users/gugatr0n1c/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gugatr0n1c/subscriptions","organizations_url":"https://api.github.com/users/gugatr0n1c/orgs","repos_url":"https://api.github.com/users/gugatr0n1c/repos","events_url":"https://api.github.com/users/gugatr0n1c/events{/privacy}","received_events_url":"https://api.github.com/users/gugatr0n1c/received_events","type":"User","site_admin":false},"created_at":"2016-02-09T09:00:28Z","updated_at":"2016-02-09T09:00:28Z","body":"Yeah, I agree.\r\nIt makes sence... when base_score is set to 0.5, and we get regression problem with np.average(label) very high and eta very small (I used eta = 0.0015 and num_tree = 5000) first hundres of trees just trying to catch this big mean - with eta = 0.0015 it take a lot of time just solving this problem... with base_score set mean(label) this is actually solved right away at begining... a next tree trying to solve original task..."}}