{"action":"created","issue":{"url":"https://api.github.com/repos/Microsoft/LightGBM/issues/126","repository_url":"https://api.github.com/repos/Microsoft/LightGBM","labels_url":"https://api.github.com/repos/Microsoft/LightGBM/issues/126/labels{/name}","comments_url":"https://api.github.com/repos/Microsoft/LightGBM/issues/126/comments","events_url":"https://api.github.com/repos/Microsoft/LightGBM/issues/126/events","html_url":"https://github.com/Microsoft/LightGBM/issues/126","id":196091853,"number":126,"title":"Dart - very poor accuracy","user":{"login":"gugatr0n1c","id":17110787,"avatar_url":"https://avatars.githubusercontent.com/u/17110787?v=3","gravatar_id":"","url":"https://api.github.com/users/gugatr0n1c","html_url":"https://github.com/gugatr0n1c","followers_url":"https://api.github.com/users/gugatr0n1c/followers","following_url":"https://api.github.com/users/gugatr0n1c/following{/other_user}","gists_url":"https://api.github.com/users/gugatr0n1c/gists{/gist_id}","starred_url":"https://api.github.com/users/gugatr0n1c/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gugatr0n1c/subscriptions","organizations_url":"https://api.github.com/users/gugatr0n1c/orgs","repos_url":"https://api.github.com/users/gugatr0n1c/repos","events_url":"https://api.github.com/users/gugatr0n1c/events{/privacy}","received_events_url":"https://api.github.com/users/gugatr0n1c/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":57,"created_at":"2016-12-16T16:00:43Z","updated_at":"2016-12-26T03:31:59Z","closed_at":"2016-12-26T03:31:59Z","body":"When I use dart as a booster I always get very poor performance in term of l2 result for regression task.\r\nEven If I use small drop_rate = 0.01 or big like 0.3.\r\n\r\nWhen I use dart in xgboost on same dataset, with similar setting (same learning rate, similiar num_trees) dart alwasy give me boost for accuracy (small but always).\r\n\r\nBut here accuracy is poor badly, like there is a bug, not just dart is not suitable for my task.\r\n\r\nCan anyone confirm that dart is working for regression task in term of better accuracy?\r\n\r\nMy setting is as follows (part of the Python code for ramdom search of params):\r\n\r\nlr = np.random.choice([0.01, 0.005, 0.0025])\r\nlist_count = np.random.choice([250, 500, 750, 1000])\r\nmin_in_leaf = np.random.choice([25, 50, 100])\r\nsubF = np.random.choice([0.15, 0.22, 0.3, 0.5, 0.66, 0.75])\r\nsubR = np.random.choice([0.66, 0.75, 0.83, 0.9])\r\nmax_depth = np.random.choice([9, 11, 15, 25, 45, 100, -1])\r\ndart_rate = np.random.choice([0, 0, 0, 0.01, 0.03, 0.1])\r\nmax_bin = np.random.choice([63, 127, 255, 511])\r\nlambda_l1 = np.random.choice([0, 1., 10., 100.])\r\nlambda_l2 = np.random.choice([0, 1., 10., 100.])\r\n    \r\niterace = 10000\r\nif only_testing:\r\n    min_in_leaf = 25\r\n    iterace = 10\r\n    \r\nboost_type = 'gbdt'\r\nif dart_rate > 0:\r\n    boost_type = 'dart'\r\n                \r\nparams = {\r\n                        'task' : 'train',\r\n                        'boosting_type' : boost_type,\r\n                        'objective' : 'regression',\r\n                        'metric' : 'l2',\r\n                        'max_depth' : int(max_depth),\r\n                        'num_leaves' : int(list_count),\r\n                        'min_data_in_leaf' : int(min_in_leaf),\r\n                        'learning_rate' : lr,\r\n                        'feature_fraction' : subF,       \r\n                        'bagging_fraction' : subR,\r\n                        'bagging_freq': 1,\r\n                        'verbose' : 0,\r\n                        'nthread' : nthread,\r\n                        'drop_rate': dart_rate,\r\n                        'max_bin': max_bin,\r\n                        'lambda_l1' : lambda_l1,\r\n                        'lambda_l2' : lambda_l2\r\n            }\r\n            \r\nmodel = lg.train(\r\n                        params,\r\n                        (matrix_learn, target_learn),\r\n                        num_boost_round = iterace,\r\n                        valid_datas = (matrix_test, target_test),\r\n                        early_stopping_rounds = 50\r\n                    )\r\n"},"comment":{"url":"https://api.github.com/repos/Microsoft/LightGBM/issues/comments/269154936","html_url":"https://github.com/Microsoft/LightGBM/issues/126#issuecomment-269154936","issue_url":"https://api.github.com/repos/Microsoft/LightGBM/issues/126","id":269154936,"user":{"login":"guolinke","id":16040950,"avatar_url":"https://avatars.githubusercontent.com/u/16040950?v=3","gravatar_id":"","url":"https://api.github.com/users/guolinke","html_url":"https://github.com/guolinke","followers_url":"https://api.github.com/users/guolinke/followers","following_url":"https://api.github.com/users/guolinke/following{/other_user}","gists_url":"https://api.github.com/users/guolinke/gists{/gist_id}","starred_url":"https://api.github.com/users/guolinke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/guolinke/subscriptions","organizations_url":"https://api.github.com/users/guolinke/orgs","repos_url":"https://api.github.com/users/guolinke/repos","events_url":"https://api.github.com/users/guolinke/events{/privacy}","received_events_url":"https://api.github.com/users/guolinke/received_events","type":"User","site_admin":false},"created_at":"2016-12-26T03:31:59Z","updated_at":"2016-12-26T03:31:59Z","body":"@gugatr0n1c  I add an parameter ```max_drop``` to limit the number of dropped trees. This can help for the speed. But I am not sure how it will affect to the accuracy. \r\nAnd if ```max_drop <= 0```, it means not limit. \r\nI will close this issue for now, feel free to open an new issue if it cannot speed up or have much loss on accuracy. "}}
{"actor":{"display_login":"guolinke"}}