{"action":"created","issue":{"url":"https://api.github.com/repos/Microsoft/LightGBM/issues/126","repository_url":"https://api.github.com/repos/Microsoft/LightGBM","labels_url":"https://api.github.com/repos/Microsoft/LightGBM/issues/126/labels{/name}","comments_url":"https://api.github.com/repos/Microsoft/LightGBM/issues/126/comments","events_url":"https://api.github.com/repos/Microsoft/LightGBM/issues/126/events","html_url":"https://github.com/Microsoft/LightGBM/issues/126","id":196091853,"number":126,"title":"Dart - very poor accuracy","user":{"login":"gugatr0n1c","id":17110787,"avatar_url":"https://avatars.githubusercontent.com/u/17110787?v=3","gravatar_id":"","url":"https://api.github.com/users/gugatr0n1c","html_url":"https://github.com/gugatr0n1c","followers_url":"https://api.github.com/users/gugatr0n1c/followers","following_url":"https://api.github.com/users/gugatr0n1c/following{/other_user}","gists_url":"https://api.github.com/users/gugatr0n1c/gists{/gist_id}","starred_url":"https://api.github.com/users/gugatr0n1c/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gugatr0n1c/subscriptions","organizations_url":"https://api.github.com/users/gugatr0n1c/orgs","repos_url":"https://api.github.com/users/gugatr0n1c/repos","events_url":"https://api.github.com/users/gugatr0n1c/events{/privacy}","received_events_url":"https://api.github.com/users/gugatr0n1c/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":54,"created_at":"2016-12-16T16:00:43Z","updated_at":"2016-12-23T08:51:34Z","closed_at":null,"body":"When I use dart as a booster I always get very poor performance in term of l2 result for regression task.\r\nEven If I use small drop_rate = 0.01 or big like 0.3.\r\n\r\nWhen I use dart in xgboost on same dataset, with similar setting (same learning rate, similiar num_trees) dart alwasy give me boost for accuracy (small but always).\r\n\r\nBut here accuracy is poor badly, like there is a bug, not just dart is not suitable for my task.\r\n\r\nCan anyone confirm that dart is working for regression task in term of better accuracy?\r\n\r\nMy setting is as follows (part of the Python code for ramdom search of params):\r\n\r\nlr = np.random.choice([0.01, 0.005, 0.0025])\r\nlist_count = np.random.choice([250, 500, 750, 1000])\r\nmin_in_leaf = np.random.choice([25, 50, 100])\r\nsubF = np.random.choice([0.15, 0.22, 0.3, 0.5, 0.66, 0.75])\r\nsubR = np.random.choice([0.66, 0.75, 0.83, 0.9])\r\nmax_depth = np.random.choice([9, 11, 15, 25, 45, 100, -1])\r\ndart_rate = np.random.choice([0, 0, 0, 0.01, 0.03, 0.1])\r\nmax_bin = np.random.choice([63, 127, 255, 511])\r\nlambda_l1 = np.random.choice([0, 1., 10., 100.])\r\nlambda_l2 = np.random.choice([0, 1., 10., 100.])\r\n    \r\niterace = 10000\r\nif only_testing:\r\n    min_in_leaf = 25\r\n    iterace = 10\r\n    \r\nboost_type = 'gbdt'\r\nif dart_rate > 0:\r\n    boost_type = 'dart'\r\n                \r\nparams = {\r\n                        'task' : 'train',\r\n                        'boosting_type' : boost_type,\r\n                        'objective' : 'regression',\r\n                        'metric' : 'l2',\r\n                        'max_depth' : int(max_depth),\r\n                        'num_leaves' : int(list_count),\r\n                        'min_data_in_leaf' : int(min_in_leaf),\r\n                        'learning_rate' : lr,\r\n                        'feature_fraction' : subF,       \r\n                        'bagging_fraction' : subR,\r\n                        'bagging_freq': 1,\r\n                        'verbose' : 0,\r\n                        'nthread' : nthread,\r\n                        'drop_rate': dart_rate,\r\n                        'max_bin': max_bin,\r\n                        'lambda_l1' : lambda_l1,\r\n                        'lambda_l2' : lambda_l2\r\n            }\r\n            \r\nmodel = lg.train(\r\n                        params,\r\n                        (matrix_learn, target_learn),\r\n                        num_boost_round = iterace,\r\n                        valid_datas = (matrix_test, target_test),\r\n                        early_stopping_rounds = 50\r\n                    )\r\n"},"comment":{"url":"https://api.github.com/repos/Microsoft/LightGBM/issues/comments/268958513","html_url":"https://github.com/Microsoft/LightGBM/issues/126#issuecomment-268958513","issue_url":"https://api.github.com/repos/Microsoft/LightGBM/issues/126","id":268958513,"user":{"login":"gugatr0n1c","id":17110787,"avatar_url":"https://avatars.githubusercontent.com/u/17110787?v=3","gravatar_id":"","url":"https://api.github.com/users/gugatr0n1c","html_url":"https://github.com/gugatr0n1c","followers_url":"https://api.github.com/users/gugatr0n1c/followers","following_url":"https://api.github.com/users/gugatr0n1c/following{/other_user}","gists_url":"https://api.github.com/users/gugatr0n1c/gists{/gist_id}","starred_url":"https://api.github.com/users/gugatr0n1c/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gugatr0n1c/subscriptions","organizations_url":"https://api.github.com/users/gugatr0n1c/orgs","repos_url":"https://api.github.com/users/gugatr0n1c/repos","events_url":"https://api.github.com/users/gugatr0n1c/events{/privacy}","received_events_url":"https://api.github.com/users/gugatr0n1c/received_events","type":"User","site_admin":false},"created_at":"2016-12-23T08:51:34Z","updated_at":"2016-12-23T08:51:34Z","body":"@guolinke my setting is this:\r\nlearning_rate = 0.006\r\nup to 10k iterations with early_stopping = 50 (stopped around 5k-7k iters with 'gdbt')\r\ndrop_rate = skip_drop = 0.3\r\nusing xDART\r\n\r\nFirst like 100 iterations are fast, but I can see still that some iterations are very fast some slower\r\nBut after 1k iterations it is like 100ms vs 20s\r\n\r\n@marugari not detaily tested in xgboost, but never have this feeling that later in the training is difference so big..\r\n\r\n@anddelu this is very big offtopic here... send me your email to my twitter PM... @gugatr0n1c"}}
{"actor":{"display_login":"gugatr0n1c"}}