{"action":"created","issue":{"url":"https://api.github.com/repos/dmlc/xgboost/issues/1756","repository_url":"https://api.github.com/repos/dmlc/xgboost","labels_url":"https://api.github.com/repos/dmlc/xgboost/issues/1756/labels{/name}","comments_url":"https://api.github.com/repos/dmlc/xgboost/issues/1756/comments","events_url":"https://api.github.com/repos/dmlc/xgboost/issues/1756/events","html_url":"https://github.com/dmlc/xgboost/issues/1756","id":188292552,"number":1756,"title":"Separate predictions are 4x slower than bulk predictions","user":{"login":"goupdown","id":23242448,"avatar_url":"https://avatars.githubusercontent.com/u/23242448?v=3","gravatar_id":"","url":"https://api.github.com/users/goupdown","html_url":"https://github.com/goupdown","followers_url":"https://api.github.com/users/goupdown/followers","following_url":"https://api.github.com/users/goupdown/following{/other_user}","gists_url":"https://api.github.com/users/goupdown/gists{/gist_id}","starred_url":"https://api.github.com/users/goupdown/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/goupdown/subscriptions","organizations_url":"https://api.github.com/users/goupdown/orgs","repos_url":"https://api.github.com/users/goupdown/repos","events_url":"https://api.github.com/users/goupdown/events{/privacy}","received_events_url":"https://api.github.com/users/goupdown/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2016-11-09T16:35:06Z","updated_at":"2016-11-11T05:16:09Z","closed_at":null,"body":"I have data containing 883k instances. Also I have ready XGBooster. I predict all the instances as one matrix and all the instances one by one with separate prediction call. For separate prediction call I need to create separate matrix which contains one instance only. Here is my code:\r\n\r\n```\r\nint main() {\r\n\tBoosterHandle h_booster;\r\n\tXGBoosterCreate(0, 0, &h_booster);\r\n\tXGBoosterLoadModel(h_booster, \"C:/ro/server33/ro_cluster/PrXgb.save/f.sit1.2.obj\");\r\n\tXGBoosterSetParam(h_booster, \"nthread\", \"1\");\r\n\r\n\tconst int instLen = 22;\r\n\tint dataLen = 883567;\r\n\tDMatrixHandle h_test;\r\n\tXGDMatrixCreateFromFile(\"C:/ro/server33/ro_cluster/cache_data/st3.sit1.h2\", 0, &h_test);\r\n\r\n\tLONG st = time();\r\n\tbst_ulong out_len;\r\n\tconst float *out;\r\n\tXGBoosterPredict((void*)h_booster, h_test, 0, 0, &out_len, &out);\r\n\tprintf(\"Bulk predict: %dms\\n\", time() - st);\r\n\r\n\tLONG tp = 0;\r\n\tst = time();\r\n\tfor (int j = 0; j < dataLen; j++) {\r\n\t\t// Just take first line of the data\r\n\t\tfloat inst[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 2, 2, 0, 0, 0, 0, 0, };\r\n\r\n\t\tDMatrixHandle hm;\r\n\t\tXGDMatrixCreateFromMat((float*)inst, 1, instLen, -1, &hm);\r\n\t\tLONG stp = time();\r\n\t\tXGBoosterPredict((void*)h_booster, hm, 0, 0, &out_len, &out);\r\n\t\ttp += time()-stp;\r\n\t}\r\n\tprintf(\"1 by 1 predict total %dms, pure %dms\\n\", time() - st, tp);\r\n\r\n    return 0;\r\n}\r\n```\r\n\r\nOutput:\r\n```\r\nC:\\>XGBoostTest.exe\r\n[23:19:25] 883566x23 matrix with 19438452 entries loaded from C:/ro/server33/ro_cluster/cache_data/st3.sit1.h2\r\nBulk predict: 550ms\r\n1 by 1 predict total 2373ms, pure 945ms\r\n\r\n```\r\nWhat we see is that one by one prediction is 2x slower. Also matrix creation gives 2x overhead. It's 4x in total. Multi-threading is not an issue here because of nthread=1.\r\n\r\nIf matrix creation is so slow isn't it better just to use float[] representation of the instance?\r\nWhy is pure prediction is 2x slower? How can existence of other instances in the matrix improve the performance of the prediction of one particular instance?\r\n\r\nIs it possible to have fixes for the above problems in the visible future?"},"comment":{"url":"https://api.github.com/repos/dmlc/xgboost/issues/comments/259883066","html_url":"https://github.com/dmlc/xgboost/issues/1756#issuecomment-259883066","issue_url":"https://api.github.com/repos/dmlc/xgboost/issues/1756","id":259883066,"user":{"login":"qingpingswiftsword","id":23352171,"avatar_url":"https://avatars.githubusercontent.com/u/23352171?v=3","gravatar_id":"","url":"https://api.github.com/users/qingpingswiftsword","html_url":"https://github.com/qingpingswiftsword","followers_url":"https://api.github.com/users/qingpingswiftsword/followers","following_url":"https://api.github.com/users/qingpingswiftsword/following{/other_user}","gists_url":"https://api.github.com/users/qingpingswiftsword/gists{/gist_id}","starred_url":"https://api.github.com/users/qingpingswiftsword/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/qingpingswiftsword/subscriptions","organizations_url":"https://api.github.com/users/qingpingswiftsword/orgs","repos_url":"https://api.github.com/users/qingpingswiftsword/repos","events_url":"https://api.github.com/users/qingpingswiftsword/events{/privacy}","received_events_url":"https://api.github.com/users/qingpingswiftsword/received_events","type":"User","site_admin":false},"created_at":"2016-11-11T05:16:09Z","updated_at":"2016-11-11T05:16:09Z","body":"Is there anyone know that how xgoost do prediction in bulk mode? I read http://scikit-learn.org/stable/modules/computational_performance.html , for logistic regression or SVM, it is easy to think the prediciton in bulk in matrix computation way. It is interesting to know how to do bulk prediction in tree mode."}}
{"actor":{"display_login":"qingpingswiftsword"}}