{"action":"created","issue":{"url":"https://api.github.com/repos/topepo/caret/issues/549","repository_url":"https://api.github.com/repos/topepo/caret","labels_url":"https://api.github.com/repos/topepo/caret/issues/549/labels{/name}","comments_url":"https://api.github.com/repos/topepo/caret/issues/549/comments","events_url":"https://api.github.com/repos/topepo/caret/issues/549/events","html_url":"https://github.com/topepo/caret/issues/549","id":194326254,"number":549,"title":"xgbTree with classProbs = TRUE is completely broken on current (Dec 7) github master branch","user":{"login":"hadjipantelis","id":4099384,"avatar_url":"https://avatars.githubusercontent.com/u/4099384?v=3","gravatar_id":"","url":"https://api.github.com/users/hadjipantelis","html_url":"https://github.com/hadjipantelis","followers_url":"https://api.github.com/users/hadjipantelis/followers","following_url":"https://api.github.com/users/hadjipantelis/following{/other_user}","gists_url":"https://api.github.com/users/hadjipantelis/gists{/gist_id}","starred_url":"https://api.github.com/users/hadjipantelis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hadjipantelis/subscriptions","organizations_url":"https://api.github.com/users/hadjipantelis/orgs","repos_url":"https://api.github.com/users/hadjipantelis/repos","events_url":"https://api.github.com/users/hadjipantelis/events{/privacy}","received_events_url":"https://api.github.com/users/hadjipantelis/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2016-12-08T12:41:48Z","updated_at":"2016-12-25T00:58:28Z","closed_at":"2016-12-25T00:58:27Z","body":"Hello,\r\n\r\nThis is directly related to [511](https://github.com/topepo/caret/issues/511) and the submit [d992fad](https://github.com/JackStat/caret/commit/d992fadc69dc7fdab3342cb1ab350ab95105ebdf) and the related commit [ed79d6d](https://github.com/topepo/caret/commit/ed79d6d41fe8cef0682a58527a6e3f85c5d3fbc0); it summarily kills `xgboost` when setting ` classProbs = TRUE`. All submits after that are broken for method `xgbTree`.\r\n\r\n``` R\r\nremove.packages('caret')\r\ndevtools::install_github('topepo/caret/pkg/caret')\r\nlibrary(caret)\r\n \r\nset.seed(42)  \r\ntrainData <- twoClassSim(500)    \r\ntestData <- twoClassSim(500)  \r\n\r\ntrCntlList <- trainControl(method = \"boot\", number = 6, classProbs = TRUE) \r\n\r\nset.seed(55)  \r\nxgbFitError1 <- train(  Class ~ ., data = trainData,  objective = \"binary:logistic\", \r\n                   method = \"xgbTree\", tuneLength = 2, trControl =  trCntlList)\r\nset.seed(55)  \r\nxgbFitError2 <- train( Class ~ ., data = trainData, objective = \"binary:logitraw\",\r\n                   method = \"xgbTree\", tuneLength = 2, trControl =  trCntlList) \r\n\r\ntrCntlList <- trainControl(method = \"boot\", number = 6, classProbs = FALSE) \r\n\r\nset.seed(55)  \r\nxgbFitOK1<- train(  Class ~ ., data = trainData,  objective = \"binary:logistic\", \r\n                   method = \"xgbTree\", tuneLength = 2, trControl =  trCntlList)\r\nset.seed(55)  \r\nxgbFitOK2 <- train( Class ~ ., data = trainData, objective = \"binary:logitraw\",\r\n                   method = \"xgbTree\", tuneLength = 2, trControl =  trCntlList) \r\n```\r\n\r\nThe error occurs within the `foreach` loop of `nominalTrainWorkflow`. In particular around Line # ~203 during the call to the `probFunction`. `probFunction` errs because the argument `modelFit$param$objective` is NULL. To that respect, `modelFit` does not have a `param` field to begin with.  That is somewhat surprising but that it can be explained by the fact that `createModel` never appended a `param` field. This is expected as `createModel` is fully agnostic about the objective function used. It should have a point where it creates a `params` field to the `modelFit` returned.\r\n\r\nI can tell that this error was introduced with `ed79d6d` as I back-tracked it; the code runs \"fine\" at the `5cc19a5` commit. (well, not totally fine, the probabilities are wrong as show in  [511](https://github.com/topepo/caret/issues/511)  but at least it works when `objective = \"binary:logistic\"`). My `sessionInfo()` is as follows:\r\n  \r\n``` R\r\nR version 3.3.1 (2016-06-21)\r\nPlatform: x86_64-redhat-linux-gnu (64-bit)\r\nRunning under: CentOS Linux 7 (Core)\r\n\r\nlocale:\r\n [1] LC_CTYPE=en_GB.UTF-8       LC_NUMERIC=C               LC_TIME=en_GB.UTF-8       \r\n [4] LC_COLLATE=en_GB.UTF-8     LC_MONETARY=en_GB.UTF-8    LC_MESSAGES=en_GB.UTF-8   \r\n [7] LC_PAPER=en_GB.UTF-8       LC_NAME=C                  LC_ADDRESS=C              \r\n[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C       \r\n\r\nattached base packages:\r\n[1] stats     graphics  grDevices utils     datasets  methods   base     \r\n\r\nother attached packages:\r\n[1] caret_6.0-73    plyr_1.8.4      xgboost_0.4-4   ggplot2_2.2.0   lattice_0.20-33\r\n\r\nloaded via a namespace (and not attached):\r\n [1] reshape2_1.4.2     splines_3.3.1      colorspace_1.2-6   testthat_1.0.2    \r\n [5] stats4_3.3.1       mgcv_1.8-12        chron_2.3-47       ModelMetrics_1.1.0\r\n [9] e1071_1.6-7        nloptr_1.0.4       withr_1.0.2        foreach_1.4.3     \r\n[13] stringr_1.1.0      MatrixModels_0.4-1 munsell_0.4.3      gtable_0.2.0      \r\n[17] devtools_1.12.0    codetools_0.2-14   memoise_1.0.0      SparseM_1.72      \r\n[21] quantreg_5.29      pbkrtest_0.4-6     parallel_3.3.1     curl_1.2          \r\n[25] class_7.3-14       Rcpp_0.12.8        scales_0.4.1       lme4_1.1-12       \r\n[29] digest_0.6.10      stringi_1.1.1      grid_3.3.1         tools_3.3.1       \r\n[33] magrittr_1.5       lazyeval_0.2.0     tibble_1.2         crayon_1.3.2      \r\n[37] car_2.1-4          MASS_7.3-45        Matrix_1.2-6       data.table_1.9.6  \r\n[41] assertthat_0.1     minqa_1.2.4        roxygen2_5.0.1     httr_1.2.1        \r\n[45] iterators_1.0.8    R6_2.2.0           nnet_7.3-12        nlme_3.1-128      \r\n[49] compiler_3.3.1     git2r_0.15.0       \r\n```\r\nImmediate work-around: I believe that adding the line `modelFit$params <- list(...)` within `createModel` will be adequate. Otherwise one would have to pull the objective function used out of the \"xgb.Booster\" output of `xgb.train` and that seems to be complicated. \r\n\r\nI will create a relevant pull request.\r\n\r\nAll the best,\r\nPantelis\r\n"},"comment":{"url":"https://api.github.com/repos/topepo/caret/issues/comments/269106084","html_url":"https://github.com/topepo/caret/issues/549#issuecomment-269106084","issue_url":"https://api.github.com/repos/topepo/caret/issues/549","id":269106084,"user":{"login":"topepo","id":5731043,"avatar_url":"https://avatars.githubusercontent.com/u/5731043?v=3","gravatar_id":"","url":"https://api.github.com/users/topepo","html_url":"https://github.com/topepo","followers_url":"https://api.github.com/users/topepo/followers","following_url":"https://api.github.com/users/topepo/following{/other_user}","gists_url":"https://api.github.com/users/topepo/gists{/gist_id}","starred_url":"https://api.github.com/users/topepo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/topepo/subscriptions","organizations_url":"https://api.github.com/users/topepo/orgs","repos_url":"https://api.github.com/users/topepo/repos","events_url":"https://api.github.com/users/topepo/events{/privacy}","received_events_url":"https://api.github.com/users/topepo/received_events","type":"User","site_admin":false},"created_at":"2016-12-25T00:58:27Z","updated_at":"2016-12-25T00:58:27Z","body":"This all works now, so I'll close the issue"}}
{"actor":{"display_login":"topepo"}}