{"action":"created","issue":{"url":"https://api.github.com/repos/mlr-org/mlrMBO/issues/439","repository_url":"https://api.github.com/repos/mlr-org/mlrMBO","labels_url":"https://api.github.com/repos/mlr-org/mlrMBO/issues/439/labels{/name}","comments_url":"https://api.github.com/repos/mlr-org/mlrMBO/issues/439/comments","events_url":"https://api.github.com/repos/mlr-org/mlrMBO/issues/439/events","html_url":"https://github.com/mlr-org/mlrMBO/issues/439","id":356559199,"node_id":"MDU6SXNzdWUzNTY1NTkxOTk=","number":439,"title":"Handling if init.design generates constant performances","user":{"login":"pfistfl","id":7313671,"node_id":"MDQ6VXNlcjczMTM2NzE=","avatar_url":"https://avatars1.githubusercontent.com/u/7313671?v=4","gravatar_id":"","url":"https://api.github.com/users/pfistfl","html_url":"https://github.com/pfistfl","followers_url":"https://api.github.com/users/pfistfl/followers","following_url":"https://api.github.com/users/pfistfl/following{/other_user}","gists_url":"https://api.github.com/users/pfistfl/gists{/gist_id}","starred_url":"https://api.github.com/users/pfistfl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pfistfl/subscriptions","organizations_url":"https://api.github.com/users/pfistfl/orgs","repos_url":"https://api.github.com/users/pfistfl/repos","events_url":"https://api.github.com/users/pfistfl/events{/privacy}","received_events_url":"https://api.github.com/users/pfistfl/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-09-03T16:24:45Z","updated_at":"2018-11-19T16:30:51Z","closed_at":null,"author_association":"NONE","body":"As per Bernd's request, here is a reproducible example:\r\n\r\nProblem: What happens if we get constant values for the performance measure in the initial design:\r\n- Maybe set `on.learner.error` to `warn` in general.\r\n\r\n- Bernd pointed me to (https://mlr-org.github.io/mlrMBO/reference/error_handling.html)\r\n  Maybe this can be linked somewhere more prominent, as I did not find it when I was searching for a\r\n  solution.\r\n\r\n```r\r\nlibrary(mlr)\r\nlibrary(OpenML)\r\n\r\ntsk = getOMLTask(3903)\r\nlrn = setPredictType(makeLearner(\"classif.rpart\"), \"prob\")\r\n# Create mlr task with estimation procedure and evaluation measure\r\nz = OpenML::convertOMLTaskToMlr(tsk, measures = auc)\r\nset.seed(201)\r\n\r\nlrn.ps =  makeParamSet(\r\n  makeNumericParam(\"cp\", lower = 0, upper = 1, default = 0.01),\r\n  makeIntegerParam(\"maxdepth\", lower = 1, upper = 30, default = 30),\r\n  makeIntegerParam(\"minbucket\", lower = 1, upper = 60, default = 1),\r\n  makeIntegerParam(\"minsplit\", lower = 1, upper = 60, default = 20)\r\n)\r\n\r\ninner.rdesc = makeResampleDesc(\"CV\", iters = 5)\r\ninit.design = data.frame(cp = c(0.729248355375603, 0.892083862796426, 0.441975111840293,  0.817802196601406,\r\n                                0.564511572476476, 0.463207047665492, 0.554985330672935,  0.443818159168586),\r\n                             maxdepth = c(27L, 18L, 18L, 6L, 17L, 17L, 1L, 13L),\r\n                             minbucket = c(30L, 30L, 26L, 49L, 20L, 20L, 56L, 27L),\r\n                             minsplit = c(18L, 60L, 20L, 39L, 34L, 52L, 14L, 29L))\r\ntune.ctrl = makeTuneControlMBO(same.resampling.instance = TRUE, budget = 10, mbo.design = init.design)\r\nlrn.tune = makeTuneWrapper(lrn, inner.rdesc, mlr::auc, par.set = lrn.ps, tune.ctrl)\r\n# Create OMLRun\r\nbmr = mlr::benchmark(lrn.tune, z$mlr.task, z$mlr.rin, measures = z$mlr.measures)\r\n```"},"comment":{"url":"https://api.github.com/repos/mlr-org/mlrMBO/issues/comments/439955332","html_url":"https://github.com/mlr-org/mlrMBO/issues/439#issuecomment-439955332","issue_url":"https://api.github.com/repos/mlr-org/mlrMBO/issues/439","id":439955332,"node_id":"MDEyOklzc3VlQ29tbWVudDQzOTk1NTMzMg==","user":{"login":"ja-thomas","id":7561944,"node_id":"MDQ6VXNlcjc1NjE5NDQ=","avatar_url":"https://avatars1.githubusercontent.com/u/7561944?v=4","gravatar_id":"","url":"https://api.github.com/users/ja-thomas","html_url":"https://github.com/ja-thomas","followers_url":"https://api.github.com/users/ja-thomas/followers","following_url":"https://api.github.com/users/ja-thomas/following{/other_user}","gists_url":"https://api.github.com/users/ja-thomas/gists{/gist_id}","starred_url":"https://api.github.com/users/ja-thomas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ja-thomas/subscriptions","organizations_url":"https://api.github.com/users/ja-thomas/orgs","repos_url":"https://api.github.com/users/ja-thomas/repos","events_url":"https://api.github.com/users/ja-thomas/events{/privacy}","received_events_url":"https://api.github.com/users/ja-thomas/received_events","type":"User","site_admin":false},"created_at":"2018-11-19T16:30:51Z","updated_at":"2018-11-19T16:30:51Z","author_association":"MEMBER","body":"```\r\nmbo.ctrl = makeMBOControl(on.surrogate.error = \"warn\")\r\nctrl = makeTuneControlMBO(budget = 100L, mbo.control = mbo.ctrl)\r\n```\r\nwould be the correct approach.\r\n\r\nThe tuneControlMBO stuff is really confusing unfortunately since we have nested control object that have very similar names...\r\n\r\n"}}
{"actor":{"display_login":"ja-thomas"}}