{"action":"opened","issue":{"url":"https://api.github.com/repos/mlr-org/mlr/issues/740","repository_url":"https://api.github.com/repos/mlr-org/mlr","labels_url":"https://api.github.com/repos/mlr-org/mlr/issues/740/labels{/name}","comments_url":"https://api.github.com/repos/mlr-org/mlr/issues/740/comments","events_url":"https://api.github.com/repos/mlr-org/mlr/issues/740/events","html_url":"https://github.com/mlr-org/mlr/issues/740","id":134321323,"number":740,"title":"subsampling ensemble variance estimator","user":{"login":"zmjones","id":1114501,"avatar_url":"https://avatars.githubusercontent.com/u/1114501?v=3","gravatar_id":"","url":"https://api.github.com/users/zmjones","html_url":"https://github.com/zmjones","followers_url":"https://api.github.com/users/zmjones/followers","following_url":"https://api.github.com/users/zmjones/following{/other_user}","gists_url":"https://api.github.com/users/zmjones/gists{/gist_id}","starred_url":"https://api.github.com/users/zmjones/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zmjones/subscriptions","organizations_url":"https://api.github.com/users/zmjones/orgs","repos_url":"https://api.github.com/users/zmjones/repos","events_url":"https://api.github.com/users/zmjones/events{/privacy}","received_events_url":"https://api.github.com/users/zmjones/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-02-17T16:00:25Z","updated_at":"2016-02-17T16:00:25Z","closed_at":null,"body":"in [this paper](http://arxiv.org/abs/1404.6473) the authors derive a random forest algorithm where the predictions can be viewed as U-statistics, which have an asymptotic normal distribution, and they give estimators for the variance parameter of this limiting distribution.\r\n\r\nthe algorithm is basically a random forest where the data for each tree are subsampled but this is nested within a sample of fixed observations which are in each subsample. i.e., select a fixed set of observations, then, for some number of iterations, take subsamples which contain this fixed set, and fit trees to each of these. then repeat the entire process for another fixed set. \r\n\r\nthere are a couple of ways to implement this i think. one would be to just create a new learner that uses either `ctree`, `rpart`, or some other tree algorithm, and does the above.\r\n\r\na second option would be to use the `BaggingWrapper`. This would require the addition of 2 parameters and would make the code a bit more complex but wouldn't be too bad. it would allow people to use other base learners for which the variance estimator's properties probably wouldn't hold. i guess we could have a warning for that though.\r\n\r\nany preferences for how to do this? they also have a hypothesis test for feature importance which i would like to implement as well. i suppose that is a point in favor of implementing it as its own learner, unless i am mistaken.\r\n\r\nthoughts?"}}