{"action":"created","issue":{"url":"https://api.github.com/repos/rasbt/mlxtend/issues/10","repository_url":"https://api.github.com/repos/rasbt/mlxtend","labels_url":"https://api.github.com/repos/rasbt/mlxtend/issues/10/labels{/name}","comments_url":"https://api.github.com/repos/rasbt/mlxtend/issues/10/comments","events_url":"https://api.github.com/repos/rasbt/mlxtend/issues/10/events","html_url":"https://github.com/rasbt/mlxtend/issues/10","id":129033937,"number":10,"title":"EnsembleRegressor","user":{"login":"rasbt","id":5618407,"avatar_url":"https://avatars.githubusercontent.com/u/5618407?v=3","gravatar_id":"","url":"https://api.github.com/users/rasbt","html_url":"https://github.com/rasbt","followers_url":"https://api.github.com/users/rasbt/followers","following_url":"https://api.github.com/users/rasbt/following{/other_user}","gists_url":"https://api.github.com/users/rasbt/gists{/gist_id}","starred_url":"https://api.github.com/users/rasbt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rasbt/subscriptions","organizations_url":"https://api.github.com/users/rasbt/orgs","repos_url":"https://api.github.com/users/rasbt/repos","events_url":"https://api.github.com/users/rasbt/events{/privacy}","received_events_url":"https://api.github.com/users/rasbt/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/rasbt/mlxtend/labels/new%20feature","name":"new feature","color":"009800"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-01-27T05:20:47Z","updated_at":"2016-03-04T04:14:48Z","closed_at":null,"body":"I opened this issue as a follow up on a discussion with \r\n\r\n@nikhilRP:\r\n\r\n> Can we have some thing like \"VotingClassifier\" for regression as well.\r\n\r\n I believe the same concept would also work for regression from a technical perspective. However, I am not sure how \"good\" or \"practical\" this would be. I think one would have to be especially careful with the averaging if there is a high gap between the predicted targets, e.g., if we have 3 regressors with predicted targets of e.g., 1.1, 1.3, 1031.4, averaging could yield some strange behavior. Maybe it would be worthwhile to include some optional outlier detection. Here are some examples\r\n\r\n1. for < 10 regressor, use Dixon's Q test to remove lower-end outliers \r\n2. for >= 10 regressor, maybe exclude everything what's outside the IQR (if the predicted scores are represented as a \"box plot\")\r\n3. as an alternative to 1. and 2., use sequential feature selection to select a k-sized subset of regressors so that the performance of the average fit is \"optimal.\"\r\n\r\nI believe that an implementation would be pretty straight-forward from a technical perspective using the Ensemble Classifier as a template.\r\n\r\nIn addition, an interesting idea by @nikhilRP was \r\n\r\n > [to] use r-squared scores (or any other measure) to determine weights of each predicted targets.\r\n\r\nHonestly, I haven't really thought about it, yet, and I suggest that we brainstorm a bit more. However, **in order to get started** we could already prepare a scaffold `EnsembleRegressor` that takes scikit-learns regressors as input estimators and outputs the average fits (similar to the `EnsembleClassifier`'s `predict_proba`). :)\r\n\r\n**Note** \r\n\r\nThe `mlxtend` is currently being slightly overhauled (see branch [https://github.com/rasbt/mlxtend/tree/03](https://github.com/rasbt/mlxtend/tree/03)). \r\n\r\nThe `mlxtend.classifier.EnsembleClassifier` from the `mlxtend.classifier.ensemble.py` module has been moved to the `mlxtend.classifier.ensemble_vote.py` module where it is now `mlxtend.classifier.EnsembleVoteClassifier`. The reason is that the new \"description\" is more specific and would help distinguishing it from a future`mlxtend.classifier.EnsembleStackingClassifier` etc.\r\nFurthermore, the `regression` subpackage has been split into 2 new subpackages `regressor` and `regression_utils` -- The former contains classes (estimators) for fitting regression models, and the latter contains auxiliary functions such as visualization tools etc.\r\n\r\nI am preparing the new documentation and website right now, and the new release (v. 0.3) will probably ready next week and ready to be merged into the current master branch.\r\n\r\nThus, an `EnsembleRegressor` would best fit into the `mlxtend.regressor` subpackage. "},"comment":{"url":"https://api.github.com/repos/rasbt/mlxtend/issues/comments/192091529","html_url":"https://github.com/rasbt/mlxtend/issues/10#issuecomment-192091529","issue_url":"https://api.github.com/repos/rasbt/mlxtend/issues/10","id":192091529,"user":{"login":"rasbt","id":5618407,"avatar_url":"https://avatars.githubusercontent.com/u/5618407?v=3","gravatar_id":"","url":"https://api.github.com/users/rasbt","html_url":"https://github.com/rasbt","followers_url":"https://api.github.com/users/rasbt/followers","following_url":"https://api.github.com/users/rasbt/following{/other_user}","gists_url":"https://api.github.com/users/rasbt/gists{/gist_id}","starred_url":"https://api.github.com/users/rasbt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rasbt/subscriptions","organizations_url":"https://api.github.com/users/rasbt/orgs","repos_url":"https://api.github.com/users/rasbt/repos","events_url":"https://api.github.com/users/rasbt/events{/privacy}","received_events_url":"https://api.github.com/users/rasbt/received_events","type":"User","site_admin":false},"created_at":"2016-03-04T04:14:48Z","updated_at":"2016-03-04T04:14:48Z","body":"No worries, @nikhilRP (sorry, but I am actually glad that it took some time because it's really busy around here :P). The paper looks cool, I haven't read it completely, yet, but their proposed idea only makes sense; it would be great to see this in action :). Btw. I just setup such a basic `StackingRegressor` that we could use as template (#12) . I'd suggest not to generalize the code, yet, but just use it a template that can be extended to the FWLS (makes debugging easier for the first implementation). "}}