{"action":"created","issue":{"url":"https://api.github.com/repos/statsmodels/statsmodels/issues/3284","repository_url":"https://api.github.com/repos/statsmodels/statsmodels","labels_url":"https://api.github.com/repos/statsmodels/statsmodels/issues/3284/labels{/name}","comments_url":"https://api.github.com/repos/statsmodels/statsmodels/issues/3284/comments","events_url":"https://api.github.com/repos/statsmodels/statsmodels/issues/3284/events","html_url":"https://github.com/statsmodels/statsmodels/issues/3284","id":190543230,"number":3284,"title":"ENH:   wrapper/prefix mixin class for robust estimation using weights","user":{"login":"josef-pkt","id":440735,"avatar_url":"https://avatars.githubusercontent.com/u/440735?v=3","gravatar_id":"","url":"https://api.github.com/users/josef-pkt","html_url":"https://github.com/josef-pkt","followers_url":"https://api.github.com/users/josef-pkt/followers","following_url":"https://api.github.com/users/josef-pkt/following{/other_user}","gists_url":"https://api.github.com/users/josef-pkt/gists{/gist_id}","starred_url":"https://api.github.com/users/josef-pkt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/josef-pkt/subscriptions","organizations_url":"https://api.github.com/users/josef-pkt/orgs","repos_url":"https://api.github.com/users/josef-pkt/repos","events_url":"https://api.github.com/users/josef-pkt/events{/privacy}","received_events_url":"https://api.github.com/users/josef-pkt/received_events","type":"User","site_admin":false},"labels":[{"id":163856,"url":"https://api.github.com/repos/statsmodels/statsmodels/labels/comp-discrete","name":"comp-discrete","color":"ededed","default":false},{"id":163677,"url":"https://api.github.com/repos/statsmodels/statsmodels/labels/comp-genmod","name":"comp-genmod","color":"ededed","default":false},{"id":163966,"url":"https://api.github.com/repos/statsmodels/statsmodels/labels/comp-robust","name":"comp-robust","color":"ededed","default":false},{"id":142961,"url":"https://api.github.com/repos/statsmodels/statsmodels/labels/type-enh","name":"type-enh","color":"99ffff","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-11-20T06:53:00Z","updated_at":"2016-12-16T00:09:07Z","closed_at":null,"body":"This is an idea for a generic class that puts outlier robust estimation outside the standard models, or possibly follow the fit_xxx pattern.\r\n\r\nrequires that models handle frequency weights (or generic iweights), or that we can overwrite loglike, score and hessian with a weighted version (which should work for parameters and generic sandwich cov_type but maybe not for all extra results). \r\n\r\nRLM uses WLS at it's core for irls. Similar, multivariate regression would need a multivariate WLS or weighted mean in the core mean function calculation.\r\nIn both cases robustness is implemented through a weighted regression/estimation.\r\n\r\nMallow's type weights for GLM for example directly put weights on the (likelihood and) score function. \r\n\r\nThis would work generically for most models, assuming we can calculate a robustness weight function. We could put the weight iteration into a wrapper class or a special `fit_robust` method and we don't need to rewrite every model to have a robust version.\r\nThe weight function could take `resid` as argument and whatever else is needed as keywords. \r\n\r\npossible problems: \r\n\r\nMessy scale separation: we might need robust estimators for scale/var, or more generally for nuisance or auxilliary parameters. Those might not be nicely exposed to being overwritten.\r\n\r\nHow much model or distribution specific information do we need in order to calculate the robustness weights, e.g. cutoffs or shape parameters for the robust weights which depend on the reference distribution?\r\n\r\nGLM would be a good case to try this out because it already has frequency weights.\r\n\r\nThis is a bit similar to maximum trimmed likelihood, which is even easier because we don't need the weights, we just drop observations. MTLE is overall messier because we do the initial global optimum search.\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/statsmodels/statsmodels/issues/comments/267482045","html_url":"https://github.com/statsmodels/statsmodels/issues/3284#issuecomment-267482045","issue_url":"https://api.github.com/repos/statsmodels/statsmodels/issues/3284","id":267482045,"user":{"login":"josef-pkt","id":440735,"avatar_url":"https://avatars.githubusercontent.com/u/440735?v=3","gravatar_id":"","url":"https://api.github.com/users/josef-pkt","html_url":"https://github.com/josef-pkt","followers_url":"https://api.github.com/users/josef-pkt/followers","following_url":"https://api.github.com/users/josef-pkt/following{/other_user}","gists_url":"https://api.github.com/users/josef-pkt/gists{/gist_id}","starred_url":"https://api.github.com/users/josef-pkt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/josef-pkt/subscriptions","organizations_url":"https://api.github.com/users/josef-pkt/orgs","repos_url":"https://api.github.com/users/josef-pkt/repos","events_url":"https://api.github.com/users/josef-pkt/events{/privacy}","received_events_url":"https://api.github.com/users/josef-pkt/received_events","type":"User","site_admin":false},"created_at":"2016-12-16T00:09:07Z","updated_at":"2016-12-16T00:09:07Z","body":"I'm looking at MNLogit and NegativeBinomial.\r\nThe main problem for adding weights that cannot be done from the outside is the Hessian. Hessian has aggregation inside and in the case of NegativeBinomial has the extra shape term. MNLogit has a tricky 4-dimensional reshape to get the Hessian.\r\nDiscrete_models also don't have the `hessian_factor` as in GLM, so either hessian_factors need to be added or the weights need to be put inside the current hessian methods.\r\n\r\nscore/score_obs and loglike/loglike_obs only differ in aggregation, but code is still duplicated."}}
{"actor":{"display_login":"josef-pkt"}}