{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/6599","repository_url":"https://api.github.com/repos/scikit-learn/scikit-learn","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/6599/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/6599/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/6599/events","html_url":"https://github.com/scikit-learn/scikit-learn/issues/6599","id":143994588,"number":6599,"title":"Implement estimator tags","user":{"login":"amueller","id":449558,"avatar_url":"https://avatars.githubusercontent.com/u/449558?v=3","gravatar_id":"","url":"https://api.github.com/users/amueller","html_url":"https://github.com/amueller","followers_url":"https://api.github.com/users/amueller/followers","following_url":"https://api.github.com/users/amueller/following{/other_user}","gists_url":"https://api.github.com/users/amueller/gists{/gist_id}","starred_url":"https://api.github.com/users/amueller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amueller/subscriptions","organizations_url":"https://api.github.com/users/amueller/orgs","repos_url":"https://api.github.com/users/amueller/repos","events_url":"https://api.github.com/users/amueller/events{/privacy}","received_events_url":"https://api.github.com/users/amueller/received_events","type":"User","site_admin":false},"labels":[{"id":68750477,"url":"https://api.github.com/repos/scikit-learn/scikit-learn/labels/API","name":"API","color":"bfd4f2","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/milestones/21","html_url":"https://github.com/scikit-learn/scikit-learn/milestone/21","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/milestones/21/labels","id":1970916,"number":21,"title":"0.19","description":"","creator":{"login":"amueller","id":449558,"avatar_url":"https://avatars.githubusercontent.com/u/449558?v=3","gravatar_id":"","url":"https://api.github.com/users/amueller","html_url":"https://github.com/amueller","followers_url":"https://api.github.com/users/amueller/followers","following_url":"https://api.github.com/users/amueller/following{/other_user}","gists_url":"https://api.github.com/users/amueller/gists{/gist_id}","starred_url":"https://api.github.com/users/amueller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amueller/subscriptions","organizations_url":"https://api.github.com/users/amueller/orgs","repos_url":"https://api.github.com/users/amueller/repos","events_url":"https://api.github.com/users/amueller/events{/privacy}","received_events_url":"https://api.github.com/users/amueller/received_events","type":"User","site_admin":false},"open_issues":183,"closed_issues":46,"state":"open","created_at":"2016-08-29T22:34:33Z","updated_at":"2016-12-12T10:57:43Z","due_on":"2017-04-01T00:00:00Z","closed_at":null},"comments":44,"created_at":"2016-03-28T14:51:08Z","updated_at":"2016-12-12T16:58:53Z","closed_at":null,"body":"Currently the common test hard-code many things, like support for multi-output, or requiring positive input, or not allowing specific kinds of predictions.\r\nThat's bad design, but also a big problem for 3rd party packages that need to add to the conditions in the common tests (you need to add to the hard-coded list of estimators with a certain property).\r\nSee https://github.com/scikit-learn-contrib/py-earth/issues/96 for an example.\r\n\r\nCurrently we have a very poor mechanism for distinguishing classifiers and regressors for similar purposes (but also to use when deciding the default cross-validation strategy), the  `estimator_type` attribute. That allows only a single tag (classifier, transformer, regressor).\r\n\r\nI think we should deprecate `estimator_type` and instead add a more flexible `estimator_properties` dictionary.\r\nThis will allow us to programmatically encode assumptions of the algorithms (like vectorizers taking non-numeric data or NB taking non-negative data) as well as clean up our act with the tests.\r\nThe people wanting to add to scikit-learn-contrib and auto-sklearn-like settings (tpot) will appreciate that ;)\r\n\r\nList of tags that I am coming up with\r\n\r\n- [ ] supports sparse data\r\n- [ ] positive data only\r\n- [ ] supports missing data\r\n- [ ] semi-supervised\r\n- [ ] multi-output only\r\n- [ ] multi-label support\r\n- [ ] multi-output regression\r\n- [ ] multi-label multi-output\r\n- [ ] 1d input only\r\n- [ ] multi-class support (or maybe \"no multi-class support\"?)\r\n- [ ] needs fitting (or maybe \"stateless\"? though the GP doesn't need fitting but is not stateless)\r\n- [ ] input dtype / dtype conversions?\r\n- [ ] sparse matrix formats / conversions\r\n- [ ] deterministic?\r\n- [ ] label transformation (not for data)\r\n- [ ] special input format? like for CountVectorizer and DictVectorizer? Or maybe we want a field \"supported inputs\" that lists ndarray, sparse formats, list, strings, dicts?\r\n- [ ] required parameters ?\r\n- [ ] integer input / categorical input supported?\r\n\r\ncc @GaelVaroquaux @ogrisel @mblondel @jnothman @mfeurer @rhiever\r\n"},"comment":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/comments/266486214","html_url":"https://github.com/scikit-learn/scikit-learn/issues/6599#issuecomment-266486214","issue_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/6599","id":266486214,"user":{"login":"amueller","id":449558,"avatar_url":"https://avatars.githubusercontent.com/u/449558?v=3","gravatar_id":"","url":"https://api.github.com/users/amueller","html_url":"https://github.com/amueller","followers_url":"https://api.github.com/users/amueller/followers","following_url":"https://api.github.com/users/amueller/following{/other_user}","gists_url":"https://api.github.com/users/amueller/gists{/gist_id}","starred_url":"https://api.github.com/users/amueller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amueller/subscriptions","organizations_url":"https://api.github.com/users/amueller/orgs","repos_url":"https://api.github.com/users/amueller/repos","events_url":"https://api.github.com/users/amueller/events{/privacy}","received_events_url":"https://api.github.com/users/amueller/received_events","type":"User","site_admin":false},"created_at":"2016-12-12T16:58:53Z","updated_at":"2016-12-12T16:58:53Z","body":"> Sparse support needs to be known prior to fit, regardless of whether the fitted model can change in its ability to do a sparse transform etc. Your argument about GridSearch says anything with respect to the ability to be fit on sparse data.\r\n\r\nI'm not sure I follow. If I grid-search and the grid involves a model that supports sparse data and one that doesn't, then it's unclear whether ``best_estimator_`` will support fitting on sparse data.\r\nI guess that is not really what we're interested in.\r\n\r\nArguably we are interested in whether fitting the grid-search on sparse data will succeed. That requires knowing whether all grid-points support fitting on sparse data, which is tricky, but not impossible.\r\nBasically we would need to iterate over the grid, instantiate the models, get the sparse support tag from all of them and call ``all`` on it.\r\n\r\nI guess I was more thinking about \"will calling ``predict`` succeed on a sparse matrix\", which we can only say for sure after ``fit``.\r\n\r\n> Yes, I think we'll need to let estimators opt out of certain tests.\r\n\r\nI implemented that in #8022."}}
{"actor":{"display_login":"amueller"}}