{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/7635","repository_url":"https://api.github.com/repos/scikit-learn/scikit-learn","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/7635/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/7635/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/7635/events","html_url":"https://github.com/scikit-learn/scikit-learn/issues/7635","id":182125204,"number":7635,"title":"Parallelize transformers?","user":{"login":"themrmax","id":8220288,"avatar_url":"https://avatars.githubusercontent.com/u/8220288?v=3","gravatar_id":"","url":"https://api.github.com/users/themrmax","html_url":"https://github.com/themrmax","followers_url":"https://api.github.com/users/themrmax/followers","following_url":"https://api.github.com/users/themrmax/following{/other_user}","gists_url":"https://api.github.com/users/themrmax/gists{/gist_id}","starred_url":"https://api.github.com/users/themrmax/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/themrmax/subscriptions","organizations_url":"https://api.github.com/users/themrmax/orgs","repos_url":"https://api.github.com/users/themrmax/repos","events_url":"https://api.github.com/users/themrmax/events{/privacy}","received_events_url":"https://api.github.com/users/themrmax/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2016-10-10T21:58:19Z","updated_at":"2016-10-13T20:20:15Z","closed_at":null,"body":"As raised in http://stackoverflow.com/questions/39948138/sklearn-featurehasher-parallelized/39951415 many (all?) transformers could be made parallel, would this make sense as a `ParallelTransformerMixin`, something like\r\n\r\n\r\n```python\r\nfrom sklearn.externals.joblib import Parallel, delayed\r\nimport numpy as np\r\nimport scipy.sparse as sp\r\n\r\nclass ParallelTransformerMixin:\r\n    def transform_parallel(self, X, n_jobs):\r\n        transform_splits = Parallel(n_jobs=n_jobs, backend=\"threading\")(\r\n            delayed(self.transform)(X_split)\r\n            for X_split in np.array_split(X, n_jobs))\r\n\r\n    return sp.vstack(transform_splits)\r\n```"},"comment":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/comments/253627258","html_url":"https://github.com/scikit-learn/scikit-learn/issues/7635#issuecomment-253627258","issue_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/7635","id":253627258,"user":{"login":"amueller","id":449558,"avatar_url":"https://avatars.githubusercontent.com/u/449558?v=3","gravatar_id":"","url":"https://api.github.com/users/amueller","html_url":"https://github.com/amueller","followers_url":"https://api.github.com/users/amueller/followers","following_url":"https://api.github.com/users/amueller/following{/other_user}","gists_url":"https://api.github.com/users/amueller/gists{/gist_id}","starred_url":"https://api.github.com/users/amueller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amueller/subscriptions","organizations_url":"https://api.github.com/users/amueller/orgs","repos_url":"https://api.github.com/users/amueller/repos","events_url":"https://api.github.com/users/amueller/events{/privacy}","received_events_url":"https://api.github.com/users/amueller/received_events","type":"User","site_admin":false},"created_at":"2016-10-13T20:20:15Z","updated_at":"2016-10-13T20:20:15Z","body":"This is very closely related to #7448 which proposed the same for ``predict``.\r\n\r\nI'm not sure how this would work as a mixin. Would all transformers get it? Then it should be in ``TransformerMixin``, right?\r\n\r\nThis wouldn't work in pipelines, unless we add this to pipelines...\r\n\r\nWhy not do it as a meta-estimator? That way we don't have to mess with ``Pipeline``."}}
{"actor":{"display_login":"amueller"}}