{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/10482","repository_url":"https://api.github.com/repos/scikit-learn/scikit-learn","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/10482/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/10482/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/10482/events","html_url":"https://github.com/scikit-learn/scikit-learn/pull/10482","id":289011100,"number":10482,"title":"[MRG] Generalize the use of precomputed sparse distance matrices, for estimators which use nearest neighbors","user":{"login":"TomDLT","id":11065596,"avatar_url":"https://avatars2.githubusercontent.com/u/11065596?v=4","gravatar_id":"","url":"https://api.github.com/users/TomDLT","html_url":"https://github.com/TomDLT","followers_url":"https://api.github.com/users/TomDLT/followers","following_url":"https://api.github.com/users/TomDLT/following{/other_user}","gists_url":"https://api.github.com/users/TomDLT/gists{/gist_id}","starred_url":"https://api.github.com/users/TomDLT/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TomDLT/subscriptions","organizations_url":"https://api.github.com/users/TomDLT/orgs","repos_url":"https://api.github.com/users/TomDLT/repos","events_url":"https://api.github.com/users/TomDLT/events{/privacy}","received_events_url":"https://api.github.com/users/TomDLT/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":9,"created_at":"2018-01-16T18:29:49Z","updated_at":"2018-02-13T23:00:43Z","closed_at":null,"author_association":"OWNER","pull_request":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/pulls/10482","html_url":"https://github.com/scikit-learn/scikit-learn/pull/10482","diff_url":"https://github.com/scikit-learn/scikit-learn/pull/10482.diff","patch_url":"https://github.com/scikit-learn/scikit-learn/pull/10482.patch"},"body":"This PR implements solution (A) proposed in #10463.\r\n\r\nIt generalizes the use of precomputed sparse distance matrices, and proposes to use pipelines to chain nearest neighbors with other estimators.\r\nMore precisely, it introduces a new estimator `NearestNeighborsTransformer`, which transforms an input X into a (weighted) graph of its nearest neighbors. The output is a sparse distance/affinity matrix.\r\n\r\nThe proposed usecase is the following:\r\n```py\r\nfrom sklearn.pipeline import make_pipeline\r\nfrom sklearn.neighbors import NearestNeighborsTransformer\r\nfrom sklearn.manifold import TSNE\r\n\r\nest_chain = make_pipeline(\r\n    NearestNeighborsTransformer(\r\n        n_neighbors=n_neighbors, mode='distance', metric=metric,\r\n        include_self=False),\r\n    TSNE(metric='precomputed', method=\"barnes_hut\"),\r\n    memory='path/to/cache',\r\n)\r\n\r\nest_compact = TSNE(metric=metric, method=\"barnes_hut\")\r\n```\r\nIn this example `est_chain` and `est_compact` are equivalent, but `est_chain` gives more control on the nearest neighbors estimator. Moreover, it can use the pipeline caching properties, to reuse the same nearest neighbors matrix for different TSNE parameters. Finally, it is possible to replace `NearestNeighborsTransformer` with a custom nearest neighbors estimator returning the graph of connections with the nearest neighbors.\r\n___\r\nThis pipeline works with:\r\n- `TSNE`, `Isomap`, `SpectralEmbedding`\r\n- `DBSCAN`, `SpectralClustering`,\r\n- `KNeighborsClassifier`, `RadiusNeighborsClassifier`, `KNeighborsRegressor`, `RadiusNeighborsRegressor`, `LocalOutlierFactor`\r\n\r\nThis pipeline cannot work with:\r\n- `LocallyLinearEmbedding` (which needs the original X)\r\n___\r\nPossible improvements:\r\n- Add a kernel parameter to transform distances into affinities\r\n- Add symmetric option for affinities (to remove warning in SpectralEmbedding, SpectralClustering)"},"comment":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/comments/365434921","html_url":"https://github.com/scikit-learn/scikit-learn/pull/10482#issuecomment-365434921","issue_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/10482","id":365434921,"user":{"login":"jnothman","id":78827,"avatar_url":"https://avatars2.githubusercontent.com/u/78827?v=4","gravatar_id":"","url":"https://api.github.com/users/jnothman","html_url":"https://github.com/jnothman","followers_url":"https://api.github.com/users/jnothman/followers","following_url":"https://api.github.com/users/jnothman/following{/other_user}","gists_url":"https://api.github.com/users/jnothman/gists{/gist_id}","starred_url":"https://api.github.com/users/jnothman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jnothman/subscriptions","organizations_url":"https://api.github.com/users/jnothman/orgs","repos_url":"https://api.github.com/users/jnothman/repos","events_url":"https://api.github.com/users/jnothman/events{/privacy}","received_events_url":"https://api.github.com/users/jnothman/received_events","type":"User","site_admin":false},"created_at":"2018-02-13T23:00:43Z","updated_at":"2018-02-13T23:00:43Z","author_association":"OWNER","body":"Test failures"}}
{"actor":{"display_login":"jnothman"}}