{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4844","repository_url":"https://api.github.com/repos/scikit-learn/scikit-learn","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4844/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4844/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4844/events","html_url":"https://github.com/scikit-learn/scikit-learn/pull/4844","id":87063800,"number":4844,"title":"[MRG] Add KNN strategy for imputation","user":{"login":"tw991","id":8785890,"avatar_url":"https://avatars.githubusercontent.com/u/8785890?v=3","gravatar_id":"","url":"https://api.github.com/users/tw991","html_url":"https://github.com/tw991","followers_url":"https://api.github.com/users/tw991/followers","following_url":"https://api.github.com/users/tw991/following{/other_user}","gists_url":"https://api.github.com/users/tw991/gists{/gist_id}","starred_url":"https://api.github.com/users/tw991/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tw991/subscriptions","organizations_url":"https://api.github.com/users/tw991/orgs","repos_url":"https://api.github.com/users/tw991/repos","events_url":"https://api.github.com/users/tw991/events{/privacy}","received_events_url":"https://api.github.com/users/tw991/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/labels/Needs%20Review","name":"Needs Review","color":"f7c6c7"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":63,"created_at":"2015-06-10T18:35:21Z","updated_at":"2016-03-27T12:50:30Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/pulls/4844","html_url":"https://github.com/scikit-learn/scikit-learn/pull/4844","diff_url":"https://github.com/scikit-learn/scikit-learn/pull/4844.diff","patch_url":"https://github.com/scikit-learn/scikit-learn/pull/4844.patch"},"body":"Trying to solve #2989.\r\n\r\nImputation strategy is following:\r\n\r\nFind all rows with full features as complete set, and impute the missing features in a row by taking the mean of those features among its K-nearest neighbors in complete set. Use scikit-learn.neighbors.NearestNeighbors to find neighbors. Now it can only handle dense matrix. \r\n\r\nBased on this paper: http://web.stanford.edu/~hastie/Papers/missing.pdf\r\n\r\nSimilar package for R: Impute(http://www.bioconductor.org/packages/release/bioc/manuals/impute/man/impute.pdf)\r\nSimilar function for Matlab:\r\nKnnimpute(http://www.mathworks.com/help/bioinfo/ref/knnimpute.html)\r\n\r\nStill working on documentation and examples."},"comment":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/comments/202057561","html_url":"https://github.com/scikit-learn/scikit-learn/pull/4844#issuecomment-202057561","issue_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4844","id":202057561,"user":{"login":"maniteja123","id":6291963,"avatar_url":"https://avatars.githubusercontent.com/u/6291963?v=3","gravatar_id":"","url":"https://api.github.com/users/maniteja123","html_url":"https://github.com/maniteja123","followers_url":"https://api.github.com/users/maniteja123/followers","following_url":"https://api.github.com/users/maniteja123/following{/other_user}","gists_url":"https://api.github.com/users/maniteja123/gists{/gist_id}","starred_url":"https://api.github.com/users/maniteja123/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/maniteja123/subscriptions","organizations_url":"https://api.github.com/users/maniteja123/orgs","repos_url":"https://api.github.com/users/maniteja123/repos","events_url":"https://api.github.com/users/maniteja123/events{/privacy}","received_events_url":"https://api.github.com/users/maniteja123/received_events","type":"User","site_admin":false},"created_at":"2016-03-27T12:50:30Z","updated_at":"2016-03-27T12:50:30Z","body":"Hi everyone, there seems to be some support for adding this feature and @rvraghav93 has also proposed working on this if it is fine. I just wanted to ask some questions based on what I understood from the discussions and having a read at the paper, while also looking at the code in MATLAB( I haven't seen the function implementation in R since it seems to be in Fortran which is new to me). Please correct me if I am understanding it in wrong way here.\r\n\r\n>\r\nThe option of non-euclidean metrics is provided by other implementations. Weighted means should be possible, etc. selecting a feature subspace in which objects are compared, rather than calculating distance over all non-missing features per sample. \r\n>\r\nRight now KNN is used as a strategy with equal weights to all neighbours with no missing values. So giving weights to the K neighbours should also be possible.\r\n\r\n>\r\nAnd yet, if this is just an application of KNNRegressor, we could substitute with any regressor -- will we land up with a generic meta-estimator?\r\n>\r\nIs this related to the idea of having ``neighbours=NearestsNeighbours(...) or LSHForest(..)`` ?\r\n\r\n>\r\nThere are too many parameters to nearest neighbor methods that we're ignoring here. Notably, the cited MatLab function imputes with what our NN module would call weights='distance', but the implementation here assumes uniform weight of neighbors. It also leaves metric configurable. And if we are only considering complete data points as the index, approximate nearest neighbors methods seem appropriate.\r\n>\r\nSince right now euclidean distance is used, similarly other distance metrics can also be used for finding neighbours ?\r\n\r\n\r\n>\r\nWe find the k nearest neighbors using a Euclidean metric, confined to the columns for which that gene is NOT missing. Each candidate neighbor might be missing some of the coordinates used to calculate the distance. In this case we average the distance from the non-missing coordinates.\r\n>\r\nIIUC right now there should be at least K samples with complete (no missing) data. But I am unable to understand this statement. Could you please elaborate on this ?\r\n\r\n>\r\nFurther, one might consider strategy, in terms of mean/median/custom, as orthogonal to knn vs whole dataset. The MatLab package allows median to be used (which may not make a whole lot of sense with euclidean distance, but is easy to justify otherwise). I suspect that if we're not doing the above, we should not be adding strategy=\"knn\" but allowing the user to choose n_neighbors (defaulting to None meaning all) as orthogonal to strategy.\r\n>\r\nRight now as @tw991 mentioned above KNN is a 'strategy', but it should be a neighbor selection which other imputation methods (eg. mean, median,..) would use.\r\n\r\nI am pretty new to this and trying to get an understanding of the methods commonly used. Thank you for your time and bearing with my questions."}}