{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-learn-contrib/categorical-encoding/issues/161","repository_url":"https://api.github.com/repos/scikit-learn-contrib/categorical-encoding","labels_url":"https://api.github.com/repos/scikit-learn-contrib/categorical-encoding/issues/161/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn-contrib/categorical-encoding/issues/161/comments","events_url":"https://api.github.com/repos/scikit-learn-contrib/categorical-encoding/issues/161/events","html_url":"https://github.com/scikit-learn-contrib/categorical-encoding/issues/161","id":395262731,"node_id":"MDU6SXNzdWUzOTUyNjI3MzE=","number":161,"title":"Multi-hot encoding for ambiguous input","user":{"login":"fullflu","id":9534465,"node_id":"MDQ6VXNlcjk1MzQ0NjU=","avatar_url":"https://avatars3.githubusercontent.com/u/9534465?v=4","gravatar_id":"","url":"https://api.github.com/users/fullflu","html_url":"https://github.com/fullflu","followers_url":"https://api.github.com/users/fullflu/followers","following_url":"https://api.github.com/users/fullflu/following{/other_user}","gists_url":"https://api.github.com/users/fullflu/gists{/gist_id}","starred_url":"https://api.github.com/users/fullflu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fullflu/subscriptions","organizations_url":"https://api.github.com/users/fullflu/orgs","repos_url":"https://api.github.com/users/fullflu/repos","events_url":"https://api.github.com/users/fullflu/events{/privacy}","received_events_url":"https://api.github.com/users/fullflu/received_events","type":"User","site_admin":false},"labels":[{"id":293058068,"node_id":"MDU6TGFiZWwyOTMwNTgwNjg=","url":"https://api.github.com/repos/scikit-learn-contrib/categorical-encoding/labels/enhancement","name":"enhancement","color":"84b6eb","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2019-01-02T14:21:36Z","updated_at":"2019-01-04T07:15:54Z","closed_at":null,"author_association":"NONE","body":"I propose to implement simple multi-hot encoding which allows ambiguous input and outputs non-negative value.\r\n\r\nLet x_j be a realization of department of a student.\r\nUsually, we assume that x_j is defined without ambiguity, such as mathematics, physics, and so on.\r\nIn real-world dataset, however, we sometimes only know the ambiguous value, such as sciences. I want to encode such ambiguous categorical features.\r\n\r\nI implemented fit and transform function and tested them by several cases (WIP).\r\nI focused on the case in which ambiguous value is represented by a delimiter (as 'mathematics|physics', which means x_j is mathematics or physics). \r\n\r\nI hope you to discuss the potential and usefulness of this type of implementation. I wonder if, at least, following updates are necessary:\r\n* implement inverse_transform function\r\n* handle impute parameter\r\n* implement not only 'or' type delimiter, but also 'and' type delimiter\r\n* reflect latest research in the field of data mining.\r\n\r\nThe difference of the related issue may be as follows:\r\n* #77 : I simply encode ambiguous|dirty feature as well as one-hot encoding. I did not consider the similarity.\r\n* #136 : Target encoding may be useful for ambiguous|dirty feature, but I focus on simple multi-hot encoding.\r\n\r\n\r\nThe merit of multi-hot encoding is its simplicity and efficiency.\r\n\r\n* Simplicity: \r\nWe only need to insert delimiter string into dirty categorical feature. \r\n* Efficiency: \r\nIf we prepare a mapping which represents relationships between ambiguous|dirty categories and feature without ambiguity, we will need a lot of memory capacity for j-th feature (O(2^C_j), where C_j is the cardinality of j-th feature).\r\nMulti-hot encoding needs O(C_j) memory.\r\n\r\n\r\nSoon I will send a pull request. I look forward to hearing from you."},"comment":{"url":"https://api.github.com/repos/scikit-learn-contrib/categorical-encoding/issues/comments/451368507","html_url":"https://github.com/scikit-learn-contrib/categorical-encoding/issues/161#issuecomment-451368507","issue_url":"https://api.github.com/repos/scikit-learn-contrib/categorical-encoding/issues/161","id":451368507,"node_id":"MDEyOklzc3VlQ29tbWVudDQ1MTM2ODUwNw==","user":{"login":"fullflu","id":9534465,"node_id":"MDQ6VXNlcjk1MzQ0NjU=","avatar_url":"https://avatars3.githubusercontent.com/u/9534465?v=4","gravatar_id":"","url":"https://api.github.com/users/fullflu","html_url":"https://github.com/fullflu","followers_url":"https://api.github.com/users/fullflu/followers","following_url":"https://api.github.com/users/fullflu/following{/other_user}","gists_url":"https://api.github.com/users/fullflu/gists{/gist_id}","starred_url":"https://api.github.com/users/fullflu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fullflu/subscriptions","organizations_url":"https://api.github.com/users/fullflu/orgs","repos_url":"https://api.github.com/users/fullflu/repos","events_url":"https://api.github.com/users/fullflu/events{/privacy}","received_events_url":"https://api.github.com/users/fullflu/received_events","type":"User","site_admin":false},"created_at":"2019-01-04T07:15:54Z","updated_at":"2019-01-04T07:15:54Z","author_association":"NONE","body":"Thank you for your response.\r\n\r\n> It would be nice if the code included a reference to some article or a blog post that would illustrate on a trivial example how the encoder works.\r\n\r\nOK, I will write a blog that uses this encoder reflecting the discussion in the Pull-Request."}}
{"actor":{"display_login":"fullflu"}}