{"action":"created","issue":{"url":"https://api.github.com/repos/AtheMathmo/rusty-machine/issues/112","repository_url":"https://api.github.com/repos/AtheMathmo/rusty-machine","labels_url":"https://api.github.com/repos/AtheMathmo/rusty-machine/issues/112/labels{/name}","comments_url":"https://api.github.com/repos/AtheMathmo/rusty-machine/issues/112/comments","events_url":"https://api.github.com/repos/AtheMathmo/rusty-machine/issues/112/events","html_url":"https://github.com/AtheMathmo/rusty-machine/issues/112","id":168504505,"number":112,"title":"Implement some basic data transformations","user":{"login":"AtheMathmo","id":10150986,"avatar_url":"https://avatars.githubusercontent.com/u/10150986?v=3","gravatar_id":"","url":"https://api.github.com/users/AtheMathmo","html_url":"https://github.com/AtheMathmo","followers_url":"https://api.github.com/users/AtheMathmo/followers","following_url":"https://api.github.com/users/AtheMathmo/following{/other_user}","gists_url":"https://api.github.com/users/AtheMathmo/gists{/gist_id}","starred_url":"https://api.github.com/users/AtheMathmo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AtheMathmo/subscriptions","organizations_url":"https://api.github.com/users/AtheMathmo/orgs","repos_url":"https://api.github.com/users/AtheMathmo/repos","events_url":"https://api.github.com/users/AtheMathmo/events{/privacy}","received_events_url":"https://api.github.com/users/AtheMathmo/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/AtheMathmo/rusty-machine/labels/help%20wanted","name":"help wanted","color":"159818"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-07-31T06:30:57Z","updated_at":"2016-08-04T19:50:01Z","closed_at":null,"body":"It would be nice to begin supporting some basic data transformations. Things like scaling, etc. A model similar to sickit learn's transform interface would be a good jumping off point.\r\n\r\nThe following transformations would be nice for a start:\r\n\r\n- Centering around zero with unit variance.\r\n- Scaling Min/Max to some range.\r\n\r\nTo achieve this we should have a new trait defining our transform API. Something like this:\r\n\r\n```rust\r\npub trait Transformer<T> {\r\n    /// Transforms the inputs and stores the transformation in the Transformer\r\n    fn transform(&mut self, inputs: T) -> Result<T, learning::Error>;\r\n\r\n    /// Reverses the transform to recover the original data\r\n    fn inv_transform(&self, inputs: T) -> Result<T, learning::Error>;\r\n}\r\n```\r\n\r\nSo our min max scaler may look like:\r\n\r\n```rust\r\npub struct MinMaxScaler {\r\n    /// Mins per column of input data\r\n    input_min: Vec<usize>,\r\n    /// Maxs per column of input data\r\n    input_max: Vec<usize>,\r\n    /// The min of the new data (default 0)\r\n    scaled_min: f64,\r\n    /// The max of the new data (default 1)\r\n    scaled_max: f64,\r\n}\r\n\r\nimpl Transformer<Matrix<f64>> for MinMaxScaler {\r\n    fn transform(&mut self, inputs: Matrix<f64>) -> Result<Matrix<f64>, learning::Error> {\r\n        // Find min and max of each column of features\r\n        // For each column:\r\n        //    Subtract the respective input_min and add the scaled_min\r\n        //    Set each value x to x * (scaled_max / input_max)\r\n    }\r\n\r\n    fn inv_transform(&self, inputs: Matrix<f64>) -> Result<Matrix<f64>, learning::Error> {\r\n        // Should fail if the Transformer has not been used to transform data yet\r\n        //\r\n        // Do the inverse of the transform:\r\n        // For each column:\r\n        //    Set each value x to x * (input_max / scaled_max)\r\n        //    Subtract the scaled_min and add input_min\r\n    }\r\n}\r\n```\r\n\r\nThe above is just to illustrate how this may look. Further requirements may mean something slightly different is needed. Some caveats we should be aware of:\r\n\r\n- How to handle constant features? (If the std dev is zero we don't want to divide by this in scaling)\r\n- How to handle class data? (i.e. values 0,1,2,3 representing one of 4 classes)\r\n\r\nThere are probably some other cases we should consider. But I think for now we can capture these as fail cases or just ignore them completely..."},"comment":{"url":"https://api.github.com/repos/AtheMathmo/rusty-machine/issues/comments/237663575","html_url":"https://github.com/AtheMathmo/rusty-machine/issues/112#issuecomment-237663575","issue_url":"https://api.github.com/repos/AtheMathmo/rusty-machine/issues/112","id":237663575,"user":{"login":"AtheMathmo","id":10150986,"avatar_url":"https://avatars.githubusercontent.com/u/10150986?v=3","gravatar_id":"","url":"https://api.github.com/users/AtheMathmo","html_url":"https://github.com/AtheMathmo","followers_url":"https://api.github.com/users/AtheMathmo/followers","following_url":"https://api.github.com/users/AtheMathmo/following{/other_user}","gists_url":"https://api.github.com/users/AtheMathmo/gists{/gist_id}","starred_url":"https://api.github.com/users/AtheMathmo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AtheMathmo/subscriptions","organizations_url":"https://api.github.com/users/AtheMathmo/orgs","repos_url":"https://api.github.com/users/AtheMathmo/repos","events_url":"https://api.github.com/users/AtheMathmo/events{/privacy}","received_events_url":"https://api.github.com/users/AtheMathmo/received_events","type":"User","site_admin":false},"created_at":"2016-08-04T19:50:01Z","updated_at":"2016-08-04T19:50:01Z","body":"Most of the logic for this is handled in #116. However we still need to define exactly which methods the `Transformer` trait will require.\r\n\r\nMy plan for now is to only implement those described above. If others see this issue and think I'm missing some crucial use case please speak up. The motivation for keeping it to the above is that it should be easy to extend the trait later."}}
{"actor":{"display_login":"AtheMathmo"}}