{"action":"created","issue":{"url":"https://api.github.com/repos/mlpack/mlpack/issues/599","repository_url":"https://api.github.com/repos/mlpack/mlpack","labels_url":"https://api.github.com/repos/mlpack/mlpack/issues/599/labels{/name}","comments_url":"https://api.github.com/repos/mlpack/mlpack/issues/599/comments","events_url":"https://api.github.com/repos/mlpack/mlpack/issues/599/events","html_url":"https://github.com/mlpack/mlpack/issues/599","id":144820057,"number":599,"title":"possible usecases for data preprocessing tool","user":{"login":"keonkim","id":10793962,"avatar_url":"https://avatars.githubusercontent.com/u/10793962?v=3","gravatar_id":"","url":"https://api.github.com/users/keonkim","html_url":"https://github.com/keonkim","followers_url":"https://api.github.com/users/keonkim/followers","following_url":"https://api.github.com/users/keonkim/following{/other_user}","gists_url":"https://api.github.com/users/keonkim/gists{/gist_id}","starred_url":"https://api.github.com/users/keonkim/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/keonkim/subscriptions","organizations_url":"https://api.github.com/users/keonkim/orgs","repos_url":"https://api.github.com/users/keonkim/repos","events_url":"https://api.github.com/users/keonkim/events{/privacy}","received_events_url":"https://api.github.com/users/keonkim/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-03-31T08:05:21Z","updated_at":"2016-04-03T05:01:34Z","closed_at":null,"body":"I am on the designing stage of the dataset and experiment tools (explained in the wiki: https://github.com/mlpack/mlpack/wiki/SummerOfCodeIdeas#dataset-and-experimentation-tools)\r\n\r\nI have a design draft, which I have submitted via GSoC, but I'd like to make the blueprint firm before getting into actual coding.\r\n\r\nIt would be a real help to know what you guys feel the most needed.\r\nIn order to find that out, I want to know possible usecases while preprocessing.\r\nMost frequently used methods, alternative tools, and all the other information regarding it help.\r\n\r\nThank you,\r\nKeon Woo Kim\r\n"},"comment":{"url":"https://api.github.com/repos/mlpack/mlpack/issues/comments/204871813","html_url":"https://github.com/mlpack/mlpack/issues/599#issuecomment-204871813","issue_url":"https://api.github.com/repos/mlpack/mlpack/issues/599","id":204871813,"user":{"login":"stereomatchingkiss","id":1801914,"avatar_url":"https://avatars.githubusercontent.com/u/1801914?v=3","gravatar_id":"","url":"https://api.github.com/users/stereomatchingkiss","html_url":"https://github.com/stereomatchingkiss","followers_url":"https://api.github.com/users/stereomatchingkiss/followers","following_url":"https://api.github.com/users/stereomatchingkiss/following{/other_user}","gists_url":"https://api.github.com/users/stereomatchingkiss/gists{/gist_id}","starred_url":"https://api.github.com/users/stereomatchingkiss/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stereomatchingkiss/subscriptions","organizations_url":"https://api.github.com/users/stereomatchingkiss/orgs","repos_url":"https://api.github.com/users/stereomatchingkiss/repos","events_url":"https://api.github.com/users/stereomatchingkiss/events{/privacy}","received_events_url":"https://api.github.com/users/stereomatchingkiss/received_events","type":"User","site_admin":false},"created_at":"2016-04-03T05:01:34Z","updated_at":"2016-04-03T05:01:34Z","body":"Following are my personal opinions, just take it as reference.\r\n\r\n>It would be a real help to know what you guys feel the most needed\r\n\r\n1 : Split a dataset into a training and test set(#523 may give you some helps)\r\n2 : Checking a dataset for loading problems and printing errors, the api/cli should be able to give users imputation strategies for missing variables, give warning to invalid, problematic values\r\n3 : converting categorical features into binary features (or numeric features)\r\n\r\nThe command line tools I feel the most needed are\r\n1 : mlpack_datatools_trans\r\n2 : mlpack_datatools_stats\r\n3 : mlpack_datatools_math\r\n4 : mlpack_datatools_eval\r\n5 : mlpack_datatools_io\r\n\r\nAbout the mlpack_datatools_io, I am not sure it is a good idea to increase dependency of mlpack or not, it would be great if you could make them as optional for users. In short, the dependency would not affect the build process if the users do not need those features which rely on external libraries.\r\n\r\nAbout the time-zone, which time-zone you want to support?How do users specify the format of the time-zone?Boost date_time may help you save some headaches.\r\n\r\n>I want to know possible usecases while preprocessing.\r\n\r\nBy usual I do not check the validity of the data, just assume they are valid, I think I can't help you about this part. \r\n\r\n>Most frequently used methods, alternative tools, and all the other information regarding it help.\r\n\r\nAbout the statistic part, armadillo and boost accumulators provide a lot of useful functions, sometimes I also use the function of opencv to do the statistic when I am doing computer vision tasks.\r\n\r\nAbout the database manipulation, I always rely on Qt5, whatever, it is too big if your purpose is provide database manipulation only. \r\n\r\nI could help you do the test on windows platform(win8, vc2015 64bits)"}}